Image cropping in the machine learning age

Georg Obermayr

Image cropping has increasingly become a pain in the ass. With content flowing more freely in the web and on different brand platforms like apps or social media more and more crops of the same image are needed. Even if you just think of a website you often need half a dozen or more different crop renditions of an image. Currently there are two technologies to crop images automatically:

There is clearly a lot for room of improvement here. Twitter is rolling out a new cropping algorithm powered by machine learning. This is based on getting to get the most salient region of an image. There has been a lot of research on this, academics did eye tracking studies to understand which parts of an image humans fixate when looking at it. Mostly this seem to be faces, text, animals, objects and regions of high contrast. Based on that insights a neural network can be trained to find the salient region—and center the crop around it.

Twitter has now developed these algorithms further to use them with high performance in a large-scale production situation. The results are stunning and show the clear advantages of this approach.

Images cropped with the old logic …
… and with the new one powered by machine learning. Source and more about the research:

I think the possibilities are endless: From automatic crops in all kinds of digital media but also in print layouts. Imagine you could just size a frame (by a layout application box, CSS or just proportions), assign an image to it and it is automatically cropped in the best way possible. I hope to see this very soon in print applications, digital infrastructure and DAM systems.

Read next The learning robots are coming

Artificial intelligence is about to change the workplace. And a lot more. Media production, used to change for years, is about to be disrupted again. Read more …

Georg Obermayr

I’m one of those guys in the media production and publishing scene, that is often labeled as a thought leader. But I’m a practitioner. Day in and day out I work as Head of Crossmedia Production in an advertising agency. I’m hands on creating content infrastructures and designing websites, apps and social media stuff that are driven by these infrastrucutures. This it what grounds me. And it is this daily business work that helps me identifying the trends and emerging topics of our field. With that kind of real world knowledge, I’m an active participant in bringing our industry forward: I write a lot about agile publishing, digital publishing, development, and media production, not just here but also in well know magazines and journals. I’m a keynote speaker at conferences and do a lot of trainings and consulting work. Since I’m originally a print person, I was involved in developing industry guidelines for PDFX-ready. I co-authored the book “Agile Publishing”, still the 400 pages reference work on how agile processes move user experience and storytelling in the spotlight of todays multichannel world. I’m living at the intersection of design, content, technology and marketing. How hypes can be moved into practical use is what drives me every day.
Buy the book "Agile Publishing" on Amazon