Another 'Starry Night'?
Imagine this: one day, you open Instagram to post a photo of your cat snoozing on an armchair, only to find a crop of dazzling new filters: Picasso, for example; van Gogh; Kandinsky. What if rendering your everyday surroundings in the familiar, yet strange, palettes of famous artists was as simple as selecting one from a menu?
Of course, such a task is far from simple, but a recent paper from a German research team suggests it's not entirely impossible.
The researchers input a photo and a work of art into their model to achieve a combination of the content of the photo (a row of houses) and the aesthetic of each painting.
In the paper, "A Neural Algorithm of Artistic Style," Leon A. Gatys, Alexander S. Ecker and Matthias Bethge demonstrate a visual model meant to allow the content and style of an image to be separated and combined in various iterations. Inputting a photo of a row of houses, the researchers used their model to blend the content of the photo with the distinctive visual styles of famous artists such as Edvard Munch and Wassily Kandinsky.
This process, of course, by no means mirrors the quick, easy application of an Instagram filter. As The Observer's Ryan Steadman points out, "the process takes about an hour to complete."
In the left-hand column, the style is emphasized and the content difficult to perceive. On the right-hand column, the emphasis on content leads to a less well-matched style to the input painting, Kandinsky's "Composition VII."
What's more, the authors write, "image content and style cannot be completely disentangled [...] there usually does not exist an image that perfectly matches both constraints at the same time."
The model works not through a simple filtration -- the authors point to non-photorealistic rendering and texture transfer as methods for stylizing photos in a more straightforward vein -- but by processing each input image in layers, extracting characteristic visual components in a hierarchy.
In other words, the authors weren't simply applying a filter, but making choices about which input -- style or content -- to weight more heavily in creating the composite image, and aiming for a balance between the two.
The results, as demonstrated in the paper, are stunning, and it's tempting to quickly leap to the possibility of an app that could make us all look like the next Picasso -- or at least give us a hint of what it would look like if Picasso painted our cat sleeping on the armchair. "The group should probably try to set up a meeting with Mark Zuckerberg," advised Steadman.
In the age of Instagram, we’re all talented photographers -- or at least we’re a little better at convincing ourselves we are. Throwing a Valencia filter on a snapshot may make your amateur pic look a little more polished, but it’s a faint imitation of the care and craft that professional photographers infuse into their art. Blending "The Scream" with your own photo, on the other hand, may bear little resemblance at all to how the real Edvard Munch would portray that scene, and it certainly wouldn't be an expression of your own creativity.
The same photo, through a Picasso's 'Femme neue assise' filter.
The original authors aren't necessarily making such startup-friendly claims. Their stated goals lean more academic, as they write that their model "provides a new, fascinating tool to study the perception and neural representation of art, style and content-independent image appearance in general." The lab has previously geared similar studies toward ends that should delight art history nerds, such as a paper on using a similar model to determine the artist of a particular work using algorithmic stylistic analysis.
Not as kitschy-fun as remixing your selfies through a Kandinsky filter, perhaps, but in the long run, probably a lot more beneficial for the art world.
H/T The Observer
Also on HuffPost:
