Can a computer learn to paint with your style?
DeepArt is a free online computer algorithm that claims to transform your photos into painterly images, using your own painting style to guide the computer.
I thought I'd try it out by uploading a photo (left) and a closely related plein-air oil painting (right) to see what the algorithm comes up with.
After waiting about 10 hours, I got an email saying my image was ready:
The result is pretty disappointing, with strange dark slashes in the sky that weren't in either the photo or the painting. The rest of the building looks really crudely painted.
I tried it once more with a photo and a painting that were really different.
The output uses the color scheme of the painting, and grafts paint-like textures to match the relative values. Beyond that it didn't make any choices that I would regard as artistic. Although the result is marginally more interesting than a Photoshop filter, it doesn't look like a painting. The algorithm doesn't do well with faces, which require a particular attention to the eyes and mouth.
Despite the shortcomings of this algorithm, it's easy to imagine the power of future software that not only patches together stylistic fingerprints, but also uses strategies of machine perception and image parsing.
----
You can also try their Visual Turing Test.
Related posts:
Fascinating. Could you show us a detailed section of the machine-generated images so we can see the quirkiness? I am especially interested in how the texture was applied to the paint of the building and how the face breaks down on closer examination.
ReplyDeleteThe computer appears to have simply repurposed chunks of your painting to try to match the visual information in the photo. Take a closer look at some of the "strange slashes in the sky" in the first outputted result -- they're pieces of the roof and wall of the buildings you painted.
ReplyDeleteI can remember when the first large format digital printing came along, I was painting as a pictorial artist in the outdoor industry. We laughed at it. We assumed the advertising world would never except the low quality, boy were ever wrong!
ReplyDeleteMachine "paints" based on past "man" decisions (reacting to a specific subject). That being, in its purest sense, an inconquerable limitation of machine. But the jury still out.
ReplyDeleteFor your first experiment with DeepArt, when you report "The rest of the building looks really crudely painted.", what caught my eye is the "looks ... painted" part.
ReplyDeleteCompute confused the electric wires with some sort of roof and didn't exactly know what to do with those triangles and rectangles the wires represent. Maybe if you'd given them a painting with poles and wires it would have done a better job. It didn't do a bad job on the building.
ReplyDeleteWe can already make a digital photo look like a convincing 19th century cabinet photo with a push of the button. It has its uses for illustrative purposes, but it doesn’t warm my heart. The same for digitally painted photographs, no matter how convincing. Not to state the obvious, but in painting, it's the performance of humans, not machines, that we're looking to connect with.
ReplyDeleteProgrammer here! I'm skeptical of DeepArt, especially since they only asked for one reference image from you. You can, for example, teach a computer to draw a cat on its own, but you have to show it thousands of cats. Similarly, you should be able to teach a computer to imitate your painting style, but you'd have to feed it hundreds, maybe thousands of paintings for it to be able to make the same decisions you would (and even then, it would be limited to what it knew from that sample). So, yeah... seems like a glorified Photoshop filter.
ReplyDeleteIt'll be a sad day when machines replace artists! Then what will we do with all this extra time put into learning and experimenting and simply enjoying the process of painting...
ReplyDeleteUntil they can teach a machine to make artistic decisions how could it replace a person? It's more than brushstrokes. How much editing does a good artist do? Move this tree over here, change the shape of that bush, leave that trash can out, move that car, emphasize the cool on those shadows, warm and brighten the focal area, etc, etc. Will a computer ever be able to make those kinds of decisions in a truly artistic way? Would a computer ever even able to decide what the image should be about? Also, computers are not experiencing machines, (at least not yet) isn't that why we sketch and paint outdoors right in front of the actual subject? We all know how that significantly effects the result.
ReplyDeleteThe computer should be able to perfectly duplicate your style James. You, of all artists, have already provided the means. The programmers just need to input all your books and videos! ;-)
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHave you taken a look at Filter Forge (filterforge.com). This is no ordinary filter/plugin. I have a copy and been able to do some things I really liked. There are ways of doing things with the programs filters that can get amazing results. This is not a plug for the company, and I get no benefit from them. Just thought you might want to take a look.
ReplyDeleteThe images are far too small in the turing test...
ReplyDelete