Thursday, October 27, 2011

Inserting elements into a photograph

Here are four photos of a real scene. In three of the images, the objects on the table are rendered by various 3D digital imaging methods.


The objects are real in only one of them. Can you tell which one?


Rendering Synthetic Objects into Legacy Photographs from Kevin Karsch on Vimeo.

(Video Link) The following video demonstrates a method called LuxRender, introduced at the 2011 SIGGRAPH convention by a team from the University of Illinois. It allows users to insert virtual elements into a pre-existing photograph.

A few easy controls allow you to input the parameters of the room and its light sources. Then the software generates all the diffuse and specular surface effects, glowing light interactions, cast shadows, and occlusion shadows.

As a traditional painter who only watches the technology from the sidelines, I find all this stuff very impressive, inspiring, and a little scary. When I’m doing a realistic painting of an imaginary scene, I know how hard it taxes my brain to figure out these complex lighting interactions. The same is true today, I suppose, for 2D digital painters.

Now our machine brothers can do the thinking for us. It’s amazing to see how the computer can make these subtle 3D judgments so effortlessly, especially given that it is inferring the light sources from an existing 2D photo image.

What are the artistic implications of this technology? Once this sort of software finds its way into the hands of everyday users and magazine editors, our visual environment will be flooded with ever more fishy photos. I can put your car in my driveway or my flying saucer on the roof of your house. And I suppose these tools will save a lot of tedious labor in the live action visual effects field.

Links
Video of LuxRender on Vimeo (with abstract)
Color and Light: A Guide for the Realist Painter
thanks,  Steve Merryman 
at

20 comments:

  1. MrCachetOctober 27, 2011 at 7:52 AM

    I guess now it will be - "You can't believe everything you see".

    ReplyDelete
  2. UnknownOctober 27, 2011 at 8:11 AM

    This comment has been removed by the author.

    ReplyDelete
  3. Torbjörn KällströmOctober 27, 2011 at 8:43 AM

    I saw it. Do I get a cookie now? :)

    The way the refraction looked in the glass on all the other pictures gave it away.

    ReplyDelete
  4. LarryOctober 27, 2011 at 9:05 AM

    I guess A

    ReplyDelete
  5. SteveOctober 27, 2011 at 10:54 AM

    I've somethimes wondered whether this capacity for altering digital images ever has an effect on law enforcement. Do artificially added ingredients to a picture leave some kind of digital fingerprint, so that an expert analyzing the image easily detects what is real and what is not? It seems there are cases which come to trial based on photographic or video evidence. Photoshop has been around a long time, but the software in this video seems to offer even greater potential for creating photos or videos of situations that don't exist -- something that could be misused by criminals or by police.

    ReplyDelete
  6. Carol Scown-RaynalOctober 27, 2011 at 11:00 AM

    I would photo "D" (shadown on brick pot).

    ReplyDelete
  7. AnonymousOctober 27, 2011 at 12:27 PM

    I want to say B...the lighting just looks a bit more natural to me.

    Also, in A, the line from the wall that shows through the glass is reversed compared to the rest. I'm not sure which way is 'supposed' to look right, so maybe it's A then, haha

    ReplyDelete
  8. Cynthia NicoleOctober 27, 2011 at 12:36 PM

    This comment has been removed by the author.

    ReplyDelete
  9. Cynthia NicoleOctober 27, 2011 at 12:37 PM

    Thank you for turning me on to this. I had not heard of it---pretty amazing, and it is the future.

    ReplyDelete
  10. MikeOctober 27, 2011 at 1:05 PM

    This is nothing new. Cottingley Fairies had A.C. Doyle hoaxed good. I am not fearful but expect that people will not be taken in so easily.

    ReplyDelete
  11. DougOctober 27, 2011 at 1:14 PM

    I'm going to say "A"

    There seems to be intelligent editing in b-d concerning the pink ball and crystal figure. In A the ball's reflection in the black case is so strong, that it almost appears like the ball is bleeding into the case, and the crystal figure almost disappears into the table, in pictures b-d there seems to be an attempt to separate both these elements. The line distortion in the cup seems to follow the cups shape more accurately in A as well.

    If I'm wrong, I don't feel bad, these are very well done!

    ReplyDelete
  12. runningheadOctober 27, 2011 at 5:01 PM

    I'd say A is real, the ball looks a little too interesting in the others and the reflections in the glass in A seem quite messy. I work a lot with both 3D apps and Photoshop, investing a lot of effort in faking natural media for the company I work for. I still can't say for certain which is real! Perhaps a higher resolution image would make it more apparent. Maybe they're all fake! I'm not at all worried about the future of commercial artists. As Syd Myers once said: "If you cross a million dollar computer with a dumb idea, you get a million dollar dumb idea." This tech needs a qualified and exlperienced pilot, and probably will for at least another 20 years.

    ReplyDelete
  13. SVSARTOctober 27, 2011 at 5:15 PM

    I think "A" is real.... The panel division (black line) shown through the water glass seems too crisp in the other photos given the distance that should be between the objects. At least that's what I am thinking, hard to tell either way though, technology sure has come a long ways.

    ReplyDelete
  14. bzyglowiOctober 27, 2011 at 8:01 PM

    I would say it's either A or D. D seems to have the most realistic lighting on the pot, but A has the more interesting reflection in the glass, which makes me lean more towards A than anything else.

    ReplyDelete
  15. Ernest Friedman-HillOctober 28, 2011 at 12:42 AM

    There are a number of different mathematical tools for detecting image alteration. Here's a cool online demo of one such technique: http://errorlevelanalysis.com/

    ReplyDelete
  16. runningheadOctober 28, 2011 at 4:21 AM

    Ah, just watched the video. I'm glad I don't work in movie FX.

    ReplyDelete
  17. davidmaasOctober 28, 2011 at 5:38 AM

    @Ernest:
    the http://errorlevelanalysis.com/ tool is very interesting, though I doubt it would be very helpful in this instance, as it is detecting pixel variations that would result in human-authored photoshop-type editing or compression-type artifacts. The digital scenes are all 'sincere' on the sense of their pixel - to - pixel values...

    I spotted the right image, but only by noting the 3:1 proportion of difference to the others. (The glass is a tad bit dirty in only one image.) Other than that? I don't think I'd be able to. A clever, well-done manipulation will not be detectable. Scary.

    ReplyDelete
  18. Chris BeatriceOctober 28, 2011 at 10:16 AM

    I say A, mostly because of the translucency of the wine glass vs. the others. Total gut feel...

    Re tech in general - this is something I think about every day! Ultimately the ability to make a good picture will separate the good and great artists from the rest, but the fact is, many of us have spent a lot of time and effort gaining knowledge and skills formerly only available to the *human* mind (i.e. "rendering") to support picture making. This can increasingly be done by machines - and that's true even if your work is not realistic in the photographic sense. 3d animated movies show how effortlessly machines can render "realistic cartoons," and once the characters and scenes are set up then infinite poses and angles are possible. Sure there are still concept artists behind those characters, but the "finished art" is done by machine. Ultimately I guess it's good for everyone that the tools and skills become less and less of a hindrance, but that does not mean there won't be casualties along the way.

    ReplyDelete
  19. Carol Scown-RaynalOctober 28, 2011 at 2:01 PM

    Here we are. We all are now virtual in a virtual world and must keep this in mind - Light and shadow : just the little detail that was missing solved by an algorithm ! Scary... ain't it ?

    ReplyDelete
  20. AnonymousOctober 31, 2011 at 8:32 AM

    The images are too small to tell, but I am 99% certain that the real objects are in photo A. I haven't time to read all comments, I am sure other posters noticed the reflections on glass too.

    ReplyDelete

Due to a high level of spam we must moderate comments. Please identify yourself by name or social media handle so we know you're not a 'bot.'