It sent a chill down my spine because I sensed the emergence of a mind.
My camera switches over to portrait-mode whenever it sees a painting or a drawing with a face in it. It stays in AUTO mode otherwise. According to Popular Mechanics: “a chip inside the camera constantly scans the image in its viewfinder for two eyes, a nose, ears and a chin, making out up to 10 faces at a time before you've hit the shutter.”
I decided to test my camera—it’s a Canon Powershot SX120—to see what it decides to regard as a face.
According to my camera, this is a face.
But this is not a face, even though, as Scott McCloud points out, it seems to fit the ideal human cognitive model. 
This portrait of Soutine by Modigliani is a face.
And Jawlensky’s Woman with a Forelock is a face.
Jawlensky’s Medusa is also a face, but my camera wasn’t sure. It switched back and forth from AUTO to PORTRAIT a few times.
There are no faces in Oskar Kokoshka’s Loving Couple with a Cat.
And there’s no face here, either.Popular Mechanics hints at what’s coming: “Sometime soon, face detection may even give way to facial identification, discerning one subject from another. For instance, the camera could retain an image tagged 'Mom' in its memory. Later, the camera would automatically recognize each subsequent picture of your mother and add the 'Mom' tag to it."
Facial identification or recognition is a fast-growing technology that uses 3-D scans or interpolates various 2-D scans to assemble a knowledge of basic structure. Wikipedia says that some of the new algorithms are “able to outperform human participants in recognizing faces and can uniquely identify identical twins.”
You can extend these forecasts to imagine future web-enabled smart cameras that can recognize makes of cars, Zagat ratings of pizza parlors, and movie preferences of strangers in your local Starbucks.
The lesson that I took away from this little experiment has nothing to do with taste in art. My thought was this: as we endow machines with more and more intelligence, we will have to get used to the idea that they will not respond to basic things as we do. Their talents and their lapses will be unlike ours.
Their thinking style— their fundamental orientation to the universe—will be different. They will see the world in a way that will seem at once chummily familiar and creepily alien.
--------
Read more about face-detecting cameras at Popular Mechanics. Wikipedia on Face Detection and Facial Recognition tech.
Scott McCloud and the ideal cartoon face on Cognition and Culture.
---
Added later: Scott's blog reflects further on intelligent machines here.
James,
ReplyDeleteI'd like to tell you I simply loved Imaginative Realism. It is inspiring, and I'm loving going through it.
This is a great post and a great deal of experimentation you did.
Google is investing a lot of time and efforts on image recognition (http://www.daniweb.com/news/post1019118.html#).
They are pretty sure this technology allied to mobility and user's preferences will be the future of web 3.0.
Best regards!
The Scott McCloud commentary is apt, precisely because the camera didn't pick it up. Abstraction & heuristic remain the strength & weakness of the wet noodle computer sandwiched in the human skull.
ReplyDeleteI found it hilarious when I discovered my brother's cell phone has a "smile detector". It first locks on to the face like any camera in portrait mode, but when you press shutter-release it doesn't take the picture until or unless the person in focus is smiling.
ReplyDeleteIt's far from foolproof, though, side-angles and bad lighting can easy confuse the program.
Still, I hope we can have intelligent robots soon...
As long as we introduce Asimovs laws I'm fine with sentient robots... Wouldn't want another Matrix or Skynet incident...
ReplyDeleteOnce they photo you for a drivers license, passport or the like (innocuous school yearbook pictures). Your tagged image will forever be in the system.
ReplyDeleteWhat's with all the "security" cameras anyway? Feeling safe in your privacy?
Magnus, where are you when we need you?
So, according to Google's Picasa face detection algorithms, Scott McCloud's head, the Jawlensky, the Modigliani and your security guard are faces, the others... not so much.
ReplyDeletetechnology is evolving at an amazing rate of speed. its scary. its like the line in jurassic park, "scientists were so preoccupied with whether or not they could, they didn't stop to think if they should"
ReplyDeleteit interesting people never think, lets enjoy what we have now, take a year off then invent the next thing in a few years. its always got to be now, immediatly dont sleep the new big thing is coming out...scary.
the thing ive noticed most is how computers and technology are slowly becoming infused with the human body. i wonder how long till we see a human brain hooked to a computer able to live forever, muhahahahahahah!
and as far as the thought that computers will have a different view then humans on different things, i think scientists will try to make them as human as possible. for instance you could possibly upload all comments and emotinal expression recorded on a site like facebook. how people react to loss, friendship, love, jokes holidays. once a computer would have this knowlege i would say he would have the human way of thinking and pattern of feeling down pretty good...scarry!
When I first came across Google Picasa's face detection utility, I was amazed at what it pulled in - I even found images I that had been in the data banks for years that I'd forgotten I had.
ReplyDeletejust to follow up my comment, i swear that my teenage neice is half computer already, i can not seen any sepration from her and her laptop/cellphone...they have become one.....
ReplyDeleteYes, I've noticed my camera doing the same thing when I photograph my drawings and paintings. Also, I think the new iPhoto already recognizes individual faces. Only a matter of time before it finds John Connor (or his mom) and does what it's really programmed to do.
ReplyDeleteYour Popular Mechanics article is already slightly behind the times. It hasn't made it into a camera yet that I know of, but Apple's photo storage program iPhoto has face recognition built in, so that you can tag a person, and it will sort through your photos and find all the photos of that person. The more photos you tag the better it learns the features and starts to recognize people from various angles. It even recognizes and differentiates cats, as I can tell you from personal experience. I'd be curious to try a sketch and a photo of the same person and see if it makes the connection... Hmm...
ReplyDeleteI wonder if the camera would recognize the "face" of Wall-E? It/he has this head that looks like a pair of binoculars, but is instantly recognizable as a face, capable of showing a lot of expression.
ReplyDeleteFascinating. Thanks so much for posting this entry and all the other wonderful entries here at you blog, I've really been enjoying reading them.
ReplyDeleteGooGoo: My older children's teachers already have a system on their computers from which they upload comments for report cards. It is unnerving to see exactly the same comment about your child from five different teachers!
ReplyDeleteThanks, James for the thought provoking post and the great sketch, ear notwithstanding.
“Sometime soon, face detection may even give way to facial identification, discerning one subject from another. For instance, the camera could retain an image tagged 'Mom' in its memory. Later, the camera would automatically recognize each subsequent picture of your mother and add the 'Mom' tag to it."
ReplyDeleteI just got my first Mac ever, and its iPhoto program does this already. I have no use for it, so I haven't used it, but the tutorial shows you how to do it. I think it's creepy.
I have recently been playing with Picassa's new face detection capabilities. It is uncannily accurate but still can't tell the difference between my non-identical (but very similar) twins. I actually found it even cooler when it mistook family members for each other. It seemed to recognize similarities between my daughter and my sons that I had missed. I have many sketches and photos of the same people, so far I guess my likenesses aren't good enough to allow it to recognize them though.
ReplyDeleteThis is really interesting, so thanks for posting it.
ReplyDeleteI have enjoyed reading the comments posted here. It's not surprising to see how sensitively and insightfuly the topic is addressed given the audience.
Your comment about it being simultaneously familiar and strange in regards to computers' reactions reminds me of the concept of "The Uncanny Valley."
http://en.wikipedia.org/wiki/Uncanny_valley
You should coin a phrase for the philosophical phenomenon...although I am currently drawing a blank on one =)
@Cully- I would be interested in hearing the results of the sketch test for the facia
Splynch, and all with Picasa: have you tried running your hand-drawn or painted portraits of your family members by Picasa? If you got the likeness pretty close, I wonder if it would make the identification.
ReplyDeleteDoes anyone know what metrics it's using for confirming identity?
I love your pure imagination, this is the type of experiment I'd be compelled to do. It's interesting and weird that the basic simple face drawing wasn't recognised but a similar face with smudgey bits added was. I'm not sure what this says but I'm sure I'll think about it for a long while.
ReplyDeleteMaureen. www.thepizzagang.com
Interesting post and discussion in the comments!
ReplyDeleteI agree that the problem resembles "the uncanny valley"; our negative reactions on increasing humanlikeness.
But the problem is that "the uncanny valley" hypothesis doen't give and guidelines for avoiding the negative reactions, other than "make it less humanlike".
I my opinion this isen't satisfying. As our tech get's more and more complex we need to ensure that it's interface get's better and better. More intuitive and easy to use. It's been done with personale computers; I'm glad that I'm not punching commandlines into DOS right now!
We need to do the same with advanced technology that increasingly become autonomous (as the automatic change to "portrait-mode" is an example on). We need to design this technology so that it doesn't trigger a creepy response.
One way to do it is to make the tech more transparent. Make the camera say (visually show or maybe tactically make you feel) "I'm changing the mode to portrait mode". Hereby you made the situation transparent and avoided a negative reaction.
But you could also add "personality" by making the camera say: "hey man, I'll just go to portrait mode". If it's done right you can trigger psychological responses in the user that will avoid a negative reaction and even ideally make it positive! And at the same time you've made the tech more intuitive and analogue; more in the way that we communicate human to human everyday (interesting discusion on this on: http://johnnyholland.org/2009/12/09/does-technology-need-personality/)
The above indicates the limitations of "the uncanny valley" because you've made the tech more humanlike and at the same time more appealing (which in contrary the hypothesis).
Human-robot Interaction is by far more complicated than this, but the above shows an example on how our future design-challenge can be solved.
For anybody is interested, you can read a shot presentation and abstract of my psychology-thesis which examined these exact problems with our increasingly more automated technology on: www.robotspodcast.com/forum/viewtopic.php?f=12&t=747
Johan-
ReplyDeleteInteresting post on the robot forum. I hesitate to broach this topic with you since you have clearly put a great deal of time and thought into the subject.
However, since I am a human with human thoughts and reactions, I figure that I am qualified enough to discuss it =)
The reflective aspect of questioning our humanity and trying to define what it means to be human or even trying to define what is alive can certainly be disquieting. However, I would respectfully submit that this is not what the Uncanny Valley is all about.
Rather, I think that the heart of it is much more primal and ancient. The repulsion reaction is just that, just a reaction. There is an article that shows that monkey's might very well fall into the Uncanny Valley too:
http://www.wired.com/wiredscience/2009/10/uncanny-monkey/
The idea, as I understand it, is that once the robot (or whatever) begins to take on enough realistic qualities our brains no longer think of it as a cute robot trying hard to be human, but instead a human that has some real problems.
Our brains will always search out subtle ques in other people so as to help us understand how to react to them. "Are they a potential mate? Are they a potential threat? Are they unhealthy?" and so forth. This is done almost subconsciously (or at least, I think that it is) and at such a base level that is not something that can be eliminated without re-wiring the brain.
Again, I am a lay person on the subject, but as a person, and an artist of sorts, I find that there is real merit to the theory.
I think that my original assertion was that James's comment reminded me of the Uncanny Valley in the sense that there was something of the familiar and something of the foriegn. As he said, the emergence of a brain from his camera recognizing faces sent "a chill down" his spine. To me, that seems like an ancient-type of reaction as opposed to an unsettling higher brain thought.
I think that I may have rambled myself into asserting no particular point.
So on that note, I will just say thanks for posting your comment, it was fun to think about =)
Did you see that Scott McCloud linked to this?
ReplyDeletehttp://scottmccloud.com/2009/12/11/the-emergence-of-a-mind/
@ Tyler J
ReplyDeleteGreat comment! You are of cause totally right! We have acquired immediate reactions (e.g repulsion) through our evolution that serve to enhance survival.
And this "the uncanny valley" can seem to explain. But the problem with "the uncanny valley" is that it's so simple and research have even indicated that it's wrong (we don't necessarily react negatively to a near humanlike face).
In the HRI-field "the uncanny valley" hypothesis have sort of gone "viral" (like a hilarious youtube video) which the Wired article with monkeys is also an example of. This have had the unfortunate consequence of overshadowing more complex theories of human reactions on robots.
Lately this trend have been broken in the professional field and reactions on many levels have been acknowledged (such as the immediate, cognitive and reflective levels of reactions as I talk about).
Still "the uncanny valley" can be a good introduction to the HRI field, but the abundance of creepy humanlike robots can make it a difficult task to find the "black swan" that falsifies the theory.
And it is precisely such a falsification that is needed so that we can begin to create more complex theories of human reactions to robots. Theories where "repulsion" is one of the many explanations of our reactions on robots.
It's good for me to try to spread and reapply my work in the above way, so I'm very glad for your comment!
Johann and Tyler, fascinating discussion. I experienced the Uncanny Valley effect when I visited the Henson Creature Shop years ago. They were working on a full size human baby animatronic with silicon skin, and one of the guys was putting the hairs into it one by one. Creepy, and as you say, repulsive, while cute at the same time.
ReplyDeleteMy personal experience of the smart camera was a little different, perhaps because it was not processed through the the same cognitive/neural pathways. It wasn't a visual experience that triggered it. It was more like a deep feeling or recognition that the machine was "thinking" or "conscious," (which of course it wasn't).
I think this same reaction (for me a mix of delight/surprise and/discombobulation) can operate with the response to verbal GPS directions, or Amazon recommendations. Not something you see that spooks you, but a sense that the machine is working above the level you expected.
The feeling passes instantly as we accept the machine's new powers.
"The feeling passes instantly as we accept the machine's new powers."
ReplyDeleteAnd THAT is the creepiest statement so far...
There used to be this website where you could upload a picture of your face and it would find which celebrities look like you. After trying it with a normal picture of me, I then tried drawings. The website liked the picture of Betty Brandt from the 1960s Spider-Man cartoon and suggested some matches, but did not like a screenshot of Megatron. My drawing of Peter Cushing from Star Wars worked, though the website did not suggest Peter Cushing as a match--I felt artistically slighted. :-P
ReplyDeleteI was using Google street view in Mexico city (for this months virtual paintout) and noticed that all the faces were blurred! Too bad - it was a great source of anonymous painting subjects. Amazing technology, though.
ReplyDeleteJohn Kaay
Nice post James - which I['m highlighting today on my blog. I'd already spotted this happening but you have said it all so much better than I could!
ReplyDeleteNice post
ReplyDeleteIt is not creepy to be able, say, to lock a file and have it unlock ONLY for your face. For instance.
ReplyDeleteHi James,
ReplyDeleteI hate to bring the future even a little closer, but I already have a camera with face recognition. I bought a Panasonic Lumix GH1 last summer; it has face detection, but also recognition. You can register and name individuals. Then when you are framing a group shot, it will recognize them (their names will pop up below the green rectangle framing their face) and it will preferentially expose and focus for them. If you register more than one person, you can even rank them...
Nick
very interesting post! i linked to it on my blog (http://didyougetthatthing.wordpress.com/), so i hope that maybe gets you a new follower or two.
ReplyDeleteThanks Katherine and Peter for the link; visitors and new readers are always welcome, and people should check out your blogs as well.
ReplyDeleteNick, the tech is moving too fast to keep up! That's really cool that you've got a camera with face recognition. What woud be even more useful for me is blink recognition. Just delay that shot a half second. Most of my once-in-a-lifetime shots are messed up with a doofussy blink-face.
@ James Gurney
ReplyDeleteVery interesting that your negative experience is on a "higher" level. It resonates with aspects of my investigation of our reactions on robots.
Just a quick commet on a recent article about "the uncanny valley": http://www.wired.com/magazine/2009/11/ff_avatar_movie/2/
On page 2 it is described how Cameron needed to make the quality of his motion-capture so good that the animations "overcame" "the uncanny valley". I think is funny that the article totally neglects that the "Na'vi" (the alien race) are very "humanlike" but NOT creepy looking.
The "Na'vi" are an example of how a negative reaction can be mediated by good character design.
james..
ReplyDeletevery interesting post. Now a days Face Recondition cameras are very smart and integrated with many other applications also.
Even there are advance face recognition cameras available in the market which will identify multiple faces even an object wear some colored spectacles or a cloth on his faces.
I have found many other details from here: Face Recognition System Feature
Hope It may help you for more details.
thanx for sharing james..
keep blogging.
The Russian guy is trying to sell an electroshocker.
ReplyDelete??? Thought you'd like to know.
Terry, sorry you clicked on it. Terry is referring to a spam comment (now gone), which unfortunately affected dozens of pages. I have to go through each one and delete them.
ReplyDelete