Tuesday, October 12, 2021

Outline vs. Tonal Shapes In Face Recognition

 Which is more important for face recognition: outline or tonal shapes?

Jim Carrey (left) and Kevin Costner.

According to vision scientists Pawan Sinha et al, "Images which contain exclusively contour information are very difficult to recognize, suggesting that high-spatial frequency information, by itself, is not an adequate cue for human face recognition processes." 

By contrast, the tonal shapes, even if they're out of focus, are relatively easy to recognize. The experts say: 
"Unlike current machine-based systems, human observers are able to handle significant degradations in face images." Shown here are Michael Jordan, Woody Allen, Elvis Presley, and Jay Leno.

That's why it's good to blur your eyes when you're capturing a likeness.
Source: Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About, Pawan Sinha, Benjamin Balas, Yuri Ostrovsky, and Richard Russell,


Raf said...

While this is certainly interesting, the choice of graphics seems a bit biased to umm 'help' us reach the proposed conclusion :)
One could say that Kevin Costner's likeness is a tad awkward due to the neckline, but it's somewhat recognizable nonetheless.
Jim Carrey though... his likeness is not an iconic image as is the case with the blurry tonal images. One would expect him to be grinning for example.

The blurred images on the other hand are chosen 'with recognition in mind' - the outlines, the tones, the expressions are iconic.

One more aspect is that the blurring of an image will probably work well enough for this exercise regardless of the type or technique of blurring applied.
It's not the case with line work though, is it? Depending on the algorithm the output image could vary from completely unrecognizable to immediately identifiable - from a haphazard group of lines to a well rendered likeness.


Terry said...

I honest to God thought #4 was Aunt Bee from the Andy Griffith Show.

James Gurney said...

Raf, glad you mentioned that; I was thinking the same thing. I suppose they wanted to come up with the line drawing through some mechanical or "objective" process, given that a human-drawn line drawing involves so many subjective choices.

Despite that shortcoming, the comparison does bring up a valid point, one that I've experienced when rendering likenesses, namely that using line alone can present challenges to achieving a likeness.

Another issue has to do with lighting. No matter how you render the image, the likeness is easier to accomplish with simple clear lighting, rather than cross-lighting or up-lighting.