Our visual system elaborates meager input

How is it that we perceive the world in such rich detail and texture, even though the eyes don't actually take in very much information?

Ivan Shishkin, In the woods, 1860s,
unfinished drawing, ink, pen on paper
State Tretyakov Gallery, Russia

Neuroscientists working with mathematicians at NYU have developed a mathematical model of vision which explains the process that the brain uses to generate elaborate visual representations.

This process is necessary because to achieve our detailed vision, we don't use our eyeballs in the same way that a camera uses its lenses and sensors. For one thing, there are relatively few neurons carrying information from the retina to the back of the brain.

The magic happens in the visual cortex, which is much better connected neurally. As the brain begins to sort out the relatively meager information it receives, it elaborates the data into a richer representation. Until recently, scientists thought this process happened in a "feed forward" direction, a one-way trip from retina to visual cortex to higher vision centers in the brain.

Oak Forest by Ivan Shishkin

But it turns out that information often loops back from higher to lower levels of processing, amplifying weak signals into richer images. This happens in real time, largely unconsciously, and sometimes inaccurately, such as when you think you see a snake, but it turns out to be a rope.

At this stage, the researchers are concentrating on the brain's preliminary decoding of the image—tasks such as edge detection and shape recognition, but they're working their way to understanding other levels of processing, such as perception of motion and color "which occurs through an entirely different and more difficult neural pathway."

Painting by Aaron Draper Shattuck

As a plein-air painter, these findings make sense to me. When I'm looking at a scene in order to paint it, I'm conscious of the impression of infinite detail, much of it suggested to me by my peripheral vision.

I'm also aware of how my visual system also screens out elements of a scene such as a pole or a foreground object,  even from a scene that I may have been looking at for over an hour. There are objects in my visual field that I'm not aware of at all until someone points them out to me or I look at a photo of the scene.

Read the articles:
Quanta Magazine: A Mathematical Model Unlocks the Secrets of Vision 
Quanta Magazine: Computers and Humans See Differently

Thanks, Keith Patton