Wednesday, April 7, 2021

How Do Our Brains Process Images—Decoding or Predicting?

For a long time, scientists believed that images were decoded in a bottom-up process.

It was thought that the image arrived on the retina and was sorted out in stages, starting with edges and shapes that were then assembled into recognizable objects, faces, or symbols by specialized areas of the brain.

You may recognize what this image represents just from the top of the picture; you may even guess who painted it.

Detail of portrait by John Singer Sargent

The alternate way of understanding visual perception is the top-down predictive model. This idea suggests that the brain contains a stockpile of representations of reality that it imposes on what you see. You don't really see reality directly; you see your brain's prediction of what's in front of you. 

According to neuroscientist Andy Clark, "the brain is essentially a prediction machine." 

Most scientists agree that these two systems interact at various stages. Bottom-up constructive image processing meets top-down prediction throughout the hierarchy of image decoding from the most basic to the most sophisticated. 

This process moves like greased lightning until you run across a prediction error or an ambiguity.

Green lines are equal. Source

Optical illusions present anomalies where the top down prediction comes into conflict with the bottom-up construction. Sometimes this conflict happens at relatively basic levels of processing, creating estimation errors of perspective, shape, or overlapping.

This skull illusion creates an ambiguity of interpretation at a higher level of interpretation. 

It's good for us as image makers to be aware of how our eyes and minds work as we decode the world around us and reconstruct it for the enjoyment of others.

1 comment:

Forrest said...

Connected to this is the Golden Ratio 1.618, which factors in. A good example, if you've ever seen someone who has a facial feature off (perhaps an eye), we tend to instantly focus on it as an aberration -- this is one reason why. All of our body parts correspond to this ratio: the length of your forearm to your upper arm, the spacing of the eyes, etc. Artists knew this and used "dynamic symmetry" which this (and other ratios) to compose their works to be more "visually pleasing".