However you define them, lines or edges are an exciting topic now because of new breakthroughs in vision science. Neuroimaging and other tools used to analyze brain function are rapidly increasing the understanding of how the visual system interprets the lines we encounter in the world around us.
The primary visual cortex, which lies at the back of the brain, has about 140 million neurons. These neurons are organized in groups that specialize in sorting the information into various properties. Some groups of neurons called orientation columns respond preferentially to vertical lines, some to horizontal lines. In the image at left, created from the visual cortex of a macaque monkey, neural clusters with different functions are grouped by color. Other clusters of neurons respond to size, color, and shape. Some are tuned to respond to vertical movement, and others to radial movement.
I asked Carl Schoonover:
Do the orientation columns perceive a vertical line as vertical even if the head is tilted or if the lines are receding in three point perspective?
“Orientation columns perceive orientation relative to the patterns of light that hit the retina. So if look at a vertical line and tilt your head 90 degrees, neurons that respond to vertical lines will go silent (aka that orientation column will go silent), whereas previously 'horizontal-preferring' silent neurons/orientation columns will then be activated.
“However, you may still perceive the vertical line as vertical, even if your head is tilted. This is because there's a lot more to the visual system than just one-to-one representation of visual space onto cortical space. In higher areas of processing, it is possible to maintain a more flexible representation of one's environment, irrespective of the exact pattern of light hitting the retina.”
“This is thanks in part to 'Helmholtzian' signals” (a centuries-old hypothesis that posits the stability of images despite our head movements). “This is useful for many reasons--for one, our heads our constantly moving as we walk, as our eyes saccade across visual space... but nonetheless our visual experience remains quite stable.”
I'll take a brief break tomorrow (Friday) and finish this up on Saturday.
------
More on orientation columns from the Journal of Comparative Neurology
Orientation columns image by Yevgeniy B. Sirotin and Aniruddha Das
Painting by John Berkey for a 1979 Brown and Bigelow calendar. Thanks, Jim Pinkowski, who has a big website of Berkey images.
Thanks, Carl Schoonover. More in his book "Portraits of the Mind"
Lines and the Brain Series,
Part 1
Part 2
Part 3
Part 4
This comment has been removed by the author.
ReplyDeleteVery interesting.
ReplyDeleteIt's already amazing that scientists can detect what part of the brain is 'triggered' by e.g. horizontal lines.
But indeed, that's just scratching the surface.
If you imagine a monkey swinging through a forest, sometimes upside down and sideways, it's clear that a lot more processing is going on than just detecting "horizontal lines".
The continuous change in orientation of the monkeys head combined with the vast amount of chaotically arranged branches gives an idea of the complexity that the brain must deal with.
Also, these monkeys aren't even concentrating on the branches. Most of the time they're more consciously involved in looking and gesticulating at each other than looking at that branch they'll have to catch, not to fall to a certain death.
In contrast, if you look at the difficult math that's involved in even simple edge detection software and if you look at the current state-of-the-art image recognition that computers barely can perform, that's quite embarrassing.
I'm sure that many scientists/mathematicians realize that image recognition through mathematical calculations is simply the wrong way to go. Our brains just don't work that way. We don't calculate G-forces when we dance in order not to fall and we don't do Fourier-analysis when listening to van Beethoven's 5th.
By studying the brain, we may be able to eventually come up with a different kind of computer logic that is less math-oriented.
Actually, I'll have to disagree with the above post. I think mathematical calculation IS the way to go when working with image recognition. I think the human brain is a very powerful 'computer' that has through millions of years of evolution developed the ability to make those calculations. I think this is quite obvious when looking at savants, people whose brains have been wired a bit differently and who can make really complex calculations with their minds alone.
ReplyDeleteInteresting, indeed. I wonder what Mr. Schoonover observes in brains when the subjects are faced with those optical illusions that make vertical parallel lines appear curved, etc...
ReplyDeleteI wonder how exactly the image processing in one's brain is affected when you view your picture upside down, leaning over, between your knees, as Chinese painters reportedly used to do. I could never decide whether this exercise really reveals something valuable I had not seen in the normal position.
ReplyDeleteWith regards to tilted heads, I suspect intorsion/extorsion may have something to do with it as well?
ReplyDeleteLayman's information regarding the higher areas of processing can be found in these books if found so far. Here excerpts from my notes.
ReplyDelete------
On Intelligence
Copyright © 2004 by Jeff Hawkins and Sandra Blakeslee
58
Natural vision, experienced as patterns entering the brain, flows like a river. Vision is more like a song than a painting.
Time needs a central place in a neuroscientific account of vision.
59
So each moment there is a new spatial pattern of stimulations along the length of the cochlea. Each moment a new spatial pattern streams up the auditory nerve. Again we see that this sensory information boils down to spatial-temporal patterns.
So touch too, is like a song. Your ability to make complex use of touch, such as buttoning your shirt or unlocking your front door in the dark, depends on continuous time-varying patterns of touch sensation.
69
the cortex creates what called "invariant representations" which handle variations in the world automatically.
70
four attributes of neocortical memory that are fundamentally different from computer memory:
• The neocortex stores sequences of patterns
• The neocortex recalls patterns auto-associatively
• The neocortex stores patterns in an invariant form
• The neocortex stores patterns in a hierarchy
71
Truly random thoughts don't exist. Memory recall almost always follows a pathway of association.
You know the alphabet. Try saying it backward. You can't because you don't usually experience it backward.
80
let's return to the sensory cortex and look at music again. (I like music as an example because it is easy to see all the issues the neocortex must solve.) Invariant representation in music is illustrated by your ability to recognize a melody in any key.
Think of the song "Somewhere over the Rainbow." You probably can't recall the key she sang it in (A flat). If I sit down at a piano and start to play the song in a key in which you've never heard it — say, in D — it will sound like the same song.
-------
"How We Decide" by Jonah Lehrer © 2009
155
MIT economists conduct an auction with business school graduates and later on executives and managers at the MIT Executive Education Program, with similar results
Students with the highest-ending SS numbers (80-99) made an average bid of fifty-six dollars. In contrast, students with the lowest-ending numbers (1-20) made an average bid of a paltry sixteen dollars. A similar trend held for every single item. On average, students with higher numbers were willing to spend 300% more than those with low numbers.
-------
My thoughts:
The rational mind has a kind of rubber band effect even when trying to apply logic which seems similar to our color perception. Learning from this blog, my own fine art painting and neuroscience, our senses seem to display the same or similar phenomena regarding all our cognition and sensory perception. Our minds are both a sloppy organic computer and a wonderfully sophisticated and nuanced pattern predictor and information stabilizer.
Fascinating thoughts and notes--thanks, everybody. I can see why vision scientists are teaming up more and more with artists.
ReplyDeleteMargaret Stratford Livingstone, a neurobiology professor at Harvard, said: “Artists are vision scientists, they just call themselves something different.”
The bottom painting, of two sets of pathways leading up to a white building, is at the Naval Academy in Annapolis. I recognize it because on Friday afternoon I was sitting right there, struggling with perspective, and painting the exact same scene in watercolor, though the trees were, of course, in their light spring green dress. What a pleasure to come home and find this picture on your blog!
ReplyDeleteI recently read the turn-of-the-last-century scifi novel "A Columbus of Space" and the author suggests a similar idea about artist-as-vision-scientist, where he describes the visual language and science of the Venusians as a more highly-developed color expression hinted at on Earth by J.M.W. Turner...
ReplyDeleteOddly enough, the brain itself has "grid cells" for navigating, which are laid out not in a rectangular grid but a hexagonal one. So: why are maps marked with latitude and longitude, rather than hexagons? Maybe because our eyes prefer rectangles...