Thursday, February 11, 2010

The Two-Streams Hypothesis

In the last two decades, new scanning technology has given us a better idea of what is happening in the brain as we process visual information.

One of the discoveries that has come out of these new data is the realization that that visual processing divides into two streams, which end up in very different parts of the brain.

According to Dr. Margaret Stratford Livingston, Professor of neurobiology at Harvard University, the visual brain processes tonal information separately from color information.

(Addendum: I changed the image thanks to a different way of processing them--Thanks, Nick! The image on the left has only luminance, but the one on the right has color information with luminance removed.)

The two streams originate in the retina, which begins some low-level processing. The information pathways route back to the optical cortex at the back of the brain. Although there is some crossover and interaction, the two streams are largely kept separate, from the level of the retina all the way to the higher-level vision centers of the brain.

The area of the brain that interprets tone is several inches away from the area that interprets color, making the experience of tone and color distinct physiological experiences, as distinct as sight and hearing.



The color stream (in blue), is also called the ventral stream or the “what” stream. It is more concerned with recognizing, identifying, and responding to objects. Color processing through the ventral stream is a capacity that is shared only by higher primates, not the bulk of other mammals.

I asked Dr. Livingstone if she would describe more differences between the "where" and "what" streams? In particular, is one stream more associated with emotional response or higher cognitive function?

“I don't know about emotions," she said, "but the dorsal (what) stream is certainly more associated with higher, conscious functions and awareness.” (Addendum: I believe Dr. Livingstone misspoke here, and meant the ventral stream.)

The difference between these two streams may explain why classically-trained artists use a strategy of planning the tonal organization of their compositions separately from the color scheme.

"The Artist at the Easel” by Sadie Dingfelder, February 2010 Monitor magazine of the American Psychological Association.


Two-Streams Hypothesis on Wikipedia

28 comments:

mordicai said...

Don't cross the streams!

sam said...

Fascinating. Thank you for your blog and your recent interview on Sidebar. How do you come up with all this stuff? Simply amazing!

James Gurney said...

Thanks, Sam, The topic today was sent to me by a blog reader, and I apologize to that person for failing to take note of your name.

Susan Adsett said...

Well, speaking of blog readers sending you stuff - have you seen this? http://www.vischeck.com/examples/daltArt.php

It's a website that will show you what a picture (or another website) looks like to someone who is colorblind. It's fascinating to see just how much information gets lost. The link above will take you to the page where they "colorblind" and then "correct" several famous paintings - but if you go here: http://www.vischeck.com/vischeck/vischeckImage.php you can run their program on any image you like. I tried it on my website - some paintings changed completely, and others hardly at all.

Mary Bullock said...

Interesting Info, Jim. It does make sense when you analyze how you see things as an artist - they really are two different things.

Nick Woolridge said...

Hi Jim,

Fascinating as usual... You demonstration image could actually show the difference more clearly, since the left side shows only luminance (tone), but the right side shows luminance + colour. A pure luminance vs colour representation looks like this:

http://www.bmc.med.utoronto.ca/wiki/Gurney_colour_demo.jpg

This is closer to what the visual system actually processes, and demonstrates why colour alone is effectively useless for most scene recognition tasks. Luminance contrast is necessary for the recognition of objects, depth, and movement.

Nick (your admirer from Biomedical Communications at the University of Toronto)

Nick Woolridge said...

My URL got trimmed, here's a shorter one:

http://www.bmc.med.utoronto.ca/wiki/Gurney.jpg

Nick

Cheri said...

I think you mistakenly called the dorsal stream the what pathway - it is the where pathway (a typo, I'm sure). The ventral stream is more associated with the temporal region that is responsible for language (defining objects), and the dorsal stream is more associated with the parietal region which is more associated with movement and spatial relationships.

I just took a sensation and perception class as part of my psychology requirements. I really think this class should be made available to all artists, and that all artists should take it. Knowing how the brain perceives the world we live in might change the way an artist represents them in his or her work!

Daroo said...

For painting, It'd be handy if we could actually manually switch between the two streams -- first perceiving only values then looking at the color information.

But wait a minute, would that color information stream be devoid of any values? Like a gel or colored glass?

If so that means our perception functions analogously to the grisaille/ glazing method of painting.

Sam Nielson said...

I love this type of information. Thanks for your ceaseless (if a little futile) effort to make me smarter!

Susan Adsett said...

Nick, your link is awesome - it actually helps explain a problem I've been having in my own paintings. I'm fine with tonal drawings, but when I work in color, I end up with something like the color representation. So I'm obviously having problems figuring out tone when using color.

Now if only I knew how to FIX that problem.

Daroo said...

Nick - Thanks for the link. That's what I was trying to say.

I think that is why, while I enjoy a lot of the pure impressionists work, I really prefer paintings that have decent color but also have a full value range -- Les valeurs as Carolus-Duran would say.

James Gurney said...

Thanks, Nick! I've switched the image for the one you sent.

Cheri, If you mean the dorsal/ventral confusion in Dr. Livingstone's quote, I think you're right. I feel fairly sure she misspoke, but I quoted her reply verbatim.

I should also mention that that the hypothesis as I've stated it is somewhat oversimplified and controversial. Not all agree that the streams are separated all the way along. But I believe the basic principle that there are many tasks of vision and they're managed by different areas of the brain is generally accepted.

OSweetNature said...

This makes sense in that some artists are stronger tonal artists, focusing on mass and form; and some artists are stronger as colorists.

Chris Whetzel said...

Yet another reason to focus on value over color. Also,I suppose the far distance between the areas of the brain could be seen as beneficial. For example, if they were closer to one another, perhaps those that are colorblind would have problems discerning value as well?

The Phillustrator said...

This is fascinating. I took the liberty of combining the separate images in photoshop. I put the Luminosity image above the Value and set it to multiply. I think you'll find the results interesting.

[URL=http://img121.imageshack.us/i/combinedy.jpg/][IMG]http://img121.imageshack.us/img121/6642/combinedy.th.jpg[/IMG][/URL]

The Phillustrator said...

Sorry, here's a proper link.

http://yfrog.com/3dcombinedyj

Lyndon said...

Phillustrator - For a strict reconstruction you want to put the colour image on to and use 'color'.

(actually I just tried that with another image and it doesn't work perfectly. odd.)

I believe you result would be almost identical to sliding down the levels on the original.

Tyler J said...

Very interesting topic and some great contributions. I am amazed at the combined image, but I suppose I shouldn't be.

I have found that my best pieces have started with working in graphite to create an interesting design first and then working on the color component second. Perhaps it's because of the split in the brain or maybe it's just the extra planning.

Roberto said...

This does not surprise me (but it is still really amazing) since the retina collects tonal and chromatic information separately (via rods and cones), and the retina is actually a little piece of brain tissue in the eye! What is really important about this is to realize that an image works on more than one level (at least two!) in our brains. Since the oldest (more primitive) part is the tonal information, it is vital that the tonal composition is coherent and as strong as possible: ie. interesting shapes (silhouettes and positive/negative relationships etc.) with balanced tonal relationships (shapewelding, rhythm, etc.).
The chromatic information is (relatively) newer and conveys a more emotional and sensual (poetic? musical?) quality.
I think it is important to realize that these two systems are not totally independent of each other: You can create ‘tonal’ relationships with pure color ( yellow is ‘lighter’ than purple, while green and red can both = gray); and the optical sensation of color can be generated by black and white (such as on a spinning disk).
Great post Jimmy G. (and Fantastic interview on Sidebar, but they never even mentioned that upside-down-drawing-with-both-hands-thing that you do! Probably just an omission due to time restrictions and all.) Congratulations on the museum show too. You-the-Man, man! -RQ

Don Cox said...

I think that should be "three decades" rather than two, as the point is well explained in the 1985 film "Colourful Notions". I mentioned this film before when the topic of relative color came up.

The traditional method of painting (Rubens, Velasquez, Goya, etc etc) in which the forms are first modelled with white and sometimes a dark pigment (such as raw umber), and then coloured with glazes, does relate well to the way the brain works. However, in my experience students find it very hard to grasp: they are used to the idea that each patch of the image can be defined by picking a spot on the Photoshop colour editor, in one operation, or by mixing paint on a palette.

Also, they are used to starting with a white background and adding darkness.

Don Cox said...

You can simulate red-green blindness in Photoshop by viewing the "channels", and copying the red channel to replace the green (or vice versa), so that red and green are identical.

Or, you can mix them to simulate the more common partial color blindness.

James Gurney said...

Don, thanks for those clarifications.

Roberto, I agree with your take on the implications for artists.

And thanks, everyone else for your interesting comments.

Frank said...

Hey James,

Fascinating post. I have question about the topic, however.

You mentioned that a lot of classical artists plan their values and colors separately, but I've always tried to do them both at the same time, as value and color seem to be inseparable when it comes to the final piece. Also, I've noticed that a strong composition in terms of value isn't necessarily the strongest solution in terms of color. Could you elaborate on that statement, or show some examples of what you meant by the separate color/value studies?

Thanks,

frank

Unknown said...

if you invert the colored half of this image, then stare at the center for about half a minute (the telephone pole crossmember, say), and then look at the same point on the black and white side, your brain will conflate the two and you will see the image with both color and value. like magic!

until you move your eyes, that is. then it'll snap back to black and white.

the longer you stare at the color side, the more vibrant and long-lasting the effect will be. gotta invert it, though, for the retinal burn-in to work (otherwise the sky will be orange, the trees blue).

James Gurney said...

Frank, you raise a very important point. While it's usually true that any good composition should be legible in tone separately from color, there are times when you may want to play areas of equal value but differing hue or chroma against each other.

In fact, the subject of Margaret S. Livingston's study was a Monet sunset effect where the orange disk of the sun and the cool sky background was painted at equal luminance or value. The effect, she suggested, was to confuse the dorsal stream image processing, and only provide information to the ventral stream. It can be a very impressive effect if it's handled right, and a good argument for your way of doing color sketches.

James Gurney said...

R. W. Kretz sent me an interesting reaction to the post as an email, and gave me permission to post it in the comments here:

"My gripe with the post basically is that its drawing a conclusion from brain area activity to the nature of the visually perceptive faculty. this conclusion is drawn mediately, namely via the thesis of causality (the premise of all natural science).

However, the immediate facts about the visually perceptive faculty can be seen when we just plain look at how looking works, taking serious how visual perception "presents itself".

This said: while the blog post (i dont know if it was your point of view or you quoting the neurologist) draws the conclusion that color and tonal perception would be two seperate functions, much like seeing and hearing, seeing still does not quite care about that. and we still see things in bright/dark and hue/saturation difference simultaneously. actually, it seems to be so simultaneous that unless you train yourself to it, its really hard to perceive the properties of color as distinct instead of mashed all into one.

As a visually trained person, you sure agree how hard it is to "unlearn" this skill, once it is in place, but at the same time, how much what we see differs from what an untrained person "naively" sees."

Unknown said...

What was interesting to me is that when I crossed my eyes to look at the two images overlapped I could only see the black and white version in my focus. I saw colors in the periphery. I wonder what the deal with that is.