Thursday, July 7, 2011

Video Eye Tracking

In some previous posts, we’ve looked at how eyetracking technology tells us something about how people look at paintings. But what about movies? How does the element of motion influence the attention of the viewer?

Eye Movements during a segment on Chilli Plasters from TheDIEMProject on Vimeo.

(Feed readers may not get the video, so link to it here)

Scientists at the DIEM Project (Dynamic Images and Eye Movement) have shown snippets from films to viewers and tracked the movements of their eyes. In the case of this clip, 48 viewers participated, so the sampling size is quite large. In addition to little ovals showing where individuals glanced, the video is overlaid with a “heatmap” which compiles viewer data to show where the vast majority of viewers were looking at a given moment.

Here are some of my observations:
1. In scenes with an even overall visual texture (such as at 1 minute: 2 seconds), the center of gaze goes to a default position in the middle of the screen.
2. People seem to anchor their gaze on the nose of the face, perhaps “reading” the rest of the face in peripheral vision from that position.
3. Viewers tend to look at the person who is speaking (not surprisingly). Getting them to look at a listener in a dramatic film is a collaboration of acting, directing, and editing.
4. When one scene cuts to another, the eye hangs in its last focal point for a bit, so editors who place the focus for the next frame in the same location will do the viewers a favor.
5. Viewers are highly goal-driven in the way they look at movie scenes. They scan for meaning.
6. Anomalies attract attention, like the goop stuck on the side of the pot at 32 seconds.
7. In fast cutting, the eye reverts to the default center (1:14-1:18)...
8. ...Which suggests that most visual information in fast-cut action scenes in movies is processed from peripheral, not foveal information. So why bother with detailed VFX, other than to give eye-candy to DVD stop-frame hounds?
9. What the heck are chili plasters?

More at the DIEM project.
Related previous posts on GurneyJourney:
Eyetracking and Composition, part 1
Eyetracking and Composition, part 2
Eyetracking and Composition part 3
Introduction to eyetracking, link.
How perception of faces is coded differently, link.
Eyetracking analysis of a scene "There Will be Blood"
University of Edinburgh, Visual Cognition Lab, Copyright 2009


Mark Harchar said...

Chili plasters? I wonder if they were like the mustard plasters my parents would put on my chest as a child when I had the croup. Those were supposed to loosen up your airways. Not sure if chili plasters are the same, but they did have mustard in them.

Colin said...

Plaster in British countries can mean bandage. I have some chili plasters from Germany. They are for things like back pain, when you put them on, they are mild, but within a half hour they begin to feel hot, then hotter, like a chemical burn. This (in theory) brings blood flow to the injury.


jeffkunze said...

I'm curious to how/if it would be different if someone were watching a movie in a language that the viewer didn't speak.

The way they try to find meaning might be brought about from looking at different elements.

P.T. Waugh said...

Ha, all I was tracking was that blue and green thing.

Torbjörn Källström said...

This is interesting. Even if for the most part it all felt very logical. I think that at the part where the guy moves the saucepan where we can't see it behind the other guy, the viewers eyes seemed to track the imagined position of the pan.

Nick said...

Good point about the fast cutting. It's why I get so bored watching films like Transformers, where the visuals should be impressing me (especially with that kind of budget!) but my eyes tend to lose focus fairly quickly.

I wonder if the results would differ if this test was given to film critics who are used to acknowledging aspects of a frame or scene that the layman may overlook?

phiq said...

I wonder if there is a difference between "looking" and "seeing." These experiments track the movement of the eyes, but the eyes are nothing without the brain, which is constantly processing the input the eyes give.

knoxblox said...

I have to agree with P.T. Waugh.

When presented with a bouncing ball (or will o' wisp), I feel compelled to follow it.

Terry said...

Interesting! I do wonder, though, if the race or nationality or ethnic background of the viewer might change the focal points a bit. It's my (possibly erroneous) understanding that in some cultures, unlike the American white middle class that I'm part of, in conversation it is the speaker who looks at the listener's face, while the listener looks away from the speaker. It's cultural, is what I was told.

Jeanie Chang said...

For some reason when he was scooping the chilli from the cutting board to the bowl I couldn't help but to look at the bowl and not at the blue green bowl thingy.

Anonymous said...

Interesting how the heat maps are very wide when there are no faces in view, or actions to which direct attention is paid. As soon as a face comes into view, the heat map immediately centres around the nose and becomes much narrower.

I wonder what the heat maps would have been if someone in a bear suit (or similar) would have walked through one of the shots.

Roberto said...

These ‘eye-tracking’ posts are very interesting. Thanks for keeping us in the loop on this evolving subject.
I think ‘phiq’ has introduced an important question: “What is the difference between ‘looking’ and ‘seeing’”?
I think eye-tracking is all about how we ‘look’… which, as a visual artist, is important to understand, and can help us create more dynamic and narrative pictures (your #8 point about VFX is well taken).
What I am struck by is what the tracking doesn’t seem to be tracking: Almost all of the conventional compositional elements!... Line, form, contrast, color ,rhythm, texture, proportion, negative space, etc.
If we want to draw attention (pun intended), put a nose on it… or stick it in the middle of the picture- that’s ‘Looking’.
If we want to create an interesting painting we must help our audience to have an aesthetic experience.
Your blog and your books are certainly helping us all to do a better job. Thanx-RQ

James Gurney said...

Nick, eye tracking tests with still images show that artists do look at images differently, and I would guess the same is true with film.

Phiq, a recording of a scanpath on still images can hint at cognition behind the movement of the focal center. You can tell when someone "zones out," for example.

Terry, you raised a number of questions that would make interesting research topics. I find I look at the other person more when I'm listening than when I'm speaking. I would doubt that such social rules of eye contact transfer to looking at faces on film.

Sirithduriel movement compels and misdirects attention—which is the magician's art.

Roberto--good thoughts. It seems to me what these eye tracking systems are missing is the shift from foveated vision (concentrating on the center of vision only) and peripheral vision, where we broaden our awareness to the big picture, without necessarily moving the center of vision. I can feel myself making that conscious adjustment while looking at a static scene, but there's no way an eye tracker would catch it.