Projects
Using eye movements to reveal memory for objects in scenes
A long-term research focus of the lab is to better understand people's ability to remember real-world objects in scenes. Previous research
had shown that many of the Visual details of an object seem to vanish from memory once gaze shifts away from that object (Irwin, 1996).
This phenomenon, however, had only been documented using relatively simple stimuli (e.g., letters) and only over a very limited range of
eye movements (e.g., one or two). With David Irwin, a gaze contingent display change methodology (meaning that online changes were
made to the display depending on the observer's gaze behavior while viewing the scene) was developed that allowed observers to view a
complex real-world scene for either 3, 9, or 15 fixations. Following the criterion number of allowed fixations, the scene was replaced by
a spatial probe at one of the previously viewed object locations and the observer was asked to report the identity of that object.
Perhaps counterintuitively, we found that memory was better after viewing the scene for only 3 fixations compared to 9 or 15 fixations
—consistent with previous work on transsaccadic memory. We also found that accuracy was extremely high (approaching 100%) for objects
that were about to be fixated when the display was terminated, suggesting that attention precedes gaze to a location in space (Irwin &
Zelinsky, 2002).
Using a similar gaze-contingent memory paradigm, another project extended the Irwin and Zelinsky (2002) logic to the question of
memory serial order effects for objects presented simultaneously in scenes. When trying to recall a list of items, such as the digits of a
telephone number, people usually find it easier to remember the last few items relative to those presented earlier in the list. This
"recency" effect has been the topic of innumerable memory studies and a cornerstone of memory theory since the dawn of modern
experimental psychology. All of these studies, however, have in common a similar methodology. List items are presented singularly one
after the other, and this serial order is later used to derive the Accuracy x Order function defining the recency effect. Unfortunately,
many everyday memory tasks require us to remember objects appearing simultaneously as part of complex scenes rather than singularly
over time. We introduced a method to remove this serial presentation constraint by using a person's eye movements during scene viewing
to serialize the order in which objects are encoded. Specifically, we systematically varied the number of objects that an observer was
allowed to fixate following the initial fixation of a target object. Using this technique, we found that memory for a target object declined
precipitously after fixating 1-3 intervening objects, but then plateaued at a level well above chance performance (Zelinsky, 1999b, 1999c,
Zelinsky & Loschky, submitted). Together with the Irwin and Zelinsky (2002) study, these findings suggest a dual-representation model
for spatial memory in multi-object scenes, with one representation being veridical but easily disrupted and a second representation being
less detailed but far more enduring. In a related ongoing project, we analyzed how often observers look back or "refixate" a previously
viewed object in a scene memory task to determine if these eye movements are being used as part of an active strategy to rehearse,
and hopefully retain, object information in memory. This work is currently under revision (Zelinsky & Loschky, in revision).
Research Philosophy
Each time we engage in a moderately complex task, we likely enlist the help of an untold number of simpler
visuo-motor operations that exist largely outside of our conscious awareness. Consider for instance the steps
involved in preparing a cup of coffee. For the sake of simplicity, assume that the coffee has already been
brewed and is waiting in the pot, and that all of the essential accessories, an empty cup, a spoon, a carton of
cream, and a tin of sugar, are sitting on a countertop in front of you. What is your first step toward accomplishing this goal? The very
first thing that you might do is to move your eyes to the handle of the coffee pot, followed shortly thereafter by the much slower
movement of your preferred hand to the same target. Because the coffee pot is hot and the handle is relatively small, this change in
fixation is needed to guide your hand to a safe and useful place in which to grasp the object. After lifting the pot, your eye may then
dart over to the cup. This action is needed, not only to again guide the pot to a very specific point in space directly over the cup, but
also to provide feedback to the pouring operation so as to avoid a spill. After sitting the pot back on the counter (an act that may or
may not require another eye movement), your gaze will likely shift to the spoon. Lagging shortly behind this behavior may be
simultaneous movements of your hands, with your dominant hand moving toward the sugar tin and your non-preferred hand moving to
the spoon. The spoon is a relatively small and slender object that again requires assistance from foveal vision for grasping; the tin is a
rather bulky and indelicate object that does not require precise Visual information to inform the grasping operation. Once the spoon is
in hand and the lid to the tin is lifted, gaze can then be directed to the tin in order to help scoop out the correct measure of sugar. To
ensure that the spoon is kept level, a tracking operation may be used to keep your gaze on the loaded spoon as it moves slowly to the
cup. After receiving the sugar, and following a few quick turns of the spoon, your coffee would finally be ready to drink (see Land et
al., 1998, for a similarly framed example).
eye movements and visual cognition
