Perception & Psychophysics1993, 54 (6), 808-813

Effects of capacity demands on picture viewing

JAMES R. ANTES and ARLINDA F. KRISTJANSONUniversity of North Dakota, Grand Forks, North Dakota

Effects of cognitive-resource demands on picture-viewing patterns were investigated. The eyefixations of 72 subjects were recorded as the subjects viewed pictures and concurrently performedone of three listening tasks. Half of the subjects were asked to remember certain objects fromthe pictures and half had no-memory instructions. Concurrent auditory monitoring increasedinterfixation distances and the frequency of fixations on regions of high informativeness, anddecreased the area of the pictures explored and the memory for objects in the scenes. It is sug­gested that the demands on cognitive resources influenced subjects' ability to encode and inte­grate fixated information and therefore prolonged the normal first phase of viewing, describedby Buswell (1935).

The pattern of eye movements exhibited by observersduring the free viewing of pictures has been well docu­mented (Antes, 1974;Buswell, 1935; Yarbus, 1967). Bus­well described two phases of viewing. The first involvesa broad survey of the scene, characterized by short fixa­tional pauses. This is followed by fixations of longer du­rations on more localized areas of the picture. Antes foundthat during the first phase observers tend to make rela­tively long saccades to areas high in information content(rated informativeness). Interfixation distances are shorterduring the second phase, during which the viewer tendsto fixate on less informative details.

The distribution of eye fixations during picture viewingappears to be an obvious example of selective attention,with the point of fixation operationalized as the locus ofattention. According to this approach, some cognitivemechanisms can be applied to only one task or input ata time. Beginningabout two decades ago, capacity modelsof attention (Kahneman, 1973; Moray, 1967) emergedto provide another way of thinking about attention. At­tention was conceived as a limited pool (or set of pools)of resources that could be allocated to different inputsor tasks. Researchers began to think in terms of howmuch of the pool of cognitive resources was drawn byvarious inputs or phases of a task, in addition to consid­ering the cognitive mechanisms that influenced which in­puts we attend to.

Although attempts have been made to integrate the twoconcepts of attention (Dark, Johnston, Myles-Worsley,& Farah, 1985), no research has been done to investigatethe capacity demands associated with picture viewing.Some mental effort is presumably involved in the pro-

The authors wish to express appreciation to Tandi Brayson, JenniferLaabs, and Robin Michael for their assistance in data collection. Re­quests for reprints should be addressed to J. R. Antes, Department ofPsychology, Box 8380, University of North Dakota, Grand Forks,ND 58202.

cesses by which the viewer gathers information from theregion around the fixation point, makes a decision aboutwhere to fixate next, and builds an evolving representa­tion of the scene. The study of the capacity associated withthese processes may provide a more complete understand­ing of the phases of picture viewing described by Buswell(1935) and Antes (1974).

The purpose of the present research, then, was to in­vestigate the capacity demands associated with pictureviewing. The strategy involved engaging the subject indifferent tasks that varied in complexity while pictureswere simultaneously being viewed. The extent to whichnormal viewing patterns were influenced by the concur­rent tasks was an indicator of the capacity normally re­quired in viewing pictures.

The eye movements of observers were recorded as theyviewed a series ofline drawings. A third of the drawingswere viewed while the subjects performed a simple lis­tening task, a third while performing a complex listen­ing task, and a third while no concurrent task was per­formed.

We were concerned that the requirement to perform aconcurrent task would encourage a strategy of not exam­ining the pictures at all. Therefore a brief memory taskwas introduced, which enabled us also to investigate whe­ther or not resources drawn away from the task of view­ing the pictures would influence what the subjects couldremember from the pictures. After each drawing wasviewed, the subjects were shown a representation of thesame scene, in which objects were replaced by geomet­ric shapes. Four of the shapes were designated by letters,and the subjects were asked to label the objects from thescene designated by the letters.

The introduction of the memory task produced a fur­ther complication, however. The study of eye movementpatterns during picture viewing has largely occurred under"free viewing" instructions, without a memory task. Ifmemory instructions influence viewing patterns, then ef­fects of the concurrent tasks would be confounded with

Copyright 1993 Psychonomic Society, Inc. 808

CAPACITY DEMANDS AND PICTURE VIEWING 809

effects of the memory instructions. Indeed, instructionsgiven prior to the presentation of the material to be viewedhas been shown to influence eye movement patterns insuch tasks as reading (e.g , Grabe, Antes, Thorson, &Kahn, 1987), picture viewing (e.g., Buswell, 1935;Yarbus, 1967), and examining X rays (e.g., Kundel &Nodine, 1978). Particularly relevant is a study by Fried­man and Liebelt (1981), who asked observers to exam­ine a series of line drawings in preparation for a difficultrecognition memory test. They found that subjects dis­tributed their eye fixations evenly across all the objectsin the drawings, regardless of the informativeness of theobjects. Therefore, the variable of instructions was in­troduced. Half the subjects received a memory test fol­lowing each picture presentation, and half did not.

The cognitive processes that are presumably requiredin scanning a picture inelude (I) encoding the fixated in­formation, (2) integrating information obtained during thecurrent fixation with information from previous fixations,(3) processing information peripheral to the point of fix­ation, and (4) making a decision regarding where to looknext. Effects on the encoding process would be indicatedby deficits in memory for fixated objects as a functionof the complexity of a simultaneous listening task. Evi­dence for disruption of the integration process by thelistening tasks would be a lengthening of Buswell's (1935)first viewing phase, which is demonstrated by relativelylong interfixation distances intervening between relativelyshort fixational pauses on highly informative regions. Ef­fects on processing peripheral information would be re­vealed by changes in the "useful field of view" (Mack­worth, 1965). One way to operationalize the useful fieldof view is to measure mean interfixation distance, withthe assumption that larger interfixation distances indicateprocessing of information farther into the visual periph­ery. Influences of the listening tasks on the decision ofwhere to look next would be revealed by the distributionof fixations across listening-task conditions.

METHOD

SubjectsThe subjects were 36 male and 36 female undergraduate psychol­

ogy students who participated for course credit. They all reportednormal vision without glasses or wore soft contact lenses.

StimuliThere were a total of 12 complex line drawings used in this ex­

periment, representing various indoor and outdoor scenes, selectedfrom a larger pool of 30 drawings. Several pilot studies were con­ducted to select the final scenes for the experiment. For each ofthe drawings from the larger pool, all objects of intermediate size(approximately 2 0 -4 0 of visual angle) were identified. In the firstpilot study, 47 subjects were shown each of the drawings and askedto make two probability judgments for each object, They were askedto rate on a scale from 0 to IDO the percentage of time they wouldexpect the object to be present in (I) "this or similar scenes," and(2) "completely different scenes." Objects with high "similar" rat­ings and low "different" ratings were designated as diagnostic ob­jects. Objects with relatively high ratings for both similar and dif­ferent scenes were called nondiagnostic objects, The 12 drawings

selected each contained two objects that were readily classified asdiagnostic (mean ratings of 80.1 % for similar scenes and 13.2 %for different scenes) and two objects classifed as nondiagnostic(meanratings of78.6% for similar scenes and 36,5% for different scenes),Figure I shows one of the scenes with the diagnostic and nondiag­nostic objects designated,

The 12 scenes were divided into three equivalent sets of four draw­ings, based upon the difficulty a second pilot group of 20 subjectshad in remembering the four critical objects (diagnostic and non­diagnostic). Informativeness ratings were obtained for each of thedrawings in a third pilot study involving 19 subjects. A 4 x 4 matrixwas drawn over each picture, creating 16 equal-sized square sec­tions. The subjects were asked to rate the informativeness of thesections by rank ordering, from I to 16, with the information con­veyed by each section ranging from least informative (I) to mostinformative (16). Mean rankings were obtained for each section andused later in the analysis of eye fixation patterns.

Listening TasksThere were three listening tasks: no task, a simple task, and a

more complex task. The subjects experienced all three listeningtasks, each during one third of the trials. For the simple listeningtask, the subjects heard a recorded list of two-digit numbers pre­sented at a rate of one number per second and were asked to pressa key when they heard a number evenly divisible by five, For thecomplex listening task, the subjects were asked to press a key whenthey heard three consecutive odd numbers. The number of timesa keypress was appropriate was equated for the two tapes and wasapproximately five per 20 sec.

The data from the present study support the selection of theselistening tasks as differing in amounts of cognitive resources re­quired for successful performance. Kahneman (1973) argued thatpupil diameter is a measure of the cognitive resources that are en­gaged during the performance of a task. Mean pupil diameter wasdetermined for each subject under the three listening-task condi­tions across memory instructions, and a one-way listening-task(none/simple/complex) analysis of variance was computed. The

Figure I. One of the drawings used in this research, with the twodiagnostic (solid lines) and nondiagnostic (dashed lines) objectsoutlined.

810 ANTES AND KRISTJANSON

listening task effect was significant [F(2. 142) = 99.70. P < .0011.Subsequent Tukey HSD analysis revealed that mean pupil diameterswhile performing no listening task (4.04 mrn), the simple listeningtask (4.37 mrn), and the complex listening task (4.46 mrn) wereall significantly different from each other.

identical listening task (none/simple/complex) x fixationblock (first/middle/last) analyses of variance with repeatedmeasures on both variables.

For the informativeness variable, there were two sig­nificant main effects and no interactions. Both listening

Figure 2. Mean informativeness of fixated regions (a), mean du­ration (b), and mean interfixation distance (c) as a function of fixa­tion block for the three listening-task conditions.

Last

IJ· ••........ ................•

....

MiddleFixation Block

-,........

...............

.........•...._... . -.

.-...-..__.--......•...... _.., ....-..-.

First

(c)

(a)

10.6

UIUI

~ 10.8Q)

.~iiiE..o'E

:>t:.c 11.0IIIa:

CIII

~ 10.4

CiQ)

~ 4.0

eIII 3.2:E

Q)ocg 3.8UI

oCo 3.6iii)(

;:s 3.4c

440

UQ)UI

.§. 420

c0.. 400III..::l0

c 380IIIQ)

:E - Simple360 ---.-. Complex

(b)

RESULTS

ApparatusEye movementswere recorded with a Gulf and Western Eye View

Monitor (Model 1994S). The instrument uses the relative locationof the corneal reflection and pupil center to determine eye posi­tion. which is sampled at a 60-Hz rate. The 1/60-sec samples wererecorded on line by a PDP-I 1/34 computer and converted later toeye fixations and saccades with a program modified after one de­scribed by Kliegl and Olson (1981).

ProcedureThe subjects were told that their eye movements would be re­

corded while they were viewing a series of line drawings. The sub­jects who had memory instructions (half the males and females)were asked to look at the drawings carefully because they wouldbe asked to identify some of the objects. A sample picture was thenshown, followed by a sample memory slide containing the geometricshapes and letters marking objects from the drawing to be identi­fied. The subjects who did not have memory instructions were sim­ply asked to look at the pictures. All subjects were also told thatwhile viewing the drawings they would be performing a listeningtask simultaneously. Specific instructions for each listeningtask weregiven just before performance on the tasks was required.

Each subject then saw the 12 drawings, one set of four (as de­fined above) while performing the simple listening task, one setwhile performing the complex task, and one set while performingno listening task. Across subjects, each picture set was matchedwith each listening task an equal number of times. Also, each listen­ing task and each picture set appeared in each ordinal position (first,second, or third) an equal number of times during the experiment.

Each picture was shown for 20 sec. The first picture of each setwas considered practice, although the subject was not told so, andthe data were not recorded. The tape containing the numbers forthe listening task was started about 2 sec before the drawing ap­peared and continued for about I sec after the drawing disappearedfrom the screen. A research assistant controlled the tape and re­corded the subject's keypressing responses to the numbers on a sheetcontaining a transcript of the recording. After each drawing, sub­jects having memory instructions were given the memory test. Thesubject had as much time as he/she wished to write the names ofthe objects represented by the four labeled shapes.

All drawings were prepared as slides and projected onto a screenapproximately 1.25 m from the subject. At that distance, the draw­ings subtended a visual angle of 160 vertically and horizontally.

Preliminary analyses were conducted to determine theeffect of memory instructions on the eye movement mea­sures. Since there were only minor effects involving in­structions, the data were combined across instruction con­ditions for the following analyses.

Effects of Listening TasksTo examine the effects of the listening tasks, the first

10 fixations, the middle 10 fixations, and the last 10 fix­ations each subject made on a drawing were analyzed.For each of these blocks of fixations, the mean rated in­formativeness of regions fixated, the mean fixation dura­tion, and the mean interfixation distance were determined.The data are depicted in Figure 2 and were subjected to

CAPACITY DEMANDS AND PICTURE VIEWING 811

task [F(2,142) = 3.51, P < .05] and fixation block[F(2,142) = 7.IO,p < .01] were significant. SubsequentTukey HSD analyses revealed that fixations during thesimple listening task were directed to significantly moreinformative regions than were those during the no­listening-task condition. The difference between the com­plex listening task and no listening task was in the samedirection but was not significant. There was no differencebetween the simple and complex listening tasks. Fixationsduring the first fixation block were directed to more in­formative regions than were those during the middle orlast block, which did not differ.

In the analysis of mean duration, the main effects ofboth fixation block [F(2,142) = 7.03, p < .01] andlistening task [F(2, 142) = 8.38, p < .001] were signifi­cant. Mean durations were shorter during the first fixa­tion block than either the middle or the last block, whichdid not differ. Longer mean durations occurred duringthe complex listening task than during the simple listen­ing task or no listening task. The latter two conditionsdid not differ.

For interfixation distance, there was a significant maineffect of listening task [F(2,142) = 4.42, p < .05] anda significant fixation block x listening task interaction[F(4,284) = 7.88, p < .01], which was characterized bya significant decrease in interfixation distance over thefirst two fixation blocks for the no-listening-task condi­tion, a significant increase for the complex listening task,and no change for the simple listening task. When com­bined over fixation block, this resulted in significantlylonger interfixation distances during the complex listen­ing task than in the no-listening-task condition. Interfix­ation distance during the simple listening task was alsogreater than that during the no-listening-task condition,but the difference only approached significance.

Interfixation distance indicates how large the saccadeswere, on the average, but does not indicate to what ex­tent the entire picture was viewed. It is thus possible tohave many large saccades, yet to focus fixations on onlya few areas of the picture. To get an indication of the ex­tent to which the entire picture was scanned, an analysisof regions viewed was made. This simply represents thenumber of the 16 picture sections viewed at least onceduring the 20-sec exposure. A one-way listening-task(none/simple/complex) analysis of variance was per­formed on these data, resulting in a significant effect forlistening task [F(2,142) = 37.02, p < .001]. Tukey HSDanalysis revealed that all three means were significantlydifferent from each other. When no listening task was in­volved, the subjects viewed 11.84 regions (74.0%); forthe simple listening task, they averaged 10.68 regions(66.8%); during the complex listening task, they viewed10.00 regions (62.5 %). Thus, as greater cognitive capac­ity was engaged, less area of the pictures was viewed.

Memory for the ObjectsTo score memory performance, a key was devised by

asking an independent group of subjects to label the criti-

cal objects. Figure 3 presents the number of objects cor­rectly remembered by subjects in the present study whohad memory instructions. A listening task (none/sim­ple/complex) x diagnosticity (diagnostic/nondiagnostic)analysis of variance was computed, with repeated mea­sures on both variables. There was a significant diag­nosticity effect [F(l,35) = 20.17, p < .001], indicatingsuperior memory for the diagnostic objects. There alsowas a significant listening-task effect [F(2,70) = 35.20,p < .001]. The Tukey HSD analysis showed that mem­ory was superior in the no-listening-task condition to thatin both the simple- and the complex-listening-task condi­tions, which did not differ.

An analysis was undertaken to determine whether thememory differences were related to the distribution of eyefixations. Loftus (1972) demonstrated a positive relation­ship between the number of eye fixations and recognitionmemory for pictures. The number of eye fixations on thediagnostic and nondiagnostic objects was determined forboth instruction conditions, and a listening task (none/simple/complex) x diagnosticity (diagnostic/nondiagnos­tic) analysis of variance was computed. The diagnostic­ity effect was significant [F(l,71) = 12.09, P < .001],with diagnostic objects receiving an average of 1.91 fix­ations, and nondiagnostic objects, 1.68 fixations. The lis­tening task x diagnosticity interaction was also signifi­cant [F(2,142) = 3.73,p < .05]. Listening demand hadlittle effect on the number of fixations on diagnostic ob­jects (1.84, 2.03, and 1.85 fixations for no listening task,the simple listening task, and the complex listening task,respectively) but did tend to reduce the number of fixa­tions on nondiagnostic objects (1.90, 1.68, and 1.46 fix­ations). This interaction is consistent with the finding re­ported earlier that viewing becomes restricted to moreinformative areas with increased task demands, given thatthe areas encompassing diagnostic objects received higherinformativeness rankings than did areas with nondiagnos­tic objects. Also, the fact that memory for diagnostic ob­jects declined with listening demand but that the numberof fixations to diagnostic objects remained constant sug­gests that the mental resources drawn during the listen-

0.9

0.8

0.1

0.6

Proportlo. 0.5

Correct 0..4

0.3

0.2

0.1

None Simple Complex

listening Test

Figure 3. Proportion of diagnostic and nondiagnostic objects cor­rectly identified as a function of listening task.

812 ANTES AND KRISTJANSON

ing tasks affected the amount of information gathered dur­ing an eye fixation.

Listening-Task PerformancePerformance on the simple and complex listening tasks

was assessed by examining d' values. The subjects per­formed significantly better on the simple listening task(d' = 3.81) than on the complex listening task [d' = 3.00;t(71) = 5.77, p < .001].

DISCUSSION

The subjects in this experiment were asked to view sceneswhile simultaneously performing listening tasks designedto draw cognitive resources. The cognitive tasks variedin difficulty, as indicated by d' values, and demandedmental effort, as indicated by pupil diameter. Of centralinterest were the effects of performing these listening taskson the scanning patterns of subjects as they viewed thedrawings.

One major effect occurred in the number of regions ofthe picture fixated. As listening demand increased, fewerregions of the picture were examined. The informative­ness analysis indicated that the regions that were viewedduring concurrent performance of the simple and com­plex listening tasks tended to be those ranked as highlyinformative. The listening tasks also had an effect on in­terfixation distance. When no listening task was being per­formed, the subjects showed the typical tendency to have,on the average, relatively large saccades early in view­ing and relatively smaller saccades as viewing progressed.When the simple listening task was being performed, in­terfixation distance remained at the relatively longer levelcharacteristic of early viewing throughout the viewing pe­riod. During the complex listening task, interfixation dis­tance actually increased from the level present early inviewing.

The listening tasks also degraded the memory for bothdiagnostic and nondiagnostic objects in the drawings. Still,in each listening-task condition, memory for diagnosticobjects was superior to that for nondiagnostic objects.When fixation location was related to memory perfor­mance, the results were consistent with the hypothesis thatthe demands for cognitive resources resulted in a reducedamount of information processed per eye fixation.

The finding that the pattern of eye movements changedas a function of concurrent task demands suggests thatmental effort is required for picture viewing. The pro­cess that was most clearly disrupted, and thus may re­quire the greatest resources, was the encoding of fixatedinformation. This is indicated by decreased memory fordiagnostic objects on simple- and complex-listening-tasktrials but no change in frequency of fixations on diagnos­tic objects.

The data are consistent with the contention that the pro­cess of integration of information across fixations also re­quired resources and was disrupted. This is suggested bythe pattern of fixation locations and interfixation distances

characteristic of early segments of normal viewing, whichwas demonstrated throughout the 20-sec exposure whenthe simple and complex listening tasks were also beingperformed.

Peripheral processing is the third picture viewing pro­cess that was identified above, and it appears not to havebeen disrupted. This is evidenced by the long saccadesto highly informative areas occurring during the simple­and complex-listening-task conditions. It is worth em­phasizing that, at the same time as longer interfixationdistances occurred, the fixations were placed selectivelyto more limited areas of the scenes, the highly informa­tive areas. This finding supports the distinction made byHockey (1970) that engaging cognitive resources causes"attentional narrowing" and not necessarily "perceptualnarrowing. "

The fourth process identified above is that of makinga decision regarding where to look next. As listening taskcomplexity increased, fixations were concentrated onmore restricted areas of the scenes, areas that were, onthe average, highly informative. This, of course, does notnecessarily indicate that the decision-making process wasdisrupted, and it may, instead, suggest that the processwas working well. That is, the distribution of fixationsis consistent with a decision-making heuristic to fixate themost highly informative area that is not fully processed.

Taken together, these results suggest that the cognitiveresources taken from normal picture viewing by the listen­ing tasks resulted in a retardation or prolongation of thenormal viewing process. The larger interfixation distancesand tendency to fixate highly informative regions charac­teristic of early phases offree viewing (Antes, 1974) con­tinued throughout the entire exposure when subjects wereperforming the simple and complex listening tasks. Therewas an apparent pattern for subjects, when faced with theconcurrent listening tasks, to fixate repeatedly the moreinformative regions. The results also suggest that each fix­ation provided less information during the simple andcomplex listening tasks than when no listening task wasbeing performed. This occurred in spite of the fact thatmean fixation duration increased with increasing task com­plexity. Loftus (1983) proposed that an eye fixation lastsat least long enough to encode the fixated picture features.The simple and complex listening tasks interfered withthis encoding process. In terms of the phases of pictureviewing that Buswell (1935) first described, perhaps theinitial phase serves to provide the viewer with a basic gen­eral understanding of the picture content that serves asa guide for later viewing. When cognitive resources arediverted to another concurrent activity, that understand­ing is delayed, and the viewer's progression to the sec­ond phase of viewing is delayed.

One implication of the present research is that viewersin situations of reduced available cognitive resources areless able to respond quickly to the changing demands ofthe visual environment. However, the subjects were notasked to respond to the scenes, and the pictures wererather mundane, far from the situation presented to, say,

CAPACITY DEMANDS AND PICTURE VIEWING 813

the operator of a motor vehicle. Also, the present sub­jects were instructed not to interrupt their performanceof the listening tasks as they viewed the drawings, a fac­tor also limiting the generalizability of these findings. Fur­ther complicating definitive conclusions from this studyis the fact that eye movement measures that are aggregatesof behavior over a period of time were used to make in­ferences about processes that presumably occur duringeach eye fixation.

REFERENCES

ANTES, J. R. (1974). The time course of picture viewing. Journal ofExperimental Psychology, 103,62-70.

BUSWELL, G. T. (1935). How people look at pictures. Chicago: Uni­versity of Chicago Press.

DARK. V. J., JOHNSTON, W. A., MYLES-WORSLEY, M., & FARAH,M. 1. (1985). Levels of selection and capacity limits. Journal ofEx­perimental Psychology: General, 114,472-497.

FRIEDMAN, A., & LIEBELT, L. S. (1981). On the time course of view­ing pictures with a view towards remembering. In D. F. Fisher, R. A.Monty, & J. W. Senders (Eds.), Eye movements: Cognition and visualperception (pp. 137-155). Hillsdale, NJ: Erlbaum.

GRABE, M., ANTES, J. R., THORSON, I., & KAHN, H. (1987). Eye fix-

ation patterns during informed and uninformed comprehensionmonitoring. Journal of Reading Behavior, 19, 123-140.

H(X'KEY. G. R. J. (1970). Effect of loud noise on attentional selectivity.Quarterlv Journal of Experimental Psychology, 22, 28-36.

KAHNEMAN, D. K. (1973). Attention and effort. Englewood Cliffs, NJ:Prentice-Hall.

KUEGL, R., & OLSON, R. K. (1981). Reduction and calibration of eyemonitor data. Behavior Research Methods & Instrumentation, 13,107-111.

KUNDEL. H .. & NODINE, C. (1978). Studies of eye movements and visualsearch in radiology. In J. W. Senders, D. F. Fisher. & R. S. Monty(Eds.). Eye movements and the higher psychological functions(pp, 241-258). Hillsdale, NJ: Erlbaum.

LOFTUS, G. R. (1972). Eye fixations and recognition memory for pic­tures. Cognitive Psychology, 3, 525-551.

LOFTUS, G. R. (1983). Eye fixations on text and scenes. In K. Rayner(Ed.), Eye movements in reading (pp. 359-376). New York: Aca­demic Press.

MACKWORTH, N. H. (1965). Visual noise causes tunnel vision. Psy­chonomic Science, 3, 67-68.

MORAY. N. (1967). Where is capacity limited? A survey and a model.Acta Psychologica, 27, 84-92.

YARBUS, A. L. (1967). Eye movements and vision. New York: Plenum.

(Manuscript received March 16, 1992;revision accepted for publication March 22, 1993.)