Concurrent schedules: reinforcer magnitude effects

351

JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2003, 79, 351–365 NUMBER 3 (MAY)

CONCURRENT SCHEDULES: REINFORCER MAGNITUDE EFFECTS

JASON LANDON, MICHAEL DAVISON, AND DOUGLAS ELLIFFE

THE UNIVERSITY OF AUCKLAND

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key pro-cedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each conditionremained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed.At a molar level of analysis, preference was described well by a version of the generalized matchinglaw, consistent with previous reports. More local analyses showed that recently obtained reinforcershad small measurable effects on current preference, with the most recently obtained reinforcerhaving a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses,and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choicehave both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitudeis exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcermagnitude is varied are likely to be due to equal frequencies of different sized preference pulses,whereas higher sensitivities when reinforcer rates are varied might result from changes in the fre-quencies of different sized preference pulses.

Key words: concurrent schedules, choice, generalized matching, reinforcer magnitude, key peck,pigeons

Much concurrent-schedule research has fo-cused on the effects on behavior of varyingthe relative frequency of reinforcement (for areview see Davison & McCarthy, 1988). Rein-forcers, however, can be varied along a num-ber of other dimensions such as magnitude,delay, and quality. Both the strict (Herrnstein,1961) and the generalized (Baum, 1974)matching laws have been extended to includesuch variations of other independent variables(Baum & Rachlin, 1969; Killeen, 1972). With-in the generalized-matching framework, if weassume that the effects of these independentvariables do not interact, we may write theconcatenated generalized matching law

B X1 1log 5 a log 1 log c, (1)O xB X2 2

where B1 and B2 are the responses emitted at

The experiment reported here was completed in partialfulfillment of a doctoral degree at The University of Auck-land by the first author. We thank William Baum for hiscontinuing contribution to this research, Mick Sibley forlooking after the subjects, and members of the Experi-mental Analysis of Behavior Research Unit for helpingconduct the experiments. The data used in the analysesare too extensive to include in an appendix, and may beobtained from the authors on receipt of a 100-Mb zip disk.

Reprints may be obtained from any author, Depart-ment of Psychology, The University of Auckland, PrivateBag 92019, Auckland, New Zealand (e-mails: [email protected]; [email protected]; [email protected]).

the two alternatives, X1 and X2 are values ofa particular independent variable at the twoalternatives, ax is the sensitivity of preferenceto changes in that independent variable, andlog c is inherent bias. Independent variablesthat are constant and equal across the re-sponse alternatives drop out of the equation,and those that are constant but unequal con-tribute to bias. Sensitivity values for differentindependent variables measure the degree ofcontrol that those variables exert over pref-erence. The effects of different independentvariables are simply summed to give overallpreference, reflecting the assumption thatthose variables do not interact. Thus, for anexperiment in which only reinforcer magni-tude is varied, Equation 1 reduces to

B M1 1log 5 a log 1 log c, (2)mB M2 2

where M1 and M2 are the reinforcer magni-tudes obtained at the two alternatives and amis sensitivity to reinforcer magnitude.

To date, relatively few studies have investi-gated the effects of reinforcer magnitude,and results remain ambiguous on how thoseeffects compare to those of reinforcer fre-quency. Catania (1963) reported an early in-vestigation of the effects of reinforcer mag-nitude using concurrent schedules. Pigeons’responses were reinforced on independent

352 JASON LANDON et al.

concurrent variable-interval (VI) 2-min VI 2-min schedules with reinforcer durations atthe two alternatives varied in a systematic wayacross four conditions. He also arranged a se-ries of conditions with a single VI schedule inwhich reinforcer magnitude was varied from3 s to 6 s. Response rate to the single VIschedule was unaffected by reinforcer mag-nitude. When, however, the equivalent datafrom one alternative during the concurrent-schedule conditions were examined, re-sponse rate was a linear function of reinforc-er magnitude.

Schneider (1973) investigated the effectsof reinforcer magnitude by varying the num-ber of food pellets presented. The procedurewas a slightly unusual two-key concurrentschedule. Reinforcers were delivered at thetwo alternatives in an irregular predeter-mined order that was changed every threesessions. Across conditions, Schneider variedboth the reinforcer-magnitude ratio and thereinforcer-frequency ratio. Response ratiosundermatched both reinforcer-frequency andreinforcer-magnitude ratios, with more ex-treme undermatching to reinforcer magni-tude. Multiple linear regression analyses ofthe log ratios of responses, reinforcer fre-quencies, and reinforcer magnitudes fromthe complete data set produced estimates ofsensitivity to reinforcer frequency and rein-forcer magnitude of 0.60 and 0.34 respective-ly. Thus, Schneider concluded that differenc-es in reinforcer frequencies exerted greatercontrol over behavior than differences in re-inforcer magnitudes.

Todorov (1973), using an even more un-usual procedure, found similar results. Hisswitching-key concurrent schedule consistedof three different VI schedules, each associ-ated with a different key color. A given colorwas associated with one schedule of reinforce-ment throughout the experiment, but the re-inforcer magnitude (defined as duration ofaccess to the food magazine) associated witheach color was varied across conditions from2 s to 8 s. Each of the three combinations ofschedules was presented once in a session fora total of 20 reinforcer deliveries each. To-dorov reported mean values for sensitivity toreinforcer frequency of 0.90 and sensitivity toreinforcer magnitude of 0.27. Again, frequen-cy exerted greater control over preferencethan did magnitude.

Keller and Gollub (1977) used a more stan-dard two-key concurrent-schedule procedure.In their Experiment 1, they varied both rel-ative reinforcer frequencies (overall constantat 60 per hour) and reinforcer durations(sum always 6 s). Keller and Gollub arguedthat their results were inconsistent with thoseof Schneider (1973) and Todorov (1973) inthat there was not ‘‘a consistently greater de-gree of behavioral control by reinforcementrate than by reinforcement duration’’ (p.149). A reanalysis reported by Davison andMcCarthy (1988), however, showed that, asacknowledged by Keller and Gollub, their re-sults were not consistent across subjects.Nonetheless, in two out of three comparisonssensitivity to reinforcer frequency was higherthan sensitivity to reinforcer magnitude, withthe group values being 0.62 and 0.50, respec-tively.

In their Experiment 2, Keller and Gollub(1977) examined the possibility that pro-longed exposure to a variety of magnitudesand frequencies of reinforcement might at-tenuate concurrent-schedule control (seealso Todorov, Oliveira Castro, Hanna, Bitten-court de Sa, & Barreto, 1983). Each subjectwas exposed to a different arrangement inwhich the reinforcer-frequency ratio, the re-inforcer-magnitude ratio, or both, were var-ied. In these conditions, relative responserates more closely approximated the relativetotal reinforcement access time. Keller andGollub interpreted this as suggesting thatcontinued exposure to variations, as in theirExperiment 1, suppresses sensitivity values.Davison and McCarthy (1988) reanalyzedthese data and reported that sensitivity to re-inforcer frequency and magnitude valueswere both 1.06, but both values had relativelylarge standard deviations (0.11 and 0.20, re-spectively). Davison and McCarthy also point-ed out that Keller and Gollub changed fromarithmetic VI schedules in Experiment 1 toexponential schedules in Experiment 2. Ex-ponential or constant-probability VI sched-ules generally produce higher sensitivitiesthan do arithmetic VI schedules (Elliffe & Al-sop, 1996; Taylor & Davison, 1983). On bal-ance, therefore, the experimental evidencesuggests that control by variations in reinforc-er frequency is greater than control by vari-ations in reinforcer magnitude (Keller & Gol-

353REINFORCER MAGNITUDE EFFECTS

lub’s Experiment 1; Schneider, 1973;Todorov, 1973).

Another relatively unusual procedure wasused by Todorov, Hanna, and Bittencourt deSa (1984) to investigate the effects of rein-forcer magnitude on concurrent scheduleperformance. They exposed pigeons to 29sessions, each 8 hr in duration, in which thereinforcement parameters changed every ses-sion. In the first nine sessions, reinforcermagnitudes were always equal and reinforcerfrequencies were varied across the two alter-natives. In the second nine sessions, both re-inforcer frequencies and reinforcer magni-tudes were varied. In the final 10 sessions,reinforcer frequencies were held constantand equal while reinforcer magnitudes werevaried. Todorov et al. showed that hour-by-hour sensitivity to reinforcer frequency values(range 0.81 to 1.13) were higher than sensi-tivity to reinforcer magnitude values (0.23 to0.62) irrespective of whether both variableswere manipulated or each was manipulatedindividually. Furthermore, these values, ob-tained using a novel procedure, were consis-tent with previous research investigating re-inforcer magnitude (Keller & Gollub, 1977,Experiment 1; Schneider, 1973; Todorov,1973), and research manipulating only rela-tive rates of reinforcement (Taylor & Davison,1983; Wearden & Burgess, 1982).

McLean and Blampied (2001) investigatedwhether the assumption made in the concat-enated generalized matching law that sensi-tivity to reinforcer frequency is independentof both absolute and relative reinforcer mag-nitudes held. A standard two-key concurrentschedule was used and relative and absolutemagnitudes of reinforcement were variedover several series of conditions. Within eachseries, the relative frequencies of reinforce-ment over the two alternatives were varied,enabling values of sensitivity to reinforcer fre-quency to be calculated. McLean and Blam-pied’s results showed that sensitivity to rein-forcer frequency was the same irrespective ofthe absolute magnitude of the reinforcers.Moreover, it was also unaffected by arrangingunequal reinforcer magnitudes for the two al-ternatives, although behavior was biased to-wards the alternative at which the larger re-inforcers were obtained. Thus, theconcatenated generalized matching law(Equation 1) was supported.

In contrast, Davison and Hogsden (1984)reported a result that is problematic for con-catenated generalized matching. In Part 5 oftheir experiment, Davison and Hogsden ar-ranged VI 120-s schedules on both keys of atwo-key concurrent schedule. They held theright-key reinforcer duration constant at 3 sand varied the left-key reinforcer durationfrom 1 to 10 s over five conditions. None ofthe previous studies had undertaken an ex-tensive manipulation of reinforcer magni-tudes in a standard procedure while retaininga constant reinforcer frequency. Davison andHogsden found a nonlinear relation betweenlog response ratios and log reinforcer-mag-nitude ratios, which is inconsistent with Equa-tions 1 and 2.

Davison and Hogsden (1984) pointed outthat the generalized matching law for rein-forcer magnitude had been generally accept-ed on the basis of relatively few data. To in-terpret their result, they first considered theirdata in terms of the amount of food con-sumed rather than the time for which accessto food was provided, because Schneider’s(1973) study is the only one reported that ar-ranged discrete amounts of food as reinforc-ers. Epstein (1981) showed that the amountof food consumed by pigeons is a negativelyaccelerated function of reinforcer duration.This, however, implies that a different kind ofnonlinearity (concave downward from the or-igin with increasing ratio rather than concaveupward) should have been evident in Davisonand Hogsden’s data. Davison and McCarthy(1988) considered the idea that the subjectstook a constant time to move from the key tothe food magazine. Adjusting the reinforcermagnitudes in this manner, however, did littleto make the data more linear. Davison andHogsden’s result suggests that sensitivity toreinforcer magnitude, rather than being aconstant, depends on the absolute magni-tudes of the reinforcers.

To summarize, research on the effects ofreinforcer magnitude on choice is bothscarce and ambiguous. First, the term rein-forcer magnitude is ambiguous. All the stud-ies above, except Schneider (1973), variedduration of access to food, and indeed some(Davison & Hogsden, 1984; McLean & Blam-pied, 2001) have used reinforcer duration asa more precise label. Because Epstein (1981)showed that the amount of food consumed is


not a simple linear function of the durationof magazine access, it is not clear how accessduration should best be translated into rein-forcer magnitude. We have retained the ge-neric term magnitude here because the ma-nipulation we have used is perhaps not mostaccurately described as one of reinforcer du-ration. The research outlined above hasshown, in general, that changes in the relativefrequency of reinforcers exert greater controlover behavior than changes in the relativemagnitude of reinforcers. That is, preferenceundermatches relative magnitude more thanit undermatches relative frequency. Some re-search (Davison & Hogsden; see also Davison,1988; Logue & Chavarro, 1987), however, hasquestioned the applicability of the concate-nated generalized matching law as a descrip-tion of behavior under these manipulations.

The present experiment also makes con-tact with recent research that has shown thatindividual reinforcers have large effects onpreference in a procedure introduced by Bel-ke and Heyman (1994) in which the rein-forcer ratio varies within sessions (Davison &Baum, 2000, 2002; Landon & Davison, 2001).Landon and Davison showed that controlover responding in this procedure was notpurely local, but also included longer-termfactors. Landon, Davison, and Elliffe (2002)reported similar effects in a more standardconcurrent schedule. They showed bothshort-term effects of individual reinforcersand long-term effects of aggregations of re-inforcers. Thus, the variables controlling re-sponding in a steady-state procedure wereneither solely local nor solely long term innature.

The present experiment extended the ap-proach taken by Landon et al. (2002). Thisapproach combines a return to conventionalexperimental manipulations with detaileddata collection. The experiment provides aparametric investigation of the effects of re-inforcer magnitude in a standard concurrentschedule with the relative frequency of rein-forcement held constant. We varied the re-inforcer magnitude ratio over five levels bychanging the number of short (1.2-s) hopperpresentations across conditions. In all condi-tions, the sum of the numbers of hopper pre-sentations per reinforcer over the two con-current alternatives was always eight. Detailed

time and event data allowed for the effects ofreinforcers to be analyzed at different levels.

METHODSubjects

The subjects were the same five homing pi-geons used by Landon et al. (2002). Theywere numbered 131, 132, 134, 135, and 136,and were maintained at 85% 6 15 g of theirfree-feeding body weights by postsession feed-ing of appropriate amounts of mixed grain.Water and grit were freely available to thesubjects at all times.

ApparatusEach pigeon was housed separately in a

cage 380 mm high, 380 mm wide, and 380mm deep. The back, left, and right walls ofeach cage were constructed of sheet metal;the top, floor, and front wall consisted of met-al bars. Each cage contained two woodenperches, the first mounted 95 mm from andparallel to the front wall, and the secondmounted 95 mm from and parallel to theright wall.

The right wall of each cage contained threetranslucent response keys, 20 mm in diame-ter, centered 100 mm apart and 200 mmabove the perches. The center key remaineddark and inoperative throughout. The leftkey could be lit yellow, and the right keycould be lit either red or green. An effectiveresponse required a force of approximately0.1 N to be applied to a lit key. A hoppercontaining wheat was located behind an ap-erture (50 mm by 50 mm) situated 145 mmbelow the center key. During reinforcer de-livery, the key lights were extinguished andthe hopper was raised to the aperture andilluminated. Reinforcement consisted of apredetermined number of successive 1.2-shopper presentations separated by 0.5-sblackouts, as described below. All experimen-tal events were arranged on an IBMt PC-com-patible computer running MED-PCt soft-ware, located in a room remote from theexperimental cages. The computer recordedthe time, at 10-ms resolution, at which everyevent occurred in experimental sessions.

ProcedureA switching-key (Findley, 1958) concurrent-

schedule procedure was used. Sessions began


Table 1

Sequence of experimental conditions and the number of1.2-s hopper presentations per reinforcer delivery foreach of the five conditions. The overall probability of re-inforcement per second was constant at .033, and therelative reinforcer probability was constant at .5.

ConditionNumber of hopper presentations

per reinforcer delivery (R : G)

12345

2:66:21:74:47:1

with the left (switching) key lit yellow, and theright (main) key lit either red or green withequal probability. Reinforcers were scheduledaccording to a single exponential VI 30-sschedule (p 5 .033 per second). Once a re-inforcer was arranged, it was allocated to ei-ther the red or green alternative with a fixedprobability of .5. Reinforcers were depen-dently scheduled (Stubbs & Pliskoff, 1969),meaning that once a reinforcer was arrangedfor one alternative, no further reinforcerswere arranged until that reinforcer had beenobtained. A 2-s changeover delay (Herrn-stein, 1961) prevented responses from pro-ducing an arranged reinforcer until 2 s hadelapsed since the last switching-key response.Reinforcers consisted of a specified numberof successive 1.2-s hopper presentations,which were varied across conditions (see Ta-ble 1). These hopper presentations were sep-arated by 0.5-s blackouts.

The sequence of experimental conditionsis shown in Table 1. Across conditions, theoverall rate of reinforcement was constant, aswas the red:green reinforcer ratio, which wasalways 1:1. The total number of hopper pre-sentations to both alternatives was alwayseight (9.6-s access to wheat), and the magni-tude ratios varied over five conditions from 7:1 to 1:7 as shown in Table 1. No stability cri-terion was in effect, but 65 sessions wereconducted for each condition to ensure suf-ficient data were collected to allow analysis ofparticular sequences of reinforcers. The datafrom the last 50 sessions of each conditionwere used in the analyses. Sessions were con-ducted daily, and ended in blackout after 80reinforcers had been obtained, or after 42min had elapsed, whichever occurred first.

RESULTS

Figure 1 shows the logarithms of the red-over-green response- and time-allocation ra-tios plotted as a function of the logarithms ofthe red-over-green reinforcer magnitude ra-tios. The magnitudes used were the total du-rations of access to wheat, with the 0.5-s pe-riods between hopper presentationsdiscarded. Equation 2 was then fitted to thedata by least-squares linear regression. Theequations for the fitted lines are shown above(time allocation) and below (response allo-cation) each line. The percentage of varianceaccounted for was always high, indicating thatthe lines fitted the data well. The biases weregenerally small, except for Pigeon 136. Theslopes of the fitted lines, which indicate sen-sitivity to reinforcer magnitude, ranged from0.70 to 0.87 (mean 5 0.76) for response al-location, and from 1.06 to 1.32 (mean 51.15) for time allocation. Thus, these valueswere higher than those estimated in previousresearch (Keller & Gollub, 1977; Schneider,1973; Todorov, 1973; Todorov et al., 1984).Table 2 shows these sensitivity values, togeth-er with sensitivities to reinforcer frequencyobtained by Landon (2002) for the same sub-jects. In nine out of ten comparisons (Pigeon136, time allocation, was the exception), sen-sitivity to magnitude was less than the corre-sponding sensitivity to frequency (binomial p, .05). All response measures of sensitivity toreinforcer magnitude were lower than thecorresponding time measures (one-tailed bi-nomial p , .05), consistent with typical find-ings for sensitivity to reinforcer frequency (El-liffe & Alsop, 1996; Taylor & Davison, 1983).

To examine the effects on current prefer-ence of recently obtained reinforcers, an an-alytic procedure described by Landon et al.(2002) was used. The data were analyzed us-ing a moving window of the eight most re-cently obtained reinforcers. Thus, 256 dis-tinct sequences of red and green reinforcerswere possible. Beginning with the eighth re-inforcer in a session, red and green responsenumbers after each successive reinforcerwere aggregated according to which of those256 sequences they followed, and a log red:green response ratio calculated as a measureof current preference. Because the presentexperiment always arranged a reinforcer-fre-quency ratio of 1:1, all the 256 possible se-


Fig. 1. Log response- and time-allocation ratios plotted as a function of the log reinforcer magnitude ratios foreach subject in each condition. The straight lines were fitted by means of least-squares linear regression, and theequations are shown on the graphs.


Table 2

Sensitivity to reinforcer magnitude for both response andtime allocation, and sensitivities to reinforcer frequencyobtained by Landon (2002) for the same subjects.

Sensitivity tomagnitude

Responses Time

Sensitivity tofrequency

Responses Time

Pigeon 131Pigeon 132Pigeon 134Pigeon 135Pigeon 136

0.740.700.710.870.77

1.161.061.091.111.32

0.920.870.991.100.99

1.241.171.111.291.04

quences of reinforcers occurred. However,there were occasional instances when prefer-ence following a particular sequence was ex-clusive to one alternative, and no log re-sponse ratio could be calculated.

For sequences following which a log re-sponse ratio could be calculated, the contri-bution of each of the immediately precedingeight reinforcers to the current log responseratio was measured using the following gen-eral linear model:

7B R 5 R: 1bR j jlog 5 log k 1 .(3)O1 2 5 6B R 5 G: 2bj50G j j

In Equation 3, j denotes reinforcer lag inthe preceding sequence of eight reinforcers,so that R0 is the most recent reinforcer. Thecoefficients bj are called log reinforcer effect(Landon et al., 2002), and are log responseratios representing the amount of currentpreference attributable to the reinforcer atLag j. Log reinforcer effect is conceptually,but not quantitatively, analogous to sensitivityto reinforcement at each lag. If the reinforcerat Lag j was obtained at the red alternative,bj is added because the log response ratioshould move in a positive direction, and con-versely is subtracted if the reinforcer was ob-tained at the green alternative. The constantlog k is also a log response ratio and measuresthe residual current preference not attribut-able to any of the eight most recently ob-tained reinforcers.

The best-fitting least-squares estimates oflog reinforcer effect and log k were obtainedby fitting Equation 3 to the log response ra-tios following each eight-reinforcer sequenceusing the Quattro Prot v. 8 Optimizer func-tion. This analysis was carried out separately

for each condition and for each subject. Fig-ure 2 shows log reinforcer effect of each ofthe preceding reinforcers plotted as a func-tion of reinforcer lag (Lag 0 is the most re-cent reinforcer) for each subject in each con-dition. The constant (log k) is also shown foreach subject in each condition.

Figure 2 shows three effects. First, the mostrecently obtained reinforcer had the largesteffect on current preference. Second, rein-forcers beyond Lag 0 had similar and smalleffects on current preference. Third, the con-stant log k became more extreme as the re-inforcer magnitude ratios were made moreextreme. A two-way repeated measures anal-ysis of variance (ANOVA) was used to exam-ine any effects of the reinforcer magnituderatio and of the sequential position of the re-inforcer on log reinforcer effect. This ANO-VA confirmed that neither the reinforcermagnitude ratio, F(3, 112) 5 1.80, p . .05,nor the sequential position of the reinforcer,F(6, 112) 5 1.30, p . .05, had a significanteffect on log reinforcer effect. The interac-tion was also not significant: F(18, 112) 50.62, p . .05. Figure 2 shows that log k be-came more extreme as the reinforcer mag-nitude ratio became more extreme, and thiswas supported by a one-way ANOVA, F(4, 20)5 100.63, p , .05.

Given that the reinforcer magnitudes ar-ranged in the present experiment were, apartfrom Condition 4, unequal, it was of interestto consider log reinforcer effect following redand green reinforcers separately. To do this,the same approach was used, but separate val-ues for log reinforcer effect were calculatedby fitting red and green reinforcers separate-ly. Thus, the following equation was fitted tothe data in the same way as Equation 3:

BRlog 5 log k1 2BG

7 R 5 R: 1bj rj1 . (4)O 5 6R 5 G: 2bj50 j gj

Equation 4 is identical to Equation 3 exceptthat a separate log reinforcer effect value wasestimated for reinforcers obtained at the red(brj) and green (bgj) alternatives.

In the interests of conserving space, Fig-ure 3 shows the results of this analysis foreach subject in Conditions 4, 1, and 3 only.


Fig. 2. Log reinforcer effect for each of the previous eight reinforcers plotted as a function of reinforcer lag (0being the most recently obtained reinforcer) for each subject in each condition. Also shown are values of log k (seeEquation 3) for each subject.

These conditions are representative of thedata from all conditions. The left and rightpanels show values of log reinforcer effectfor the red and green alternatives, respec-tively. The right panels also show values oflog k. The data from Condition 4 (4:4) showthat a reinforcer at Lag 0 had a large effecton preference, while reinforcers beyond Lag0 had small, generally positive, and similarlysized effects on current preference. In Con-

ditions 1 (2:6) and 3 (1:7) clear and regularchanges occurred in the values of log rein-forcer effect. Within conditions, reinforcersobtained at the alternative providing thelarger reinforcers had both larger and morepositive effects on current preference. Also,in the right panels, as the reinforcer mag-nitude (and the ratio) was increased therewas an increase in the values of log reinforc-er effect at all lags. In the left panels, as the


Fig 3. Log reinforcer effect for each of the previous eight reinforcers plotted separately for the two alternatives(brj and bgj, see Equation 4) as a function of reinforcer lag (0 being the most recently obtained reinforcer) for eachsubject in Conditions 4, 1, and 3. Also shown are values of log k for each subject.

reinforcer magnitude decreased (as the ratiowas changed) there was a corresponding de-crease in the values of log reinforcer effect.Reinforcers at Lag 0 continued to have apositive, but progressively smaller, effect oncurrent preference. Reinforcers beyond Lag0 in Condition 3 (1:7, M 5 1) had negativeeffects on current preference, and in Con-dition 1 (2:6, M 5 2) this was so in 33 of 35estimates. Moreover, with the exception of

Condition 4 (4:4), log reinforcer effect be-yond Lag 0 was more positive for a largerreinforcer than it was negative for a smallerreinforcer. This effect was significant acrossall conditions on binomial tests (p , .05).Values of log k were less extreme and con-tained less between subjects variability thanthose shown in Figure 2. They still becamemore extreme as the reinforcer magnituderatio became more extreme, however, and


Fig. 4. Log response ratios in interreinforcer intervals following successive same-alternative reinforcers (solidlines) in each condition. The broken lines join ‘‘discontinuations,’’ where a reinforcer was obtained from the otheralternative following sequences of successive same-alternative reinforcers. A sliding window nine reinforcers in lengthwas used throughout.

this was again supported by a one-way AN-OVA, F(4, 20) 5 100.63, p , .05.

A more local analysis was used to examinethe effects of reinforcers on behavior at a re-inforcer-by-reinforcer level. The data were de-composed into log response ratios emitted ininterreinforcer intervals following every se-quence of reinforcers obtained in a condi-tion, using a sliding window nine reinforcersin length. Thus, before the first reinforcer ina sequence, one log response ratio could becalculated. After the first reinforcer, and be-fore the second, two log response ratios wereavailable (one following a red reinforcer, andone following a green reinforcer). After tworeinforcers in a sequence, four log response

ratios were available, one for each possibletwo-reinforcer sequence, and so on.

Figure 4 shows the log response ratios fol-lowing sequences of red or green reinforcersobtained in succession, and the effects of asingle discontinuation at each sequential po-sition in each condition. As has been shownelsewhere (Davison & Baum, 2000, 2002; Lan-don & Davison, 2001; Landon et al., 2002),substantial local effects of individual reinforc-ers were again evident in the present data. Ingeneral, successive reinforcers obtained fromthe same alternative moved preference to-wards the alternative from which they wereobtained, irrespective of the reinforcer-mag-nitude ratio arranged in that condition. Dis-


continuations, in contrast, had comparativelylarge effects on preference.

While some of the tree structures shown inFigure 4 appear slightly asymmetrical, anysuch asymmetry is unrelated to the directionof the reinforcer magnitude ratio (e.g., Con-dition 2 vs. Condition 3). Across conditions,no systematic differences in changes in thelog response ratios following identical se-quences of red and green reinforcers wereevident. Thus, the shape of the tree struc-tures was similar across conditions. The effectof the differences in reinforcer magnitude ra-tios arranged across conditions was seen inthe trees as a whole shift towards the alter-native that arranged the larger reinforcers.These shifts were ordered in the same way asthe reinforcer magnitude ratios.

The data were then aggregated across sub-jects into successive 5-s bins in interreinforcerintervals following the four possible two-re-inforcer sequences. Separate log response ra-tios were calculated for the successive 5-s binsfollowing each two-reinforcer sequence. Fig-ure 5 shows these log red:green response ra-tios plotted as a function of time since thesecond reinforcer. Note that preference inthe first 5-s bin following a reinforcer deliverywas often exclusive, and thus no data pointsappear. In Condition 4 (4:4), the effects ofred- and green-alternative reinforcers mir-rored one another; in the first two 5-s binsafter a reinforcer delivery, a preference wasevident for the just-reinforced response. Asmall preference reversal occurred in Bins 4and 5, and for subsequent bins the log re-sponse ratios were relatively stable at a levelclose to zero.

As the reinforcer magnitudes were madeunequal across conditions, several regularchanges were evident in Figure 5. First, thestable levels of behavior in the interreinforcerintervals changed so that they favored the al-ternative providing the larger reinforcers.These changes were ordered in the same wayas the reinforcer magnitude ratios. Second,the durations of the preference pulses follow-ing reinforcer deliveries increased when re-sponses to that alternative were reinforcedwith larger reinforcers. In Condition 5 (7:1),the transitory preference following a large(red) lasted for about five 5-s bins, and inCondition 2 (6:2) it was reduced to aboutfour 5-s bins. In the same conditions, the

preference pulses following a small (green)reinforcer were much shorter (one or two 5-s bins). Still, preference did not stabilize untilabout the fifth to sixth 5-s bin following a re-inforcer delivery because preference shiftedtowards the red (large reinforcer) alternativebeyond the stable levels evident later in in-terreinforcer intervals. Conditions 1 (2:6)and 3 (1:7) provided results that were reason-ably symmetrical with Conditions 2 (6:2) and5 (7:1), respectively.

DISCUSSION

The present results (Figure 1) were de-scribed well by the generalized matching law(Equation 2; Baum, 1974; Killeen, 1972). Logresponse ratios were a linear function of logreinforcer-magnitude ratios. At more locallevels of analysis, regularities were evident inthe effects of individual reinforcers onchoice.

The present values of sensitivity to rein-forcer magnitude were significantly lowerthan the values of sensitivity to reinforcer fre-quency previously obtained from the samesubjects (Landon, 2002; see also Table 2).Thus, although sensitivity to magnitude washigher than reported previously (Keller &Gollub, 1977, Experiment 1; Schneider, 1973;Todorov, 1973; Todorov et al., 1984), the pre-sent findings agree with the general findingof previous research that varying reinforcermagnitude exerts less control over choicethan varying reinforcer frequency.

The present results were inconsistent withthe previous study by Davison and Hogsden(1984) that had arranged a systematic andparametric variation of reinforcer magnituderatios. Unlike Davison and Hogsden, wefound a linear relation between log responseand log magnitude ratios, as predicted byEquation 2. The reason for this difference ismost likely a procedural one. Davison andHogsden held the reinforcer duration con-stant at one alternative and varied the dura-tion available at the other alternative. Thus,the overall reinforcer duration availableacross the two alternatives changed acrossconditions. In contrast, the overall reinforcermagnitude in the present experiment washeld constant at a total of eight hopper pre-sentations. As Davison and Hogsden pointedout, their result limits the applicability of the


Fig. 5. The log response ratio in successive 5-s time bins in each condition following the four possible two-reinforcer sequences. Also plotted are reference lines indicating zero on each y-axis.

generalized matching law. The result does,however, parallel the effect of overall rein-forcer rate in concurrent schedules (Alsop &Elliffe, 1988; Elliffe & Alsop, 1996). In Davi-son and Hogsden’s study, log response ratiosbecame more extreme as the overall reinforc-er duration was increased. If, as would be ex-pected, the linear relation evident in the pre-sent data holds, this implies that sensitivity to

reinforcer magnitude would increase as theoverall reinforcer magnitude was increased.

Figures 2 and 3 show that both recently ob-tained reinforcers and reinforcers obtainedin the more distant past affected current per-formance, with the most recently obtained re-inforcer having the largest effect (cf., Landonet al., 2002). Values of log k also changed asthe reinforcer-magnitude ratio was varied,


indicating a longer-term effect of the rein-forcer-magnitude ratio. Unlike the results ofLandon et al., who varied the reinforcer-fre-quency ratio, Figure 2 showed no effect of thereinforcer-magnitude ratio on log reinforcereffect. When log reinforcer effects for rein-forcers at the two alternatives were consid-ered separately (Figure 3), however, an effectwas found. Log reinforcer effect, for reinforc-ers obtained at the alternative providing thelarger reinforcers, increased as the reinforc-er-magnitude ratio became more extreme(and the magnitude of the reinforcers at thatalternative increased). Similarly, log reinforc-er effect decreased for reinforcers obtainedat the other alternative and, with the excep-tion of the most recently obtained reinforcer,became negative.

These negative values of log reinforcer ef-fect were almost always smaller than the cor-responding positive values of log reinforcereffect at the other alternative obtained in thatcondition when reinforcer magnitudes wereunequal; smaller reinforcers beyond Lag 0had negative effects on current preference,but the positive effects of larger reinforcerswere greater (Figure 3; significant in all con-ditions on binomial tests, p , .05). Becausethese effects were averaged in Figure 2, smallpositive values of log reinforcer effect appearthere beyond Lag 0. As mentioned above, logreinforcer effect (Figure 3) increased as re-inforcer magnitude was increased at one al-ternative, and decreased as reinforcer mag-nitude was decreased at the other. Given theoverall constancy shown in Figure 2, it seemslikely that these changes were contextual innature. That is, they were driven by thechange in the relative magnitude of rein-forcement rather than the change in the re-inforcer magnitude at an alternative itself.This could be investigated more thoroughlyby arranging a constant reinforcer magnitudeat one alternative and varying reinforcermagnitude at the other alternative across aseries of conditions as was done by Davisonand Hogsden (1984), but with detailed datacollection.

The analyses of preference during interre-inforcer intervals (Figure 5) also provided ev-idence of both short- and long-term effects ofreinforcers. Across conditions, preference sta-bilized during interreinforcer intervals at lev-els that were ordered in the same way as the

reinforcer magnitude ratios. This longer-termcontrol was evident at periods well in excessof the typical interreinforcer interval. Short-er-term effects were seen in the preferencepulses following a reinforcer delivery whichwere different at the two alternatives whenunequal reinforcer magnitudes were ar-ranged. At the alternative providing the larg-er reinforcers, these movements were large,lasting approximately 25 s before preferencestabilized at a level that also favored that al-ternative. At the alternative providing thesmaller reinforcers, the preference pulseswere similar in duration, but consisted of aninitial shift in preference towards the just-re-inforced response that lasted about 5 s (inCondition 3 no absolute preference for thatresponse occurred). This was followed by aperiod in which preference moved towardsthe alternative providing the larger reinforc-ers, beyond the stable levels evident later inthe interreinforcer intervals, before return-ing to those stable levels.

The substantial short-term effects of largereinforcers and lesser effects of small rein-forcers are initially difficult to reconcile withthe reinforcer-by-reinforcer analyses (Figure4), in which any asymmetry in the effects ofreinforcers at the two alternatives appears tobe unrelated to reinforcer magnitude. Se-quences of continuations and continuationsfollowed by a discontinuation had similar ef-fects on preference in each condition relativeto the levels at which preference began. Thisperhaps suggests that the effects of varyingreinforcer magnitude were more molar in na-ture, with the tree as a whole moving towardsthe alternative providing the larger reinforc-ers, but the local effects of sequences of re-inforcers remaining unchanged.

These apparent discrepancies can be rec-onciled by closer inspection of the tree dia-grams (Figure 4). Consider Condition 5 (7:1), in which the smaller reinforcers werearranged at the green alternative. Preferencefollowing a sequence of successive green re-inforcers in this condition was similar to thestable levels that preference reached in be-tween reinforcers (Figure 5). For example,following sequences of three to eight succes-sive green reinforcers, the average log re-sponse ratio was 0.44 (range 0.32 to 0.47; anoverall preference for red) in Figure 4. Theaverage log response ratio in Bin 7 and be-


yond (Figure 5) irrespective of the precedingtwo-reinforcer sequence was 0.42.

Thus, the two analyses show that the largerreinforcers moved preference away from thestable levels approached within interreinforc-er intervals. In contrast, the net effect of asmaller reinforcer was to leave preference rel-atively unchanged at these stable levels (Fig-ure 5). A similar result was reported by Lan-don et al. (2002) with respect to low-ratealternative reinforcers. This description is, ofcourse, an oversimplification as it misses themore local changes occurring. Nonetheless,the preference pulses shown here are similarto those reported by Landon et al. when re-inforcer rates were varied. At the reinforcer-by-reinforcer level, the structure of the treediagrams (Figure 4) changed little across con-ditions, in contrast to those shown by Landonet al. This is because in the present study, thesmall and large preference pulses occurredwith equal frequency in each condition,whereas when reinforcer rates were varied,larger preference pulses also occurred morefrequently.

To summarize, the present experimentconfirmed previous findings that changes inreinforcer magnitude exert less control overchoice than changes in the relative frequencyof reinforcement. The difference betweenthe amounts of control these variables exert,however, may be less than suggested by pre-vious researchers (Schneider, 1973; Todorov,1973; Todorov et al., 1984). In addition, logresponse ratios were a linear function of thelog reinforcer-magnitude ratios when thesum of the reinforcer magnitudes was heldconstant, unlike one previous study in whichthe sum of the magnitudes was also varied(Davison & Hogsden, 1984). The local effectsof reinforcers were similar to those shown inrapidly changing procedures (Davison &Baum, 2000, 2002; Landon & Davison, 2001).Preference within interreinforcer intervals(Figure 5) showed evidence of both short-and long-term effects of reinforcers (Landonet al., 2002), and these were also seen in re-inforcer-by-reinforcer analyses (Figure 4). Noevidence suggested, however, that control be-came more local as reinforcer magnitude wasvaried. Rather, control by changes in rein-forcer magnitude seemed to be manifested ina similar way to control by changes in rein-forcer frequency. The lower sensitivities when

reinforcer magnitudes are varied might bethe result of the constant frequency of differ-ent sized preference pulses. In contrast, whenreinforcer rates are varied (Landon et al.),the different sized preference pulses (largerat the higher-rate alternative) also occur withdiffering frequencies.

REFERENCES

Alsop, B., & Elliffe, D. (1988). Concurrent-schedule per-formance: Effects of relative and overall reinforcerrate. Journal of the Experimental Analysis of Behavior, 49,21–36.

Baum, W. M. (1974). On two types of deviation from thematching law: Bias and undermatching. Journal of theExperimental Analysis of Behavior, 22, 231–242.

Baum, W. M., & Rachlin, H. C. (1969). Choice as timeallocation. Journal of the Experimental Analysis of Behav-ior, 12, 861–874.

Belke, T. W., & Heyman, G. M. (1994). Increasing andsignaling background reinforcement: Effect on theforeground response-reinforcer relation. Journal of theExperimental Analysis of Behavior, 61, 65–81.

Catania, A. C. (1963). Concurrent performances: A base-line for the study of reinforcement magnitude. Journalof the Experimental Analysis of Behavior, 6, 299–300.

Davison, M. (1988). Concurrent schedules: Interaction ofreinforcer frequency and reinforcer duration. Journalof the Experimental Analysis of Behavior, 49, 339–349.

Davison, M., & Baum, W. M. (2000). Choice in a variableenvironment: Every reinforcer counts. Journal of theExperimental Analysis of Behavior, 74, 1–24.

Davison, M., & Baum, W. M. (2002). Choice in a variableenvironment: Effects of blackout duration and extinc-tion between components. Journal of the ExperimentalAnalysis of Behavior, 77, 65–89.

Davison, M., & Hogsden, I. (1984). Concurrent variable-interval schedule performance: Fixed versus mixedreinforcer durations. Journal of the Experimental Analysisof Behavior, 41, 169–182.

Davison, M., & McCarthy, D. (1988). The matching law: Aresearch review. Hillsdale, NJ: Erlbaum.

Elliffe, D., & Alsop, B. (1996). Concurrent choice: Effectsof overall reinforcer rate and the temporal distribu-tion of reinforcers. Journal of the Experimental Analysisof Behavior, 65, 445–463.

Epstein, R. (1981). Amount consumed as a function ofmagazine-cycle duration. Behaviour Analysis Letters, 1,63–66.

Findley, J. D. (1958). Preference and switching underconcurrent scheduling. Journal of the Experimental Anal-ysis of Behavior, 1, 123–144.

Herrnstein, R. J. (1961). Relative and absolute strengthof response as a function of frequency of reinforce-ment. Journal of the Experimental Analysis of Behavior, 4,267–272.

Keller, J. V., & Gollub, L. R. (1977). Duration and rate ofreinforcement as determinants of concurrent re-sponding. Journal of the Experimental Analysis of Behav-ior, 28, 145–153.

Killeen, P. R. (1972). The matching law. Journal of the Ex-perimental Analysis of Behavior, 17, 489–495.


Landon, J. (2002). Choice behaviour: Short- and long-termeffects of reinforcers. Unpublished doctoral dissertation,The University of Auckland, Auckland, New Zealand.

Landon, J., & Davison, M. (2001). Reinforcer-ratio varia-tion and its effects on rate of adaptation. Journal of theExperimental Analysis of Behavior, 75, 207–234.

Landon, J., Davison, M., & Elliffe, D. (2002). Concurrentschedules: Short- and long-term effects of reinforcers.Journal of the Experimental Analysis of Behavior, 77, 257–271.

Logue, A. W., & Chavarro, A. (1987). Effect on choice ofabsolute and relative values of reinforcer delay,amount, and frequency. Journal of Experimental Psychol-ogy: Animal Behavior Processes, 13, 280–291.

McLean, A. P., & Blampied, N. M. (2001). Sensitivity torelative reinforcer rate in concurrent schedules: In-dependence from relative and absolute reinforcer du-ration. Journal of the Experimental Analysis of Behavior,75, 25–42.

Schneider, J. W. (1973). Reinforcer effectiveness as afunction of reinforcer rate and magnitude: A com-parison of concurrent performance. Journal of the Ex-perimental Analysis of Behavior, 20, 461–471.

Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent re-sponding with fixed relative rate of reinforcement.

Journal of the Experimental Analysis of Behavior, 12, 887–895.

Taylor, R., & Davison, M. (1983). Sensitivity to reinforce-ment in concurrent arithmetic and exponentialschedules. Journal of the Experimental Analysis of Behav-ior, 39, 191–198.

Todorov, J. C. (1973). Interaction of frequency and mag-nitude of reinforcement on concurrent performanc-es. Journal of the Experimental Analysis of Behavior, 19,451–458.

Todorov, J. C., Hanna, E. S., & Bittencourt de Sa, M. C.N. (1984). Frequency versus magnitude of reinforce-ment: New data with a different procedure. Journal ofthe Experimental Analysis of Behavior, 41, 157–167.

Todorov, J. C., Oliveira Castro, J. M., Hanna, E. S., Bit-tencourt de Sa, M. C. N., & Barreto, M. Q. (1983).Choice, experience, and the generalized matchinglaw. Journal of the Experimental Analysis of Behavior, 40,99–111.

Wearden, J. H., & Burgess, I. S. (1982). Matching sinceBaum (1979). Journal of the Experimental Analysis of Be-havior, 38, 339–348.

Received June 26, 2002Final acceptance January 5, 2003

Date post:	15-May-2023
Category:	Documents
Upload:	independent
View:	1 times
Download:	0 times

Concurrent schedules: reinforcer magnitude effects

Documents