This article was downloaded by: [107.10.133.26] On: 24 March 2015, At: 15:30 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Click for updates Journal of Clinical Child & Adolescent Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hcap20 Evidence-Based Psychosocial Treatments for Children and Adolescents with Attention-Deficit/Hyperactivity Disorder Steven W. Evans a , Julie Sarno Owens a & Nora Bunford a a Department of Psychology , Ohio University Published online: 18 Nov 2013. To cite this article: Steven W. Evans , Julie Sarno Owens & Nora Bunford (2014) Evidence-Based Psychosocial Treatments for Children and Adolescents with Attention-Deficit/Hyperactivity Disorder, Journal of Clinical Child & Adolescent Psychology, 43:4, 527-551, DOI: 10.1080/15374416.2013.850700 To link to this article: http://dx.doi.org/10.1080/15374416.2013.850700 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
Transcript
This article was downloaded by: [107.10.133.26]On: 24 March 2015, At: 15:30Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK
Click for updates
Journal of Clinical Child & Adolescent PsychologyPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/hcap20
Evidence-Based Psychosocial Treatments for Childrenand Adolescents with Attention-Deficit/HyperactivityDisorderSteven W. Evans a , Julie Sarno Owens a & Nora Bunford aa Department of Psychology , Ohio UniversityPublished online: 18 Nov 2013.
To cite this article: Steven W. Evans , Julie Sarno Owens & Nora Bunford (2014) Evidence-Based Psychosocial Treatments forChildren and Adolescents with Attention-Deficit/Hyperactivity Disorder, Journal of Clinical Child & Adolescent Psychology,43:4, 527-551, DOI: 10.1080/15374416.2013.850700
To link to this article: http://dx.doi.org/10.1080/15374416.2013.850700
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Evidence-Based Psychosocial Treatments for Children andAdolescents with Attention-Deficit=Hyperactivity Disorder
Steven W. Evans, Julie Sarno Owens, and Nora Bunford
Department of Psychology, Ohio University
The purpose of this research was to update the Pelham and Fabiano (2008) review ofevidence-based practices for children and adolescents with attention-deficit=hyperactivitydisorder. We completed a systematic review of the literature published between 2007 and2013 to establish levels of evidence for psychosocial treatments for these youth. Our reviewincluded the identification of relevant articles using criteria established by the Societyof Clinical Child and Adolescent Psychology (see Southam-Gerow & Prinstein,in press) using keyword searches and a review of tables of contents. We extend theconceptualization of treatment research by differentiating training interventions frombehavior management and by reviewing the growing literature on training interventions.Consistent with the results of the previous review we conclude that behavioral parenttraining, behavioral classroom management, and behavioral peer interventions arewell-established treatments. In addition, organization training met the criteria for awell-established treatment. Combined training programsmet criteria for Level 2 (ProbablyEfficacious), neurofeedback training met criteria for Level 3 (Possibly Efficacious), andcognitive training met criteria for Level 4 (Experimental Treatments). The distinctionbetween behavior management and training interventions provides a method for consider-ing meaningful differences in the methods and possible mechanisms of action for treat-ments for these youth. Characteristics of treatments, participants, and measures, as wellas the variability in methods for classifying levels of evidence for treatments, are reviewedin relation to their potential effect on outcomes and conclusions about treatments.Implications of these findings for future science and practice are discussed.
Numerous studies document that children and adolescentswith attention-deficit=hyperactivity disorder (ADHD)experience poor outcomes across several domains of func-tioning, including education, vocation, interpersonal rela-tions, and health risk. These problems lead to substantialimpairment (Wehmeier, Schacht, & Barkley, 2010), parent
distress (Wymbs, Pelham, Molina, & Gnagy, 2008), andextensive costs to society (Pelham, Foster, & Robb,2007; Robb et al., 2011). Research on the developmentand evaluation of psychosocial treatments1 for childrenand adolescents (hereafter children) with ADHD has beenfocused on improving these outcomes for almost 40 years(see Antshel & Barkley, 2011, for a historical review).Reports of progress with this work have been highlightedin two special issues of the Journal of Clinical Child andAdolescent Psychology (JCCAP). In 1998, Pelham,Wheeler, and Chronis published the first in this series ofliterature reviews of psychosocial treatments for ADHD,and Pelham and Fabiano updated that review in 2008.The current article provides an updated review and
During the preparation of this article, Steven Evans was partially sup-
ported by a grant from the National Institute of Mental Health
(MH074713) and both Steven Evans and Julie SarnoOwenswere partially
supported by grants from theDepartment of Education, Institute forEdu-
cation Sciences (IES; R324C080006, R305A110059, R324A120272). We
appreciate the assistance of the students and staff in the Center for Inter-
vention Research in Schools at Ohio University and Greg Fabiano and
Saskia van der Oord who read an earlier version of this manuscript.Correspondence should be addressed to Steven W. Evans, Center
for Intervention Research in Schools, Department of Psychology, Ohio
1The terms treatment and intervention are used synonymously
throughout the article.
Journal of Clinical Child & Adolescent Psychology, 43(4), 527–551, 2014
Copyright # Taylor & Francis Group, LLC
ISSN: 1537-4416 print=1537-4424 online
DOI: 10.1080/15374416.2013.850700
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
follows the current version of the JCCAP Evidence BaseTreatments Updates (EBT) evaluation criteria (seeTable 1; hereafter EBT Evaluation Criteria).
Pelham and Fabiano (2008) evaluated 46 treatment
studies and sorted the interventions into one of three
categories: behavioral parent training (BPT), behavioral
classroom management (BCM), and behavioral peer inter-
ventions (BPI). Consistent with the 1998 review, BPT and
BCM met criteria for well-established treatments for
ADHD. Pelham and Fabiano (2008) reported two conclu-
sions regarding BPI, with one pertaining to traditional,
weekly, social skills training groups provided in a clinic
(BPI-C) and the other pertaining to interventions targeting
peer relationships and functioning in recreational settings
(BPI-R) mostly provided in the context of summer treat-
ment programs (STP; Pelham, Fabiano, Gnagy, Greiner,
& Hoza, 2005). BPI-C did not have adequate evidence
to be considered well-established or probably efficacious.
In contrast, BPI-R met criteria for a well-established
treatment. Other reviews published since 2008 havereported similar findings about BPT, BCM, and BPI-R(e.g., Fabiano et al., 2009; Owens, Storer, &Girio-Herrera,2011; Sadler & Evans, 2011), but some have reachedvery different conclusions (Sonuga-Barke et al., 2013).The purpose of the current review is to critically evaluatethe empirical literature of treatment studies publishedduring the last 5 years and incorporate the findings withthose in the Pelham and Fabiano (2008) review to
1. Determine current levels of evidence for psycho-social interventions for children with ADHD, and
2. Report and review characteristics of interventions,participants, and measures that may influence theoutcomes of psychosocial treatment research.
APPROACH TO UPDATED REVIEW
Although it has been only 5 years since the latest review,the literature has continued to expand at a rapid pace.
M.1. Group design: Study involved a randomized controlled design
M.2. Independent variable defined: Treatment manuals or logical equivalent were used for the treatment
M.3. Population clarified: Conducted with a population, treated for specified problems, for whom inclusion criteria have been clearly delineated
M.4. Outcomes assessed: Reliable and valid outcome assessment measures gauging the problems targeted (at a minimum) were used
M.5. Analysis adequacy: Appropriate data analyses were used & sample size was sufficient to detect expected effects
Level 1: Well-Established Treatments
1.1 Efficacy demonstrated for the treatment in at least two (2) independent research settings and by two (2) independent investigatory teams
demonstrating efficacy by showing the treatment to be either:
1.1.a. Statistically significantly superior to pill or psychological placebo or to another active treatment
OR
1.1.b. Equivalent (or not significantly different) to an already well-established treatment in experiments
AND
1.2. All five (5) of the Methods Criteria
Level 2: Probably Efficacious Treatments
2.1. There must be at least two good experiments showing the treatment is superior (statistically significantly so) to a wait-list control group
OR
2.2. One or more good experiments meeting the Well-Established Treatment level with the one exception of having been conducted in at least two
independent research settings and by independent investigatory teams
AND
2.3. All five (5) of the Methods Criteria
Level 3: Possibly Efficacious Treatments
3.1. At least one good randomized controlled trial showing the treatment to be superior to a wait list or no-treatment control group
AND
3.2. All five (5) of the Methods Criteria
OR
3.3. Two or more clinical studies showing the treatment to be efficacious, with two ore more meeting the last four (of five) Methods Criteria, but none
being randomized controlled trials
Level 4: Experimental Treatments
4.1. Not yet tested in a randomized controlled trial
OR
4.2. Tested in 1 or more clinical studies but not sufficient to meet level 3 criteria.
Level 5: Treatments of Questionable Efficacy
5.1. Tested in good group-design experiments and found to be inferior to other treatment group and=or wait-list control group; i.e., only evidence
available from experimental studies suggests the treatment produces no beneficial effect.
In 2008, Pelham and Fabiano reported that three typesof treatment (BPT, BCM, and BPI-R) met criteria forwell-established treatment. We maintain these threeclassifications with a couple of modifications. First, weclassify these treatments into the larger category ofbehavior management (BM) because all treatments inthis category involve training parents, teachers, or pro-gram staff to modify the behavioral contingencies inthe environments within which the children functionand outcomes are measured. Second, we eliminated thedistinction within the BPI category that distinguishedbetween settings including clinic-based BPI (BPI-C)and recreational settings-based BPI (BPI-R). We pro-pose that the setting is not the most critical distinctionbetween these two types of treatment. Instead, BPI-Rinvolves staff members manipulating contingencies toimprove the social behavior of the youth in the sameenvironment in which outcomes are measured. In con-trast, BPI-C involves training participants to exhibitnew prosocial behaviors and to discontinue exhibitingnegative behaviors in environments other than the onewhere treatment is provided. Although some studies ofBPI-C include encouraging parents or teachers to rewardthe participants when they exhibit desired changes inbehavior, the main focus of the intervention is training.Thus, to capture this distinction, we propose a secondlarge category: Training Interventions (TI). The TI labelapplies to social skills training programs that wereformerly categorized as BPI-C, as well as several newtreatments that have emerged in the last decade. Forexample, neurofeedback and cognitive training do notinvolve manipulating contingencies in the environmentswhere the behavior change is intended to occur. Thus,the TI category rather than the BM category better fitsthese treatments. Finally, some of the organization TIand school-based treatment programs (e.g., ChallengingHorizons Program; Evans, Schultz, DeMars, & Davis,2011) also fit into the TI category, as the skills are taughtand their use is rewarded in environments other thanwhere change is intended and outcomes are measured.
The distinction between BM and TI is important forthe way in which we conceptualize and study these twotypes of treatment. For example, there is considerableresearch in the area of treatment integrity (Perepletchi-kova & Kazdin, 2005). For BM interventions, treatmentintegrity applies to those who train the parents and tea-chers, as well as to the parents and teachers who providethe behavioral interventions strategies. In TI, treatmentintegrity applies only to those training the children, asthere are no secondary implementers of strategies. BMtreatments are intended to lead to behavior change bymanipulating contingencies in the target environment.Once targeted behaviors are changed, then generaliza-tion and maintenance of behavior change may occurand is achieved by fading the modified contingencies
and connecting the child to naturally occurring contin-gencies (Stokes & Baer, 1977). TI lead to behaviorchange by improving the skill set of the child and eitherhoping for generalization (Stokes & Baer, 1977) (e.g.,cognitive TI) or providing reinforcement and punish-ment in the training setting for behavior change thatoccurs outside of that setting. Given that treatments inthe BM and TI categories have unique presumedmechanism of action, as well as unique implications forrelationships between participant characteristics, integ-rity, and outcomes, we organize treatments in this reviewin accordance with these two overarching categories.Within the BM classification, we retain the categoriesused in the Pelham and Fabiano (2008) review of BPT,BCM, and BPI. Within the TI classification, we includeneurofeedback training, cognitive training (includingtraining of working memory, attention, and executivefunctioning), and organization skills training. We wouldhave also included traditional social skills training(formerly labeled BPI-C) in TI; however there were nostudies since 2008 of this intervention that met thecriteria for inclusion in this review.
CHARACTERISTICS AFFECTING OUTCOMES
The previous review concluded that all of the BM treat-ments were well-established. Research questions in studiestesting these treatments were thus likely to change fromdoes the treatment work to how does it work, for whom doesit work, or how can outcomes be enhanced. We examinedthe extent to which these new questions have beenaddressed in the last 5 years of research. In addition, wealso examined several characteristics of participants andmeasures that may influence the results and conclusionsof a study. For example, given that participant character-istics that influence treatment outcomes have been ident-ified (see Hoza, Johnston, Pillow, & Ascough, 2006;Ollendick, Jarrett, Grills-Taquechel, Hovey, & Wolff,2008), we reviewed some research methods that lead tovariations in sample characteristics and discuss how suchcharacteristics may influence treatment effects. Inaddition, characteristics of measurement may also impactoutcomes, making it difficult to compare results acrossstudies (De Los Reyes & Kazdin, 2009). One measure-ment issue related to eligibility criteria involves the choiceof informants and decision rules used to determine a diag-nosis of ADHD. Both have been shown to influence thediagnostic decisions (Rowland et al., 2008; Valo &Tannock, 2010) and we examined the variability acrossstudies in this area. Another measurement issue involvesthe choice of outcome measures. As with diagnoses, thesource of outcome data varies considerably across studiesand could influence outcomes depending on a variety offactors. Outcomes may depend on the construct chosen
PSYCHOSOCIAL TREATMENTS FOR ADHD 529
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
as an outcome variable of interest (e.g., symptoms orfunctional impairment), on the way in which such con-struct of interest is defined and measured (e.g., objectivevs. subjective measurement or informant type), andwhether informants are aware of the treatment condition(Jadad et al., 1996). For example, as noted in the EBTEvaluation Criteria (see Table 1), outcome measuresshould map onto the problems targeted in treatment.Thus, one goal of our review was to highlight characteris-tics of participants and measures that may impact treat-ment outcomes with the aim of generating hypothesesfor the next generation of research in this area.
CRITERIA FOR EVALUATING TREATMENTS
The criteria used to select rigorous studies for review andto determine whether treatments are evidence based orwell established are generally consistent across reviewsin special issues of JCCAP. The only differences betweenthe presently-employed criteria and those used in the2008 special issue are minor wording changes that shouldnot change the classification of the research (Southam-Gerow & Prinstein, in press). However, these criteriahave not been used consistently in other reviews,contributing to inconsistent conclusions across studies.For example, a recent review and meta-analysis bySonuga-Barke and colleagues (2013) concluded thatthe mean effect size for ADHD symptoms across well-controlled studies of behavioral interventions for chil-dren with ADHD was zero. To calculate the mean effectsize, Sonuga-Barke et al. excluded studies wherein raterswere aware of treatment condition and combined resultsfrom very different types of psychosocial interventions.Further, although many behavioral interventions focuson changing functional impairment, Sonuga-Barke andcolleagues relied solely upon ratings of ADHD symp-toms as the outcome variable. The authors acknowledgedthat this focus on symptoms may be inconsistent with thegoals of many psychosocial interventions; however, theynoted that this requirement was necessary to obtain acommon metric to facilitate conducting a meta-analysis.Nevertheless, as a result of this criteria, most of thebehavioral treatment literature was excluded from con-sideration and, consequently, the conclusions reachedby Sonuga-Barke and colleagues are different from thosereached by most other reviews or meta-analytic studies.As is apparent in this example, the conclusions of anyreview, including this one, should be considered in thecontext of the criteria used to evaluate the literature.
Consistent with the two aims of this study, we classifiedthe treatment research reported during the last 5 yearsaccording to the EBT Evaluation Criteria for classifyingpsychosocial treatments (Southam-Gerow & Prinstein,in press) and organized the studies into two major
categories based on the treatments evaluated (BM &TI). We began each section of the results by reportingthe conclusions of the most recent review (Pelham &Fabiano, 2008) and then followed with an updated sum-mary of the studies published since 2007 that meet theEBT Evaluation Criteria. In addition, we examined thevariability across studies pertaining to characteristics oftreatments, participants, and measurement. Finally, wehighlighted issues pertaining to the classification of treat-ments according to the level of scientific evidence. Ourreview concludes with recommendations pertaining tofuture research and practice guidelines.
METHOD
To determine which articles to include in our review, weconducted a three-wave procedure. The first (keywordsearch) and second (table of contents search) wavesinvolved the identification of articles that met our prede-termined set of inclusion criteria. The third waveinvolved coding of the included articles to identify thosethat met the EBT Evaluation Criteria.
Procedure
Wave 1: Keyword search. To conduct our keywordsearch, we followed methods proposed by Cooper andHedges (1994) for completing keyword searches inPsycINFO and Medline. Namely, we compiled and usedthe following Boolean string: (‘‘attention deficit hyperac-tivity disorder’’ OR ADHD OR ADD OR hyperkinesisOR ‘‘attention deficit disorder’’ OR ‘‘attention deficitwith hyperactivity’’) AND (treatment OR interventionOR training) NOT (adult) NOT (pharmacological ORmedical). Using these terms, we identified 1,544 articlesvia the PsycINFO search and 2,479 via the Medlinesearch published since 2007. We conducted a separatesearch for articles reporting results of the MTA Study,with the following Boolean string: (MTA OR ‘‘Multi-modal Treatment of Attention Deficit HyperactivityDisorder’’) and obtained an additional 646 articlesvia our Medline search (and 0 via PsycINFO), yieldinga total number of 4,669 studies.
Wave 2: Table of contents (TOC) search. Wesearched the tables of contents of issues published since2007 of well-known journals that publish studies of psy-chosocial interventions: Behavior Modification, BehaviorTherapy, Child and Family Behavior Therapy, Cognitiveand Behavior Practice, Journal of Abnormal Psychology,Journal of Abnormal Child Psychology, Journal of theAmerican Academy of Child and Adolescent Psychology,Journal of Applied Behavior Analysis, Journal of Consult-ing and Clinical Psychology, Journal of Emotional and
530 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
Behavioral Disorders, Journal of School Psychology,Attention Research Update, School Mental Health,Journal of Attention Disorders, School PsychologyReview, School Psychology Quarterly, and Journal ofClinical Child and Adolescent Psychology. The searchwas conducted either by accessing the journal websitesor by searching two electronic journal index databases(Alice and The OhioLINK Elec Journal Center). Weobtained 163 articles in this manner.
Thus, we obtained 4,669 articles via the keywordsearch process and 163 identified via the TOC search pro-cess, yielding a total number of 4,832 articles. Of these4,832, we then limited our scope to those articles thatwere (a) empirical studies, (b) published in peer-reviewedjournals between 2007 and August 2012 or in press byAugust 2012, (c) available in English, (d) treatment stu-dies with children and adolescents with ADHD (up to17 years), and (d) evaluated at least one psychosocialtreatment only group (i.e., evaluates a psychosocial treat-ment alone or in comparison to another treatment). Wedefined psychosocial treatment as any intervention thatis not medication or diet. Based on our final criterion, stu-dies of multimodal treatments compared to medicationbut not to psychosocial treatment alone were excluded.Using these criteria, 122 studies remained, and we codedthese studies using the EBT Evaluation Criteria.
Wave 3: Study coding per EBT Evaluation Criteria.The 122 articles were categorized based on the fiveEBT Evaluation Criteria (see Table 1), each of whichwas judged either as characteristic or as not character-istic of the methodology employed. Of the 122 articles,101 were excluded because they violated at least one ofthe EBT Evaluation Criteria. Twenty-one met all fivecriteria and are discussed in detail in our upcomingResults section. Although a reduction from 122 studiesto 21 eliminates many studies from consideration inthis review, it is worth noting that, in the 2008 review,only 29 between-group or crossover design studieswere included from a period that covered twice asmany years as this one. Similarly, the recently publishedreview by Sonuga-Barke and colleagues included only15 studies out of all psychosocial treatment researchdating back to at least 1973. Thus, it appears that oursample of studies is not disproportionally small for theperiod covered.
RESULTS
Our review is based on 21 studies that were publishedsince October 2007, met the five EBT EvaluationCriteria, and were not included in the previous reviewby Pelham and Fabiano (2008) (see Tables 2 and 3).Table 2 provides a summary of the reviewed studies.
For each study, we extracted data on the total samplesize, the age range of the sample, and the ethnicity, race,and sex distribution in the sample. We also describedoutcome domains assessed, the methods or informantswho provided information about those outcomes, andthe quality of the study according to the standards ofNathan and Gorman (2002) and the What Works Clear-inghouse Evidence Standards for Reviewing Studies.2
Because diagnostic assessment procedures varied acrossstudies, we provided a summary of the measures thatwere reportedly used to determine ADHD diagnosis ineach study, as well as the process for combiningsymptom-based data across informants (i.e., And=OrRule). More specifically, the process was categorizedas using the ‘‘And Rule’’ if symptom rating of bothinformants (parents and teachers) had to meet thethreshold of six symptoms for inclusion in the ADHDgroup. The process was categorized as using the ‘‘OrRule’’ if the threshold of six symptoms could beachieved using symptoms endorsed by either the parentor the teacher. If only one rater was used to obtaininformation about symptoms and=or impairment, wecategorized the process as ‘‘Parent Only.’’ Last, if thedescription provided by the authors of the article wereinsufficiently detailed, we categorized the process as‘‘Unclear.’’
We also summarized outcome data for each study (seeTable 3). Some studies included a midpoint assessment
2Per the What Works Clearinghouse standards (Institute of
Education Sciences, 2011), a study that met criteria for either Meets
Evidence Standards or Meets Evidence Standards with Reservations
was conducted within a relevant time frame, tested a relevant inter-
vention with a relevant sample, employed relevant and adequate
(i.e., valid and reliable) outcomes measures, provided enough infor-
mation to calculate an effect size for at least one outcome measure,
and was a randomized controlled trial or a quasi-experiment. For a
study to be categorized as Meets Evidence Standards, the study also
had to employ random assignment or functionally random haphazard
assignment, the research team had to demonstrate the absence of high
overall or of high differential attrition, groups had to be equated on a
pretest of the outcome measure, and the intervention had to be free of
intervention contamination. If a study failed to meet one or more of
the criteria for Meets Evidence Standards but employed a quasi-
experimental design, group assignment, equating and baseline equival-
ence; had no severe overall or differential attrition or, if it did have
severe attrition, such attrition is accounted for in the analysis, and
had no intervention contamination; it was categorized as Meets Evi-
dence Standards with Reservations. All studies that met the five task
force method criteria used in this review met one of these two WWC
standards. The Nathan and Gorman categorization ranges from 1 to
6 and all studies that met criteria for being included in this review
met criteria for either Type 1 or 2. Type 1 studies employ the most
rigorous scientific evaluations and are randomized, prospective clinical
trials with comparison groups, blind assessments, state-of-the-art
diagnostic procedures, clear inclusion and exclusion criteria, an
adequate sample size and a clear description of statistical method-
ology. Type 2 studies are clinical trials wherein an intervention is tested
but the study lacks one component of Type 1 criteria.
and some included a follow-up assessment well after thetreatment phase; however, because the focus of thisarticle is on immediate outcomes of a given treatment,we reported only the outcomes that represent pre- toposttreatment change. Table 3 includes the effect sizesfor the psychosocial intervention relative to a controlcondition and for the psychosocial treatment relative toan alternative active treatment for which there is evidenceof a positive effect on outcomes. In cases where theauthors of the article provided effect sizes for pre-postoutcomes, we extracted the effect sizes they providedand have highlighted via superscripts the type of effect sizereported. In cases where the authors did not provide theeffect sizes for pre–post outcomes, we calculated an effectsize using data provided in the study (i.e., means, standarddeviations, and sample sizes, F values, or t values andcorresponding degrees of freedom) and highlight viasuperscripts the type of effect size reported and=or theequation used to calculate the effect size. Given the vari-ability in how effect sizes were calculated, readers shouldnot attempt to make direct comparisons across studies.
In determining the level of evidence for each type oftreatment, some judgments about the quality of theoutcome measures had to be made. Broadly speaking,the following principles were used to consider quality:(a) outcome measures assessing change in functioningwere considered to be of greater importance thanmeasures assessing symptoms, (b) ratings provided byinformants who were not involved in the treatmentwere considered to be of higher quality than ratingsprovided by informants who were involved in treat-ment, (c) objective measures obtained within thecontext of typical functioning (e.g., observations inthe classroom) were considered to be of higher qualitythan objective measures obtained devoid of context(e.g., neuropsychological measures), and (d) studiesthat provided outcomes across multiple domains and=or multiple informants were considered to be morecompelling than those that provided outcomes in onlyone domain or by a single informant. Last, we indi-cated whether the authors of the article reported theclinical significance of outcomes (e.g., reported percent-age of participants falling below a clinical threshold ormeeting a reliable change index). Because very fewstudies (n¼ 3) included an analysis of moderating ormediating variables, the results of such analyses arebriefly reviewed in the Results and Discussion sectionsbut not presented in Table 3.
We begin our review with the BM category and thethree subcategories of BPT, BCM, and BPI. In addition,because some studies used a combination of thesetreatments we have a Combined Category for BM treat-ments. The TI category is reviewed next and includescognitive, neurofeedback, and organization trainingfollowed by a Combined Category for TI.
Behavior Management
Behavioral parent training. Both of the previoustreatment reviews (Pelham & Fabiano, 2008; Pelhamet al., 1998) concluded that BPT was a well-establishedtreatment for youth with ADHD. Six studies that meetthe EBT Evaluation Criteria for this review have beenpublished since the last review. All of the BPT programsfocused on BM procedures that are consistent withthose that achieved well-established status such as theCommunity-Oriented Parenting Education program(Cunningham, Bremner, & Secord-Gilbert, 1993) andthe Defiant Children program, second edition (Barkley,1997). In four of the six studies, BPT was conducted ingroups with weekly sessions lasting between 2 and2.5 hr, over 8 to 12 weeks (Chacko et al., 2009; Fabianoet al., 2009; Fabiano et al., 2012; van den Hoofdakkeret al., 2007). The other two studies evaluated individualBPT sessions, with one study evaluating the efficacy ofa single session of treatment (Meyer & Kelley, 2007) andthe other providing 12 sessions (McGrath et al., 2011).
With regard to outcomes, these six studies documen-ted significant benefits on parent ratings of child symp-toms and=or impairment for BPT when compared toa waitlist or routine care condition (Chacko et al.,2009; Fabiano et al., 2012; McGrath et al., 2011;Meyer & Kelley, 2007; van den Hoofdakker et al.,2007) and when compared to active alternative treatmentconditions (e.g., Meyer & Kelley, 2007). Fabiano andcolleagues (Fabiano et al., 2009; Fabiano et al., 2012)as well as Chacko and colleagues (2009) evaluated anenhanced BPT to address the needs of a specific popu-lation (i.e., fathers, single mothers) and reported thatthe adapted version of BPT was equivalent, and in thecase of some outcomes, better than the standardwell-established version. As a result, these studies extendthe foundation of research that led Pelham and Fabiano(2008) to conclude that BPT was a well-establishedtreatment for youth with ADHD.
It is noteworthy that five of these six studies of BPTevaluated unique adaptations of the structure of BPT(e.g., single session, phone session) to better addressthe needs of a unique group of individuals who do nottypically attend BPT (e.g., single mothers, fathers). Intheir program, Strategies to Enhance Positive Parenting(STEPP), Chacko and colleagues modified traditionalparent training sessions by increasing the length of the ses-sions to 2.5 hr and included opportunities for singlemothers to observe staff modeling behavior managementand incentive procedures. Mothers participating in theStrategies to Enhance Positive Parenting programreported improvements in their children’s oppositionaldefiant disorder (ODD) symptoms and functioning (i.e.,parent–child relations, family functioning) relative to tra-ditional BPT services and to no treatment. Similarly,
540 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
Fabiano and colleagues (Fabiano et al., 2009; Fabianoet al., 2012) modified a traditional BPT program to makeit appealing for fathers. The Coaching Our Acting-outChildren: Heightening Essential Skills (COACHES) pro-gram delivered behavior management skills training inthe context of fathers coaching their children to play soc-cer. The investigators reported little difference betweentraditional BPT and COACHES in father and motherratings of child symptoms except that fathers in the COA-CHES program reported greater perceived improvementin their child’s behavior, relative to fathers in the tra-ditional BPT program (Fabiano et al., 2009). In thesecond study of COACHES (Fabiano et al., 2012), theinvestigators reported improvements over a waitlistgroup in observed rates of fathers’ making positive andnegative statements to their child and in fathers’ ratingsof child behavior. These studies indicate that adaptationsof traditional BPT engages individuals not typicallyserved while maintaining the treatment gains of BPT.
Two of the other studies also included unique appli-cations of BPT including a single-session intervention(approximately 90min with four weekly follow-uptelephone calls) with young adolescents (Meyer & Kelley,2007) and telephone-based BPT (McGrath et al., 2011).The one-session BPT targeted homework compliance andthe authors reported significant improvements in parentratings of homework completion and objective measuresof percent of submitted homework. The telephone-basedBPT included twelve 40-min telephone calls in additionto handbooks and videos that parents read and viewedat home. Although BPT typically targets impairment,McGrath and colleagues examined change in participants’ADHD diagnostic status. Both the one-session BPTtargeting homework compliance and the telephone-based BPT represent treatment models that removebarriers to treatment attendance that are commonly foundin multisession clinic-based parent training programs.
Having established the evidence base for BPT(Pelham & Fabiano, 2008), investigators appear to havemoved toward modifying procedures to improve accessand engage individuals who previously showed lowparticipation rates or less desirable outcomes. The pushtoward innovative delivery models can extend the reachof well-established BPT practices and moves the sciencebeyond a primary focus on efficacy to one of dissemi-nation. Some limitations of these studies include an over-reliance on ratings of outcomes from those receivingservices (i.e., parents), a low number of participants fromminority groups (see Chacko et al. for an exception), andan exclusive focus on elementary school-age children.As additional adaptations and enhancements to BPTare made, it may be important to follow the models ofChacko and Fabiano by comparing enhanced BPT totraditional BPT so that the exact benefits offered byenhanced models can be understood. For example, some
enhancements may produce child outcomes that aresimilar to and not better than traditional BPT, yet theyserve to engage new populations that otherwise wouldnot receive services. In contrast, other enhancementsmay provide benefits both in terms of service engagementand in child and adolescent outcomes. This contrasthelps to highlight important mediators of treatmentoutcomes for future study (mediators and moderatorswere not examined in any of these studies).
Behavioral classroom management. Both of theprevious treatment reviews (Pelham & Fabiano, 2008;Pelham et al., 1998) determined that BCM interventionswere well-established treatments. Since the last review,there were two published studies that meet the EBTEvaluation Criteria for the current review. The first is astudy of BCM by Fabiano and colleagues (2010) whoevaluated BCM in elementary schools in the context ofspecial education services. Namely, the effectiveness ofa Daily Report Card (DRC) intervention in combinationwith ongoing teacher consultation (DRCþ consultation)throughout the entire academic year, relative to specialeducation ‘‘business as usual’’ was examined. Resultsindicated that the DRCþ consultation services con-dition led to statistically significant improvements inclassroom rule violations and teacher ratings of ODD=conduct disorder symptoms, classroom behavior, andacademic productivity, as well as teacher-rated improve-ment on behavior goals compared to the business asusual condition. The results of this study demonstratethat the DRC can be feasibly implemented by school-employed classroom teachers to produce meaningfulgains in the behavior of students with ADHD.
The second study of BCM was conducted by Mikamiand colleagues (2012), who presented an innovativeapproach to BCM by leveraging specific factors (i.e.,student–teacher interactions) within the classroomcontext. The investigators contrasted two methods ofmanaging classroom behavior of elementary school-agedchildren in an analogue classroom setting. Both methodsincluded the most common core components ofclassroom-wide behavior management but differed inthe way in which teachers applied some of the behaviormanagement techniques, such as praise, individualattention, and direct and indirect messages of acceptanceof others. The additive benefit of Making SociallyAccepting Inclusive Classrooms (MOSAIC) over a well-established treatment was evaluated. In MOSAIC, thegoal was to reduce rejection, social devaluation, andexclusion of children with ADHD within the peer group.By the end of the 2-week program, behavior problemsdid not differ between the two groups. However, relativeto the traditional BCM condition, children with ADHDin MOSAIC were significantly less rejected by their peersand had more reciprocated friendships; yet this outcome
PSYCHOSOCIAL TREATMENTS FOR ADHD 541
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
was moderated by child sex; the effect was stronger forboys than for girls. This innovative intervention extendsthe research on BCM to include the manipulation ofsubtle behavior management techniques and outcomesrelated to peer acceptance.
Overall, the two studies of BCM that met EBTEvaluation Criteria increase the support for BCM as awell-established treatment for ADHD and add to theliterature by evaluating BCM in a naturalistic setting(Fabiano et al., 2010) and by challenging BCM research-ers to consider teacher and student behaviors in a newlight (Mikami et al., 2012). Although BCM has met thecriteria for being a well-established treatment since1998, the literature supporting this claim only includeselementary school aged children. Given the developmen-tal changes occurring within children as they progressthrough puberty and transition into young adulthood,as well as the differences between the contexts of middleand high schools (compared to elementary schools), it isunclear whether the findings just described generalize toadolescent populations.
Behavioral peer interventions. In the previousreview interventions targeting social impairment weresorted into two categories. The first included traditionalsocial skills training and that has been reclassified as a TIin this review. The second category included behavioralpeer interventions in recreation settings with most ofthese occurring in Summer Treatment Programs (STP;Pelham & Hoza, 1996). Based on two large, between-group studies conducted in the STP (Pelham et al.,2008, and one of the MTA studies, Pelham et al.,2000), Pelham and Fabiano (2008) indicated that BPIsin recreational settings were a well-established treatmentfor ADHD. The rationale for this type of treatment isthat by training staff in specific settings to manipulatecontingencies in those settings, children will demonstrateimprovements in social functioning. One study of BPIwas published since the 2008 review and the treatmentevaluated in this study (Mikami, Lerner, Griggs,McGrath, & Calhoun, 2010) involved training parentsto be social coaches and to modify contingencies whentheir children were in social situations to facilitate appro-priate social behavior. Although not in a recreational set-ting, the manipulation in Parent Friendship Coaching(PFC) is the same as in the studies of STP; adults aretaught to manipulate contingencies in a target settingto improve the social behavior of children with ADHD.PFC consisted of eight 90-min weekly group sessions andparticipants were families of 124 children (half diagnosedwith ADHD) between the ages of 6 and 10 years.Participants with ADHD were randomly assigned toeither receive PFC or to a no-treatment control condition.In addition to significant improvements in parents’ ratingsof social skills and quality of play, the investigators also
reported significant improvements for those receivingPFC compared to controls on teacher ratings of peerliking and acceptance. The investigators asked parentsto not inform the teachers about their involvement intreatment so the teachers’ ratings were completed withoutawareness of condition. Further, although support wasnot found for many hypothesized mediators, the authorsfound that changes in some parenting behaviors duringpeer interactions, specifically parent facilitation ofsuccessful behaviors, correction of child behavior, andreductions in criticisms, mediated the effect of PFC onchild peer functioning. Little support was found forpossible moderating effects of sex, ADHD subtype,ODD comorbidity or medication status, suggestingthat the intervention effects are applicable across severalsubgroups. Thus, this study extends previous findings ina number of ways. First, participants achieved gains insettings other than the one in which contingencies weredirectly manipulated. Second, raters who were unawareof treatment condition confirmed these improvements.Last, some of the results support the hypothesizedmechanism of change (i.e., change in parenting behaviorsduring playdates). Although it is questionable whether thestudies reported in the 2008 review were conducted by twoindependent research teams (as is required for a designationof well-established), the addition of this study by Mikamiand colleagues (2010)3 yields adequate evidence for BPIto be considered a well-established treatment.
Combined behavioral treatment studies. Pelhamand Fabiano (2008) noted that some studies, such asthe MTA, included a combination of BPT, BCM, and=or BPI preventing them from reaching conclusions aboutthe degree to which each treatment individually contrib-uted to outcomes. For this reason, we added a fourthcategory for BM studies that evaluated treatments thatwere a combination of any of the preceding three cate-gories. We identified six studies that reported the resultsof treatments that combine aspects of BPT, BCM, and=or BPI. Given prior evidence supporting BPT andBCM, it is not surprising that these studies reportednumerous benefits for the combined treatment relativeto a no-treatment condition or to an active psychosocialsupport intervention (Abikoff et al., 2013; Kern et al.,2007; Langberg et al., 2010; Pfiffner et al., 2007; Poweret al., 2012; Webster-Stratton, Reid, & Beauchaine,2011). Possible mediators and moderators were onlyexamined in the study by Langberg and colleagues(2010). Specifically, at the 14-month assessment point
3We understand that this study may have been classified in the BPT
section; however, the purpose of the intervention was to train adults to
modify contingencies in the environments with which children socially
interacted with peers for the purpose of enhancing their social
functioning, therefore, we judged that it fit better in the BPI category
than BPT.
542 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
in the MTA study, the benefits of the combined inter-vention on homework problems (relative to all othertreatment conditions) were strongest for children withmoderate (rather than severe) parent-rated ADHDsymptoms. Variables that did not moderate the outcomesincluded child sex, learning disability status, medicationstatus, and receipt of school services. These outcomeshighlight the impact of combining well-establishedtreatments to improve ADHD symptoms and func-tioning in areas that may not be adequately addressedby any individual treatment alone (e.g., homeworkmanagement, organizational skills).
Training Interventions
Cognitive training. There were two studies of cogni-tive training that met all five EBT Evaluation Criteria(Beck, Hanson, Puffenberger, Benninger, & Benninger,2010; van der Oord, Ponsioen, Geurts, Brink & Prins,in press). In the study conducted by Beck and colleagues,participants (ages 7–17) were randomly assigned toeither a trial involving twenty-five 30- to 40-min sessionsof a computerized cognitive training task (Cogmed RM)or to a waitlist control condition over a 5-week period.The sessions took place in the participants’ homes, andparents were instructed to monitor and reward childrenfor completing sessions on a computer. Investigatorsgathered parent and teacher ratings of ADHD symptomsand behaviors thought to be related to executive func-tioning at pretreatment, posttreatment, and 4-monthfollow-up. The results of the study were mixed; manyfactors on the parent rating scales revealed significantbenefits for the intervention at posttreatment and follow-up relative to the control condition; however, only one of10 (10%) factors on the teacher rating scales indicateda statistically significant advantage for treatment overcontrol. Reconciling these large rater-specific differencesraises questions about the degree to which improvementsin parent ratings may have been partially attributable toparents’ awareness of the treatment and investment intheir child’s practice.
In the second study of cognitive training, conductedby van der Oord and colleagues (in press), training proce-dures that were more varied than those used by Beck et al.were evaluated. Specifically, the cognitive training inter-vention tested by van der Oord et al. included a novelcomputer game feature that may have helped with treat-ment engagement. Participants completed twenty-five40-min training sessions over a 5-week period. Similarto Beck and colleagues’ findings, results indicated thatparent ratings of ADHD symptoms and parent ratingson two of five subscales of a behavioral measure of execu-tive functioning were improved for the treatment groupcompared to the waitlist control group. Ratings fromteachers revealed no differences between the groups. As
van der Oord and colleagues acknowledged, the findingof differences only on some parent ratings without anydifferences on teacher ratings raises questions aboutthe validity of the reported effects. If the improved beha-viors reported by the parents were not detected atschool, then the clinical utility of this treatment is ques-tionable. Namely, the demands on working memory areoften greater at school than at home, and teachers arefrequently monitoring and measuring student function-ing in ways related to working memory. Yet, similar towhat was found by Beck et al., teachers did not noticeimprovements in symptoms or in behaviors related toexecutive functioning after children completed the treat-ment. As a result and consistent with the conclusions ofother recent reviews (Shipstead, Redick, & Engle, 2012),cognitive training must be considered an experimentaltreatment per the EBT Evaluation Criteria because,although two randomized trials have been conducted,the results are equivocal.
Neurofeedback training. Since 2008, only one studythat met all five EBT Evaluation Criteria evaluated neu-rofeedback training (Gevensleben et al., 2009). This ran-domized trial included 102 children with ADHD betweenthe ages of 8 and 12 years. One group received neurofeed-back training that was designed to help children acquireself-control of specific brain activity patterns to reduceADHD symptoms and improve daily functioning. Theother group completed a computerized attention trainingintervention. Participants completed eighteen 50-mincomputer sessions at a clinic over a 3- to 4-week period.Investigators reported benefits for the group receivingneurofeedback training on parent ratings of ADHDand ODD symptoms, aggression, and the total score ofthe Strengths and Difficulties Questionnaire (Goodman,1997). In addition, significant benefits were also reportedfor teacher ratings of inattention, hyperactivity, andoverall ADHD symptoms. The investigators also gath-ered parent and teacher ratings of social, academic,and home functioning, and there were no significant dif-ferences between the groups on any of these measures. Ofnote, parents and teachers were unaware of treatmentcondition, reducing the possibility of rater bias in theresults. Given that the treatment led to reductions inlevels of symptoms without significant gains in function-ing, neurofeedback training meets task force criteria fora Level 3 treatment or one that is a possibly efficacioustreatment for ADHD.
Organization Training
Investigators have developed and evaluated interven-tions that focus on training children with ADHD toovercome their difficulties organizing school materials.There were two studies of organization training that
PSYCHOSOCIAL TREATMENTS FOR ADHD 543
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
met all EBT Evaluation Criteria; one evaluating a clinic-based intervention for elementary school aged children(Abikoff et al., 2013) and one evaluating school-basedinterventions for young adolescents (Langberg, Epstein,Becker, Girio-Herrera, & Vaughn, 2012). The approachfor training organization of materials and the trackingof assignments is similar across these two studies.Participants were taught organization rules, and theorganization of their materials was regularly measuredagainst a checklist. Although contingent rewards wereprovided for organization and for participant self-correction during the training sessions, consistent withother studies of training interventions, there was minimalto no manipulation of contingencies in the environ-ments outside of the training setting (i.e., classroomsand homes).
Abikoff and colleagues (2013) compared the Organi-zation Skills Training (OST) intervention to a waitlistcontrol condition. OST involved 20 hour-long sessionsheld at a clinic twice per week after school. Parentsattended approximately 10min of each session and,although they were encouraged to monitor theirchildren’s use of the skills, no explicit proceduresfor such monitoring were provided. Children learnedtechniques for tracking assignments and materials andreceived in-session prizes for the successful use of thetechniques between sessions. The results indicated that,relative to the waitlist condition, OST produced signifi-cantly better parent and teacher ratings of organization,academic functioning, homework completion, andfamily conflict. Based on a similar model of trainingstudents to improve the organization of materials andtime, Langberg and colleagues (2012) evaluated theHomework, Organization, and Planning System(HOPS) provided by school mental health professionals(SMHP) in middle schools. The intervention involvedtraining students to organize their materials, track andmonitor assignments, and plan evening homework com-pletion. The SMHP met with students for sixteen 20-minsessions over 11 weeks. Results indicated that HOPSproduced significantly better parent (but not teacher)ratings of organization, homework, and family conflictand that these gains were maintained at 3-monthfollow-up. Measures of feasibility and integrity alsoindicated that the HOPS could be feasibly implementedwith integrity by SMHPs.
Overall, the effects of organization training appearsto vary as a function of sample characteristics. Thereare a number of noteworthy distinctions between thestudy conducted by Abikoff and colleagues (2013) andthe one conducted by Langberg et al. (2012). First,Abikoff et al.’s sample comprised elementary schoolaged children with a higher mean IQ (113), bettereducated parents with approximately one third ofparents having obtained a graduate or professional
degree, and better resourced families who had the meansto attend a clinic twice per week. Conversely, parti-cipants in Langberg and colleagues’ study were middleschool students with a mean IQ of 98 who attendedthe intervention sessions at school. Both studies evaluatedtreatments consisting solely of organization interventions.Thus, organization training has been evaluated by twoindependent research teams with both demonstratingstatistically significant benefits over a waitlist or no-treatment control condition. Thus, organization inter-ventions meet criteria for a well-established treatment.
Combined Training
The remaining two studies in this section conducted anevaluation of a combined training program (ChallengingHorizons Program [CHP]; Evans et al., 2011; Molinaet al., 2008). The CHP is a school-based treatmentprogram for adolescents with ADHD that targetsimpairment related to organization (see earlier), aca-demic skills, and social functioning. It has been modifiedand evaluated as a mentoring program in a middleschool setting (Evans, Serpell, Schultz, & Pastor, 2007)and a coaching intervention in a high school setting(Sadler, Evans, Schultz, & Zoromski, 2011), but mostof the research including the two studies describedhere have evaluated it as an after-school program thatoperates in 2.5-hr sessions, 2 days per week at the part-icipants’ middle school. The study conducted by Molinaet al. was a small trial (11 participants in CHP and 12in community care) that evaluated the benefits of theCHP provided over a 10-week period of the school year.The study by Evans et al. study was slightly larger(31 participants in CHP and 18 in community care),and the intervention was provided over a 5-monthperiod. Molina et al.’s results indicated significantimprovements in parent ratings of internalizing symp-toms, delinquency, and school adjustment. The resultsobtained by Evans et al. revealed significant benefitsin teacher ratings of academic and classroom func-tioning and parent ratings of hyperactivity=impulsivitysymptoms. CHP has been evaluated in two randomizedcontrolled studies since 2008, but not by two inde-pendent research teams. Both studies reported statisti-cally significant parent- and teacher-reported benefitsto the CHP. Given this level of evidence, we classifiedCombined Training (i.e., CHP) as meeting criteria forLevel 2 or probably efficacious treatment.
DISCUSSION
The purpose of the current review was to critically evalu-ate the empirical literature published during the last fiveyears to determine levels of evidence for psychosocial
544 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
interventions for youth with ADHD (see Table 4) and toidentify factors that may influence the outcomes of thesetreatments. Considering the EBT Evaluation Criteria,the conclusions of the 2008 review and the literaturepublished in the last 5 years, we confidently concludethat Behavior Management interventions includingBPT, BCM, and BPI, as well as their use in combination,are well-established treatments. In addition, one of theTraining Interventions, organization training, met thesecriteria. The other Training Interventions includingcognitive training met criteria for Level 4 (ExperimentalTreatments), neurofeedback training met criteria forLevel 3 (Possibly Efficacious), and the combined trainingprogram (Challenging Horizons Program) met criteriafor Level 2 (Probably Efficacious). Next we criticallydiscuss factors that are important to consider wheninterpreting the outcomes of these treatments, includingcharacteristics of the interventions, participants, andmeasurement, as well as the characteristics of the systemfor classifying interventions.
Characteristics of the Interventions
The addition of TI to the arsenal of psychosocialtreatments has been an important shift in the focus oftreatment development for youth with ADHD.Although early efforts at training, such as social skillstraining, were not successful, current efforts focusingon organization and the development of other competen-cies are showing promise. For example, Gevenslebenet al. (2009) reported beneficial effects of neurofeedbacktraining that are equivalent to outcomes reported instudies of well-established behavioral treatments (e.g.,Cohen’s d range from .30 to .64). The obvious advan-tages of TI are that such treatments do not necessitatereliance on adults in the home and school environmentsto consistently implement modified contingencies withintegrity. Indeed, this aspect of TI may render themparticularly useful with adolescents. Given the numerousteachers encountered by adolescents over the course ofthe day, the fact that teens are monitored by adults lessclosely than younger children, and the challenges asso-ciated with identifying salient rewards for adolescents;
it may be that training is the preferred treatmentmodel for youth in this age group.
It is important to note that there is an assumption thatTI produce change in competencies that will persist overtime and across settings, given that these interventionsare not context specific as are traditional behavioralinterventions. However, this potential generalizationadvantage has not been demonstrated. Given thatAbikoff and colleagues (2013) reported success with theirorganization TI with elementary school aged childrenand that both parents and teachers observed the success,there is some promising evidence in support of thisassumption. If generalization of skills developed in TIcan be generalized across time and setting, then provid-ing TI to youth early in their academic careers certainlyhas advantages.
Another novel characteristic of the recent treatmentliterature is that many studies that tested treatmentspreviously identified as well-established focused onimproving access or increasing involvement of popula-tions who do not usually use these interventions.Fabiano and colleagues (Fabiano et al., 2009; Fabianoet al., 2012) modified BPT procedures to improve theengagement of fathers. Chacko and colleagues (2009)attempted to meet the needs of single mothers, andMcGrath et al. (2011) conducted BPT over the telephoneto reduce travel demands on clients. In both the Fabianoet al. and Chacko et al. studies, modified BPT did notyield notably better outcomes than traditional BPT butdid result in better engagement and satisfaction offathers and single mothers, respectively, than traditionalBPT. Of note, although these studies of BPT reportedoutcomes better than no treatment or equivalent totraditional BPT with the same subgroup of participants,we cannot conclude whether the treatment effects wereequivalent to those obtained by families who are not partof such subgroups. The modifications to BPT implemen-ted in the study by McGrath and colleagues involvedconducting the intervention over the telephone and withhandbooks and videos provided to the families. Reportsof satisfaction with ‘‘telephone coaches’’ indicated thatproviding BPT remotely may increase access to thiswell-established treatment for many families who may
TABLE 4
Summary Table of Levels of Evidence
Level 1: Well-Established
Level 2: Probably
Efficacious
Level 3: Possibly
Efficacious
Level 4:
Experimental
Level 5:
Not Effective
Behavioral Parent Training Combined Training
Interventions
Neurofeedback
Training
Cognitive
Training
Social Skills
Training
Behavioral Classroom Management
Behavioral Peer Intervention
Combined Behavior Management Interventions
Organization Training
PSYCHOSOCIAL TREATMENTS FOR ADHD 545
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
not obtain it otherwise. All three groups of investigatorsdescribed implications for further modifications toBPT that may further enhance the efficacy of the inter-vention with the targeted subgroups. For example,Chacko and colleagues noted a need to enhance servicesfor maternal personal problems and to help motherswith communicating with school staff. Continuedinvestigation of parent and child characteristics thatmoderate response to BPT or engagement with BPTare warranted and can provide additional guidancefor those working to extend the reach of thesewell-established services.
Characteristics of Participants
As noted in the previous review (Pelham & Fabiano,2008), very little research has been conducted with ado-lescents with ADHD. Of the well-established treatments,only the organization training included one study target-ing adolescents and these were young adolescents(Langberg et al., 2012; ages 11–14). Given the develop-mental differences between children and adolescents andthe large differences across these age groups in terms ofschool settings, peer relations, and relationships with par-ents; our conclusions about the levels of evidence for BMtreatments are restricted to children between approxi-mately 4 and 12 years of age. There continues to be a needto develop and evaluate treatments for adolescents.
There were two studies of combined BM treatmentsthat included preschool-aged children (Kern et al.,2007; Webster-Stratton et al., 2011). These investigatorstook very different approaches to children in this youngage group. Kern and colleagues combined parenteducation and individualized home and preschoolinterventions based on the results of functionalbehavior analyses. This procedure was contrasted withparent education alone over 18 months. Although attend-ance at parent education was poor in both groups(M percentages¼ 37% and 29%), both groups improvedon 16 of the 18 primary outcome measures. There wereno significant treatment advantages for those in the activetreatment group relative to those in the control group.Given the poor attendance at the parent sessions it isunclear what led to improvement in the parent educationonly group that yielded improvements that were equiva-lent to those obtained by participants in the active treat-ment group. In contrast,Webster-Stratton and colleaguescompared the combination of the Incredible YearsProgram (BPT) and a child-focused group training inter-vention (TI) to a waitlist control and reported significanttreatment effects for those receiving the combinedtreatment. Attendance at parent training sessions wasmuch higher in this study than in the Kern and collea-gues’ study (M percentage attendance¼ 93 [mothers]and 85 [fathers]) and the mean age of the sample was
approximately 11 months older. There is an extensiveliterature demonstrating treatment effects for the Incred-ible Years Program, and little to no evidence supportingthe efficacy of a child-focused training intervention.Based on the extensive literature on BM approaches withyoung children prior to 2008, Pelham and Fabiano con-cluded that these approaches were well-established forthis age group, and these two studies add to that evidence.
Another difference between participants recruited forthe studies just reviewed involves recruitment proce-dures. Participants recruited from clinic settings arelikely to have parents attending the clinic with them,and parental presence indicates a degree of involvementand resources that are not always present among famil-ies recruited from the community. For example, asnoted previously, participants in the Abikoff et al.(2013) study were recruited at a clinic and had an aver-age IQ estimate of 113. Participants in the Power et al.(2012) study were also recruited from a clinic and thesocioeconomic status of 98% of the participants was inthe middle to high range. These figures can be con-trasted with those obtained by two studies, whereinparticipants were recruited from schools (Evans et al.,2011; Langberg et al., 2012). In these studies the averageIQ estimate was 95 and 98, respectively. The averagefamily income was approximately $45,000 in the Evanset al. study, and Langberg et al. reported that more thanhalf of their families had incomes less than $75,000 peryear (15% had less than $25,000). To the extent thatcognitive ability and income may influence outcomesand=or parent involvement (e.g., Owens et al., 2003;Rieppi et al., 2002), these differences need to be notedwhen interpreting findings and explicitly explored infuture studies. Indeed, only three of 21 studies includedanalyses examining moderators of treatment outcomes.Important differences in conclusions may be a functionof participant characteristics that could be related torecruitment methods.
Finally, it is noteworthy that the reviewed researchdid not directly address questions relating treatmentresponse to the racial and ethnic backgrounds of parti-cipants. Although there continues to be an emphasison the importance of these research questions and fund-ing agencies continue to require diverse samples, thescience addressing these issues is very shallow.
Characteristics of Measurement
There are two assessment-related issues that we believeshould be considered when interpreting findings andthese pertain to diagnostic decisions and measurementsources. First, as can be seen in Table 2, investigatorsof some studies based diagnoses on parent report only,whereas others used both parent and teacher report.Among those that based diagnoses on both parent and
546 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
teacher report, some counted symptoms as present basedon an ‘‘and’’ rule and others used an ‘‘or’’ rule. Many ofthe studies did not indicate the basis for deciding whensymptoms were considered present. Two studies in thepast 5 years have revealed that these subtle decisionscan lead to important differences in terms of whichchildren are diagnosed with ADHD and which are not(Rowland et al., 2008; Valo & Tannock, 2010). Theresults of treatment outcome studies may also be affectedby these variations in how diagnoses are determined. It isunclear if these differences are important and whethervariations in samples due to diagnostic proceduresmay influence the populations to whom findings mightgeneralize.
Second, the vast majority of the measures used todetermine the level of evidence for the treatments wereratings completed by parents and=or teachers who wereaware of the child’s treatment condition. There is evi-dence indicating that awareness of treatment conditioninflates effect sizes (Jadad et al. 1996). This factor alonemay account for much of the difference between the con-clusions of this review and the recent publication bySonuga-Barke and colleagues (2013). Researchers con-ducting treatment development and evaluation researchwith behavioral treatments typically recruit the adultsin a child’s life to implement the modified contingenciesin the natural settings where the child’s problematicbehavior occurs. As a result, it may not be possible tofind knowledgeable sources for ratings who are unawareof treatment status. Further, research has demonstratedthat a large portion of the variance in teacher ratings isdue to rater-related effects as opposed to variability inchild behavior (Briesch, Chafouleas & Riley-Tillman,2010). Alternatives to ratings can be difficult toimplement. For example, direct observations have manylimitations including expense and time (see Pelham,Fabiano, & Massetti, 2005). Briesch and colleagues(2010) reported that three to five observations eitherwithin or across days are needed to assess task engage-ment at school in order to obtain dependable estimatesof the target behavior. Further adding to the costs ofdirect observation, these authors conducted 8 hr of train-ing with their observers. Although raters can sometimesbe unaware of treatment conditions, conducting enoughobservations to obtain valid indices of outcomes,tracking infrequent behavior, costs of observers, andmeasuring constructs that are not easily observable(e.g., reciprocal peer relationships) make it difficult torely on observations. Tracking objective criteria relatedto a permanent product is another assessment optionand was used in the organization and CHP studies. Forexample, staff tracked organization progress based ona set of objective criteria pertaining to the participants’school binders. Although staff who completed the track-ing forms were aware of the treatment condition, staff
simply marked whether each criterion was met or notmet. The items described concrete choices (e.g., an itemis present or absent) and thus were less likely to be influ-enced by rater effects than items on parent and teacherrating scales. Nevertheless, systems like these used totrack organization may not be possible when assessingsome of the constructs targeted in treatments for childrenwith ADHD (e.g., social functioning). Last, schoolrecords (e.g., grades, office referrals) often offer ecologi-cal validity but are not entirely immune from teacherbias, leading to limited reliability across teachers, schoolbuildings, and time.
To counter some of these challenges in measurement,it has been recommended that investigators take amultisource and multimethod approach to assessing theconstructs that are intended to change as a functionof a treatment (American Academy of Pediatrics, Com-mittee on Quality Improvement and Subcommittee onAttention-Deficit=Hyperactivity Disorder, 2011); how-ever, this approach creates other problems. As describedby De Los Reyes and Kazdin (2006), there is no standardfor identifying how many of the multiple measures andwhich ones need to indicate treatment effects in orderfor the study to be regarded as supporting the efficacyof the treatment. For many of the studies in this reviewand the two previous reviews completed by Pelham andcolleagues (Pelham & Fabiano, 2008; Pelham et al.,1998), relatively few of the possible outcomes measuredindicated statistically significant differences between thetreatment and comparison groups. Reliable and validindices of both symptoms and impairment related toADHD that are not compromised by sources aware oftreatment conditions are sorely needed along with guide-lines for interpreting findings from studies with multiplemeasures of outcomes.
Method for Classifying Treatments
The substantial differences between this review and themeta-analysis published by Sonuga-Barke and colleagues(2013) underscore the lack of a clear consensus for howwe determine levels of evidence for a treatment. Theareas of inconsistency begin with the selection of studiesto be considered in a review. The criteria for selection ofstudies in this review are listed as M1 to M5 in Table 1.Sonuga-Barke and colleagues eliminated studies thatcontrasted a treatment with another active treatmentwithout a no-treatment control group. For example,the Fabiano and colleagues (2009) study compared themodified BPT program for fathers (COACHES) toa standard BPT condition and this study was excludedby Sonuga-Barke and colleagues due to ‘‘no appropriatecontrol.’’ The criteria used in the present reviewconsiders demonstrating equivalence to another well-established treatment as evidence supporting the efficacy
PSYCHOSOCIAL TREATMENTS FOR ADHD 547
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
of an intervention, whereas the criteria employed bySonuga-Barke et al. did not.
Another factor contributing to the selection ofresearch studies to consider in a review pertains to theoutcome measures selected and this choice pertains toanother key difference between our findings and thoseof Sonuga-Barke and colleagues. The outcome measurecriterion used in this study indicates that an outcomemeasure must be reliable, be valid, and gauge the prob-lems targeted (see M4 in Table 1). As a result, the socialfunctioning outcome measures used in the Mikami et al.(2010) study of a parent friendship coaching interventionwere acceptable in our review because social impairmentis a very common problem for youth with ADHD.Although they also noted that impairment may be amore relevant outcome for psychosocial interventions,Sonuga-Barke et al. excluded this study from their meta-analyses due to ‘‘no ADHD outcomes.’’ We includedmeasures of symptoms and impairment and suggest thatdrawing conclusions about levels of evidence for psycho-social treatments based solely on symptoms is likely toseriously underestimate their effects. As noted by Pelhamand Fabiano in their review, impairment predictslong-term outcomes better than symptoms (Mannuzza& Klein, 1999), and impairments are the primary reasonsthat parents pursue treatments for their child. Change insymptoms is related to change in impairment, but thereare large differences when considering children improvedon one or the other (J. S. Owens, Johannes, & Karpenko,2009). Furthermore, conclusions about treatmentresponse based only on symptom changes (e.g., MTACooperative Group, 1999) may end up misrepresentingthe benefits of psychosocial treatments (Conners et al.,2001). Thus we consider the inclusion of measures asses-sing both symptoms and impairment related to ADHDas critical for assessing treatment response.
Finally, we were challenged during the review andclassification of the TI studies with regards to determininglevels of evidence when studies reported mixed outcomes.For example, as previously noted, both studies of cogni-tive training (Beck et al., 2010; van der Oord et al., inpress) reported gains across parent ratings of symptoms,mixed improvements across parent ratings of executivefunctioning, and only one instance of improvement outof multiple comparisons of teacher ratings of symptomsand executive functioning. Although both studies metall of five EBT Evaluation Criteria, the lack of clarityin the larger literature regarding the necessary propor-tion of measures on which improvement is to be demon-strated (De Los Reyes & Kazdin, 2006, 2009) madeclassification difficult. This issue, along with many relatedlimitations to our systems for classifying treatmentsaccording to their evidence base is described in verythoughtful articles by De Los Reyes and Kazdin (2006,2009), who propose a classification system to address
some of these limitations: the Range of Possible ChangesModel. De Los Reyes and Kazdin (2006, 2009) describedthe difficulties associated with comparing inconsistentfindings obtained on the same outcome measure acrossstudies, as well as inconsistent findings obtained withinthe same study across outcome measures, and proposeda process that considers a proportional index of findingsthat is to be contrasted with study hypotheses. Other toolsfor advancing our science of identifying evidence-basedtreatments may involve a diminished reliance on p valuesand statistical significance. In fact, there has been anincreased reliance on effect sizes during the last decadeas well as on the use of indices of clinically significantchange (Jacobson & Truax, 1991). Nine of the 21 studiesreviewed in this article reported some indicator of clini-cally significant change. It may also be time to considerother alternatives for analyzing and conceptualizingresponse to treatment, including Bayesian analysesthat provide effect sizes indicating the odds of responsebetween treatment conditions. In any event, methodsfor analyzing and interpreting outcome research need toadvance if we are going to be able to identify reliableclassification systems of treatments.
Implications for Practice
If practitioners are going to begin prioritizing the use ofwell-established treatments, dramatic transformationsare needed in two areas within our systems of care. Thefirst involves the integration of training protocols for stu-dents in graduate programs who have the potential tobecomemental health practitioners in schools and clinics.The evidence suggests that many of the professionalmental health practitioners are not being trained inevidence-based practices (Kelly et al., 2010; Shernoff,Kratochwill, & Stoiber, 2003). This lack of trainingmay be related to the lack of accountability for practi-tioners to provide evidence-based practices. In manysystems of care, including schools and clinics, there isno direct accountability on individual clinicians toprovide evidence-based practices with integrity. Insteadthe focus of accountability is often on patient quotasand billable units (regardless of quality of care). Studiesshow that without supervision and accountability, clini-cians drift and adherence to best practices diminish(Schoenwald, Henggeler, Brondino, & Rowland, 2000).Thus, without a quality assurance system that trains,monitors, supervises, and incentivizes use of evidence-based practices, there may be little likelihood of wide-spread adoption.
Although the gap between science and practicehas been thoroughly discussed in both the researchand practice settings of many disciplines, we are notaware of evidence that the gap is meaningfully shrinking.For example, when we conduct treatment development
548 EVANS, OWENS, BUNFORD
Dow
nloa
ded
by [
107.
10.1
33.2
6] a
t 15:
30 2
4 M
arch
201
5
and evaluation research in schools, we are frequentlyintroducing school mental health professionals (counse-lors and social workers) to the basic techniques involvedin cognitive behavioral therapy and behavioral parenttraining, for the first time. Conducting treatment researchin the settings intended for implementation will forceinvestigators to continue to face some of these challengingimplementation issues, and some of the studies consideredin this review provide examples of this research practice.However, it may be that the professional silos providingthe greatest obstacle to consistent implementation ofevidence-based practices are those between science, policy,and practice and not necessarily those between disciplines.
In summary, this review provides an update on thestate of the science for psychosocial interventions foryouth with ADHD. It highlights the innovations thathave occurred in the last 5 years including innovationsto existing well-established treatments to reach newpopulations, an increase in research on adolescentsand preschool children with ADHD, and the develop-ment of a new category of interventions (i.e., TI). Wealso highlighted several critical issues to be incorporatedinto the next generation of research, such as attention tocharacteristics of participants, diagnostic procedures,outcome measures, and the system classifying levels ofevidence. We look forward to observing and partici-pating in advancements that take place in the next5 years and the impact that those scientific advancesmay have on practice and policy.
REFERENCES
Abikoff, H. B., Gallagher, R., Wells, K., Murray, D. W., Huang, L.,
Lu, F., & Petkova, E. (2013). Remediating organizational function-
ing in children with ADHD: Immediate and long-term effects from
a randomized controlled trial. Journal of Consulting and Clinical
Psychology, 81, 113–128.
American Academy of Pediatrics, Committee on Quality Improvement,
& Subcommittee on Attention-Deficit=Hyperactivity Disorder.
(2011). ADHD: Clinical practice guideline for the diagnosis,
evaluation, and treatment of attention-deficit=hyperactivity disorder
in children and adolescents. Pediatrics, 128, 1–16.
Antshel, K. M., & Barkley, R. (2011). Overview and historical back-
ground of attention deficit hyperactivity disorder. In S. W. Evans
& B. Hoza (Eds.), Treating attention-deficit=hyperactivity disorder:
Assessment and intervention in developmental context (pp. 1–1 to
1–30). New York, NY: Civic Research Institute.
Barkley, R. A. (1997). Defiant children: A clinician’s manual for assess-
ment and parent training. New York, NY: Guilford.
Beck, S. J., Hanson, C. A., Puffenberger, S. S., Benninger, K. L., &
Benninger, W. B. (2010). A controlled trial of working memory
training for children and adolescents with ADHD. Journal of
Clinical Child and Adolescent Psychology, 39, 825–836.
Briesch, A. M., Chafouleas, S. M., & Riley-Tillman, T. C. (2010).
Generalizability and dependability of behavior assessment methods
to estimate academic engagement: A comparison of systematic direct
observation and direct behavior rating. School Psychology Review,
39, 408–421.
Chacko, A., Wymbs, B. T., Wymbs, F. A., Pelham, W. E., Swanger-
Gagne, M. S., Girio, E., . . .O’Connor, B. (2009). Enhancing
traditional behavioral parent training for single mothers of children
with ADHD. Journal of Clinical Child and Adolescent Psychology,
38, 206–218.
Conners, C. K., Epstein, J. N., March, J. S., Angold, A., Wells, K. C.,
Klaric, J., . . .Wigal, T. (2001). Multimodal treatment of ADHD in
the MTA: An alternative outcome analysis. Journal of the American
Academy of Child and Adolescent Psychiatry, 40, 159–167.
Cooper, H., & Hedges, L. V. (1994). Research synthesis as a scientific
enterprise. In H. Cooper & L. V. Hedges (Eds.), The handbook
of research synthesis (pp. 4–14). New York, NY: Russell Sage
Foundation.
Cunningham, C. E., Bremner, R., & Secord-Gilbert, M. (1993).
Increasing the availability, accessibility, & cost efficacy of services
for families of ADHD children: A school-based systems-oriented
parenting course. Canadian Journal of School Psychology, 9, 1–15.
De Los Reyes, A., & Kazdin, A. E. (2006). Conceptualizing changes in
behavior in intervention research: The range of possible change
model. Psychological Review, 113, 554–583.
De Los Reyes, A., & Kazdin, A. E. (2009). Identifying evidence-based
interventions for children and adolescents using the range of possible
changes model: A meta-analytic illustration. Behavior Modification,
33, 583–617.
Evans, S. W., Schultz, B. K., DeMars, C. E., & Davis, H. (2011).
Effectiveness of the Challenging Horizons After-School Program
for young adolescents with ADHD. Behavior Therapy, 42, 462–474.
Evans, S. W., Serpell, Z. N., Schultz, B., & Pastor, D. (2007).
Cumulative benefits of secondary school-based treatment of
students with ADHD. School Psychology Review, 36, 256–273.
Fabiano, G. A., Pelham, W. E., Coles, E. K., Gnagy, E. M., Chronis-
Tuscano, A. M., & O’Connor, B. C. (2009). A meta-analysis of
behavioral treatments for attention- deficit=hyperactivity disorder.
Clinical Psychology Review, 29, 129–140.
Fabiano, G. A., Pelham, W. E., Cunningham, C., Yu, J., Gangloff, B.,
Buck,M., . . .Gera, S. (2012). A waitlist-controlled trail of behavioral
parent training for fathers of children with ADHD. Journal of
Clinical Child and Adolescent Psychology, 41, 337–345.
Fabiano, G. A., Vujnovic, R. K., Pelham, W. E., Waschbusch, D. A.,
Massetti, G. M., Pariseau, M. E., . . .Volker, M. (2010). Enhancing
the effectiveness of special education programming for children with
attention deficit hyperactivity disorder using a daily report card.
School Psychology Review, 39, 219–239.
Gevensleben, H., Holl, B., Albrecht, B., Schlamp, D., Kratz, O.,
Studer, P., . . .Heinrich, H. (2009). Distinct EEG effects related to
neurofeedback training in children with ADHD: A randomized
controlled trial. International Journal of Psychophysiology, 74,
149–157.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A
research note. Journal of Child Psychology and Psychiatry, 38, 581–586.
Hoza, B., Johnston, C., Pillow, D. R., & Ascough, J. C. (2006).
Predicting treatment response for childhood attention-deficit=
hyperactivity disorder: Introduction of a heuristic model to guide
research. Applied and Preventive Psychology, 11, 215–229.
Institute of Education Sciences. (2011). What Works Clearinghouse:
Procedures and Standards Handbook (version 2.1). Washington,
DC: Author. Available from http://ies.ed.gov/ncee/wwc/pdf/
