Background: The Autism Diagnostic Interview-Revised (ADI-R) is considered a first choice assessment tool in autism spectrum disorder. Nevertheless, despite its wide use in psychiatric practice and recommendations by various clinical guidelines, its interrater reliability has predominantly been confirmed in research settings by specially trained, research reliability interviewers. The reliability of ADI-R assessments among clinicians has not yet been established. Therefore, this study examined the spontaneous interrater reliability of the ADI-R in a naturalistic clinical multicenter setting. Sampling and Methods: Ten video-recorded ADI-R administrations were rated by 5 different raters each from a pool of 11 raters affiliated to 8 different clinical sites. Results: The interrater reliability for the 12 diagnostic criteria operationalizing autism spectrum disorders according to DSM-IV/ICD-10 in the ADI-R algorithms ranged between G(q,k) (analogous to intraclass correlations) = 0.96 and 0.99 for reciprocal social interaction, 0.96 and 1.00 for communication, and 0.91 and 0.97 for repetitive and restricted behavior. Reliability of diagnostic classification was ĸCohen 0.83. Conclusions: The findings endorse the psychometric properties of ADI-R in terms of interrater reliability previously reported from research settings and support their generalization to common clinical settings. Limitations of this study include an unbalanced sample composition.

Keywords:
Autism spectrum disorder, Interrater reliability, Diagnostic instrument, Cross-cultural transferability, Psychometrics
1.
American Psychiatric Association (APA): Diagnostic and Statistical Manual of Mental Disorders: DSM-IV, ed 4. Washington, APA, 1994.
2.
American Psychiatric Association (APA): Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR, ed 4, text rev. Washington, APA, 2000.
3.
American Psychiatric Association (APA): Diagnostic and Statistical Manual of Mental Disorders: DSM-5, ed 5. Arlington, APA, 2013.
4.
World Health Organization (WHO): The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva, WHO, 1993.
5.
Atladottir HO, Gyllenberg D, Langridge A, Sandin S, Hansen SN, Leonard H, et al: The increasing prevalence of reported diagnoses of childhood psychiatric disorders: a descriptive multinational comparison. Eur Child Adolesc Psychiatry 2015;24:173-183.
6.
Zablotsky B, Black LI, Maenner MJ, Schieve LA, Blumberg SJ: Estimated prevalence of autism and other developmental disabilities following questionnaire changes in the 2014 National Health Interview Survey. Natl Health Stat Report 2015;87:1-20.
7.
Hinkka-Yli-Salomäki S, Banerjee PN, Gissler M, Lampi KM, Vanhala R, Brown AS, et al: The incidence of diagnosed autism spectrum disorders in Finland. Nord J Psychiatry 2014;68:472-480.
8.
Idring S, Lundberg M, Sturm H, Dalman C, Gumpert C, Rai D, et al: Changes in prevalence of autism spectrum disorders in 2001-2011: findings from the Stockholm youth cohort. J Autism Dev Disord 2015;45:1766-1773.
9.
Kim YS, Leventhal BL, Koh YJ, Fombonne E, Laska E, Lim EC, et al: Prevalence of autism spectrum disorders in a total population sample. Am J Psychiatry 2011;168:904-912.
10.
Kaufman J, Birmaher B, Brent B, Rao UMA, Ryan N, Axelson D, et al: Kiddie-SADS-Present and Lifetime Version (K-SADS-PL), 2009 Working Draft (accessed June 3, 2016). http://www.psychiatry.pitt.edu/sites/default/files/Documents/assessments/KSADS-PL_2009_working_draft_full.pdf.
11.
Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al: Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 1997;36:980-988.
12.
Goodman R, Ford T, Richards H, Gatward R, Meltzer H: The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry 2000;41:645-655.
13.
World Health Organization (WHO): Composite International Diagnostic Interview (CIDI): researcher's manual. Geneva, WHO, 1994.
14.
First MB, Spitzer RL, Gibbon M, Williams JBW: Structured Clinical Interview for DSM-IV Axis I Disorders SCID-I: clinician version, administration booklet. Washington, D.C., American Psychiatric Press, 1997.
15.
Gillberg C, Gillberg IC, Råstam M, Wentz E: The Asperger Syndrome (and high-functioning autism) Diagnostic Interview (ASDI): a preliminary study of a new structured clinical interview. Autism 2001;5:57-66.
16.
Stone WL, Hogan KL: A structured parent interview for identifying young children with autism. J Autism Dev Disord 1993;23:639-652.
17.
Skuse D, Warrington R, Bishop D, Chowdhury U, Lau J, Mandy W, et al: The developmental, dimensional and diagnostic interview (3di): a novel computerized assessment for autism spectrum disorders. J Am Acad Child Adolesc Psychiatry 2004;43:548-558.
18.
Nygren G, Hagberg B, Billstedt E, Skoglund A, Gillberg C, Johansson M: The Swedish version of the Diagnostic Interview for Social and Communication Disorders (DISCO-10). Psychometric properties. J Autism Dev Disord 2009;39:730-741.
19.
Wing L, Leekam SR, Libby SJ, Gould J, Larcombe M: The Diagnostic Interview for Social and Communication Disorders: background, inter-rater reliability and clinical use. J Child Psychol Psychiatry 2002;43:307-325.
20.
Rutter M, Le Couteur A, Lord C: ADI-R. Autism Diagnostic Interview Revised. Los Angeles, Western Psychological Services, 2003.
21.
Huerta M, Bishop SL, Duncan A, Hus V, Lord C: Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. Am J Psychiatry 2012;169:1056-1064.
22.
NICE Clinical Guidelines. Autism: Recognition, Referral, Diagnosis and Management of Children and Young People on the Autism Spectrum. Leicester, British Psychological Society/Royal College of Psychiatrists, 2011.
23.
Swedish Council on Health Technology Assessment (SBU): Autism Spectrum Disorder. Diagnostic Assessment and Interventions, Organization of Services and Participation - A Systematic Review. Stockholm, Swedish Council on Health Technology Assessment, 2013.
24.
Volkmar F, Siegel M, Woodbury-Smith M, King B, McCracken J, State M, et al: Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 2014;53:237-257.
25.
Ashwood KL, Buitelaar J, Murphy D, Spooren W, Charman T: European clinical network: autism spectrum disorder assessments and patient characterisation. Eur Child Adolesc Psychiatry 2015;24:985-995.
26.
de Bildt A, Sytema S, Zander E, Bölte S, Sturm H, Yirmiya N, et al: Autism Diagnostic Interview-Revised (ADI-R) algorithms for toddlers and young preschoolers: application in a non-US sample of 1,104 children. J Autism Dev Disord 2015;45:1-16.
27.
Havdahl KA, von Tetzchner S, Huerta M, Lord C, Bishop SL: Utility of the Child Behavior Checklist as a screener for autism spectrum disorder. Autism Res 2016;9:33-42.
28.
Lampi KM, Sourander A, Gissler M, Niemela S, Rehnstrom K, Pulkkinen E, et al: Brief report: validity of Finnish registry-based diagnoses of autism with the ADI-R. Acta Paediatr 2010;99:1425-1428.
29.
Saemundsen E, Magnusson P, Smari J, Sigurdardottir S: Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: convergence and discrepancy in diagnosing autism. J Autism Dev Disord 2003;33:319-328.
30.
The National Board of Health and Welfare (Socialstyrelsen): The Methods used in Child and Adolescent Psychiatry. A National Inventory. Västerås, Socialstyrelsen, 2009.
31.
Zander E, Sturm H, Bölte S: The added value of the combined use of the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule: diagnostic validity in a clinical Swedish sample of toddlers and young preschoolers. Autism 2015;19:187-199.
32.
Kim SH, Lord C: New Autism Diagnostic Interview-Revised algorithms for toddlers and young preschoolers from 12 to 47 months of age. J Autism Dev Disord 2012;42:82-93.
33.
de Bildt A, Oosterling IJ, van Lang ND, Kuijper S, Dekker V, Sytema S, et al: How to use the ADI-R for classifying autism spectrum disorders? Psychometric properties of criteria from the literature in 1,204 Dutch children. J Autism Dev Disord 2013;43:2280-2294.
34.
de Bildt A, Sytema S, Ketelaars C, Kraijer D, Mulder E, Volkmar F, et al: Interrelationship between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic Interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) classification in children and adolescents with mental retardation. J Autism Dev Disord 2004;34:129-137.
35.
Gray KM, Tonge BJ, Sweeney DJ: Using the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule with young children with developmental delay: evaluating diagnostic validity. J Autism Dev Disord 2008;38:657-667.
36.
Le Couteur A, Haden G, Hammal D, McConachie H: Diagnosing autism spectrum disorders in pre-school children using two standardised assessment instruments: the ADI-R and the ADOS. J Autism Dev Disord 2008;38:362-372.
37.
Le Couteur A, Rutter M, Lord C, Rios P, Robertson S, Holdgrafer M, et al: Autism Diagnostic Interview: a standardized investigator-based instrument. J Autism Dev Disord 1989;19:363-387.
38.
Lord C, Pickles A, McLennan J, Rutter M, Bregman J, Folstein S, et al: Diagnosing autism: analyses of data from the Autism Diagnostic Interview. J Autism Dev Disord 1997;27:501-517.
39.
Lord C, Rutter M, Le Couteur A: Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord 1994;24:659-685.
40.
Mazefsky CA, Oswald DP: The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism 2006;10:533-549.
41.
Papanikolaou K, Paliokosta E, Houliaras G, Vgenopoulou S, Giouroukou E, Pehlivanidis A, et al: Using the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule-Generic for the diagnosis of autism spectrum disorders in a Greek sample with a wide range of intellectual abilities. J Autism Dev Disord 2009;39:414-420.
42.
Sappok T, Diefenbacher A, Budczies J, Schade C, Grubich C, Bergmann T, et al: Diagnosing autism in a clinical sample of adults with intellectual disabilities: how useful are the ADOS and the ADI-R? Res Dev Disabil 2013;34:1642-1655.
43.
Kim SH, Thurm A, Shumway S, Lord C: Multisite study of New Autism Diagnostic Interview-Revised (ADI-R) algorithms for toddlers and young preschoolers. J Autism Dev Disord 2013;43:1527-1538.
44.
Howlin P, Moss P, Savage S, Rutter M: Social outcomes in mid- to later adulthood among individuals diagnosed with autism and average nonverbal IQ as children. J Am Acad Child Adolesc Psychiatry 2013;52:572-581 e1.
45.
Gilchrist A, Green J, Cox A, Burton D, Rutter M, Le Couteur A: Development and current functioning in adolescents with Asperger syndrome: a comparative study. J Child Psychol Psychiatry 2001;42:227-240.
46.
Bölte S, Poustka F: Factor structure of the Autism Diagnostic Interview-Revised (ADI-R): a study of dimensional versus categorical classification of autistic disorders (in German). Z Kinder Jugendpsychiatr Psychother 2001;29:221-229.
47.
Lecavalier L, Aman MG, Scahill L, McDougle CJ, McCracken JT, Vitiello B, et al: Validity of the Autism Diagnostic Interview-Revised. Am J Ment Retard 2006;111:199-215.
48.
Tanguay PE, Robertson J, Derrick A: A dimensional classification of autism spectrum disorder by social communication domains. J Am Acad Child Adolesc Psychiatry 1998;37:271-277.
49.
Snow AV, Lecavalier L, Houts C: The structure of the Autism Diagnostic Interview-Revised: diagnostic and phenotypic implications. J Child Psychol Psychiatry 2009;50:734-742.
50.
Hill A, Bölte S, Petrova G, Beltcheva D, Tacheva S, Poustka F: Stability and interpersonal agreement of the interview-based diagnosis of autism. Psychopathology 2001;34:187-191.
51.
Cicchetti DV, Lord C, Koenig K, Klin A, Volkmar FR: Reliability of the ADI-R: multiple examiners evaluate a single case. J Autism Dev Disord 2008;38:764-770.
52.
Cicchetti DV, Lord C, Koenig K, Klin A, Volkmar FR: Reliability of the ADI-R for the single case. Part II. Clinical versus statistical significance. J Autism Dev Disord 2014;44:3154-3160.
53.
Lord C, Storoschuk S, Rutter M, Pickles A: Using the ADI-R to diagnose autism in preschool children. Infant Ment Health J 1993;14:234-252.
54.
Poustka F, Lisch S, Ruhl D, Sacher A, Schmotzer G, Werner K: The standardized diagnosis of autism, Autism Diagnostic Interview-Revised: interrater reliability of the German form of the interview. Psychopathology 1996;29:145-153.
55.
Tsuchiya KJ, Matsumoto K, Yagi A, Inada N, Kuroda M, Inokuchi E, et al: Reliability and validity of Autism Diagnostic Interview-Revised, Japanese version. J Autism Dev Disord 2013;43:643-662.
56.
Chakrabarti S, Fombonne E: Pervasive developmental disorders in preschool children. JAMA 2001;285:3093-3099.
57.
Putka DJ, Le H, McCloy RA, Diaz T: Ill-structured measurement designs in organizational research: implications for estimating interrater reliability. J Appl Psychol 2008;93:959-981.
58.
Conger AJ: Integration and generalization of kappas for multiple raters. Psychol Bull 1980;88:322-328.
59.
Hallgren KA: Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 2012;8:23-34.
60.
Cicchetti DV, Sparrow SA: Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic 1981;86:127-137.
61.
Shavelson RJ, Webb NM: Generalizability Theory. Handbook of Complementary Methods in Education Research. Mahwah, Erlbaum, 2006, pp 309-322.
62.
Fleiss JL: Measuring nominal scale agreement among many raters. Psychol Bull 1971;76:378-382.
63.
Cohen J: A Coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
64.
de Vet HC, Terwee CB, Knol DL, Bouter LM: When to use agreement versus reliability measures. J Clin Epidemiol 2006;59:1033-1039.
65.
Cicchetti DV: Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 1994;6:284-290.
66.
Cicchetti DV, Volkmar F, Klin A, Showalter D: Diagnosing autism using ICD-10 criteria: a comparison of neural networks and standard multivariate procedures. Child Neuropsychol 1995;1:26-37.
67.
Klin A, Lang J, Cicchetti DV, Volkmar FR: Brief report: interrater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: results of the DSM-IV autism field trial. J Autism Dev Disord 2000;30:163-167.
68.
Leekam SR, Libby SJ, Wing L, Gould J, Taylor C: The Diagnostic Interview for Social and Communication Disorders: algorithms for ICD-10 childhood autism and Wing and Gould autistic spectrum disorder. J Child Psychol Psychiatry 2002;43:327-342.
69.
Stone WL, Coonrod EE, Pozdol SL, Turner LM: The Parent Interview for Autism-Clinical Version (PIA-CV): a measure of behavioral change for young children with autism. Autism 2003;7:9-30.
70.
Gillberg IC, Gillberg C: Asperger syndrome - some epidemiological considerations: a research note. J Child Psychol Psychiatry 1989;30:631-638.
71.
Zavaleta-Ramirez P, Nafate-Lopez O, Villarreal-Valdes G, Ulloa-Flores RE, Albores-Gallo L: K-SADS-PL-2009/autism spectrum disorders (ASD) inter-rater reliability. Salud Ment 2014;37:461-466.
72.
Lord C, Rutter M, DiLavore P, Risi S, Gotham K, Bishop S: Autism Diagnostic Observation Schedule, ed 2 (ADOS-2). Manual (Part I). Torrance, Western Psychological Services, 2012.
73.
Kim SH, Lord C: Combining information from multiple sources for the diagnosis of autism spectrum disorders for toddlers and young preschoolers from 12 to 47 months of age. J Child Psychol Psychiatry 2012;53:143-151.
74.
Risi S, Lord C, Gotham K, Corsello C, Chrysler C, Szatmari P, et al: Combining information from multiple sources in the diagnosis of autism spectrum disorders. J Am Acad Child Adolesc Psychiatry 2006;45:1094-1103.
© 2017 S. Karger AG, Basel
2017
Copyright / Drug Dosage / Disclaimer
Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.
You do not currently have access to this content.