Background/Aims: The optimal method of establishing test cutoffs or cutpoints for cognitive screening instruments (CSIs) is uncertain. Of the available methods, two base cutoffs on either the maximal test accuracy or the maximal Youden index. The aim of this study was to compare the effects of using these alternative methods of establishing cutoffs. Methods: Datasets from three pragmatic diagnostic accuracy studies which examined the Mini-Mental State Examination (MMSE), the Addenbrooke's Cognitive Examination-Revised (ACE-R), the Montreal Cognitive Assessment (MoCA), and the Test Your Memory (TYM) test were analysed to calculate test sensitivity and specificity using cutoffs based on either maximal test accuracy or the maximal Youden index. Results: For ACE-R, MoCA, and TYM, optimal cutoffs for dementia diagnosis differed from those in index studies when defined using either the maximal accuracy or the maximal Youden index method. Optimal cutoffs were higher for MMSE, MoCA, and TYM when using the maximal Youden index method and consequently more sensitive. Conclusion: Revision of the cutoffs for CSIs established in index studies may be required to optimise performance in pragmatic diagnostic test accuracy studies which more closely resemble clinical practice.

1.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, De Vet HCW, Lijmer JG: The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem 2003;49:7-18.
2.
Noel-Storr AH, McCleery JM, Richard E, Ritchie CW, Flicker L, Cullum SJ, Davis D, Quinn TJ, Hyde C, Rutjes AWS, Smailagic N, Marcus S, Black S, Blennow K, Brayne C, Fiorivanti M, Johnson JK, Kopke S, Schneider LS, Simmons A, Mattsson N, Zetterberg H, Bossuyt PMM, Wilcock G, McShane R: Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem Initiative. Neurology 2014;83:364-373.
3.
Dubois B, Feldman HH, Jacova C, Hampal H, Molinuevo JL, Blennow K, DeKosky ST, Gauthier S, Selkoe D, Bateman R, Cappa S, Crutch S, Engelborghs S, Frisoni GB, Fox NC, Galasko D, Habert MO, Jicha GA, Nordberg A, Pasquier F, Rabinovici G, Robert P, Rowe C, Salloway S, Sarazin M, Epelbaum S, de Souza LC, Vellas B, Visser PJ, Schneider LS, Stern Y, Scheltens P, Cummings JL: Advancing research diagnostic criteria for Alzheimer's disease: the IWG-2 criteria. Lancet Neurol 2014;13:614-629, erratum in Lancet Neurol 2014;13:757.
4.
Bartlett JW, Frost C, Mattsson N, Skillback T, Blennow K, Zetterebrg H, Schott JM: Determining cut-points for Alzheimer's disease biomarkers: statistical issues, methods and challenges. Biomarkers Med 2012;6:391-400.
5.
Youden WJ: Index for rating diagnostic tests. Cancer 1950;3:32-35.
6.
Larner AJ: Pragmatic diagnostic accuracy studies. www.bmj.com/contents/345/bmj.e3999/rr/599970 (accessed September 30, 2014).
7.
Mioshi E, Dawson K, Mitchell J, Arnold R, Hodges JR: The Addenbrooke's Cognitive Examination Revised (ACE-R): a brief cognitive test battery for dementia screening. Int J Geriatr Psychiatry 2006;21:1078-1085.
8.
Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, Chertkow H: The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695-699.
9.
Brown J, Pengas G, Dawson K, Brown LA, Clatworthy P: Self administered cognitive screening test (TYM) for detection of Alzheimer's disease: cross sectional study. BMJ 2009;338:b2030.
10.
Larner AJ: Addenbrooke's Cognitive Examination-Revised (ACE-R): pragmatic study of cross-sectional use for assessment of cognitive complaints of unknown aetiology. Int J Geriatr Psychiatry 2013;28:547-548.
11.
Larner AJ: Screening utility of the Montreal Cognitive Assessment (MoCA): in place of - or as well as - the MMSE? Int Psychogeriatr 2012;24:391-396.
12.
Hancock P, Larner AJ: Test Your Memory (TYM) test: diagnostic utility in a memory clinic population. Int J Geriatr Psychiatry 2011;26:976-980.
13.
Moons KGM, Stijnen T, Michel BC, Büller HR, Van Es GA, Grobbee DE, Habbema DF: Application of treatment thresholds to diagnostic-test evaluation: an alternative to the comparison of areas under receiver operating characteristic curves. Med Decis Making 1997;17:447-454.
14.
Folstein MF, Folstein SE, McHugh PR: ‘Mini-Mental State.' A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189-198.
15.
Larner AJ: Comparing diagnostic accuracy of cognitive screening instruments: a weighted comparison approach. Dement Geriatr Cogn Disord Extra 2013;3:60-65.
16.
Larner AJ, Hancock P: ACE-R or MMSE? A weighted comparison. Int J Geriatr Psychiatry 2014;29:767-768.
17.
Davis DH, Creavin ST, Noel-Storr A, Quinn TJ, Smailagic N, Hyde C, Brayne C, McShane R, Cullum S: Neuropsychological tests for the diagnosis of Alzheimer's disease dementia and other dementias: a generic protocol for cross-sectional and delayed-verification studies. Cochrane Database Syst Rev 2013;3:CD010460.
18.
Larner AJ: Dementia in Clinical Practice: A Neurological Perspective. Pragmatic Studies in the Cognitive Function Clinic, ed 2. London, Springer, 2014, pp 94, 105, 117, 123, 124, 139, 181, 182.
19.
Larner AJ: Effect size (Cohen's d) of cognitive screening instruments examined in pragmatic diagnostic accuracy studies. Dement Geriatr Cogn Disord Extra 2014;4:236-241.
20.
Larner AJ: Impact of the National Dementia Strategy in a neurology-led memory clinic: 5-year data. Clin Med 2014;14:216.
21.
Alzheimer's Society: Dementia UK, ed 2, overview. London, Alzheimer's Society, 2014.
22.
Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement 2013;9:63-75.e2.
Copyright / Drug Dosage / Disclaimer
Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.
You do not currently have access to this content.