Sensitivity and Specificity of the Saint Louis University Mental Status Examination to Detect Mild Cognitive Impairment and Dementia in Chinese Population

With the increasingly aging population, the incidence of dementia also continues to increase. According to the International Alzheimer’s Disease Association, 1 person suffers from dementia every 3 s [1], and the World Health Organization estimates that there will be 152 million people with dementia worldwide in 2050, with approximately half living in Asia [2]. In Taiwan, 270,000 people were reported to suffer from dementia in 2017, and the number is expected to reach 830,000 in 2051 [3]. The total amount spent on dementia care was 1 trillion US dollars in 2018, and this figure is estimated to increase to 2 trillion US dollars in 2030 [2]. The increasing cost of healthcare for dementia impacts families and society and will be an economic burden for younger generations.

Mild cognitive impairment (MCI) is the transitional period between normal cognitive functioning and dementia. MCI often progresses to dementia with an annual conversion rate of approximately 14% [4]. According to the recommendations of the Alzheimer’s Association, an early diagnosis of cognitive impairment in a primary care setting can allow for early treatment and monitoring for the symptoms of dementia [5]. An early therapeutic intervention beginning during the year in which dementia is detected has been estimated to result in a 9.8% reduction in the cost of dementia care over 10 years through a longer duration in the mild stages and reduced time in the more costly moderate and severe stages. Therefore, dementia care costs can be significantly reduced through earlier detection and treatment [6].

The Saint Louis University Mental Status (SLUMS) examination is commonly used as a screening instrument to detect MCI in Western countries [4, 7, 8]. It has been shown to be easy to administer and to not require collateral informants to allow for the early detection of MCI among community-dwelling older adults in various countries including China, Poland, Egypt, and Turkey [9-12]. However, the SLUMS was originally developed using a US Veteran population, and the cutoff values may not be applicable to Chinese populations due to cultural diversity and different education levels between Western and Eastern countries. To date, no previous study has used specialty diagnoses with neurological testing to confirm the sensitivity, specificity, and discriminatory ability validity of the SLUMS in Chinese population. Although the SLUMS has been translated into Chinese, no cutoff points to detect dementia and MCI in Chinese population have been established [10, 13]. Cao et al. [10] found high correlations among the Montreal Cognitive Assessment (MOCA) [14], Mini-Mental State Examination (MMSE) [15], and SLUMS-Chinese version (SLUMS-C), but they did not determine cutoff SLUMS-C scores, and they suggested that further work is needed to explore optimal cutoff SLUMS scores for screening MCI and dementia in Chinese population. Therefore, the aim of this study was to evaluate the utility and diagnostic accuracy of the SLUMS-C in the diagnosis of dementia and MCI in Chinese population.

A cross-sectional design was conducted, and convenience sampling was applied; participants were recruited from the outpatient department of the neurology and psychiatric clinics during a doctor’s appointment from the 3 participated hospitals until the number of subjects reached a satisfactory amount. Totally, 450 patients who met the study criteria were invited and 367 subjects agreed to participate in the study with only 4 subjects excluded due to repeated data (n = 2) and taking antipsychotics (n = 2), separately. The inclusion criteria were individuals with the ability to speak Mandarin or Taiwanese and communicate with the researcher. Subjects who were unable to interact with the researcher due to dementia and emotional and other problems were excluded. Data were collected from March to December 2018.

After approval from the Institutional Review Board for human subject protection, the researcher contacted the administrators of 1 regional hospital and 2 medical centers for approval. The research purpose was explained, and informed consent was obtained from each patient and patient’s surrogate (if the patient was unable to make a decision due to advanced dementia) before the interview was performed.

Medical diagnoses of MCI and dementia from 3 certified neurologists and psychiatrists based on the clinical criteria of the Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) [16] and related neuropsychological testing including the Chinese version of MMSE [17] and Cognitive Abilities Screening Instrument (CASI) [18] served as the gold standard of SLUMS-C [13]. Inclusion and exclusion criteria of diagnosis for cognitive status are given in Table 1. The consistency of the diagnosis process was established prior to the beginning of the study. To avoid overestimating the SLUMS-C score for the subjects who were taking cholinesterase inhibitors to control dementia or antipsychotics, the research assistants confirmed with the subjects’ physicians the accuracy of the diagnosis before undergoing the SLUMS examination. Missing data were inspected by the researcher through calculating the total score of the SLUMS.

The consistency of administering MMSE and SLUMS-C by each researcher was established prior to data collection. In addition, 2 researchers observed 29 patients simultaneously to assess interrater agreement. Kappa values of 0.79–1 for MMSE and 0.97–1 for SLUMS were obtained with only the item “Which floor am I on?” for MMSE presenting a value of 0.79.

Demographic characteristics, including age, gender, educational level, marital status, religious belief, presence of chronic disease, and psychiatric medication use, were collected. The CASI and MMSE were used as the related neuropsychological tests for clinical criteria.

The SLUMS examination is a 30-point screening scale containing 3 categories with 18 questions: 3 in orientation, 9 in reasoning, and 6 in memory. Based on its original development in the USA, for individuals with less than high school education, the cutoff scores for MCI range from 20 to 24 with cutoff yield sensitivity and specificity values of 0.92–0.81 and 1.0–0.98, respectively. The cutoff score for dementia is <20, with cutoff yield sensitivity/specificity values of 1.00/0.98, respectively. For individuals with high school education and higher, the cutoff scores for MCI range from 21 to 26, with cutoff yield sensitivity and specificity values of 0.95–0.98 and 1.0–0.76, respectively. The cutoff score for dementia is <21, with cutoff yield sensitivity/specificity values of 0.96/1.00, respectively [4].

Data analyses were conducted using SPSS v17 (Chicago, IL, USA). Demographic characteristics and MMSE and SLUMS scores were presented using means, standard deviations, frequencies, and percentages. Receiver operating characteristic (ROC) curves were plotted using sensitivity and 1−specificity values. The diagnostic accuracy of the SLUMS was calculated as the area under the ROC curve (AUC). The AUC represents the mean sensitivity value for all possible values of specificity. The AUC was estimated using the Wilcoxon method and used to distinguish diseased and normal groups. It was also evaluated for statistical significance against the null hypothesis that the true AUC was 0.50. An AUC ranging from 0.7 to 0.8 was considered to be acceptable, 0.8–0.9 excellent, >0.9 outstanding, and 1 completely correct [19-21]. Cutoff SLUMS scores that best differentiated diagnostic groups were determined using the Youden index, which maximizes trade-offs between sensitivity and specificity [22]. We analyzed the data according to different educational levels as follows: university, senior high school, junior high school, and elementary school.

In total, 363 participants were recruited from 3 outpatient clinics of 1 local hospital and 2 medical centers, of whom 155 (42.7%) had normal cognitive function, 87 (24.0%) had been diagnosed with MCI, and 121 (33.3%) had been diagnosed with dementia. More than half of them were female (59.5%), and the mean age was 73.32 ± 10.55 years. Regarding educational level, 12.1% were illiterate and 37.5% had >9 years of education. The mean MMSE score was 24.16 ± 5.45, and the mean SLUMS score was 18.13 ± 7.46 (Table 2).

The SLUMS did not show satisfactory AUCs for the preliminary detection of normal cognitive status and MCI by different educational levels (Fig. 1). All AUCs <0.7 were as described below: (1) the AUCs were 0.32/0.47 and the optimal cutoff scores to determine MCI were 24/25.5, while these cutoffs yield sensitivity/specificity values of 1.00/0.08 and 0.90/0.19 for those who have ≤6 years of education and >6 years of education, respectively (Fig. 1a, b). (2) The AUCs were 0.36/0.48 and the optimal cutoff scores to determine MCI were 25.5/26.5, while these cutoffs yield sensitivity/specificity values of 1.00/0.06 and 0.86/0.20 for those who have ≤9 years of education and >9 years of education, respectively (Fig. 1c, d). (3) The AUCs were 0.40/0.46 and the optimal cutoff scores to determine MCI were 25.5/25.5, while these cutoffs yield sensitivity/specificity values of 0.98/0.09 and 0.77/0.27 for those who have ≤12 years of education and >12 years of education, respectively (Fig. 1e, f). (4) The AUCs were 0.42/0.54 and the optimal cutoff scores to determine MCI were 25.5/21, while these cutoffs yield sensitivity/specificity values of 0.93/0.14 and 0.57/0.71 for those who have ≤16 years of education and >16 years of education, respectively (Fig. 1g, h).

The SLUMS showed acceptable AUCs for the preliminary detection of dementia by different educational levels (Fig. 2). All AUCs > 0.7 were as described below: (1) the AUCs were 0.79/0.78 and the optimal cutoff scores to determine dementia were 14.5/18.5, while the cutoffs yield sensitivity/specificity values of 0.82/0.71 and 0.70/0.72 for those who have ≤6 years of education and >6 years of education, respectively (Fig. 2a, b). (2) The AUCs were 0.80/0.79 and the optimal cutoff scores to determine dementia were 14.5/17.5, while the cutoffs yield sensitivity/specificity values of 0.73/0.81 and 0.60/0.84 for those who have ≤9 years of education and >9 years of education, respectively (Fig. 2c, d). (3) The AUCs were 0.80/0.81 and the optimal cutoff scores to determine dementia were 15.5/19.5, while the cutoffs yield sensitivity/specificity values of 0.68/0.82 and 0.71/0.74 for those who have ≤12 years of education and >12 years of education, respectively (Fig. 2e, f). (4) The AUCs were 0.81/0.73 and the optimal cutoff scores to determine dementia were 15.5/22.5, while the cutoffs yield sensitivity/specificity values of 0.64/0.85 and 0.80/0.65 for those who have ≤16 years of education and >16 years of education, respectively (Fig. 2g, h).

The results of this study showed that the discriminatory ability of the SLUMS-C to detect dementia was excellent in our Chinese population. The SLUMS [4] could detect dementia by different educational levels, and an educational level of senior high school showed the best cutoff, sensitivity, and specificity. Dementia was identified at cutoff points of 18-19 for individuals with high school education or above and 15-16 for individuals with an education level below high school. The SLUMS-C has been shown to be able to distinguish various stages of cognitive status; however, previous studies have indicated that further work is needed to explore optimal SLUMS cutoff scores for screening MCI and dementia in Chinese populations [10, 23]. Using physician’s diagnoses with neurological testing as the golden standard to detect the sensitivity, specificity, and discriminatory validity of the SLUMS-C could be a good tool for dementia detection.

The original SLUMS has shown acceptable ROC curves for MCI in Western populations [4]. Cao et al. [10] reported that SLUMS scores could reflect normal cognitive function, MCI, and dementia in Chinese population and the SLUMS was shown to be better than the MMSE to determine cognitive impairment in an elderly Turkish population [9]. However, Cao et al. [10] suggested that the optimal cutoff point should be shifted from using 12 years of education as established in the original Western population to 6 years of education in Chinese population. Our findings were inconsistent with previous studies, and we found that the AUCs of the SLUMS to detect normal cognitive function and MCI were 0.58–0.60 by different educational levels. In addition, the optimal cutoff scores for SLUMS to detect MCI were 21.5 and 23.5 for <12 years of education and 12 years of education or above, respectively, which were lower than reported in an American study [4]. The optimal cutoff scores for the SLUMS to detect MCI were 17 and 25.5 for <6 years of education and 6 years of education or above, respectively. These cutoffs yielded sensitivity/specificity values of 0.70/0.64 for those with 6 years or less in education. The sensitivity was 0.9 but the specificity was only 0.27 for those who had 6 years or less in education. Therefore, based on the current study, we do not suggest that the optimal point cutoff should be shifted from using 12 years of education as in the original English version to 6 years of education in Chinese population [10]. Although the SLUMS was developed as a screening instrument to detect MCI in Western countries, it did not demonstrate acceptable AUCs for the preliminary detection of MCI in the present study. This may be due to cultural and educational differences in the test [23-25].

The original cutoff scores of the SLUMS for MCI in the USA were 21–26 for individuals with at least high school education and 19–24 for individuals with less than high school education [4]. However, the scores to detect MCI in the current study were 19–24 and 16–22 for individuals with at least high school education and less than high school education, respectively, which were lower than the original cutoff scores of the SLUMS for MCI in the USA. The prevalence of MCI will be higher if we used the original cutoff points to detect MCI in our Chinese population. A previous study also reported that non-white patients were more likely to be identified as having cognitive impairment by the SLUMS [23]. Racial and ethnic differences should be considered when using a screening scale in different countries, and the criteria and fitness of the scale need to be confirmed accordingly. This study acknowledged the use of the convenience sampling strategy to collect data from 3 hospitals in Taiwan which may limit the generalizability of the study.

Cognitive screening instruments using pencil and paper tests are easily applicable in community settings and can also assist in the differential diagnosis of cognitive status before medical care is required for dementia [26]. We suggest that the SLUMS-C could be a beneficial and convenient screening instrument to detect dementia in Chinese populations, although caution should be taken when using the SLUMS to detect MCI. Before the Chinese version can be confirmed to have satisfactory sensitivity/specificity and good ROC curves for cutoff scores to detect MCI, we recommend that the original cutoff scores should be used to detect MCI. After community screening, a comprehensive clinical evaluation including cognitive assessment, functional status, corroborative history, and imaging confirmation is needed.

The Institutional Review Boards of National Cheng Kung University Hospital in Tainan and Kaohsiung Medical University Hospital in Kaohsiung gave ethical approval for this study (B-ER-107-024 and KMUHIRB-SV(I)-20180003). All subjects and their guardians (if the patient was unable to make a decision due to advanced dementia) have given their written informed consent.

The authors have no conflicts of interest related to this work.

This project was funded by a grant from the Kaohsiung Medical University (KMU-Q107017).

We warrant that this manuscript has been read by all co-authors, and all authors contributed to this study and manuscript accordingly. Ya-Ping Yang: data collection and manuscript writing. Ying-Che Huang and Cheng-Sheng Chen: establishment of the gold standard of MCI and dementia, study design, and manuscript review. Yi-Ching Yang: study design, statistical consultation, and manuscript review. Jing-Jy Wang: project director, study design, and manuscript review.