The Prognostic Test Accuracy of the Short and Standard Forms of the Montreal Cognitive Assessment

Cognitive impairment, a common and challenging aspect of post-stroke recovery, affects up to two-thirds of survivors [1, 2]. The type of cognitive impairment can vary widely among individuals [3]. Given its multifaceted impact, early identification of cognitive impairments is crucial [4‒6].

Cognitive impairment is a critical concern in stroke care, and international guidelines recommend cognitive screening before hospital discharge [7]. However, standardized cognitive assessments have not yet become a standard component of stroke care in either clinical practice or research settings [8]. The Montreal Cognitive Assessment (MoCA) is a commonly used screening tool in clinical practice [9]. Originally developed to distinguish mild cognitive issues from dementia [10], the MoCA became a popular instrument in stroke care due to its high sensitivity in detecting mild cognitive impairment [11]. Administering the MoCA, especially in acute settings, can be resource-intensive, prompting the development of shorter versions [12]. Among these options is a short-form Swedish version of the MoCA (s-MoCA-SWE) that was developed based on data from cognitive screenings performed within 2–14 days post-stroke [13]. Demand for the use of the s-MoCA-SWE in acute stroke units has been high, but the instrument’s ability to classify cognitive impairment at a later stage after stroke needs to be evaluated.

We aimed to determine the prognostic accuracy of both the short and standard forms of the MoCA in predicting long-term cognitive recovery following a stroke. We investigated both objective and subjective cognitive outcomes at 6 months post-stroke.

This cohort study used data from the Efficacy of Fluoxetine – a Randomized Controlled Trial in Stroke (EFFECTS) trial (URL: https://www.clinicaltrials.gov; unique identifier: NCT02683213). EFFECTS was a multicenter, double-blind RCT involving 1,500 stroke survivors across Sweden (2014–2019) [14]. The EFFECTS trial was neutral in terms of the modified Rankin Scale as the primary outcome and for other outcomes such as cognition [14]. Hence, the data set could be used as one cohort for secondary analyses.

Patients eligible for our study were adults (≥18 years) diagnosed with stroke (ICD-10 codes: ischemic stroke I63 [classified based on the Oxfordshire Community Stroke Project Classification [15] or intracerebral hemorrhage I64), presenting with persistent neurological deficits at enrollment. Only those with complete baseline and 6-month MoCA cognitive assessments were included.

Baseline (2–15 days post-stroke) and 6-month follow-up data (±14 days) were collected through face-to-face interviews at local centers, where trained assessors performed cognitive assessments before randomization to EFFECTS. Physicians, nurses, and occasionally occupational therapists conducted these assessments.

Cognitive function was assessed using the Swedish translation of the MoCA (version 7.1) [10]. The score range on the MoCA is 0–30 points, with a higher score indicating better cognitive function. Cognitive impairment was defined as a MoCA score ≤25 [10].

The s-MoCA-SWE score was derived from the baseline MoCA assessment [13]. The s-MoCA-SWE score ranges from 0 to 16, and a score of ≤12 indicates impaired cognition [13]. The Stroke Impact Scale (SIS [version 3.0]) was used to identify the self-perceived impact on cognitive function [16]. The memory and thinking domain (mtSIS) was used as a proxy for self-reported cognitive function. A raw score on the mtSIS domain was transformed as follows: $domainscore=meanitemscore-1/5−1×100$⁠. The cutoff for good recovery was set at >81.5 [16].

Stroke severity was assessed with the National Institutes of Health Stroke Scale (NIHSS) [17]. Reperfusion treatment for ischemic stroke was defined as having intravenous thrombolysis, thrombectomy, or both.

We analyzed the baseline s-MoCA-SWE (threshold ≤12) and the MoCA (threshold ≤25) scores to distinguish between objective (MoCA-assessed) and subjective (mtSIS-evaluated) cognitive impairments at 6 months post-stroke. We constructed two contingency tables and calculated sensitivity, specificity, negative predictive values, and positive predictive values, along with 95% confidence intervals. For sensitivity analyses, we adjusted baseline and 6-month follow-up MoCA values for age and education level [18].

Using binary logistic regression, we assessed the ability of early post-stroke (2–15 days) s-MoCA-SWE scores to predict both objective and subjective impairments at 6 months. We developed separate regression models for the s-MoCA-SWE and the MoCA, with crude models including only the primary variables, and multivariable models adding sex, age, education, and stroke severity. We evaluated the regression models using calibration plots and the Brier score to verify their practical effectiveness. Additionally, ROC curve analysis evaluated the prognostic accuracy of baseline s-MoCA-SWE and MoCA scores for predicting cognitive function at 6 months post-stroke.

Our study cohort included 1,141 of the 1,500 EFFECTS participants who underwent baseline and follow-up MoCA assessments (online suppl. eFig. 1; for all online suppl. material, see https://doi.org/10.1159/000540372). Assessing those with and without MoCA data for analyses, there was no significant difference in sex distribution (p = 0.46), but the included patients were younger (mean age 70.8 vs. 72.7 years, p = 0.001) and had milder strokes (average NIHSS score 4.1 vs. 6.0, p < 0.001).

Demographic characteristics of the study participants are summarized in online supplementary eTable 1. At baseline, the prevalences of cognitive impairment were 71.7% (cutoff ≤12) according to the s-MoCA-SWE and 67.0% per the MoCA (cutoff ≤25) (Table 1). At 6 months after stroke, the prevalences of cognitive impairment were 55.6% according to the s-MoCA-SWE and 48.6% with the MoCA. The mtSIS showed a considerably lower prevalence of subjective cognitive impairment (23.5%) at the 6-month follow-up.

The sensitivity of the baseline s-MoCA-SWE score ≤12 for detecting objective cognitive impairment at 6 months (MoCA score ≤25) was 92.3% (Table 2). For comparison, the sensitivity of the baseline MoCA score ≤25 for detecting cognitive impairment at 6 months was 89.7% (Table 2). When the MoCA score was adjusted for sex and age, the sensitivity of the baseline s-MoCA-SWE remained higher compared with the sensitivity of the baseline MoCA (online suppl. eTable 2).

The sensitivity of the baseline s-MoCA-SWE score ≤12 for detecting subjective cognitive impairment at 6 months, with the mtSIS (threshold for impairment ≤81.5) as reference, was 81.5% (Table 3). By comparison, the sensitivity of the baseline MoCA score ≤25 for detecting objective cognitive impairment at 6 months was 78.1% (Table 3). When the MoCA score was adjusted for sex and age, the sensitivity of the baseline s-MoCA-SWE remained higher compared with the sensitivity of the baseline MoCA (online suppl. eTable 3).

For objective cognitive impairment at 6 months post-stroke, each one-point decrease in s-MoCA-SWE scores reduced impairment odds by 32% and MoCA scores by 28% (Table 4). After adjusting for demographics and stroke severity, these associations slightly decreased (s-MoCA-SWE OR = 0.70, MoCA OR = 0.74). For subjective impairment, initial analyses showed a significant, albeit weaker, association for the s-MoCA-SWE (OR = 0.88) and the MoCA (OR = 0.91), which persisted after adjustment. Calibration plots and ROC analysis results are in online supplementary eFigures 2–4 respectively.

Administered 2–15 days after a stroke, the s-MoCA-SWE showed sensitivity comparable to or greater than the MoCA for detecting those patients likely to experience cognitive impairments at 6 months post-stroke. While the s-MoCA-SWE had higher sensitivity for objective cognitive impairments, the MoCA maintained a better balance between sensitivity and specificity, especially in detecting subjective cognitive impairments. Overall, any differences seen between the prognostic utility of the two approaches were modest. In a busy acute stroke setting, given the potential time and resource savings, the comparable properties would favor greater use of the short-form assessment. Early use of cognitive screening in stroke can help to identify at-risk individuals, allowing for timely patient management, rehabilitation, and follow-up. Both the traditional and s-MoCA-SWE showed a high sensitivity for detecting those at risk of future cognitive issues. The ideal balance of test properties in this situation is debatable. Although a high sensitivity, risks overclassification, and the potential for distress and resource overuse related to false positive classification, it could be argued that an inclusive approach at baseline is preferable to a more specific tool that misses people who will go on to have later issues with cognition.

We observed an overall improvement in cognitive function over time, as evidenced by the decrease in cognitive impairment prevalence from 71.7% at baseline with the s-MoCA-SWE to 55.6% at 6 months and from 67.0% at baseline with the MoCA to 48.6% at 6 months. The observed prevalence rates and their decrease over time provide valuable insights into post-stroke recovery patterns. The mtSIS showed a considerably lower prevalence of cognitive impairment (23.5%) at the follow-up. In line with other studies, our results highlight differences in the assessment of subjective versus objective cognitive impairments [19]. The discrepancies between subjective and objective cognitive assessments in stroke survivors can be attributed to several factors such as psychological well-being, metacognitive abilities, or the nature of the assessments [19]. While objective tests quantitatively measure specific cognitive functions through standardized tasks, subjective assessments rely on personal experiences and perceptions, which can vary widely based on the lived experience of the stroke survivor. These factors underscore the complexity of post-stroke cognitive impairment and the need for multifaceted assessment approaches.

This study has both strengths and limitations. We utilized the same cohort and baseline data previously employed to develop the s-MoCA-SWE [13]. This approach may result in enhanced test performance compared to applying the scale in a truly independent cohort. However, it is important to note that the cognitive function at the 6-month follow-up was not analyzed in our study concerning the development of the s-MoCA-SWE. Having the MoCA as an outcome for the 6-month objective measure of cognition entails some inherent limitations. Although the MoCA is widely used for cognitive screening [20], it primarily assesses global cognitive function and may not capture more subtle changes in specific cognitive domains that can be important in a longer-term mild stroke population. In addition, we also used the mtSIS domain to capture subjective cognitive impairment. This domain mainly focuses on memory function, and it is possible that patients had other subjective cognitive problems that we did not identify. Additionally, subjective cognitive assessments have been shown to lack accuracy as measures of cognition since they are affected by other factors such as mood [21]. In this study, we used an indirect method to calculate the s-MoCA-SWE score, summing tasks from the MoCA according to a previously developed method [13]. It is worth noting that it is unclear whether the performance of patients directly taking the short version would yield the same psychometric measures as our derived data. Therefore, further research is needed to validate the direct administration of the s-MoCA-SWE and confirm its reliability and accuracy. Such a study would also allow for an assessment of the feasibility and acceptability of the shorter test.

The exclusion of patients with missing MoCA data was a practical necessity, though these patients were often older and had more severe strokes. This selection bias might limit the generalizability of our findings and affect the study’s external validity, as cognitive screenings early post-stroke are unsuitable for severely affected, bedridden, or cognitively impaired patients [22, 23]. Additionally, the lack of information on pre-stroke cognitive status – which significantly influences post-stroke recovery – requires careful consideration, especially since trajectories of recovery can differ significantly between patients with and without pre-existing cognitive issues [24]. Our study performed a secondary analysis of data from the EFFECTS trial, a randomized controlled trial where patients received either fluoxetine or placebo. The primary outcome, the modified Rankin scale, showed no difference between groups, nor did fluoxetine significantly impact cognitive function [14]. Therefore, we treated the EFFECTS data as a single cohort.

Our results suggest that the s-MoCA-SWE, administered 2–15 days post-stroke, maybe a time-efficient alternative to the full MoCA, allowing for quicker cognitive screening without significantly compromising the detection of cognitive impairment. However, prospective studies are needed to validate our results.

We would like to acknowledge the patients, their families, and the healthcare professionals involved in EFFECTS.

This study protocol was reviewed and approved by the Swedish Ethical Review Authority, Approval No. 2023-00356-02. The EFFECTS protocol was reviewed and approved by the Research Ethics Committee in Stockholm, Approval No. 2013/1265-31/2 and by the Swedish Medical Agency, Approval No. 5.1-2014-43006. The Declaration of Helsinki was followed. Written informed consent was obtained from participants to participate in the study.

The authors have no conflicts of interest to declare.

The study was supported by NEURO Sweden (F2023-0096), the Swedish state under the ALF agreement between the Swedish government and the county councils (ALFGBG-983604, ALFGBG-965653), the Sahlgrenska University Hospital’s Foundation (SU-997824), Promobilia, the Swedish Brain Foundation, Norrbacka-Eugenia Foundation, Hjalmar Svensson Foundation (HJSV2024018), Insamlingsstiftelsen för neurologisk forskning (2024-00034), and Wilhelm and Martina Lundgren’s Vetenskapsfond (2024-SA-4384). The funder had no role in the design, data collection, data analysis, and reporting of this study.

Design: T.A., T.Q., and E.L. Methods: T.A., O.B., A.M., E.Li, K.S.S., and T.Q. Subject recruitment and data collections: E.L. and K.S.S. Analysis: T.A., O.B., and A.M. Drafting of the manuscript and preparation of paper: T.A. Revising the manuscript critically for important intellectual content: O.B., A.M., E.Li., E.L., K.S.S., and T.Q. Final approval of the version to be published: all authors.

The data file contains potentially identifying and sensitive patient information. According to the Swedish Ethical Review Authority (https://etikprovning.se/for-forskare/ansvar/), the datasets generated in this study cannot be made publicly available for ethical and legal reasons. Further inquiries can be directed to the corresponding author.