The External Validation of the Nursing Homes Short Depression Inventory in Older Adults with Major Neurocognitive Disorders in Long-Term Care Centers

Depression is characterized by psychological distress accompanied by somatic and cognitive manifestations [1]. In long-term care (LTC) centers, depression affects up to 37% of patients [2] and is associated with decreased quality of life [3, 4] and increased morbidity [5‒8], mortality [9, 10], and costs [5, 11].

Identifying depressive manifestations is often difficult in patients with major neurocognitive disorders (MNCD), as the latter presents overlapping manifestations and may limit an individual in communicating or recalling his symptoms [12]. To facilitate assessments, the Cornell Scale for Depression in Dementia (CSDD), a patient-reported and observer-rated measure, was developed [13]. However, although being the benchmark scale in patients with mild-to-moderate neurocognitive disorders [14], the CSDD’s mixed method of administration prolongs its completion time (20–30 min [13]), and the patient-reported component limits its applicability in MNCD patients [15]. To reduce these challenges, the CSDD has been used solely as an observer-rated scale, but with limited accuracy [16].

To overcome these limitations, Prado-Jean et al. [17] developed the Nursing Homes Short Depression Inventory (NH-SDI), a short observer-rated scale designed for cognitively impaired LTC patients. In a sample of 99 French LTC patients, the NH-SDI demonstrated excellent screening accuracy, outperforming the CSDD [17, 18]. However, to the best of our knowledge, the NH-SDI’s validity was assessed in only one study, and little information on its reliability exists [16‒18], making it unclear whether it may be applied to other clinical contexts or instead of the CSDD. This study aimed to establish the external validity and reliability of the NH-SDI in LTC patients with MNCD and to compare its estimates to those of the CSDD, the most used scale for detecting depressive manifestations in MNCD patients.

We conducted a validation study in two Quebec (Canada) LTC. LTC-1 contained three regular continuing care units (128 beds) and one secured continuing care unit for ambulating patients with MNCD (24 beds); LTC-2 contained four regular continuing care units (156 beds). Patients at both sites had altered cognitive and/or mobility functions and required help with activities of daily living [19, 20]. Bedside staff included registered nurses (RN), licensed practical nurses, and orderlies [21]. Patient-knowledgeable RNs and physicians from both sites collaborated on data collection (Fig. 1).

A convenience sample of experts in geriatrics or measurements was assembled. Then, we recruited a convenience sample of patients based on the inclusion criteria of residing in a participating LTC for at least 1 month, having a diagnosis of Alzheimer MNCD, vascular MNCD, or a mix of both, and having their legal representative consent for them to participate in the study. The Research Ethics Committee at CIUSSS de l’Estrie-CHUS authorized this study.

The Mini Mental State Examination (MMSE) is an 11-item scale designed to detect the severity of cognitive impairment based on patient interviews [22]. The MMSE measures 6 aspects of cognition, and the total score indicates whether a patient has severe (0–9), moderate (10–19), mild (20–25), or no cognitive impairment (≥26) [22‒24]. The MMSE has a pooled sensitivity of 88.3% and specificity of 86.2% [25].

The Neuropsychiatric Inventory Questionnaire (NPI-Q) is a 12-item observer-rated scale designed to capture the severity of neuropsychiatric manifestations in MNCD (also known as behavioral and psychological symptoms of dementia [BPSD]) and caregiver distress [26‒29]. Items on the BPSD severity subscale are measured on a four-point scale (0 = absent; 1 = mild; 2 = moderate; and 3 = severe). The total score ranges from 0 to 36, with higher scores indicating greater severity [26‒29]. The NPI-Q presents moderate-to-strong correlations with the NPI [26, 30‒32].

The CSDD is a 19-item observer-rated and patient-reported scale [13, 33‒35] aiming to assess the severity of depressive manifestations over the previous week on a four-point Likert scale (a = unable to evaluate; 0 = absent; 1 = moderate/intermittent; 2 = severe). The total score ranges from 0 to 38, and higher scores suggest clinical depression [13, 33].

The NH-SDI is a 16-item observer-rated scale [17, 18] aiming to assess the presence of depressive manifestations over the previous 2 weeks on a dichotomous scale (0 = absent; 1 = present). The total score ranges from 0 to 16, and higher scores suggest clinical depression [17, 18].

Significant clinical depression was used as the reference standard, which was defined as enough depressive manifestations for a physician to decide on beginning treatment [36]. Although physicians were instructed to use Diagnostic and Statistical Manual of Mental Disorders criteria, Fifth Edition (DSM-V) [1], the severity of the cognitive impairments limited their applicability.

First, a focus group discussion was conducted with domain experts to examine the NH-SDI’s content validity. Next, RNs received training on the use of the NH-SDI, CSDD, and NPI-Q by the first author (É.T.) and physicians on diagnosing depression (based on DSM-V criteria) in MNCD elderly patients by the fourth author (J.R.D.), a geriatric psychiatrist. From May 2022 to the end of September 2022, participating patients were assessed by the first author with the French version of the MMSE and observer-rated by a RN with the French versions of the NH-SDI, CSDD, and NPI-Q. To assess interrater reliability, 52 patients were assessed by a second RN. Within 2 weeks after RNs’ assessments, patients were assessed by their family physician for significant clinical depression. Physicians’ and RNs’ assessments were blinded.

A content analysis [37] of the experts’ discussions on the relevance, comprehensibility, exhaustiveness [38], and applicability of the NH-SDI was conducted. Then, descriptive statistics were used to summarize patient characteristics. Sensitivity, specificity, positive (PPV) and negative (NPV) predictive values of the NH-SDI and CSDD were generated against the reference standard and their cut-off yielding the highest combined sensitivity and specificity was identified [39]. An area under the receiver operating characteristic curve (AUC) was generated [40] for both scales and compared [41]. Spearman rank correlations [42] were used to assess the scales’ construct validity and intra-class correlations (ICC) for interrater reliability [43]. For all analyses, p < 0.05 was the criterion for statistical significance. VassarStats [44] was used to generate the 95% confidence intervals (CIs) [45] of screening accuracy estimates and IBM SPPS statistics version 28.0 for other analyses [46].

Six experts (1 MSc in measurement, 5 in geriatrics [1 PhD; 1 RN; 2 MDs; and 1 nurse practitioner]) assessed the content validity of the NH-SDI. Overall, experts agreed that measuring depression in the elderly with MNCD is complex and that some items on the NH-SDI may be difficult to comprehend or may overlap with other conditions (e.g., BPSD) (online suppl. material; for all online suppl. material, see https://doi.org/10.1159/000533357). Nonetheless, some mentioned that the NH-SDI could enhance the reliability of RNs’ assessments and structure their reports of manifestations to physicians. They recommended minor linguistic revisions to adapt items such as “is worried” and “is worse in the morning” for French-Canadian users and requested that user instructions indicate that signs captured must be newly present (2 weeks) or deteriorated to be considered positive.

Of 332 patients screened for eligibility (166 at LTC-1; 166 at LTC-2), 186 met our eligibility criteria (97 at LTC-1; 89 at LTC-2); however, legal representatives for 68 patients declined participation or did not respond. The final sample, thus, consisted of 93 patients (Fig. 1) who did not differ significantly from other eligible patients regarding the MNCD type. The typical patient in our sample was an 89-year-old female with Alzheimer’s disease, ten comorbidities, and apathy as her most frequent BPSD (Table 1).

The NH-SDI’s median (Md) (interquartile range [IQR]) completion time was 3 min (2–5) and shorter (p < 0.001) than the CSDD’s (10 min [6‒11]). Sixteen patients had missing values on the NH-SDI or the CSDD for items that could not be assessed. These patients were more cognitively impaired (p = 0.003). The NH-SDI’s missing values were “expresses despair and pessimism” (n = 15), “is worried” (n = 13), and “has delusional ideation” (n = 16). The CSDD’s missing values were “suicide” (n = 16), “self-depreciation” (n = 16), “pessimism” (n = 15), and “mood-congruent delusions” (n = 16).

The NH-SDI achieved a Spearman’s rho correlation (r_s) of 0.913 (p < 0.001) with the CSDD and r_s = 0.303 (p = 0.003) with the NPI-Q. The CSDD achieved a r_s = 0.416 (p < 0.001) with the NPI-Q (see online suppl. material 2).

A reference standard assessment was available for 57 of the 93 participating patients (Fig. 1). Compared to the whole sample, this subsample comprised fewer females and psychiatric comorbidities, and more cases of severe cognitive impairment (Table 1). Best accuracy estimates were found when missing values in the NH-SDI and CSDD were replaced by 0.

Within the subsample of 57 patients, five (8.8%) had significant clinical depression (Table 1). Using the cut-off ≥3, the NH-SDI identified 12 (21.0%) patients as depressed (Table 1) and achieved 100% (95% CI: 46–100%) sensitivity, 83% (95% CI: 69–91%) specificity, 36% (95% CI: 14–64%) PPV, and 100% (95% CI: 90–100%) NPV (Table 2). The NH-SDI performed significantly better than the chance at discriminating depressed from nondepressed patients (AUC: 0.923 [95% CI: 0.846–0.100]; p < 0.001).

Using the cut-off ≥3, the CSDD identified 16 (28.1%) patients as depressed (Table 1) and achieved 100% (95% CI: 46–100%) sensitivity, 75% (95% CI: 61–86%) specificity, 28% (95% CI: 11–54%) PPV, and 100% (89–100%) NPV (Table 2). The CSDD performed significantly better than the chance at discriminating depressed from nondepressed patients (AUC: 0.846 [95% CI: 0.733–0.959]; p = 0.000). No significant difference in both scales’ AUCs was identified (dAUC: 0.077 [−0.008–0.162]; p = 0.077) (Fig. 2).

A second nurse assessment was available for 52 patients. The ICC was 0.765 (0.59–0.87; p < 0.001) for the NH-SDI, 0.774 (0.61–0.87; p < 0.001) for the CSDD, and 0.729 (0.53–0.85; p < 0.001) for the NPI-Q.

This study aimed to expand Prado-Jean et al.’s [17, 18] work by examining the external validity and reliability of the NH-SDI in LTC patients with MNCD, and comparing estimates to those of the CSDD. Overall, our experts agreed that depression in MNCD patients is characterized by atypical signs and that some items on the NH-SDI may overlap with other conditions (e.g., BPSD). This concern was corroborated by the significant correlation found between the NH-SDI and NPI-Q. Moreover, experts recommended minor changes to item wording and user instructions to improve comprehensibility. Good interrater reliability scores suggest that these concerns were adequately addressed.

As for criterion validity, the NH-SDI and the CSDD both presented excellent sensitivity, good specificity, and equivalently discriminated depressed from nondepressed patients. However, considering the NH-SDI’s significantly shorter completion time, an occupied LTC nurse could favor its use over the CSDD. In addition, we observed that both scales have high NPV but low PPV, which suggests that they might be better for ruling out a depressive state than ruling it in [47]. Indeed, while they are likely to capture nearly all true-positive cases of depression, this will be at the expense of many false-positives, who may undergo further unnecessary diagnostic workup. However, unless antidepressant prescribing is initiated [48], such investigations are unlikely to be of any harm, and non-pharmacological interventions may be beneficial to patients.

Globally, our results are consistent with those of other studies on the accuracy of observer-rated depression-screening scales in LTC patients with MNCD [16, 49]. Our findings contrast, however, with those of Prado-Jean et al. [17, 18], who reported an 86% PPV and 85% NPV for the NH-SDI, a discrepancy likely attributable to variations in depression prevalence [47] (52.5% in Prado-Jean et al. [17, 18] vs. 8.8% in this study).

While low, the prevalence of depression in our study is in line with that reported in a recent Quebec (Canada) study (7.6%) [50]. Nonetheless, both statistics are 2 to 3 times less than those reported in other Canadian or abroad LTC (approximately 30% [2, 49]). Although we may not exclude the possibility of missed diagnoses [51], the low prevalence of depression in Quebec LTC could potentially be explained by different operational definitions of depression [16], positive impacts of governmental efforts in offering daily recreational activities in LTC [52] or in delaying LTC admissions [53], and high usage of antidepressants [51], all of which warrant further investigation.

Last, there is variability in the most optimal cut-off values found across studies (CSDD: ≥2 to ≥6 [54‒56]; NH-SDI: ≥3 to ≥5 [17, 18]). Variability could be attributable to the flexibility/strictness of the reference standard [57] (e.g., significant clinical depression vs. diagnostic criteria) or patient characteristics (e.g., MNCD type) [58]. Given this variability, future validation studies should keep reporting results for a variety of cut-off scores and patient characteristics, and criterion validity should be reassessed prior to using the NH-SDI or the CSDD.

A physician shortage and heavy workloads implied that a reference standard was obtained for only a subset of our sample. Since these patients differed to some extent from our whole sample (e.g., sex, psychiatric comorbidities), our criterion validity results may be generalizable to only a subset of our study population. Moreover, the small sample available for our reliability and criterion validity analyses resulted in wide CIs, thus imprecise estimates. Validation of the NH-SDI and CSDD using larger and more representative samples of LTC patients is warranted.

In conclusion, the NH-SDI is quick to administer, shows evidence of validity and reliability, and has the potential to become a useful and better observer-rated scale than the CSDD to detect depressive signs in LTC patients with MNCD. Further validation of the instrument is warranted prior to recommending its use in LTC.

We would like to thank all RNs and physicians who collaborated on patient assessments.

This study protocol was reviewed and approved by the Research Ethics Committee at the CIUSSS de l’Estrie-CHUS, approval number 2022-4512. Written informed consent was obtained from the expert participants, as well as from the legal guardians of the patient participants.

The authors have no conflicts of interest to declare.

This study was funded by Pr. Christian M. Rochefort’s research laboratory, which receives funding from the Canadian Institutes of Health Research. É.T. holds masters’ degree scholarships from the Quebec Ministry of Education (Ministère de l’Éducation et de l’Enseignement Supérieur du Québec) and from the Quebec Network on Nursing Intervention Research. D.C. holds masters’ degree scholarships from the Canadian Institutes of Health Research (Frederick-Banting and Charles-Best Canada Graduate Scholarship), the Research Center on Aging (Centre de recherche sur le vieillissement de l’Université de Sherbrooke), the Ministry of Education (Ministère de l’Éducation et de l’Enseignement Supérieur du Québec), and the Quebec Network on Nursing Intervention Research. These funding sources were not involved in study conception and design, data acquisition, analyses, and interpretation, or in the final decision to submit this manuscript for publication.

Conception and design of the study: É.T., D.C., M.P.V., J.R.D., and C.R.; acquisition of data: É.T. and D.C.; analysis and interpretation of data: É.T. and C.R.; drafting, critically revising for intellectual content, and approving the final version of this article: É.T., D.C., M.P.V., J.R.D., and C.R.; agreeing to be accountable for all aspects of the study: É.T., D.C., M.P.V., J.R.D., and C.R.

Research data are not available due to privacy or ethical restrictions. Further inquiries can be directed to the corresponding author.