Criterion-Related Validity of the Cognitive Function Score with the Revised Hasegawa’s Dementia Scale and the Bedriddenness Rank with the Barthel Index and the Katz Index: A Multi-Center Retrospective Study

The cognitive function score (CFS) is a public scale for assessing the activities of daily living (ADL) in older adults with dementia in Japan. It is widely used in national healthcare and long-term care settings [1]. Our previous study established that the CFS exhibits high inter-rater reliability and demonstrated significant criterion-related validity compared with established dementia assessment scales, such as the mini-mental state examination (MMSE) and the ABC dementia scale [2, 3]. In essence, the CFS is a valuable objective scale for assessing dementia. In contrast, the scores of the revised Hasegawa’s dementia scale (HDS-R), an easy-to-use dementia screening tool developed in Japan, are significantly correlated with MMSE scores, which can assess dementia severity and are widely utilized in various countries [4, 5]. However, the HDS-R requires participants to indicate their age, date, and time, as well as recall words and name objects. Therefore, it is reliant on the cooperation of the participants and takes approximately 10 min to complete [6]. In contrast, the CFS offers a rapid assessment, requiring only a few steps to determine the ability of an individual to perform daily activities independently or their need for caregiving. Furthermore, it can be completed within a few minutes. There have been no prior reports on the criterion-related validity between the CFS and HDS-R. However, if significant criterion-related validity is established between these scales, it may facilitate the broader utilization of the convenient tool, CFS, and alleviate the burden on busy healthcare settings.

The Bedriddenness Rank (BR) is a public scale for assessing the independence of ADL in older adults routinely used in both Japanese healthcare and caregiving settings [1]. This is an easy assessment tool that can be completed in a few steps. However, there have been no reports on the inter-rater reliability of the BR or establishing the criterion-related validity between the BR and other ADL scales other than our previous research [2, 3], in which the BR demonstrated high inter-rater reliability [2] and significant criterion-related validity [3] compared with the Barthel index (BI) [7] and Katz index (KI) [8]. Nevertheless, these studies were conducted in only one middle-sized acute care hospital and one chronic care hospital. Therefore, to account for diverse backgrounds and contexts, it is essential to validate the BR in various hospital settings, including tertiary hospitals and other acute and chronic care facilities. This novel study aimed to elucidate the previously unexplored criterion-related validity of the CFS and HDS-R. Additionally, we verified the criterion-related validity of the BR with the BI and KI using data from multiple facilities, including tertiary hospitals and other acute and chronic care hospitals, which differ from the target population in previous research.

This was a multicenter retrospective study and a secondary analysis of the previous study [9]. The study included patients aged ≥20 years hospitalized in eight hospitals, ranging from chronic to tertiary care settings between April 1, 2018, and March 31, 2021. Patients meeting the following criteria were excluded from the analysis: those with BR categorized as normal but BI <10; BR categorized as normal but KI ≤1; BR categorized as B or C but BI = 100; BR categorized as B or C but KI = 6; normal CFS but HDS-R <5; and CFS of levels 3 or 4 but HDS-R >25, as these were considered input errors. Additionally, patients with missing data for the following variables: BR, BI, KI, CFS, or HDS-R, were excluded from the analysis.

We collected data from medical charts on admission, including age, sex, BR, CFS, BI, KI, and HDS-R scores. Additionally, we noted whether basic ADLs (including eating, moving, personal maintenance, going to the toilet, bathing, walking, going up and down stairs, dressing, defecation, and urination) were performed independently. The BR, comprising nine categories, was divided into five major categories (normal, J, A, B, and C). Furthermore, the CFS, comprising eight categories, was divided into six major categories (normal, I, II, III, IV, and M).

This study conformed to the Declaration of Helsinki and the Ethical Guidelines for Medical and Health Research Involving Human Subjects issued by the Japanese Ministry of Education, Culture, Sports, Science, and Technology. Written informed consent was not obtained, and an opt-out informed consent protocol was used for the use of participant data for research purposes. This consent procedure was reviewed and approved by the Ethics Committee of the Saga University Hospital, Approval No. 2021-07-R-07, date of decision; September 29, 2021. Consent was obtained from all patients using the comprehensive agreement method of the hospital, and their patients’ anonymity was protected.

Continuous variables are presented as medians and interquartile ranges, whereas categorical variables are expressed as absolute numbers and percentages (%). Correlations between the CFS and HDS-R score, as well as between the BR and BI or KI, were analyzed using Spearman’s rank correlation coefficients. The CFS and BR were treated as ordinal variables; however, the CFS category “M” was excluded from the ordinal analysis due to its distinct definition, which includes “significant psychiatric symptoms, behavioral problems, or serious medical conditions requiring specialized medical care,” deviating from the definitions of categories I to IV. The significance level was set at p < 0.05. Statistical analyses were performed using SPSS version 25 (IBM, Armonk, NY, USA).

During the study period, 150,278 patients aged ≥20 years were hospitalized. A total of 10,770 individuals were excluded due to suspicion of input errors. A total of 749 participants were included in the analysis of criterion-related validity between the CFS and HDS-R after excluding 18,021 patients with missing CFS data and 138,759 with missing HDS-R data. Furthermore, 93,432 patients were included in the analysis of criterion-related validity between the BR and the BI or KI after excluding 56,846 patients with missing data for the BR, BI, or KI.

In the CFS cohort analysis, 202 patients (27.0%) were categorized as having normal CFS, and the median HDS-R score was 18 (range: 6–26) (Table 1). The correlation coefficient between the CFS and HDS-R scores was −0.834 (p < 0.001) (shown in Fig. 1).

In the BR cohort, 4,717 patients (5.0%) were categorized as having normal BR; the median BI was 100 (range: 65–100), and the median KI was 6 (range: 2–6) (Table 1). The correlation coefficient between the BR and BI was −0.741 (p < 0.001), while that between the BR and KI was −0.740 (p < 0.001) (shown in Fig. 2).

In this study, the CFS demonstrated significant criterion-related validity, as evidenced by its alignment with the widely used HDS-R. Furthermore, the BR, in conjunction with either the BI or KI, exhibited significant criterion-related validity in populations with diverse backgrounds. These findings indicate that the CFS is an objective assessment tool for the severity of cognitive dysfunction, whereas the BR is an objective ADL scale that is available across various settings.

This study revealed a significant correlation between each CFS category and the HDS-R scores. As the HDS-R scores tend to be higher for milder levels of dementia and lower for severe dementia [4], the correlation between the HDS-R scores and CFS suggests that the CFS may be similarly used in assessing the severity of dementia. The CFS does not classify a state as normal unless it is clearly evident to any observer. If patients have mild cognitive impairment without a significant impact on daily life, they are assessed as having a CFS of 1 or worse. Therefore, it is likely that the median HDS-R score for the CFS group, which was higher than the dementia cutoff of 20 points, reflects this difference. Depending on whether the CFS is better or worse than level II, it indicates whether the dementia screening results obtained through the widely used HDS-R, with a cut-off score of 20 points, are positive or negative. Although the HDS-R presents operational burdens in assessment, the CFS allows for a simplified assessment process for the severity of cognitive dysfunction in a few steps. Previous studies have indicated the high inter-rater reliability of CFS assessments [2]. With the additional finding of criterion-related validity between the CFS and established dementia scales in this study, it may be concluded that the CFS is a highly valuable scale for the severity of cognitive dysfunction in healthcare and long-term care settings.

The BR revealed a significant correlation with both BI and KI in a diverse population, consistent with prior research findings. Similarly, in our studies [2, 3], patients with the BR categories of normal and J had approximately identical median BI and KI scores, specifically 100 and 6, respectively. This result aligns with previous research [3], which demonstrates that the BR provides a detailed evaluation and classification of patients who are closer to independent ADLs than the BI and KI. Furthermore, the BR is more advantageous as an efficient and practical assessment tool than the time-consuming BI and KI [3]. Therefore, the BR may be an objective and easy-to-use scale for assessing ADL in greater detail. However, the correlation between the BR and the BI or KI was slightly weaker than that reported in previous studies [2, 3]. This discrepancy may be attributed to the predominant acute care hospital patient population in this study, in which many patients experienced rapid changes in their health conditions. This leads to inaccuracies in ADL assessments due to significant and rapid fluctuations.

This study has some limitations. First, this was a retrospective study, and data accuracy may vary between facilities. Second, the administration of the HDS-R was conducted in only two facilities, potentially leading to a limited pool of patients and a selection bias. Third, similar to previous studies, the categorization of the BR and CFS into detailed subgroups was not achieved. Fourth, this study did not assess the inter-rater reliability of the CFS and BR. Fifth, the causes of cognitive impairment were not assessed, and the use of HDS-R for evaluating patients other than those with Alzheimer’s disease may have influenced the results. Sixth, although our study considered the actual medical context and focused on BR and CFS in adult patients aged 20 or older, these indicators were originally developed for older adults with disabilities or dementia, which could have potentially impacted the results. Lastly, most input errors were detected and excluded within the BR group due to issues with each hospital’s data input system. This fact may indicate a potential lack of reliability in the data for this study.

In conclusion, the CFS revealed significant criterion-related validity compared with the established cognitive assessment scale, the HDS-R. Additionally, the BR, in conjunction with either the BI or KI, showed significant criterion-related validity, even when applied to populations with diverse backgrounds. Furthermore, the CFS may be used as an objective cognitive function scale, whereas the BR is a valuable ADL scale in diverse settings.

We thank Miho Hayashida and Naoko Otsubo from Saga University Hospital; Kenta Yamaguchi, Yuka Hisamoto, Yasuhiro Chibu, and Toshinobu Eguchi from Yuai-Kai Foundation and Oda Hospital; Tomokazu Ichibakase from National Hospital Organization Ureshino Medical Center; Yoshihiko Nakashima and Kaori Hamai from Karatsu Municipal Hospital; and Yuriko Takao, Mika Tokushima, Yoshiro Nakayama, and Dr. Kozo Naito from Saga-Ken Medical Centre Koseikan for their assistance with data acquisition.

This study was approved by the Ethics Committee of Saga University Hospital (Approval ID. 2021-07-R-07, date of decision: September 29, 2021). This study conforms to the Declaration of Helsinki and the Ethical Guidelines for Medical and Health Research Involving Human Subjects issued by the Japanese Ministry of Health, Labour, and Welfare and the Ministry of Education, Culture, Sports, Science, and Technology. Written informed consent was not obtained, and an opt-out informed consent protocol was used for the use of participant data for research purposes. This consent procedure was reviewed and approved by the Ethics Committee of Saga University Hospital, Approval No. 2021-07-R-07, date of decision; September 29, 2021. Consent was obtained from all patients using the comprehensive agreement method of the hospital, and their patients’ anonymity was protected.

Masaki Tago is supported by grants from the Japan Society for the Promotion of Science, JSPS KAKENHI (Grant No. JP21H03166).

This work was supported by JSPS KAKENHI (Grant No. JP21H03166). The funder had no role in the design, data collection, data analysis, and reporting of this study.

Risa Hirata: conceptualization, methodology, validation, formal analysis, investigation, data curation, and writing – original draft. Naoko E. Katsuki: conceptualization, methodology, validation, formal analysis, data curation, and writing – review and editing. Hitomi Shimada: data curation and writing – review and editing. Eiji Nakatani: methodology, validation, formal analysis, writing – review and editing, and supervision. Kiyoshi Shikino, Maiko Ono, Chihiro Saito, Kaori Amari, Kazuya Kurogi, Mariko Yoshimura, Tomoyo Nishi, Shizuka Yaita, Yoshimasa Oda, Midori Tokushima, Yuka Hirakawa, Masahiko Nakamura, Shun Yamashita, Yoshinori Tokushima, Hidetoshi Aihara, and Motoshi Fujiwara: data curation and writing – review and editing. Masaki Tago: conceptualization, methodology, validation, writing – original draft, supervision, project administration, and funding acquisition.

The study was registered at the University Hospital Medical Information Network (UMIN) at www.umin.ac.jp (UMIN ID: UMIN000045420). The datasets generated and analyzed during the current study are available in the UMIN-ICDR repository, https://center6.umin.ac.jp/cgi-bin/icdr_e/ctr_view.cgi?recptno=R000050831.