The Better Assessment of Illness Study for Delirium Severity: Study Design, Procedures, and Cohort Description

Delirium, characterized by acute decline in attention and cognitive functioning, is associated with functional decline, prolonged length of hospital stay, institutionalization, higher healthcare costs, greater caregiver burden, accelerated cognitive decline, and higher mortality [1-3]. Research in delirium has advanced substantially over the past 2 decades, imparting in-depth understanding of its epidemiology, risk factors, and preventive strategies. Now, refinements in measurement approaches to delirium severity are critically needed to provide sensitive outcome measures for clinical trials and pathophysiologic investigations needed to advance treatments for delirium. Measures of delirium severity are essential to capture changes over time and therefore allow for monitoring of response to treatment, tracking clinical course and prognosis, and correlating with potential mechanistic factors, such as pathophysiologic biomarkers [4]. Quantifying delirium severity requires nuanced, finely grained, multi-domain measurements to capture its many features and estimate its effects on clinical outcomes that are meaningful to patients, family caregivers, healthcare providers and systems, and society at large [4].

Several delirium severity measures are in current use. The most widely used include the Delirium Rating Scale-Revised-98 (DRS-R-98) [5], the Memorial Delirium Assessment Scale (MDAS) [6], and the Confusion Assessment Method-Severity (CAM-S) score [4, 7]. While these measures are all useful, they have features that limit their utility in large sample size clinical epidemiologic research. Some are time consuming to administer, over-emphasize hyperactive features, require highly skilled clinician raters, or are not validated against clinical outcomes [4, 7, 8]. New delirium severity measures that address these limitations and fully cover important domains of the delirium construct, have strong performance characteristics, and predict important clinical outcomes are greatly needed [9, 10]. Thus, a group of interdisciplinary delirium experts launched the Better ASsessment of ILlness (BASIL) study to evaluate existing delirium severity measures and to develop new measures to address these unmet needs. Here we describe the BASIL study, including the study design, variables, procedures, characteristics of the cohort at enrollment, and data quality.

The BASIL study is an ongoing prospective cohort study of 352 hospitalized older adults, with planned 1-year follow-up of study participants. The focus of the BASIL study is the evaluation of existing delirium severity measures and subsequently the development of new measures utilizing expert panel and advanced measurement approaches in future work. The current paper focuses on the methods and description of the prospective cohort study.

Detailed description of the development and validation approaches for the new delirium severity measures is outside the scope of the present paper. In brief, we will follow rigorous approaches including expert panel processes to identify key items, advanced statistical methods to select and combine items, and examination of face and content validity and field testing in a new cohort to evaluate construct and predictive validity. The current paper is intended to provide a description of the cohort for the validation.

Patients age 70 years or older, who were English speaking, admitted or transferred to the medical or surgical services as either emergency or elective admissions and residing within 40 miles of the Beth Israel Deaconess Medical Center (BIDMC) in Boston were eligible for participating in the study. BIDMC is a large academic medical center with 673 beds, over 40,000 admissions and 10,000 operations per year. Eligible participants were identified initially by reviewing their medical records. Subsequently, approval to approach patients for potential enrollment into the study was obtained from participating hospitalists and surgeons. Exclusion criteria included inability to perform cognitive testing due to legal blindness or severe deafness, active alcohol abuse (more than 5 drinks per day for men, 4 drinks per day for women), alcohol withdrawal within the last 6 months, diagnosis of schizophrenia or active psychosis, nonverbal condition (e.g., aphasic, intubated), immediate discharge plans, or imminently terminal condition. To assure a clinically diverse study population in -BASIL, we monitored enrollment to include a minimum of 100 patients with dementia and 100 surgical patients in the final study cohort. Written informed consent was obtained from participants whenever possible. For participants who failed a standard capacity assessment but assented to participate, informed consent was obtained from the health care proxy either in person or by telephone. All study procedures were approved by the Institutional Review Boards of BIDMC and Hebrew SeniorLife, the study coordinating center.

Patients were enrolled between October 20, 2015 and March 15, 2017. Trained lay interviewers conducted initial evaluations within 48 h of hospital admission, followed by daily assessments during hospitalization, and follow-up interviews at 1-month and 12-month after discharge. Caregiver interviews were also conducted once during hospitalization, and at 1- and 12-month after discharge. The 12-month follow-up assessments are ongoing and are expected to reach completion during 2018. Study variables, time points of assessments, and sources of data are described in Table 1.

The initial assessment lasted about 45 min and resulted in the collection of demographic information, delirium status, and other study variables. After the initial assessment during participants’ index hospitalization, delirium and delirium severity were assessed daily with 10–15 min face-to-face interviews. Once 4 daily interviews had been completed (and if delirium negative), interviews went to every other day; if delirium was present, interviews continued on a daily basis. Brief telephone interviews were conducted with patients and caregivers 5–10 days after discharge for those participants who remained delirious at the time of hospital discharge, in order to assess for persistent delirium. All participants underwent 45-min interviews in their homes at 1- and 12-month after the index hospitalization to evaluate for delirium, cognition, and other study variables (described below).

Participants were asked to provide the name and contact information of a family member or caregiver who saw them regularly at home, who were familiar with their functional status, and who would participate in an interview either in-person or by telephone. During the participant’s hospitalization, a 10-min interview was conducted with this family member or caregiver to establish the participant’s pre-admission cognitive and physical functioning, assess for evidence of dementia, and determine any recent changes in mental status. At the 1- and 12-month follow-up assessments, family members or caregivers also underwent a 10 min interview in-person or by telephone to determine interval changes in the participant’s mental and functional status.

The BASIL assessment battery, described in Table 1, includes demographic and clinical characteristics, delirium and cognitive function measures, and other study variables including physical functioning, subjective health and well-being, clinical outcomes, and ratings of subjective distress (delirium burden) from nurses, patients, and caregivers. These variables will be described further below.

We used the CAM, CAM-S, MDAS, and DRS-R-98 for assessment of delirium and delirium severity. Multiple measures were obtained to assess for convergent validity. Brief cognitive tests were used to score all of the delirium severity measures, and included the Montreal Cognitive Assessment [11, 12] supplemented with additional cognitive tests (e.g., days of the week and months backwards) that have been widely used for delirium assessment [11, 12].

The CAM [13] consists of 10 operationalized items originally derived from the Diagnostic and Statistical Manual of Mental Disorders: acute onset and fluctuation, inattention, disorganized thinking, altered level of consciousness, disorientation, memory impairment, perceptual disturbances, psychomotor agitation or retardation, and altered sleep-wake cycle. The CAM diagnostic algorithm for delirium requires the presence of both (1) acute onset and fluctuation and (2) inattention, and either (3) disorganized thinking or (4) altered level of consciousness [13]. The CAM has been demonstrated to have a sensitivity of 94% (95% CI 91–97), specificity of 89% (95% CI 85–94), and inter-rater reliability of 0.70–1.00 in studies involving over 1,070 participants [14]. The CAM-S long form (based on ratings of 10 items from the full CAM) and CAM-S short form (based on ratings of 4 items from CAM diagnostic algorithm) scoring systems allow for the quantification of delirium severity. The CAM-S score is a sum of ratings of individual CAM features on a 3-point scale: 0 (absent), 1 (mild), or 2 (marked), except for acute onset or fluctuation, which is rated 0 (absent) or 1 (present). The CAM-S severity scores range from 0 to 19 for the long form, and 0 to 7 for the short form, with higher scores indicating greater delirium severity [4]. Diagnosis of delirium was made using the CAM diagnostic algorithm (above), without use of any specific CAM-S cut point.

The MDAS rates the severity of delirium using 10 items on a 4-point scale (0–3) with a possible total range of 0–30. MDAS items include reduced level of consciousness, disorientation, short-term memory, impaired digit span, reduced ability to maintain and shift attention, disorganized thinking, perceptual disturbance, delusions, psychomotor activity, and sleep-wake cycle disturbances. A score of 13 or higher is used to indicate delirium, and higher scores indicate greater delirium severity [6].

The DRS-R-98 utilizes all available sources of information including family, chart, and nurses to identify and rate the severity of delirium according to 13 severity items: sleep-wake cycle disturbance, perceptual disturbances and hallucinations, delusions, lability of affect, language, thought process abnormalities, motor agitation, motor retardation, orientation, attention, short-term memory, long-term memory, and visuospatial ability. For the purposes of the BASIL study, the DRS-R-98 scoring instructions were modified slightly (with more details added) for administration by trained lay interviewers rather than physicians. The severity ratings for each item range from 0 (no impairment) to 3 (severe impairment). An overall severity score of greater than 15 of a total 39 points from the original publication was used to indicate delirium, and higher scores indicated greater severity of delirium [5].

The Informant Questionnaire on Cognitive Decline in the Elderly was administered to family members to determine pre-hospital cognitive status for patients [15-17]. The Family CAM was also administered to caregivers to capture additional information relevant to the assessment of delirium and delirium severity [18, 19].

The physical function measures include basic Activities of Daily Living Scale (ADLs) [20], Instrumental ADLs (IADLs) [21], and the Physical Function Summary score derived from the Medical Outcomes Study Short Form-12 (MOS SF-12) [22]. The ADL scale assesses the ability to perform 7 basic care skills (bathing, dressing, grooming, feeding, using the toilet, transferring, and walking). The IADL scale assesses the ability to perform seven complex activities: using the telephone, arranging transportation, shopping for groceries, cooking, housekeeping, taking medications, and managing finances. The MOS SF-12 Physical Function Summary score evaluates moderate activities, such as moving a table, pushing a vacuum cleaner, bowling, or playing golf; climbing several flights of stairs; and regular work or volunteer activities [23, 24]. Falls, incontinence, poor oral intake, and grip strength were also assessed at multiple time points in the study (Table 1).

Subjective health and well-being measures included the General Health (MOS SF-12) [22] assessed at the initial patient interview and both follow-up time points. Subscales derived from the Neuro-QOL short form [25, 26], including Depression and Emotional and Behavioral Dyscontrol [26], were assessed initially, daily during hospitalization, and at each follow-up time point via patient interviews. Subjective ratings of pain levels and sleep quality were obtained daily during hospitalization. The Posttraumatic Stress Disorder scale [27, 28] was administered to the patient at 1- and 12-month follow-up interviews. Ratings of subjective distress (delirium burden) were obtained with closed and open-ended responses from nurses, patients, and family members.

Medical records were reviewed initially to screen for eligibility criteria. Subsequently, a comprehensive and standardized medical record review was conducted after hospital discharge by an experienced research physician to abstract demographic characteristics (e.g., age, gender, race, ethnicity, education, marital status, and living situation) and clinical information, including reason for hospitalization, admission diagnoses, medical comorbidities, abnormal laboratory results, surgical type, precipitating factors for delirium (e.g., medications, iatrogenic events, catheters or physical restraints), postoperative complications, intercurrent illnesses, length of stay, and death (Tables 1, 2). Chart evidence of delirium was collected using a validated approach [29, 30]. Based on the medical record information, Charlson Comorbidity Index score [31] and Acute Physiology and Chronic Health Evaluation II score were calculated at initial assessment [32].

Additional study variables included self-reported general health [22], collected at initial assessment and all follow-up time points (Table 1). Subsequently, information on vital status, re-hospitalization, and institutionalization, was obtained at all follow-up interviews with both participants and family members. At a future date, study data will be merged with healthcare utilization data to be obtained from the Centers for Medicare and Medicaid Services and death data from the National Death Index.

All study interviewers underwent at least 4 weeks of intensive training and standardization [29]. After didactic instruction by an expert trainer on how to administer questionnaires, trainees practice with peers and older adult volunteers, shadow an experienced interviewer during 4–5 interviews with study participants, and finally perform 4–5 interviews under observation by an experienced interviewer. Training continues until a trainee has conducted 2 consecutive interviews without errors. Coding questions are encouraged and discussed at mandatory weekly staff meetings to ensure standardized coding by all interviewers.

Data variables are described in a codebook located within a web-based database management system, Research Electronic Data Capture (REDCap). Derived variables are defined in Variable Definition Sheets with explanations of missing records and descriptions of any changes. All missing data were closely monitored to assess for any coding errors and to verify the absence of any systematic errors in data collection. In addition, ongoing and published analyses are cataloged and updated regularly.

REDCap was used to collect and track study data, provide -follow-up timelines for interviewers, and produce completion -reports that are reviewed weekly at each staff meeting [33]. All missing interviews and data are addressed at these meetings. The REDCap data entry forms are programmed to detect out-of-range values and avoid erroneous entries (e.g., skip patterns). Paper forms are used to collect parts of the interviews that require written tasks (e.g., clock drawing) or when Internet access is limited and REDCap cannot be accessed. All interview data must be checked immediately by interviewers prior to finalization. Finalized forms (both REDCap and paper forms) are then checked by a second independent rater to ensure completeness, accuracy, and internal consistency. Data quality reports are reviewed quarterly by the study investigators.

In total, 42 paired inter-rater reliability assessments of in-person interviews were conducted with 2 interviewers observing each participant simultaneously, and completing ratings independently, blinded to each other’s ratings. Functional measures (ADLs) and delirium (CAM, MDAS, and DRS-R-98) ratings were included in these assessments. Degree of agreement, kappa (measure of agreement), and weighted kappa (measure of agreement weighted by degree of disagreement) for all interview variables were assessed by individual item and for overall ratings (or score thresholds).

For the CAM, reliability was assessed for the 10 individual CAM features and the overall CAM rating. For the overall rating, an exact agreement was required for the presence or absence of each of the CAM features: acute change, inattention, disorganized thinking, and altered level of consciousness. For the individual CAM features, an agreement was required on the exact level (not present, mild, or marked) for each of the 10 features rated by the CAM. Exact agreement was required for the ADLs for each level of dependency (no help needed, help needed, completely unable to perform task), for the MDAS rating of each item (none, mild, moderate, severe) and for the DRS-R-98 rating of presence of each item. For the DRS-R-98, the rating was tailored to each question according to the DRS instructions. For example, for the item “abnormalities of thinking processes based on verbal or written output,” answering choices included: normal thought processes, tangential or circumstantial, associations loosely connected occasionally but largely comprehensible, and associations loosely connected most of the time. The presence of “abnormalities of thinking processes” was assigned for any answering choice other than “normal thought processes.”

All analyses were performed using SAS, version 9.3 (SAS Institute, NC, USA). All statistical tests were 2-tailed, and a p value of less than 0.05 was considered to indicate statistical significance.

A total of 352 participants were enrolled between -October 20, 2015 and March 15, 2017 (Fig. 1). The estimated proportion of eligible participants who were enrolled (response rate) was 67%, calculated according to standard procedures [30]. This response rate is in line with other observational studies of this type [29, 30]. Enrollment characteristics of the BASIL cohort are shown in Table 2. The average participant was 80.3 years old (SD 6.8), 203 (58%) were female, 48 (14%) non-white race, and 6 (2%) of Hispanic ethnicity. The mean number of years of education was 14.5 (SD 3.0) with the majority of patients (67%) having completed some college. Only 13 (4%) patients lived in nursing homes prior to admission. A total of 102 participants (29%) were admitted to a surgical service or underwent any type of surgical procedure during their hospital stay. On average, the patients were medically complex with Charlson Comorbidity Scores ≥2 (192 patients, 55%). One hundred and two participants (29%) had dementia at the initial assessment, defined by Informant Questionnaire on Cognitive Decline in the Elderly score of greater than 3.5 or ICD-10 code for dementia.

Each participant provided between 1 and 15 daily assessments while in the hospital. Only 8.5% of patient days that were eligible for delirium assessment were missed. The leading reasons for missed daily assessments include participant’s or family member’s refusal and participant’s unavailability due to procedures related to clinical care. The original cohort (n = 352) was reduced by 44 deaths between enrollment and 1-month interviews. The cohort that reached 1 month (n = 308) was then further reduced by 24 deaths, 25 unobtainable or refused interviews, and 6 completely lost to follow-up, yielding 256 remaining for the 12-month interview. Thus far, 132 (53%) of the 12-month interviews have been completed, and these are still ongoing. Overall, there have been 89 deaths (25%) and only 8 (2%) were completely lost to follow-up at 12 months. The major reasons for losses are lack of time, declining health and memory, or family member requests. The 25 initial refusals at 1-month are still being tracked for 12-month follow-up.

Missing data for major outcomes are detailed in Table 3. Only 1–3% of most major outcomes had missing data (i.e., delirium, basic activities of daily living, or readmission). On the IADL measure, 11% had missing data for strictly procedural reasons, since IADLs were not initially included in the 1-month patient interview.

In total, 42 paired interviews were completed for inter-rater reliability assessments. Inter-rater reliability was calculated on overall scores for all delirium measures (CAM, MDAS, and DRS-R-98) and ADLs. For delirium presence or absence across all measures and ADLs, agreement was perfect with a kappa of 100% (Table 4). Inter-rater reliability was also calculated on individual items, and ranged from moderate to complete agreement. Weighted kappas ranged from 0.79 (psychomotor agitation) to 1.00 (for 3 out of 10 items) for the 10 individual CAM features. For the 4 core features of delirium that are part of the CAM diagnostic algorithm, weighted kappas were 0.85 for inattention, 0.96 for disorganized thinking, and 1.00 for both acute change and altered level of consciousness. For physical functioning, all items had a weighted kappa of 1.00. For the MDAS, weighted kappa ranged from 0.65 (reduced level of consciousness) to 1.00 (for 2 out of 10 items), and for DRS-R-98 from 0.69 (lability of affect) to 1.00 (for 5 out of the 12 items).

This paper provides the first comprehensive description of the BASIL study methods and cohort, which we hope will prove useful for clinicians and researchers to interpret the study results, and to guide or assist future related investigations in older adults. This study will enable the evaluation and comparison of existing delirium severity measures, and ultimately, the development and validation of new instruments.

Unique strengths of the BASIL study include the measurement of delirium severity based on formal cognitive testing and using multiple, rigorous approaches. The study is also unique in utilizing qualitative approaches to explore patient and caregiver distress associated with delirium. The BASIL cohort is diverse in terms of racial and ethnic inclusion, in representing both surgical and medical patients, and in including patients with cognitive disorders (29% met dementia criteria for our study), substantial medical comorbidity (55% had Charlson Comorbidity Index ≥2), and functional impairment (80% had at least one ADL dependency). Death accounted for the majority of study attrition (n = 88, 25%) with only 8 patients (2%) completely lost to follow-up. Clinical outcomes have been and will continue to be carefully tracked through multiple data sources. Several important limitations are worthy of comment. First, there was no pre-admission baseline assessment of the study participants. Since the focus was to measure delirium severity during hospitalization, a pre-admission assessment was not considered a priority for the present study. Moreover, some study variables, such as detailed neuropsychological testing and frailty assessment would not be feasible in this acutely ill study population. Finally, given the detailed assessments and long-range follow-up required, the cohort is of moderate size at a single site, and thus, generalizability to larger samples in other settings will need to be assured in future studies.

This novel prospective study holds great potential to advance the conceptualization and measurement of delirium severity through evaluating and comparing existing measures, and ultimately testing the performance characteristics of new measures, including convergence with existing measures, and predictive validity for important clinical and healthcare utilization outcomes. Ultimately, we hope such new measures will provide responsive and finely grained outcome measures to advance the development of new treatment and management approaches through clinical trials, as well as to progress our mechanistic understanding of the complex pathophysiology of delirium. Thus, this study represents a critical next step that holds great promise to help move the field ahead.

We thank the patients, family members, nurses, and physicians at BIDMC who made this study possible. This paper is dedicated to the memory of Joshua Bryan Inouye Helfand.

This manuscript was funded by grants no. R01AG044518 (SKI/RNJ), R24AG054259 (SKI), K07AG041835 (SKI), P01AG031720 (SKI). Dr. Marcantonio’s time was supported in part by grants no. R01 AG030618 (ERM), R01AG051658 (ERM), and K24AG035075 (ERM); Dr. Fong’s time in part by R21AG057955 (TGF); and Dr. Hshieh’s time in part by R24AG054259 02S1 (TTH); all grants from the National Institute on Aging. Dr. Inouye holds the Milton and Shirley F. Levy Family Chair at Hebrew SeniorLife/Harvard Medical School.

(Presented in alphabetical order; individuals may be part of multiple groups, but are listed only once under major activity).

Sharon K. Inouye, MD, MPH (Overall PI, HSL, BIDMC, HMS); Richard N Jones, ScD (BRN).

Tamara Fong, MD, PhD (HSL, BIDMC, HMS); Tammy Hshieh, MD (BWH); Edward R. Marcantonio, MD, SM (BIDMC, HMS); Annie Racine, PhD (HSL, HMS); Eva M. Schmitt, PhD (HSL); Dena Schulman-Green, PhD (Yale University); Patricia A. Tabloski, PhD, GNP-BC, FGSA, FAAN (Boston College); Thomas Travison, PhD (HSL, HMS).

Tatiana Abrantes, BS (HSL); Brett Armstrong, MPH (BIDMC); Sylvie Bertrand, BA (HSL); Angelee Butters, MA (BIDMC); Madeline D’Aquila, BS (HSL); Jacqueline Gallagher, MS (BIDMC); Jennifer Kettell, BS (HSL); Jacqueline Nee, BA (HSL); Katelyn Parisi, BA, (HSL); Margaret Vella, BS (HSL); Guoquan Xu, MD, PhD (HSL); Lauren Weiner, MA (BIDMC).

Yun Gou, MA (HSL); Douglas Tommet, MPH (BRN).

Charles H. Brown, M.D. (Johns Hopkins);^{a, b} Sevdenur Cizginer, M.D. (BRN);^a Diane Clark, PT, DScPT, MBA (University of Alabama);^a Joseph H. Flaherty, MD (St. Louis University);^a Anne Gleason, B.S. (HSL);^{a, b} Ann M. Kolanowski, Ph.D., RN (Penn State);^{a, b} Karen J. Neufeld, MD, MPH (Johns Hopkins University);^a Margaret G. O’Connor, PhD (BIDMC);^a Margaret A. Pisani, M.D., MPH (Yale);^{a, b} Thomas Robinson, M.D. (University of Colorado);^{a, b} Joe Verghese, M.B., B.S. (Albert Einstein);^{a, b} Heidi Wald, M.D., MPH (University of Colorado);^{a, b} Sharon M. Gordon, Psy.D. (Vanderbilt)^b:

(a) Participated in the expert panel to identify delirium severity items; (b) participated in the expert panel to identify delirium burden items. Abbreviations: BIDMC, Beth Israel Deaconess Medical Center; BWH, Brigham and Women’s Hospital; BRN, Brown University; HMS, Harvard Medical School; HSL, Hebrew SeniorLife; PI, principal investigator.

T.T.H. and T.G.F. are Co-First authors. R.N.J. and S.K.I. are Co-Senior authors.