Performance on Cognitive Screening Tests and Long-Term Substance Use Outcomes in Patients with Polysubstance Use Disorder

The prevalence of cognitive impairments among patients with substance use disorders (SUDs) is estimated to be between 30 and 80% [1, 2]. Such impairments may cause a loss of cognitive and behavioural flexibility and capacity to assimilate and engage in treatment programmes that often have an educative and cognitive emphasis [3, 4]. Indeed, previous findings suggest that cognitive impairments are associated with poorer SUD-treatment outcomes through lower recognition of problem use [5], lower treatment adherence [6], lower outpatient therapy attendance [7], a high dropout rate [8], relapse proneness [9], lower self-efficacy [10], and reduced disposition to change and desire for help [11, 12].

Abstinence is often considered a safe approach and the ultimate goal of SUD treatment [13]. Studies also suggest abstinence improves the quality of life, psychological distress, and executive function [14]. However, abstinence-based treatment may not be considered realistic for all patients with a SUD. For some, the primary clinical goal of treatment may rather be harm reduction, sustaining or improving daily life functioning, and preventing debilitating heavy substance use. In a clinical context, identifying long-term risk factors for abstinence and heavy substance use is paramount to tailor treatment to the patients’ needs. Although cognitive impairments have been associated with adverse short-term treatment outcomes from SUD treatment, the ultralong-term outcome trajectories and recovery patterns of patients with cognitive impairments are still largely unknown. In fact, longitudinal studies investigating cognitive predictors rarely exceed 12 months.

Treatment retention is considered a key predictor of treatment outcome and constitutes a considerable challenge in treating patients with SUDs [15, 16]. Treatment may significantly reduce substance intake, but remission with or without abstinence and treatment is common [17, 18]. McKellar, Harris, and Moos [19] found that cognitive impairment predicted treatment dropout but not substance intake 5 years after dropping out of treatment. Thus, predictors of treatment dropout or early relapse may not correspond to predictors of long-term outcomes. Further longitudinal studies on associations between cognitive function and treatment outcome are therefore strongly called for.

Accurate identification of cognitive impairments may be vital to enable personalized interventions. However, identifying such impairments is challenging in nonspecialized clinical settings. Performance on cognitive screening tests or self-reports of cognitive functioning may not give an accurate impression of the patient’s cognitive functioning and may even be more indicative of psychological distress than neurocognitive function [20‒22]. In addition, there may be discrepancies between a therapist’s clinical evaluation of neurocognitive status and performance on neuropsychological tests [23]. Although the gold standard entails a comprehensive neuropsychological examination, time constraints and the availability of personnel specialized in designing and interpreting results from these examinations are limited. Furthermore, anamnestic information may not be readily available, and patients may show variable motivation and attendance that impede the assessment efforts. As a result, service providers commonly rely on short screening instruments measuring broad cognitive domains. However, the criterion-related and, in particular, the ecological validity of such cognitive screening instruments in terms of long-term clinically relevant outcomes in patients treated for a SUD is not well established [24].

The overall objective of the present study was to evaluate the clinical value of including a set of well-known and commonly used cognitive screening instruments when patients with a polysubstance disorder (pSUD) are enrolled in a treatment program. Specifically, the present study aimed to (1) establish associations between cognitive impairments measured by the Montreal Cognitive Assessment^® (MoCA^®), Wechsler Abbreviated Scale of Intelligence (WASI), and Behaviour Rating Inventory of Executive Function – Adult version (BRIEF-A) and substance intake at follow-ups 1 and 5 years after enrolling in a treatment programme, and (2) examine the ability of the MoCA^®, WASI, and BRIEF-A to predict substance abstinence and heavy substance use in patients with pSUD at the two follow-ups. Accordingly, we hypothesize that cognitive impairment according to at least one of the screening instruments will be associated with increased substance use and predict non-abstinence and/or heavy substance use at 1- and 5-year follow-ups after enrolment.

This study is based on data from the Stavanger Study of Trajectories of Addiction (STAYER), a prospective longitudinal cohort study of neurocognitive, psychological, and social recovery in patients with SUD who started a new treatment sequence in the Stavanger University Hospital catchment area in Norway.

A total of 208 patients with SUD were recruited at convenience from 10 specialized outpatient and residential SUD-treatment facilities within the Stavanger University Hospital catchment area between March 2012 and January 2016. To be eligible for treatment in the Norwegian specialized SUD-treatment services, patients must meet the criteria for either a diagnosis of F1x.1 harmful use, F1x.2 dependency syndrome, or F63.0 pathological gambling as defined by the ICD-10 [25]. After a minimum of 2 weeks, a baseline assessment was performed to minimize contamination from drug withdrawal and acute neurotoxic effects from psychoactive substances [26]. Follow-up assessments were conducted after 1 and 5 years. Participants were compensated approximately EUR 40 for their participation. Trained research personnel of the STAYER research group collected all data. Clinicians working with the patient were naïve to the assessment results obtained in the current study.

The inclusion criteria were as follows: (a) patients enrolled in the treatment program to which they were admitted for at least 2 weeks; (b) patients who met the diagnostic criteria for F1x.1 or F1x.2; (c) patients over 16 years of age; and (d) patients who reported polysubstance use defined as the consumption of multiple substances within the last year before inclusion.

Demographic and neurocognitive data were obtained by conducting semi-structured interviews by asking the patients to fill out questionnaires and perform the selected cognitive tests at baseline. Substance intake was measured as part of the 1- and 5-year follow-up assessments [27].

The Montreal Cognitive Assessment (MoCA^®) gives an overall measure of cognitive function [28]. It samples behaviour across 14 performance tasks that engage multiple cognitive domains and is scored in integers to obtain a total score between 0 and 30. MoCA^® has demonstrated excellent sensitivity and acceptable specificity to identify mild cognitive impairment at a sum score equal to or below 25 [28]. MoCA^® has demonstrated good test-retest reliability, good internal consistency, and sensitivity in detecting mild cognitive impairment according to this cut-off value among patients with SUD [2, 24].

The Wechsler Abbreviated Scale of Intelligence (WASI) was included to estimate intellectual function [29]. The WASI comprises four subtests, two verbal measures of crystallized intelligence (vocabulary and similarities) and two nonverbal tests of fluent intelligence (block design and matrix reasoning). WASI subtests are similar to their Wechsler Adult Intelligence Scale – Third Edition [30] counterparts but include different items. The full-scale IQ (FSIQ) was selected to reflect a general intellectual function (“g-factor”). Cognitive impairment was defined as a FSIQ <86, which classifies participants with borderline intellectual disability as cognitively impaired [31].

The BRIEF-A, a self-report questionnaire with high ecological validity, was included to assess executive functioning in real-life situations [32, 33]. The BRIEF-A comprises nine subscales and three composite scores. We examined the validity scales of the BRIEF-A and utilized the cut-off scores, age norms, and validation criteria proposed by the original authors [32]. Elevated scores are associated with substance use status and numerous social adjustment indicators in patients with SUDs [34]. A t-score of ≥65 on the BRIEF-A Global Executive Composite (GEC) score was used to identify participants with cognitive impairment.

The Drug Use Identification Test (DUDIT) is a self-report screening tool to assess substance consumption, substance behaviours, and substance-related problems [35]. The DUDIT comprises 11 items that are reported on a five-point Likert scale ranging from “never” to “four or more times a week.” We used the four consumption items from the DUDIT (DUDIT-C) to gauge substance intake [27] and the DUDIT-C continuous scores when examining the association between substance intake and cognitive performance. In addition, we defined two substance intake categories: total abstinence DUDIT-C score = 0 and heavy substance use DUDIT-C score ≥7. In the original DUDIT protocol, subjects reported substance use over the past 12 months. In the current study, participants were enquired about substance intake pertaining to the past 3 months.

Assumptions of normality were evaluated by inspection of Q-Q plots and the Shapiro-Wilks test. To obtain optimal statistical power, we opted not to exclude cases listwise when some cognitive measures were missing or invalid. The DUDIT-C continuous scores were significantly skewed at the 1- and 5-year follow-ups (z-scores 4.56), and Mann-Whitney U test was performed to evaluate differences between-group means. The χ² test of independence was used to analyse group differences for the categorical variables. As multiple comparisons were made, Bonferroni adjusted p values were used to evaluate the statistical significance of study dropout and the outcome variables of abstinence and heavy use at the 1- and 5 year follow-ups. We ran separate logistic regression models with abstinence and heavy use at the follow-ups as the dependent variables and cognitive impairment defined according to the specific cognitive screening tool (MoCA^®, WASI, or BRIEF-A), age, and gender as predictors. Statistics were conducted using the statistical software package SPSS version 26 (IBM Corp., released 2019).

Of the 164 participants included in this study, 144 participants were available for the 1-year follow-up assessment, and 108 participants were available for the 5-year follow-up assessment. The flow of participants and available data are presented in Figure 1. Note, only one participant scored in the IQ range below 70 (IQ = 67).

Table 1 shows the demographic features of the sample at baseline, presented separately for the cognitively impaired and non-impaired groups. Patients with cognitive impairment were younger (Mdn = 24.0) than patients without cognitive impairment (Mdn = 27.0), U = 5,808.5, p = 0.028 when impairment was defined according to the GEC scale from BRIEF-A.

Regarding cognitive performance at baseline, 33% of the sample met the criterion for cognitive impairments according to MoCA^®, 18% according to WASI, and 60% according to BRIEF-A. At the 1-year follow-up, 54% of the sample reported total abstinence from substances. Conversely, 31% presented heavy substance use. At the 5-year follow-up, 64% of the sample reported abstinence from substances, while 25% presented heavy substance use. Nine (6%) participants dropped out of the study before the 1-year follow-up assessment, and 38 (23%) dropped out from the study before the assessment at the 5-year follow-up.

A statistically significant association between continuous DUDIT-C scores and overall cognitive performance measures was only found for MoCA^® at 1-year follow-up, where a Mann-Whitney U test showed a significant difference between patients with cognitive impairment (Mdn = 4) and cognitively non-impaired patients (Mdn = 0), U = 25,777, p = 0.043. We found no differences in abstinence or heavy substance use between patients defined with and without cognitive impairments according to the included cognitive screening tests. At α = 0.05, patients with cognitive impairment measured by WASI were more likely to drop out of the study than patients without cognitive impairment at the 5-year follow-up measurement χ² (1, N = 163) = 4.1, p = 0.043. However, this result lost statistical significance after Bonferroni correction (0.05/18 = 0.003). Table 2 presents substance use and study dropout at 1- and 5-year follow-ups, stratified according to cognitive impairment measured by MoCA^®, WASI, and BRIEF-A, and the total sample.

None of the predictors, including age and gender, were statistically significant in the logistic regression models exploring associations between the categorical substance use outcome variables (abstinence and heavy use at year 1 or year 5 follow-up) and cognitive impairment defined according to each of the cognitive screening tests (MoCA^®, WASI, or BRIEF-A) (shown in Table 3).

We examined the ability of three standard cognitive screening instruments to predict substance use 1 and 5 years after treatment initiation. As cognitive impairments are well-established risk factors for adverse SUD-treatment processes and outcomes [7, 8], we expected to find negative clinical outcome behaviour among patients defined as cognitively impaired according to at least one of the screening instruments. The present results partly confirmed this by showing a statistically significant association between cognitive impairment according to MoCA^® and substance consumption at the 1-year follow-up. Surprisingly, but in line with McKellar, Harris, and Moos [19], we did not find any disparities between cognitive impairment according to any of the cognitive screening instruments and any long-term clinically relevant substance use outcomes. Furthermore, according to MoCA^®, WASI, and BRIEF-A scores, cognitive impairment did not predict substance abstinence or heavy substance use at the 1- and 5-year follow-ups.

In this study, the frequency of cognitive dysfunction varies between screening instruments. MoCA^® identified a frequency rate of 33%, comparable to previous studies in SUD populations [2]. All participants with an impairment defined according to WASI had a FSIQ in the range of 50–85, labelled mild to borderline intellectual disability [31]. A frequency of 18% within this range is somewhat lower than prevalence rates of 30–39% reported in previous studies of patients with SUD [12, 31]. Lastly, the frequency of cognitive impairment defined according to BRIEF-A was 60%, comparable to the 63% frequency reported by McKowen et al. [36].

Abstinence was found to be common among all participants, regardless of cognitive impairment (54–64%). Conversely, 25–31% of all participants reported heavy substance use regardless of cognitive impairment. These findings align with some previous studies that have demonstrated a disconnection between cognitive impairment and behaviour considered relevant for successful SUD treatment, such as treatment retention, attendance, and substance use outcomes [19, 37].

Cognitive impairment is shown to be a risk factor for relapse during or shortly after treatment [31, 38, 39]. Furthermore, cognitive impairment has been shown to predict treatment dropout, which is a risk factor for relapse per se [8]. However, the current study suggests a limited value of using sum scores from standard screening instruments designed to assess broad cognitive domains, such as the WASI, MoCA^®, and BRIEF-A, as predictors of long-term substance use. Moreover, the inability of MoCA^® to predict long-term outcomes is of particular interest because it is commonly utilized in clinical settings, and studies are emphasizing its ability to detect cognitive impairments in SUD populations [24]. Other studies have suggested that performance on MoCA^® can be used to predict several clinically relevant outcome variables, such as dropout from residential treatment facilities [40]. However, the MoCA^® was not developed specifically to detect cognitive impairments in SUD populations. Some items may be redundant, and MoCA^® may not adequately test all cognitive functions relevant for SUD recovery. For example, the instruments utilized in the current study did not assess decision-making and emotion-driven response inhibition, although these cognitive components may be vital in predicting long-term substance intake [9, 41]. Indeed, this study classified participants as cognitively impaired or non-impaired based on a single aggregated cut-off which may have oversimplified the multidimensional nature of the neuropsychological functions and abilities required for SUD recovery [42]. Additional studies including more detailed information within MoCA^®, and the two other instruments included in the present study are thus called for.

The predictive value of cognitive impairment may be attenuated in patients with pSUD as they may have a more severe clinical profile than patients with a mono-substance use disorder. Compared with mono-substance users, polysubstance users have an earlier onset of substance use [43], are younger [44], have higher levels of psychological distress and personality disorders [43, 45], poorer social adjustment [46], and lower socioeconomic status [47]. Studies suggest that these characteristics are associated with an increased risk of dropout and relapse [8, 48‒52]. Generally, individuals with pSUD may be burdened and lack psychosocial resources to such an extent that cognitive functioning plays a subordinate role in long-term recovery. Alternatively, other psychosocial factors may play a more prominent role in later phases of the recovery process.

There are few screening instruments for cognitive impairment in adults with SUDs, and the predictive validity of current instruments related to key clinical variables is not sufficiently established [24]. SUDs are recognized as persistent diseases and limit the validity of outcome measurements from longitudinal studies of short duration. The current study is among the few studies examining long-term clinical outcomes among patients with co-occurring SUD and cognitive impairments. Moreover, this is the first study that has compared the predictive value of three standard clinical screening instruments on long-term substance intake in a representative cohort of patients with a SUD. The current study provides additional insight by utilizing clinically significant substance outcome categories.

The STAYER cohort represents a heterogeneous patient group recruited from several specialized and diverse SUD-treatment facilities. The universal access to health care in Norway allows for the collection of a more comprehensive sample relative to countries where care is privatized and costly. The study targets polysubstance users, representing up to 91% of treatment-seeking patients [53]. Thus, the study utilizes a highly representative and clinically relevant sample. This allows the results to be generalizable to the broader clinical SUD services. The STAYER research group has also been well funded and utilized elaborate tracking and follow-up strategies to ensure a high retention rate and few missing data entries [54].

The main limitation still concerns missing data, particularly at the 5-year follow-up. As with all longitudinal studies, missing data and a high attrition rate compromise the internal and external validity of the current study. Despite the efforts to ensure high retention, the research group could not obtain data on several participants at follow-up measurements, which adds to the study dropout attrition rate. This leaves a total of 34% of the patients without DUDIT-C results at the 5-year follow-up.

Although the cognitive assessments were performed a minimum of 2 weeks after substance cessation, the timeframe from detoxification to assessment may be too short for some participants to measure stable neurocognitive impairment. However, studies of long-term recovery have not always required 2-week substance abstinence [55]. In addition, the frequency of cognitive dysfunction according to MoCA^® and BRIEF-A found in the current study is comparable to results reported in previous studies in SUD populations.

The current study does not include data on substance intake dynamics before and between the assessments. Thus, substance use measurements reflect the participants’ substance intake only at a particular moment in time and may fail to capture the dynamic nature of recovery and relapse [56]. Although substance use may reflect a comprehensive understanding of recovery in substance use treatment, it is an insufficient requirement to conceptualize long-term recovery. Extensive changes pertaining to dimensions of connectedness, identity, meaning in life, occupation, and meaningful positive social relations are essential to handle the moderation of substance use [57] and should be considered a treatment goal per se.

We have not controlled for comorbid mental disorders. Affective states, such as dysphoria, depression, and anxiety, may be an integral and core functional element in SUDs [58]. Indeed, findings have shown that scores on WASI, MoCA^®, and BRIEF-A are associated with psychological distress in SUD populations [21, 22, 59]. However, this issue is of limited relevance here, as the purpose of the current study was to determine the extent to which results on the cognitive screening instruments can be used to predict long-term substance use outcomes, independent of aetiology.

The selected DUDIT-C cut-off score defining heavy substance use resulted in a modest sample size for that subsample. This might have contributed to driving the null findings at the 5-year follow-up. Furthermore, the sample included few patients with impaired intellectual functioning. The STAYER cohort was recruited using convenience sampling from a clinical setting, and is thus vulnerable to ascertainment biases by under-sampling patients in the lower end of intellectual functioning and with weak motivation for change. We did not find an increased dropout rate for participants with cognitive impairment according to WASI after Bonferroni correction. Nevertheless, it is still possible that a greater study dropout rate masks true differences in substance use behaviour in patients with impaired intellectual functioning due to low statistical power or that patients with impaired intellectual functioning and worse substance use outcomes had an increased risk of study dropout.

According to our findings, cognitive impairment's predictive value for long-term treatment outcomes may be limited. In a clinical context, this is an optimistic outcome due to the high frequency of cognitive deficits in the SUD population. The results from screening with instruments assessing broad cognitive domains at treatment initiation should be interpreted with caution when informing treatment strategies. Conclusions must be supported by medical history, psychiatric and functional assessment, and a more comprehensive neuropsychological assessment. Recent efforts to develop cognitive screening instruments for the SUD population are promising [60]. However, the predictive validity of clinically relevant variables is not sufficiently established. Consequently, there is a need to establish clinically viable instruments with well-established clinical and ecological validity to assess cognitive functions in the SUD population. In addition, trajectories in SUD recovery among individuals with and without cognitive impairments warrant further investigation.

The authors would like to extend their sincere gratitude to the participants in STAYER, and the staff of the participating clinical services, the KORFOR staff, and in particular, Thomas Solgård Svendsen, Anne-Lill Mjølhus Njaa, and Janne Aarstad, who collected all the data.

This study protocol was reviewed and approved by the Regional Ethics Committee West, University of Bergen, approval reference REK 2011/1877. The research was conducted according to its guidelines and those of the Helsinki Declaration (1975). All participants gave written informed consent.

The authors have no conflicts of interest to declare.

The Western Norway Regional Health Authority, Strategic Initiative for Substance Use Research, grant number 912003, and the Department of Research and Education, Division of Psychiatry, Stavanger University Hospital, funded this research.

Jens Hetland, Egon Hagen, and Aleksander Hagen Erga conceptualized and designed the study. Jens Hetland wrote the first draft and revised the manuscript. Jens Hetland and Aleksander Hagen Erga performed the analyses. Egon Hagen, Aleksander Hagen Erga, and Astri Johansen Lundervold made critical revisions of the manuscript. Aleksander Hagen Erga and Astrid Johansen Lundervold supervised the study. All authors contributed to the article and approved the submitted version.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. Further enquiries can be directed to the corresponding author.