Psychometric Properties of the Cannabis Abuse Screening Test in Hungarian Samples of Adolescents and Young Adults

Cannabis is the most widespread illegal substance in the world [1]. In 2008, global marihuana production was estimated to be between 13,300 and 66,100 tons, while the production of hash reached figures of between 2,200 and 9,900 tons [2]. According to the United Nations survey, 2.8-4.9% of the world's population had used the substance in 2009. Accordingly, cannabis is also the most widely used illegal substance in Europe. Irrespective of significant differences between countries, studies have reported 10-30% lifetime prevalence rates; moreover, it has been found that young people are most likely to use the substance [3]. Between 2003 and 2008, an increasing trend of stagnation was observed in European countries but regional differences were also identified. While prevalence rates are typically higher in Western European countries, increase has been more notable in Eastern European countries [1]. In the second half of the 1990s, the increase in prevalence was significantly higher in post-communistic countries than in Western European countries (10.3% on average vs. 2.9% in Western Europe) [4]. Characteristically, we have only anecdotal evidence of cannabis consumption from Eastern European countries from the 1960-1970s. In addition, the lack of systematic data collection marked the 1980s as well. Following the rapid political and economic transformation in many of these countries (including Hungary) new drugs became easier to access and afford. As a result, cannabis use has become more or less normalized among youths in most Eastern European countries [5]. Since the tetrahydrocannabinol content of marihuana increased significantly in the past years, the use of marijuana became even more problematic [6].

Based on epidemiologic studies conducted in the last two decades, cannabis use seemed to increase in Hungary as well. According to the results of the Hungarian study of the European School Survey Project on Alcohol and Other Drugs (ESPAD), lifetime prevalence of cannabis use among 16-year-old students was 4% in 1995, 11% in 1999, and 16% in 2003; however, in 2007, parallel with the decrease observed in some Western countries, lifetime prevalence was 13% [7,8,9,10]. According to the 2011 ESPAD study, lifetime use of cannabis was 19% in Hungary, with no significant gender difference. Hungarian data followed the European trends again: more ESPAD countries showed significant increase in lifetime use of cannabis from 2007 to 2011 than reporting lower figures. The European average in 2011 was 17%, with a minimal prevalence rate of 4% and a maximal rate of 42% [11].

Problematic cannabis use has no consensual definition, and definitions are often limited to the description of negative social and health consequences [12,13]. For the screening and for the assessment of problematic cannabis use no consensual, internationally accepted standard measure is available. However, a few short screening tests have been developed for the use in clinical settings and epidemiologic surveys to recognise and identify risk factors and differentiate persons with high risk in order to develop effective interventions [13]. It is important to note that these questionnaires are designed to be used as screeners, not as diagnostic tools. Among these scales, the Severity of Dependence Scale [14] and its modified version [15] have been developed to assess the severity of cannabis use. Another measurement tool is the Cannabis Use Disorder Identification Test [16], developed along with the modification of Alcohol Use Disorder Identification Test [17]. The Problematic Use of Marijuana [18] contains questions regarding the interpersonal context of cannabis-related problems while the Cannabis Abuse Screening Test (CAST) [19] measures several aspects of harmful use (table 1). These instruments have generally good psychometric properties. Their validity is typically analyzed with the help of a relevant cannabis use related section of M-CIDI (Munich-Composite International Diagnostic Interview) [20] that assesses abuse and dependence according to the DSM-IV criteria.

The CAST was developed in France in 2002, where it is used every year as part of the annual ESCAPAD studies. CAST was an optional module for the ESPAD studies too, and was applied to measure cannabis-related problems in 13 countries in 2011 [11]. According to a 2007 study of Legleye et al. [19], the CAST is effective in identifying the high risk of abuse. Persons scoring at least 4 are likely to have worse physical mental and educational status. In addition, the authors obtained good psychometric results and confirmed the one-dimensional structure of the CAST. In a more recent study, Legleye et al. [21] cross-validated the CAST with M-CIDI. Besides the one-dimensional structure, good internal consistency, and satisfactory concurrent validity, they identified 95.1% specificity (successfully identified valid negative cases) and 45.21% sensitivity (successfully identified valid positive cases). The test did not differentiate between abuse and dependence. However, this latter result can be explained not only by weaker discriminative performance but also by the problematic nature of the DSM-IV concept. Namely, a growing number of studies support the notion that abuse and dependence are not two separate diagnoses but refer to a common background disorder. Applying the DSM-IV concept to adolescents and young adults seems problematic [22]. Some characteristic symptoms may not be experienced by adolescent substance users, others may occur very rarely, while still others can be interpreted only after a specific age [23]. Moreover, adolescents are often less accurate in judging their own substance use because different distorting factors might be present [24]. Their self-awareness is also lower than that of older persons [25]. Then again, adults may also have trouble differentiating between abuse and dependence, and for this reason, DSM-V will supposedly not contain the concept of abuse, and these two concepts will be merged together [26]. It is also important to note that there are considerable gender differences among adolescents and young adults in cannabis use patterns [27].

The aims of the present research were to verify the factor structure of the CAST and to estimate its sensitivity and specificity in an adolescent and young adult population. Because there are only Western European data available so far, this study contributes to international research on the CAST. Moreover, there are significant differences between the Central-Eastern European region and Western Europe regarding the spread of illicit substances, including cannabis as well as characteristic patterns of use [5]. Thus, validating the CAST in this region is highly reasonable. Further importance of the study is that it planned to analyze the CAST on an adolescent (high school) and a young adult (college) sample simultaneously because several studies suggest that measures do not work necessarily the same way for both populations because of differences in adolescent and adult substance use [25,28]. Among adolescents, for example, experimenting with illicit substances is (statistically) an ordinary phenomenon that most adolescents simply ‘outgrow', while this experimenting is less characteristic of adults. For adolescents, but not necessarily for older persons, substance use might have special functions (e.g. expressing autonomy). Adolescent substance use might however be an obstacle in solving particular developmental tasks; therefore, it can contribute to long-term negative consequences. This is less characteristic for adults who are likely to lead a more evolved lifestyle and reached a more stable identity. Conclusively, it felt reasonable to validate the CAST simultaneously on high school student and college student populations.

College Sample

Our first sample consisted of college students who finished at least 3 years of their university studies, studying in Budapest and living in dormitories. We contacted the five biggest universities in Budapest and obtained their permission to collect the data in the 39 dormitories. All universities and dormitories collaborated in the research. In the dormitories, we visited all rooms in an attempt to reach all students. We made two additional attempts to contact the residents if we could not contact them initially. To be included in the study, students had to start at least their fourth year at the university. Out of the 10,190 contacted persons, 3,556 students met the inclusion criteria. Among them, 238 students studied abroad, 542 students could not be reached, and 381 persons rejected participation in the study. Thus, 2,395 persons (67.4%) completed the questionnaire. Among them, 439 participants indicated that they have used cannabis in the past year; therefore, they were included in the present analysis. This sample comprised 65.1% males. The mean age was 23.9 years (SD = 1.56 years). Although data collection was in all cases based on personal contact, anonymity was assured. Respondents completed the questionnaires individually and returned them to the interviewers in sealed envelopes. Interviewers were psychology students.

High School Sample

We recruited high school students from high schools attended by most members of the university sample. Those high schools were selected which at least by 6 of the university students were attended. The high school sample consisted of 12th grade students graduating in 2010. Based on this selection, we selected 95 schools from 54 cites outside of Budapest. The number of classes we chose represented proportionally the university sample. From 132 randomly selected classes, 14 classes rejected participation in the study. Concerning the remaining 118 classes, we collected the data for 3,124 students. Based on the data, 476 (56.3% males, mean age of 19.0 years, SD = 0.65 years) of these students indicated having used cannabis during the preceding year.

We collected all data in class on a previously agreed date. Data collection took place in March 2010 in the high school sample. Data from college students were collected from November 2009 to February 2010. All participating students returned completed questionnaires in sealed envelopes. Due to the timeframe provided by a class, a short section of the questionnaire had to be completed on a second occasion; however, we asked only one fourth of students to participate again. Concerning the present analysis, this decision affected only the M-CIDI, which was thus completed only by 90 persons. The subsample was nevertheless representative of the total sample in terms of age (t = 1.68, n.s.), the age of first use (t = 0.66, n.s.), the last month of use (χ² = 0.05, n.s.), and Cannabis Abuse Screening Test score (t = 0.87, n.s.). However, the proportion of girls in the subsample was higher than the proportion in the total sample (58.9 and 40.3%, χ² = 10.3, p < 0.05).

Procedure

The Institutional Review Board of the Eötvös Loránd University approved the study. All participants signed an informed consent. Parental consent was also sought for students younger than 18 years of age.

Cannabis Use Questions

In the questionnaire, we asked about the time of the last cannabis use, frequency of use in the past year and past month and the age of first cannabis use.

Cannabis Abuse Screening Test [19]

The short self-rating questionnaire was developed for screening of problematic cannabis use among adolescents and young adults. The six items of the questionnaire (table 3) target many aspects of harmful use: (1) nonrecreational use; (2) memory disturbances; (3) being encouraged to reduce or stop using cannabis; (4) unsuccessful attempts to quit, and (5) problems linked to cannabis consumption. All items are measured on a 5-point Likert scale (0 ‘never', 1 ‘rarely', 2 ‘from time to time', 3 ‘quite often', 4 ‘very often'). Positive response thresholds vary across items. The threshold was set at ‘from time to time' for the first two questions, as they do not screen problems but frequencies of use in different contexts, and at ‘rarely' for the remaining items. Using this algorithm, individual test scores can range from 0 to 6. The translation process involved two native speakers independent from each other who translated the original version. After reaching a consensus, a third person back-translated the test, and the two English versions were then compared. This cycle was repeated until there was no significant discrepancy between the compared versions.

DSM-IV Diagnoses

Diagnostic assessments for the past 12 months were based on the paper-and-pencil version of the M-CIDI [29]. The M-CIDI is an updated version of the World Health Organisation's CIDI version 1.2 [30], which incorporates 19 items to cover DSM-IV diagnostic criteria. At least 1 out of 4 criteria defines cannabis abuse: role impairment, hazardous use, legal problems, and social-interpersonal problems. Cannabis dependence is defined by the presence of at least 3 out of 7 criteria: tolerance, withdrawal, using more or longer than intended, impaired control, much time spent using, reduced activities, and use despite problems. Although withdrawal is not a criterion for cannabis dependence in the current DSM-IV classification, it was included in this study as increasing evidence suggests a clinically significant cannabis withdrawal syndrome [31]. The reliability and validity of the M-CIDI have been reported elsewhere [21].

To test the factor structure of the Hungarian version of the CAST, two confirmatory factor analyses were conducted on high school and college student samples separately. We tested the one-factor model with the weighted least-squares mean and variance adjusted estimation method [32,33]. For the confirmatory factor analyses, a satisfactory degree of fit requires the comparative fit index (CFI) and the Tucker-Lewis Index (TLI) to be close to 0.95. The model should be rejected when these indices are < 0.90 [32]. The next fit index, root mean squared error of approximation (RMSEA), indicates excellent fit if its value is < 0.05, an adequate fit if its value is around 0.08, and poor fit if its value is > 0.10. Closeness of model fit using RMSEA (CFit of RMSEA) is a statistical test [34], which evaluates the statistical deviation of RMSEA from the value 0.05. Nonsignificant probability values (p > 0.05) indicate acceptable model fit.

For sensitivity analysis based on the M-CIDI as a gold standard, we calculated the sensitivity and specificity values for all CAST cut points. Thus, we could assess the accuracy of the CAST by calculating the proportion of subjects classified as nondependent versus dependent or as nonproblematic versus abuser using the M-CIDI. We defined sensitivity (the proportion of true positives that are correctly identified by the CAST) and specificity (the proportion of true negatives that are correctly identified by the CAST) as suggested by Altman and Bland [35] and Glaros and Kline [36]. In order to explore the probability that the CAST will give the correct ‘diagnosis', we calculated the positive predictive values (PPV), the negative predictive values (NPV), and the accuracy values for each possible CAST cut points. We defined PPV as the proportion of patients with positive test results who are correctly diagnosed [36,37]. We defined NPV as the proportion of patients with negative test results who are correctly diagnosed [35,36].

A receiver operating characteristic curve analysis (ROC analysis) was obtained by plotting sensitivity (true-positive test results) against the false-positive rate (1 - specificity) for all possible CAST cut-off points. This curve plots all sensitivity/specificity pairs resulting from continuously varying the decision threshold over the entire range of observed results; thus, it is the comprehensive representation of pure accuracy [38].

These statistical analyses were performed using the MPLUS 6.1. [39] and SPSS 16.0. [40].

Table 2 summarizes the demographic characteristics and description of cannabis-related variables for both samples. The high school sample consisted of 467 persons (43.7% female). Mean age was 16.41 years (SD = 1.37). Prevalence of last month cannabis use was 40.1%; 13.3% fulfilled the criteria of cannabis abuse, and 8.9% met cannabis dependence criteria. The college sample consisted of 439 persons (34.9% female). Mean age was 23.9 years (SD = 1.56). 33.7% used cannabis in the previous month. Prevalence of abuse was 10.6%, while 9.8% met dependence criteria, based on M-CIDI. Last year's prevalence was 100% in both of our samples, since it was one of the inclusion criteria of our study. The two samples differ significantly regarding age of first use of cannabis (t = 14.97, d.f. = 913, p < 0.0001), but they do not differ in CAST score (t = 1.72, d.f. = 913, p < 0.09). As for the comparison of proportions, the prevalence of cannabis use is significantly higher in high school students than in college students (χ² = 4.03, d.f. = 1, p < 0.05). However, the proportions of cannabis abuse and cannabis dependence are not statistically different in the two groups (χ² = 0.52, d.f. = 1, p > 0.05, and χ² = 0.07, d.f. = 1, p > 0.05, respectively).

One factor model was tested with the weighted least-squares mean and variance adjusted estimation method in high school and college samples separately. The items and factor loadings are presented in table 3.

Concerning high school students, the degree of fit was excellent (χ² = 15.6, d.f. = 9, p < 0.08, RMSEA = 0.038 [0.000-0.072], CFit = 0.66, CFI = 0.99, TLI = 0.99) and the standardized factor loadings were between 0.73 and 0.95. Regarding college students, the one-factor model yielded also an excellent fit (χ² = 15.8, d.f. = 9, p < 0.07, RMSEA = 0.044 [0.000-0.079], CFit = 0.57, CFI = 0.99, TLI = 0.98), with standardized factor loadings ranging from 0.71 to 0.93. Internal consistencies measured with Cronbach's α's were 0.76 for the high school sample and 0.71 for the college sample.

Based on M-CIDI as a ‘gold standard', we calculated the validity indicators: sensitivity and specificity as well as the feasibility indicators: PPV, NPV, and accuracy of the CAST at all possible cut-off points in high school (table 4) and in college students (table 5).

In high school students, the sensitivity, specificity, NPV, and accuracy were optimal at cut-off point 2 when considering cannabis dependence; however, the PPV was lower than optimal. Higher cut-offs (e.g. 4) resulted in optimal PPV, but all other indicators appeared unacceptable. Similarly, the cut-off point for cannabis use disorder could be detected at 2 points with somewhat limited sensitivity. Sensitivity is higher at 1 point, however specificity and PPV is below acceptable. The area under the curve value (AUC) of the ROC curve was used to measure the overall performance of the CAST in this high school sample. The AUC was 0.95 (95% CI 0.90-1.00) in case of cannabis dependence and 0.86 (95% CI 0.72-0.99) in case of CUD. The confidence intervals indicate that these areas are significantly different from 0.50. It is important to note, that these analyses were conducted on the representative subsample of 90 high school students who completed the M-CIDI.

In college students, we found similar cut-off points. Concerning cannabis dependence defined by the gold standard, the CAST cut-off point 2 yielded adequate levels of sensitivity, specificity, NPV, and maximum level of accuracy; however, the PPV was moderate. PPV was moderately higher (61 and 67 instead of 59) at cut-off 3 and 4; however, sensitivity significantly decreased below acceptable (46 and 27 instead of 78). Regarding CUD, again, the cut-off point 2 provided the adequate specificity, PPV as well as NPV, and maximal accuracy; however, with moderate sensitivity. Sensitivity is lower at all other cut-offs. The AUC of ROC curve was 0.93 (95% CI 0.89-0.98) in the case of cannabis dependence and 0.89 (95% CI 0.84-0.95) in the case of CUD. The confidence intervals indicate that these areas are significantly different from 0.50.

The aim of our study was to analyze psychometric properties of the CAST, a well-known tool for screening problematic use of cannabis. We conducted our research on two samples: Hungarian high school and college students. Besides the CAST, M-CIDI was also part of the questionnaire to assess concurrent validity. ROC analysis was conducted to obtain cut-off scores.

Based on the results, the Hungarian version of the CAST has adequate psychometric properties suitable for screening wider populations, proving to be an ideal tool for assessing problematic cannabis use among adolescents and young adults. However, the internal consistency is moderate for high school students. Moreover, favorable characteristics of the CAST were confirmed on two independent samples comprising participants of different age groups, which increased the robustness and generalizability of our results. In addition to using Western European data to validate this measure, it is important to assess psychometric properties in culturally different samples, as the results of some previous studies on validating various screening instruments suggest that their reliabilities and factor structures may not always fit the requirements in populations other than those for which they were designed. Gu et al. [41], for instance, were unable to confirm the reliability and unidimensionality of the widely used Severity of Dependence Scale [14] in a Chinese sample, in spite of the numerous positive results achieved in other populations. The Hungarian - and in a wider context, Eastern European - cannabis situation have some unique features which cannot be detected in Western European societies. Before the political transformation in 1989, cannabis was almost unavailable in Hungary, but the shift to the market economy brought a convertible national currency and cannabis - among other drugs - became available abruptly. This led to a generation gap, whereas the older generations tend to perceive all drugs as equally dangerous, younger generations tend to consider cannabis less harmful than other drugs [5].

It is worth to compare our results to Western European data. Legleye et al. [21] found prevalence rates of 13.8% for abuse, and 22.1% for dependence using M-CIDI among 17-year-old 12-month cannabis users. Their finding on abuse is close to ours (13.3% for the high school sample and 10.6% for the college sample), but the Hungarian dependence rates are lower (8.9 and 9.8%, respectively) than the French rates. Cultural differences can be hypothesized behind the differences. Our results, in line with previous studies [19,21,42], confirmed the one-dimensional structure of CAST. Our results regarding cannabis dependence showed that applying a cut-off score of 2 results in optimal sensitivity, specificity, NPV, and accuracy, but less than optimal PPV. In the case of cannabis use disorder, the same cut-off score proved ideal in terms of specificity, PPV, NPV, and accuracy. Sensitivity in this case is, however, not satisfactory. Similarly, Legleye et al. [21] obtained higher values of specificity than sensitivity with an optimal cut-off score of 3 or 4, when testing CAST against DSM-IV among adolescents 17 years of age. A subsequent study [42] found that the screening properties of the questionnaire were unsatisfactory in case of cannabis dependence; however, with regard to cannabis use disorder they obtained good sensitivity and specificity with a cut-off score of 3 (for the binary version) and 6 (for the full version) among the adolescent population of cannabis users seeking treatment.

It is observable that an ideal cut-off depends on sample characteristics and purpose of utilization; that is why no single optimal cut-off can be established. For example - according to the ESPAD recommendations -, a cut-off of 2 should be used to detect cannabis-related problems among adolescents [11].

Even though our two - high school and college - samples differ in view of the main demographic variables, results - most importantly the cut-off scores - are similar. We interpret this finding as a hallmark of the stability and generalizability of the Hungarian version of the CAST.

We have to emphasize that the lack of standard diagnostic scores does not present a real problem. We have to consider that screening is only the first step in problem assessment; its objective is the identification of the potential problem and the assessment of its severity. Scores above the defined cut-off score can be used as indicators rather than evidence for a diagnosis. Setting cut-off points results in dichotomous classification that implicitly reduces predictive force [28]. For this reason, it is worth defining the cut-off points suitable for the specific objective separately for every sample and population. Namely, the sample and the objective of screening can define the ‘adjustments' (sensitivity, accuracy, etc.) that would allow professionals to apply the instrument optimally.

Systematic analysis of the relationship between cut-off scores and psychometric indices allows professionals to define specific cut-off scores when performing research and screening, and helps them to apply appropriate cut-offs, depending on the actual aim of data collection [43].

Our study is not without limitations. Selection of the samples may be biased. For example, students characterized by problematic and risky behaviors (including cannabis consumption) are more likely to be absent from the class during data collection. Uncontrolled factors (e.g. personality characteristics, attitudes toward drugs or previous participation in drug-related studies) may all have had an influence on refusal in the college sample. We conducted our analysis on a representative subsample (90 students) of the high school sample. Our results would have been more reliable, if we had the possibility to perform the analysis on the whole sample. For future studies, it is desirable to collect data from clinical samples for further validation of the test.

In conclusion, our study is not without limitations, but on the whole, its strengths exceed its weaknesses. The Hungarian version of the CAST can serve as a basis for exploring substance use habits in a relatively understudied Eastern European adolescent population, and promote internationally comparable statistical data collection. The questionnaire helps early screening and intervention as well. These are important goals, given the growing body of evidences on the association between persistent use of cannabis and negative psychological (dysphoria, impaired attention, memory and psychomotor performance, and tolerance) and somatic (e.g. cardiovascular and respiratory) effects [44]. Nowadays, the debate is intensive on the nature of the dose-response relationship between long-term use of cannabis and increased risk of psychotic outcomes [45]. Notable evidence is available on the irreversible neurotoxic effects of tetrahydrocannabinol on the adolescent brain [46].

The present work was supported by the Hungarian Ministry of National Resources Grant KAB-KT-10-0013 and KAB-KT- 10-0016, and the Hungarian Scientific Research Fund Grant 83884. Zsolt Demetrovics and Gyöngyi Kökönyei acknowledge financial support of the János Bolyai Research Fellowship awarded by the Hungarian Academy of Science. The project was also supported by the European Union and the European Social Fund under the grant agreement No. TÁMOP 4.2.1./B-09/1/KMR-2010-0003.