Background: Existing cannabis treatment programs reach only a very limited proportion of people with cannabis-related problems. The aim of this systematic review and meta-analysis was to assess the effectiveness of digital interventions applied outside the health care system in reducing problematic cannabis use. Methods: We systematically searched the Cochrane Central Register of Controlled Trials (2015), PubMed (2009-2015), Medline (2009-2015), Google Scholar (2015) and article reference lists for potentially eligible studies. Randomized controlled trials examining the effects of internet- or computer-based interventions were assessed. Study effects were estimated by calculating effect sizes (ESs) using Cohen's d and Hedges' g bias-corrected ES. The primary outcome assessed was self-reported cannabis use, measured by a questionnaire. Results: Fifty-two studies were identified. Four studies (including 1,928 participants) met inclusion criteria. They combined brief motivational interventions and cognitive behavioral therapy delivered online. All studies were of good quality. The pooled mean difference (Δ = 4.07) and overall ES (0.11) give evidence of small effects at 3-month follow-up in favor of digital interventions. Conclusions: Digital interventions can help to successfully reduce problematic cannabis use outside clinical settings. They have some potential to overcome treatment barriers and increase accessibility for at-risk cannabis users.
An estimated 14.6 million Europeans aged 15-34 have used cannabis in the past year, and 3 million of those use it daily or near-daily [1, 2]. The European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) estimates that existing cannabis treatment programs reach only a very limited proportion of the population in need of treatment . Across the European Union, between only 1 out of 10 and 1 out of 20 daily and near-daily users are currently in outpatient treatment . In some European countries, the proportion of users receiving treatment is even lower, at around or below 1 in 100 daily or near-daily users . At the same time, the number of cannabis users enrolled in specialised drug treatment is rising and might reflect an increasing demand for treatment .
Digital media interventions (i.e. treatment programs based on information and communication technology, or ICT) have the potential to reach young people with problematic cannabis use. This channel may be particularly effective for users who cannot be reached by means of traditional approaches in clinical settings, for a number of reasons. First, the internet plays an increasingly important role in the daily life of the majority of adolescents and young adults. Around 87.9% of North Americans, 73.2% of the population of Australia and Oceania and 73.5% of Europeans use the internet, totaling over 945.2 million web users across the three continents . Smartphones, tablets and laptop computers make information retrieval easier than ever before. Often the internet is the first source of information for any health-related concern. In Western countries, up to 83% of 16-18 years use the internet to find information, 85% communicate through social media and 76% chat with friends or family .
The growth in the availability and use of digital media has a strong potential to expand and alter the landscape of substance use disorder treatment . First, digital media offers treatment providers the opportunity to reach users globally, given that digital interventions can also be provided as ‘unguided interventions', which do not require professionals to deliver the programs in person. Geographical coverage can therefore be extended in areas where traditional drug-treatment facilities are scarce. Even in instances where digital interventions are combined with some form of contact or facilitation from professionals, the burden on therapists is still reduced . Digital media may provide a way to offer private, convenient, and low-cost (or free) access to substance-use-disorder treatment programs [5, 7]. Additionally, self-paced and personalized learning processes may further enhance the motivation to undertake treatment. Moreover, the perception of increased privacy may help address the issue of stigmatization .
In the past decade, a number of ICT-based treatment programs have been developed and tested in relation to various mental health problems [9, 10]. Several literature reviews examined the overall effectiveness of internet-based interventions, which focused on the treatment of alcohol, tobacco and illegal drug use [5, 6, 8, 10, 11, 12, 13, 14, 15]. Only one meta-analysis tested exclusively the effectiveness of internet and computer-based interventions for various groups of cannabis users . This review featured a diverse sample of cannabis users, for example, with respect to age, type of intervention undertaken (general, selective and indicated prevention), and treatment setting (school, university, clinic, primary care). The meta-analysis indicated that there are small, but significant, short-term effects of internet, computer or CD-ROM-based interventions in reducing cannabis use. Generalizing from these findings remains difficult, however, due to the heterogeneity of the approaches tested. Moreover, it remains unclear whether digital cannabis interventions would be equally effective when applied exclusively outside of a clinical context. This is a pertinent question, as the majority of problematic cannabis users are adults who are not reached by traditional addiction treatment facilities [1, 2]. This study thus aims at complementing the literature review of Tait et al.  by exploring the effectiveness of internet- and computer-based cannabis interventions applied in non-clinical settings.
We aim to assess the effectiveness of digital interventions (i.e. internet- and computer-based interventions) in the treatment of late adolescent and adult cannabis users recruited from non-clinical settings.
We conducted this systematic review and meta-analysis in accordance with the Cochrane Handbook for Systematic Reviews of Interventions . We developed a detailed search strategy for identifying studies to include in the review. In November and December 2014, and again in June 2015, we searched the Cochrane Central Register of Controlled Trials, which includes the Cochrane Drugs and Alcohol Group Register of Trials, PubMed (2009-2015), Medline (2009-2015) and Google Scholar (prior to 2015), as well as several reference lists taken from various articles. We also contacted researchers in this field to obtain further information on their studies. The following search terms were employed to find potentially eligible articles: substance-related disorders OR addiction OR abuse OR dependence OR illicit; cannabis OR marijuana OR hashish; internet OR web OR online OR computer OR CD-ROM; and treatment OR intervention. We reviewed publications written in English, French and German. The PRISMA checklist is available upon request.
Inclusion and Exclusion Criteria
Types of Studies
We reviewed randomized controlled trials (RCTs) that tested the effect of CD-ROM, internet- or computer-based interventions (with or without additional therapeutic guidance) for the treatment of cannabis use disorders.
Types of Participants and Setting
We included all participants who were current users of cannabis, regardless of age, motivational stage, gender or nationality. We analysed if the studies reported diagnostic criteria for cannabis use disorders as set out in the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) 4 , the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) 5  and the International Classification of, 10th revision Mental and Behavioral Disorders, 10th revision (ICD 10) , but we did not use them as eligibility criteria. Eligible participants must have been approached in other than clinical settings, such as in schools, universities, job centres or via the internet. Studies in which cannabis users were approached through the health care system (e.g. clinics, drug treatment centres, general practices) were excluded.
Types of Interventions
We included treatment programs for the management of problematic cannabis use, cannabis abuse or dependence, which were delivered via the internet, computer or CD-ROM. However, programs aiming solely at cannabis prevention, self-assessment websites and interventions that were designed to improve the skills of therapists or of users (such as improving job prospects) were excluded. Similarly, studies that only measured participants' knowledge of, attitude towards or intentions regarding cannabis use were excluded.
The control groups in the selected studies consisted of cannabis users who had had not undergone any treatment (untreated control group), a delayed treatment control group, a group who had received therapist-delivered intervention, a group who was only assessed for cannabis use and related outcomes, or any other intervention different from the experimental condition (see Experimental Condition).
The outcome measure employed in this study is participants' self-reported cannabis use (per day, in past week, past month, past three months, quantity of cannabis used), assessed by a questionnaire.
Two researchers independently screened the titles and abstracts of all the studies identified in the search process. Any potentially eligible studies were obtained as full articles and screened for inclusion and exclusion criteria. If any two or more studies revealed the use of overlapping samples, only the largest study was to be selected to avoid duplication of data. Data were selected and assessed independently by two researchers. In instances of missing or ambiguous data, clarification was sought from the original authors. The review excluded publications of studies that did not report data, such as study protocols.
Meta-Analytical Data Synthesis
Effect size (ES) was calculated by Cohen's d and Hedges' g, using the Cochrane software . Calculations were based on the relevant outcome data (i.e. mean difference in frequency of cannabis use in the last 30 days). Potential publication bias was assessed with a funnel plot.
Assessment of Methodological Quality
In the context of a systematic review, study validity is the extent to which the design and implementation of a study are capable of preventing systematic errors or bias. In order to gauge the methodological quality of the selected studies, two researchers independently assessed the implementation of each study according to the criteria set out in the Cochrane Handbook for Systematic Reviews of Interventions . We employed the following criteria: (1) Allocation llocation concealment: Studies were considered to have (a) adequate allocation concealment if participants were assigned to treatment groups using central randomization (i.e. allocation by a central office unaware of subject characteristics), an on-site computer randomization system, which stored allocation data in a locked computer file accessible only after the characteristics of an enrolled participant had been entered, or if they described any other allocation method offering adequate concealment; (b) Inadequate allocation concealment if using alternation; making reference to case numbers, dates of birth, day of the week; or using any procedure that is entirely transparent before allocation, such as an open list of random numbers; or if the description of the allocation process contained elements suggestive of inadequate concealment; (c) Unclear allocation concealment if authors did not report an allocation concealment approach, or the approach reported did not fall into either category (a) or (b). (2)Blinding: Studies were categorized as (a) double blind; (b) single blind (blinding of participants only); (c) unclear blinding; or (d) no blinding. (3) Attrition bias: For each study, it was determined if (a) losses to follow-up were completely recorded; (b) losses to follow-up were incompletely recorded; or (c) it was unclear to what extent losses to follow-up were recorded. (4) Detection bias: for each study, it was determined if (a) the assessor was blind to treatment allocation during outcome assessment; (b) the assessor was not blind to treatment allocation during outcome assessment; or (c) it was unclear whether or not the assessor was blind during outcome assessment. (5) Intention-to-treat (ITT) analysis: studies were assessed to determine if (a) a statistical ITT analysis was performed; (b) no ITT analysis was performed; or (c) it was unclear whether or not a statistical ITT analysis was performed.
We also assessed the strength of evidence of each study, based on the methodological quality and design of the study and according to the criteria set out by the Oxford Centre for Evidence-Based Medicine  (table 1). The two researchers independently assessed the quality of evidence furnished by each study. The agreement between both raters was strong (Cohen's kappa 0.80).
The search resulted in 140 records. A total of 52 studies were considered eligible and four of those met the inclusion criteria [22, 23, 24, 25] (fig. 1). Sufficient data to enable the calculation of the ES and OR were available for these four studies. The total number of study participants covered in this review is 1,928.
Of the four studies selected for inclusion in this review, one was conducted in the United States , one in Australia and Oceania  and two in Europe [25, 26]. The studies were carried out in high schools or colleges  or through information and help websites [23, 24, 25]. Participants were recruited via offline advertisement [24, 25], online advertisement [23, 24] and drug-prevention websites [23, 24, 25] (in some studies, more than one recruitment approach was used; table 2).
Types of Interventions
All of the tested treatment programs were web-based [22, 23, 24, 25] (table 2). One RCT  examined the effectiveness of a web-based screening followed by a brief motivational intervention, called ‘e-checkup to go', aimed at problematic cannabis users. Approximately 4,000 incoming students (aged 17-19) at a large American public university were recruited to participate in an online screening survey the summer prior to beginning college. Students who recorded any cannabis use in the past 3 months received an invitation letter and email. They were asked to log on to the study website (http://www.echeckuptogo.com/usa/), where they completed a baseline screening questionnaire. Based on their initial screening responses, a brief, web-based, personalized feedback intervention was provided. The motivational enhancement intervention was primarily text-based, but it also incorporated pictures, figures and graphs to engage users' interest . The intervention group (n = 171) was compared to an assessment-only control group (n = 170).
In an RCT by Tossmann et al.  adult problematic alcohol users (n = 235) and cannabis users (n = 67) were recruited via both a German addiction prevention website (www.drugcom.de) and through online advertisements. Participants in the cannabis intervention group (n = 33) were asked to complete a web-based screening on alcohol or cannabis use, and subsequently received one session of web-based motivational interview. This brief intervention, which lasted 29 minutes on average, was delivered in a private online chat room by a drug counselor trained in motivational interviewing (MI). The control group (n = 34) completed a web-based screening on the addiction prevention website, and were provided with information on the self-test via a private online chat (11 min average duration).
The Australian treatment program ‘Reduce Your Use'  (https://reduceyouruse.org.au/sign-up/) is a fully self-guided, web-based intervention for individuals who have used cannabis at least once during the past month and expressed a desire to cease using cannabis. This program consisted of six modules of motivational enhancement and cognitive behavioral therapy (CBT), which were delivered at specific intervals set by the participant. Participants were able to track their progress throughout the course of the program via cannabis-use graphs. The website also featured a personalized space for each participant, blogs written by former cannabis users, quick assist links, and weekly automated encouragement emails. Individuals using the website could choose to either read its content as text or watch videos of actors reading the text. Study participants were recruited by means of both online and offline advertisements. The intervention group (n = 119) was compared to a control group (n = 111), who instead received six cannabis information sessions on a control condition website.
‘Quit the Shit' (https://www.quit-the-shit.net) is a German online chat-based intervention. To test its effectiveness in a RCT, problematic [..] cannabis users were recruited from the general population via a website on drugs and drug abuse . Participants in the intervention group (n = 863) were asked to keep an online diary of their cannabis use over a period of 50 days. The program began with a 50-minute interview with a qualified psychotherapist, aiming to identify each participant's particular situation and determine his or her individual consumption goals and coping strategies. Participants subsequently received weekly feedback on their diary entries by their counselors. The program structure and counseling sessions were based on the principles of self-regulation and self-control. Participants in the control group (n = 429) had to wait three months before starting treatment.
Types of Comparisons
One study carried out a comparison between an intervention group and an assessment-only group . Two studies used active comparisons, in which control groups were provided with information on the self-test  or used a control condition website . One study used delayed-treatment controls as a comparison to the intervention group  (table 2).
Risk of Bias
Allocation concealment: all of the selected studies were RCTs featuring adequate allocation concealment (central randomization). Performance bias: all of the studies were intervention studies. Neither intervention providers nor recipients were blind to treatment conditions. Attrition bias: loss to follow-up was unclear in one study , while the other three studies provided adequate information on the flow of patients through the trial. ITT: ITT analysis was performed in three of the four selected studies [22, 23, 24] (table 2).
Level of Evidence
The principal outcome taken into account in this review is ‘reduction in cannabis use'. This was measured as users' self-reported last month cannabis use in all of the selected studies. Other measures were also reported, such as ‘number of joints smoked per typical week'  or ‘standard cannabis units in the past month' . Urine toxicology tests were not employed in any of the studies.
Overall Comparison of Treatment Effects
Figure 2 and table 3 show the results of the meta-analysis across studies. All selected studies reported consumption results after a 3-month follow-up. The study of Tossmann et al. (2011) reported the highest mean difference (Δ = -9.5; 95% CI -12.52 to -6.48, ES 0.24), while the study of Lee et al. (2010) revealed the lowest (Δ = -0.81; 95% CI -3.87 to 2.25, ES 0.08). The pooled mean difference (pooled Δ = -4.07; 95% CI -5.80 to -2.34) and pooled ES (ES 0.11) provide evidence of small effects in favor of web- and computer-based interventions. A funnel plot is available upon request.
This study aimed at undertaking a systematic search and review of literature on the effectiveness of digital interventions in reducing cannabis use in cannabis users. As nine out of ten problematic cannabis users typically do not have any contact with traditional addiction treatment facilities , our literature review placed special emphasis on cannabis users who were not sampled from clinical settings. Four RCTs met inclusion criteria and were included in our analysis, for a total of n = 1,928 study participants covered by this meta-analysis. All studies were designed to assess the effectiveness of digital interventions for reducing cannabis use, and were conducted in the US, Australia and Oceania as well as Europe. Participants were recruited from the general population through internet-based advertisements and via other channels such as advertisements distributed by post, email and in print media. The risk of bias in the selected studies was assessed to be moderate as they featured adequate allocation concealment, central randomization, and attrition bias. Furthermore, undertaking ITT analysis was conducted in all trials. Only in one study, inadequate information on attrition bias and ITT analysis was provided . Using the ‘Oxford Centre of Evidence-Based Medicine (CEBM)' (2001) criteria , one study  is an individual RCT with low drop-out. Thus, it can be classified as ‘LoE' 1b (high quality RCT). The other four studies are also RCTs, but the potential ‘LoE' 1b needs to be downgraded to ‘LoE' 2b (low quality RCT) due to high attrition (<80% follow-up). Therefore, the quality of the evidence provided by all of the studies included in this meta-analysis is rated as ‘good'. All of the studies were published within the last three years.
Effects of Digital Cannabis Interventions on Cannabis Use: Meta-Analysis Results
The pooled ES and mean difference (Δ) were calculated for all four RCTs, which combined elements of both motivational and CBT. The results of these trials indicate small ESs suggesting that web- and computer-based interventions can be effective at reducing cannabis use. All of these studies were conducted among participants who were drawn from the general population. They targeted older adolescents and adults with problematic cannabis use. The largest treatment effects were found for the web-based online chat with a trained psychotherapist, plus online diary with weekly personalized, written feedback based on CBT/MI . The ES we calculated for this study (ES 0.24; 95% CI 0.04-0.51) is slightly higher than that calculated in the review of Tait et al. . They used a combined measure of ‘days and grams of cannabis used in the past 30 days' as outcome variable their we choose the item ‘frequency of days in the last 30 days', which seemed more comparable to the outcome measures of the other three included studies (table 3). The smallest treatment effects were found in relation to a study that provided a brief web-based screening followed by a brief web-based personalized feedback intervention . The intervention was probably less effective in reducing cannabis use due to the short duration of the intervention and the lack of contact between students and a drug counselor. On the whole, the pooled ES (0.11) we found for digital cannabis interventions aiming at reducing cannabis use among older adolescents and adults in non-clinical settings was slightly smaller than the pooled ES (0.16) reported by Tait et al. . They focused on the effectiveness of more heterogeneous types of internet and computer-based interventions (in terms of duration, intensity) conducted in a number of different settings (schools, colleges, clinics, primary care, and general population) . Although the primary outcome measure chosen in this study was ‘reduction in cannabis use', the selected interventions also show promise in relation to certain secondary outcomes (not analysed in this study) such as the reduction of cannabis dependence symptoms  and beneficial effects relating to use-related self-efficacy, anxiety, depression and life satisfaction . Digital interventions thus show promise in complementing more traditional interventions that aim to prevent and reduce problematic cannabis use, such as school-based prevention programs , media campaigns  and treatment programs in clinical settings .
Public Health Implications
The small pooled ES we found for digital cannabis interventions in non-clinical settings have public health implications. Cannabis is the most widely used illegal drug in Europe and a majority of Western countries . Intensive cannabis use is associated with an increased risk of psychosocial, mental and physical health problems [28, 29]. Given that very few problematic cannabis users make use of the traditional addiction treatment system [2, 30], public health systems are faced with the challenge of reaching out to individuals who use cannabis to a risky or harmful degree, but are not motivated to seek treatment [2, 9]. Gates et al.  analysed the possible reasons for low levels of treatment-seeking among cannabis users. These include the feeling that treatment is not necessary to reduce cannabis use, no motivation to stop, and a lack of treatment option awareness. Furthermore, a frequently reported barrier to entry into treatment is a preference for avoiding the stigma stemming from being labeled as an illicit drug user.
Given the high level of internet use in Western societies, digital interventions have the potential to reach adolescents and young adults everywhere, and especially in areas where physical treatment facilities are scarce, such as in rural communities . Digital interventions can be communicated and promoted through various channels, including: (1) post or email campaigns advertising cannabis use programs targeted to at-risk groups such as high school graduates, military conscripts, students in their first year of tertiary education, and unemployed young people at job centres; (2) online advertisement through social media, search engines, or music- and video-sharing websites; (3) mobile advertising (including SMS, MMS and advertisements served and processed via online channels); (4) other information sources (e.g. print media such as newspaper and magazines in addition to TV and radio); (5) mobile applications (free or low cost software designed to run on smartphones and other mobile devices).
There are several limitations to this study, which may restrict us from generalizing our findings. (1) First, in all of the trials we reviewed, cannabis use was measured only on the basis of self-reported data. No collection of urine samples for drugs screenings was undertaken. The inclusion of this type of screening in future studies would help to increase confidence in the effectiveness of digital interventions . (2) The selected studies were only brief interventions for older adolescents and adults, with some variability in intensity, duration, and intervention materials. (3) While all of the selected studies measured cannabis use as ‘consumption in the past 30 days', this variable was operationalized in different ways. An international consensus needs to be reached on the standards of outcome measures in order to improve the comparability of findings across clinical trials. (4) Finally, it must be noted that these studies provided evidence only on minor levels of effectiveness of digital interventions in relation to regular or at-risk cannabis users. The effects of digital interventions in individuals with heavier cannabis use or multiple substance use, cannabis use disorders, comorbid mental disorders (e.g. depression, psychosis or other substance use disorders) or severe psychosocial problems and other problem behaviors were not explored, but may differ among subgroups of users. Information on the long-term stability of treatment effects is also missing. (5) The studies did not consider the existence of adverse or undesirable effects of digital cannabis interventions, which call for further investigation. Lastly, since the number of included trials is small, the presented results need to be confirmed by additional single studies and subsequent meta-analyses.
Conclusion and Perspectives
The principal finding in this study is that digital treatments are a promising way of reducing problematic cannabis use outside clinical settings. Such treatments have the potential to make treatment more accessible to cannabis users in the general population, and may help overcome existing barriers to treatment.
E.H., U.W.P., M.F. and R.S. all contributed to the development of the study design as well as to the collection, extraction, analysis and evaluation of data. Additionally, all authors contributed to the drafting of this manuscript and approved the final version.
The authors have no conflicts of interest.
This study was funded by a grant from the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA).