Evaluations of State Medical Cannabis Programs in the USA: A Narrative Review

The use of medical cannabis (MC) continues to grow in the USA [1‒6]. In response, individual states have established MC programs that develop policy, implement protocols, and provide regulatory guidance for MC use to facilitate commerce, define eligibility criteria, establish product content and quality standards, and optimize patient experience [7‒10]. In this rapidly expanding field, there are little data to guide program implementation and evidence-based practices, leading to variation in guidelines and recommendations regarding policy and regulations across states [8, 11‒14]. For example, procedures to obtain authorization for MC use, resources to guide program participants, guidelines for laboratory and safety testing – even the medical conditions that qualify for MC treatment – are all unique to each state [15‒17]. These disparities create uncertainty regarding which approaches will ensure optimal program outcomes [8, 16].

To determine which aspects of MC programs are successful, evaluations are an important tool [18‒21]. Evaluations can engender state accountability and ensure that participants, providers, and the public are informed about program performance [19, 22]. Evaluations can generate knowledge, identify gaps in process and knowledge, and encourage improvement by determining the prevalence of growing issues within an MC program [23]. Such issues include the need to determine that the increasingly diverse products are safe, prevent product diversion, curb the misuse of MC and, by extension, adverse events among consumers [24‒31]. Evaluations are also critical given noted concerns regarding disparate access among underserved populations [32‒36]. Given myriad ways that evaluations could be conducted by state MC programs, a successful evaluation should, at a minimum, provide relevant, unbiased insights to program goals, impact, data validity, and participants’ experience [23].

In order to determine key components and approaches that have been leveraged for state MC program evaluations, our team conducted a narrative review of the current literature in the field. Determining the methods, structure, and outcomes of independent evaluations has significant potential to identify approaches that may be undertaken in other states and ultimately improve the safety and outcomes of MC programs nationally. In this narrative review, we provide a rationale for our search strategy, describe processes and key results relating to qualitative analysis of MC program evaluations identified, and synthesize common themes present among them. To our knowledge, this will be the first review of MC program evaluations.

Our narrative review strategy followed guidelines established by Green et al. [37], which are well suited for synthesizing a wide range of information sources. We queried three primary sources between December 2024 and April 2024: (1) scientific literature, (2) gray literature, including Google and all 38 states’ MC program websites, and (3) expert outreach. For the scientific literature, we searched PubMed, APA PsychINFO (EBSCOhost), Medline, Embase, and Scopus with the terms “cannabis OR marijuana” and “state program OR state run” and “[state]” (i.e., the query ran multiple times, each for a different state until all states were queried), as shown in the online supplementary Table 1A (for all online suppl. material, see https://doi.org/10.1159/000542472). We did not include the term “evaluation” in the searches, as the queries returned too few results, and we wanted to include reports that may be labeled otherwise but still met our criteria. Google Scholar was also utilized using the search terms “[state] cannabis evaluation” and “[state] medical marijuana program evaluation,” wherein each state was individually searched. The first 30 results for each state query were screened. After the first 30, there was repetition of results between state searches, thus removing the individuality of each state’s search results.

The second approach included gray literature studies and reports of state MC programs. Two authors (C.H. and C.R.) screened the first 100 search results of Google advanced search using the terms “medical cannabis program evaluation,” “medical marijuana program evaluation,” “cannabis program evaluation,” and “marijuana program evaluation” and restricting results to .gov, .org, and .edu websites. Another author (L.F.) screened the first 50 Google results for medical marijuana program evaluations by state using the search terms “medical marijuana program evaluation:.ST.gov.” Saturation of relevant results was reached within 50 search results but was extended to 100 in the beginning to ensure a comprehensive search.

For states with an MC program (38 states and Washington, D.C., as of April 2024), we also performed a review of each state’s MC program website and manually searched their reports for a program evaluation. If the website had an internal search function, a search for “evaluation” was conducted within each state’s cannabis state program’s site. Another author (A.T.K.) sent out emails in September 2023 to national and international MC researchers and experts to locate other unpublished or in-progress evaluations. Finally, two authors (C.H. and C.R.) reviewed reference lists of included articles.

For inclusion, the following criteria had to be met. (1) The focus was solely on an MC program and not adult/recreational cannabis use; if an evaluation was conducted during a time and place that contained elements of both, it was excluded to focus on MC. (2) The evaluation was conducted by an independent – state or non-state (i.e., non-state employees) – entity. Purely internal reports were excluded. (3) The evaluation included a consideration of patient experiences, including surveys, focus groups, and/or interviews. (4) The evaluation was for a single-state/district MC program within the USA, including Washington, D.C. (i.e., not a merging of data from multiple states, given the difficulty of extracting each state’s policies in a multistate evaluation). There were no restrictions on publication dates.

All abstracts and gray literature reports were screened between three authors (L.F., C.R., and C.H.), and a short list of potential full-text inclusions was screened by two other authors (G.C. and A.T.K.) for relevance. One author (G.C.) reviewed titles of excluded full-text articles to reduce error. Results were synthesized in the following ways: evaluations were thoroughly read to determine commonalities and differences. Basic descriptive information regarding each evaluation was extracted, and themes were qualitatively identified. Results based on these themes were synthesized into tables to describe each evaluation.

A flow diagram of search results is shown in Figure 1. We identified 272 abstracts to be screened through scientific literature searches and 1,300 titles through gray literature searches. We found no additional evaluations by contacting MC researchers and experts via email. Of abstracts screened, we excluded 223 from the scientific literature and 1,268 from the gray literature that did not meet our inclusion criteria. Of the 69 full scientific articles and 32 gray literature reports retrieved, three scientific articles and four gray literature reports ultimately satisfied all inclusion criteria as evaluations of single-state MC program evaluations conducted by an external party. Results were synthesized by summarizing the overarching goals and structure of each MC program evaluation and determining commonalities and differences between evaluations. We report results according to three overarching themes: evaluation characteristics, program experience, and assessment of cannabis use.

Characteristics of MC program evaluations included in our study are shown in Table 1, which represent six US states, namely, Maryland [38], Minnesota [39], New Mexico [40], Ohio [41], Rhode Island [42], and Pennsylvania, which had two evaluations [43, 44]. All evaluations were published before legalization of recreational cannabis use, if ever subsequently legalized; thus, only MC was legally available at the time of each included evaluation. We identified variation in several evaluation characteristics. Published versions of each evaluation were made available in one of three ways: the state MC program website, the website of an academic institution, or in a peer-reviewed journal. Similarly, evaluations were conducted by one of three types of evaluators: independent researchers from an academic institution or nonprofit entity, a private research consulting group, or a separate individual/organization within states’ Department of Health. Data collection and analysis approaches likewise varied. Patient-facing methods included surveys, interviews, or focus groups, while analytic strategies included both qualitative and quantitative elements.

All evaluations used convenience methods of sampling; however, samples sizes varied widely, from 15 to 13,011 participants. Recruitment strategies likewise varied substantially. For example, the Pennsylvania focus group study recruited on a statewide level but recruited only 27 MC program participants from one dispensary, while Maryland recruited 13,011 participants whose demographic distribution was similar to that of the MC program population. Some states utilized links to surveys distributed via email, while others advertised evaluation participation via flyers and direct contact in MC dispensaries. We found that qualitative evaluations tended to have smaller samples and quantitative evaluations tended to have larger samples, but no specific hypothesis testing or power calculations were mentioned in any of the evaluation reports. Most evaluations did not include a significant comparison group of nonprogram enrollees who may use cannabis for medical purposes. As an exception, Ohio recruited over 2,700 participants, 6% of which were not enrolled in the MC program.

Program goals were generally aligned with an overall objective to learn about the current status of the MC program. However, specific aspects of interest and their associated goals were unique to each state. New Mexico’s evaluation goal was to assess the current status of the program and investigate choices made by participants and medical providers, and they undertook three separate surveys/interviews to do so. Ohio focused on examining program growth and participant satisfaction levels with components of their program. In Maryland, the evaluation examined patterns of use, perceptions of risk and benefit, and occurrence of high-risk behavior with the goal of informing future program and policy efforts to ensure safe use of MC. Minnesota had a general goal to “describe the experiences from the first year of the program” but mainly collected data on frequency and chemotype (low, high, or equal THC:CBD ratio) of cannabis use in patients per different conditions. They stated that they are mandated to review their program every year, albeit only one evaluation was found. Rhode Island had a similar assessment with a general goal to ascertain how their program has met or failed to meet its objectives with their “exploratory program evaluation.” The Pennsylvania evaluations were geared toward assessing efficacy of cannabis on qualifying conditions and determining barriers of access while its other evaluation addressed the certification process.

State MC programs assessed experiences of enrolled program participants, per our criteria, and two states also assessed provider experience. Table 2 summarizes program experience results. All evaluations asked participants about aspects of program navigation, including difficulty of obtaining a card to access MC (seven evaluations), processing time (two evaluations), and information adequacy and accessibility (five evaluations). Six evaluations also asked about program costs; two evaluations about the need for flexibility in allowing caregiver participation; and four evaluations about MC cultivation at home.

With one exception (Minnesota), most participants found their state programs to be either very easy or very difficult to navigate. In Rhode Island, 100% of participants evaluated said program navigation was a simple and clear process. In Pennsylvania’s survey, 71.8% of participants evaluated summarized the process as “fairly easy.” In focus groups, participants were satisfied but wished the program was further integrated into the healthcare system. Half of participants evaluated in Minnesota found the program easy to navigate. For the states that found their program difficult to navigate, 43.6% of the participants in Ohio said the program is not responsive to inquiries, while in New Mexico, 90% were neutral or concerned about the difficulty of obtaining an MC card. Regarding cost, three states (Pennsylvania, Ohio, and Minnesota) reported program costs as a major or important issue. Over half of these participants claimed that either the registration costs or the cost of a qualifying medical exam was a financial burden. Regarding information adequacy, most participants evaluated in Pennsylvania and Rhode Island reported that they had trouble finding information about the state’s MC program. Conversely, most participants evaluated in Maryland and Minnesota found their state’s information sources to be adequate.

Additionally, participants (in two to five evaluations depending on the concern area) were asked about their level of agreement with six areas of concern, as shown in Figure 2. These issues included (1) legal protections, (2) issues with qualifying medical providers, (3) stigma, (4) access to dispensaries, (5) issues with MC products, and (6) sources of information.

Among areas of concern, evaluations in Ohio and Rhode Island asked participants regarding legal protections against arrest as they related to program satisfaction in their states. In Ohio, 75.7% of participants stated that providing legal protections would be the policy change that would most positively affect patient satisfaction. Rhode Island initially created their MC program legislation to protect program participants and their caregivers from arrest as it was cited as the most expected protection; however, some participants or caregivers were arrested if they exceeded the statutory limit of cannabis supply.

Additionally, three states (Rhode Island, New Mexico, and Pennsylvania) asked program participants regarding issues with qualified medical professionals (QMPs) – i.e., providers who may authorize access to MC for their patients. Of note, Minnesota included a question about the types of QMPs used for certification but did not assess interactions between program participants and QMPs. In New Mexico, 28% of program participants reported two primary concerns for obtaining MC authorization were difficulties (a) securing an in-person visit with a QMP and (b) obtaining a recommendation for MC treatment from a QMP. In Pennsylvania, less than half of survey participants reported difficulty finding a recommending QMP, while only a few focus group participants reported difficulty obtaining an MC treatment recommendation from an individual QMP encounter and no difficulty from a QMP group certification event.

Three evaluations addressed the issue of stigma. In Maryland, approximately 75% of program participants reported they were open to discussing their MC use and did not feel stigmatized. However, in New Mexico, 10% of program participants reported stigma as common, both internally (self-stigma) and externally (social and healthcare settings). Likewise, participants in the Pennsylvania focus group evaluation consistently reported they felt uncomfortable discussing MC use with their healthcare providers.

For every evaluation that asked about dispensary access, over 60% of participants reported no issues accessing a dispensary and between 2% and 13% said they lived too far from a dispensary. Rhode Island did not have dispensaries at the time of the evaluation, which program participants reported as a major issue in accessing safe MC.

Two evaluations asked about whether participants had issues relating to dispensaries or dispensary products. In Ohio, 67% of participants bought their MC solely from dispensaries. Of the participants that chose not to purchase from an Ohio dispensary, 45% stated that dispensaries were too expensive. However, 88% of participants who purchased from dispensaries stated they trusted the safety of their products. In the Pennsylvania focus group evaluation, participants reported primarily purchasing from dispensaries but having problems with product availability and consistency.

Four evaluations asked about where participants sourced their MC information. The Maryland, New Mexico, and the Pennsylvania focus group evaluations asked about individual sources of information and found that dispensaries and the internet were most widely relied upon among program participants. One state, Maryland, required a clinical director to help provide MC information at each dispensary; however, only 10% of their program participants reported utilizing this resource. Minnesota asked only about their state MC program website and call center and found that 52% reported these sources of information as helpful or very helpful.

Only two states, Minnesota and New Mexico, included a survey for qualified medical providers, as shown in online supplementary Table 2A in the supplemental material. The purpose of each survey differed, however, with Minnesota’s survey asking providers to answer questions regarding their patients’ outcomes and New Mexico’s survey surveying medical providers about their own experiences within the state’s MC program. The healthcare providers in Minnesota reported that 60% of their patients had at least a moderate improvement in their condition. Providers (33%) also mentioned a decrease in their patients’ other medications, particularly pain medications. New Mexico’s healthcare professionals responding to the survey were 63% nurse practitioners, 24% physicians, and the remaining were physician assistants, psychologists, and pharmacists. In reporting their experience with the program, 46% said the most difficult aspect of being an MC provider is lack of scientific information on MC. They also reported that the two most important factors in considering recommending MC to their patients are the patient’s condition and the patient’s previous experience with cannabis.

State MC program evaluations consistently assessed several aspects of cannabis use, including therapeutic efficacy as reported by program participants for medical and psychiatric conditions authorized for MC use (qualifying conditions), patterns of cannabis use (medical and nonmedical), and participant risk and mitigation factors associated with MC use (risk assessment and mitigation).

Evaluations included in our study consistently asked about MC use (all seven evaluations) and efficacy (four evaluations) for qualifying conditions. Five evaluations asked about MC use for conditions other than the qualifying condition, as shown in Table 3. The qualifying conditions of PTSD, pain, and anxiety/depression were reported most commonly among participants in Maryland, New Mexico, Pennsylvania, and Ohio. None of these conditions was authorized in Minnesota (only FDA-approved indications for THC-based medications were authorized), and Rhode Island’s qualifying conditions were not consistently organized by specific symptoms or diagnoses (e.g., “chronic disease/condition”), making comparison to other states difficult. Evaluations that asked about MC use for conditions other than the participants’ qualifying condition reported that participants also used MC for sleep (three evaluations), pain (three evaluations), nausea (one evaluation), and anxiety/depression (four evaluations). When asked about efficacy, most participants reported that MC improves symptoms and/or is effective/beneficial. Similarly, providers in Minnesota reported that 60% of their patients had at least moderate improvement in their condition.

With the exception of Rhode Island, all states included in our study assessed cannabis use patterns in their evaluations (see online suppl. Table 3A). Five evaluations asked about spending on MC, ranging from rating the severity of problems associated with MC costs to the dollar amount spent in a given timeframe. Every evaluation, excluding the Pennsylvania focus groups, asked about patterns of MC use, and of those, only Ohio did not follow up with questions about specific types or chemotypes of MC products. Three evaluations asked about MC for nonmedical use.

The format of questions pertaining to frequency of MC use varied substantially across evaluations. Maryland’s evaluation asked how many times per month MC was used, Ohio asked how many times per week, and the Pennsylvania survey asked how many times per day. New Mexico asked whether MC was used daily and Minnesota’s evaluation logged frequency of purchases. In all cases, participants’ frequency of MC use was high.

Additionally, Maryland reported frequency of use based on routes of administration, while New Mexico, Minnesota, and the Pennsylvania surveys were based on the chemotypes of MC used. Only evaluations in Maryland, New Mexico, and Ohio asked about nonmedical use, with 36% of participants in Maryland and 59% of participants in New Mexico reporting using MC in a nonmedical capacity. Ohio, however, asked if participants used MC primarily medicinally or recreationally.

Questions regarding risk factors associated with MC use were less common among program evaluations, as shown in online supplementary Table 4A. Three asked about polysubstance use, and two asked about cannabis use disorder. One state, Maryland, probed about problematic cannabis use. Maryland questioned whether participants drove under the influence (80% said they drove within 3 h), whether they stored MC safely (78% said they had), and if they used MC responsibly by avoiding smoking MC when around children (52% said they did). The Maryland evaluation also asked about high-risk behaviors that could lead to cannabis use disorder or driving under the influence and found one in ten participants reported engaging in high-risk behavior with cannabis. New Mexico asked whether participants were counseled about cannabis use risk factors by their primary care physician. In evaluations that asked about polysubstance use, over 50% of participants used either alcohol or tobacco concomitantly with MC. While specific questions about participant opioid use were not mentioned, a third of providers in Minnesota reported a decrease in their patients’ other medications, particularly pain medications.

In conducting our evaluation search for this review, we found that while many states invest substantial resources to build high-quality MC programs with robust internal reporting, very few have conducted evaluations with the purpose of assessing their own programs utilizing participants’ feedback [43, 44]. Such evaluations are critically valuable to MC program officials, researchers, and policy makers who desire to maximize benefit and minimize harm from MC use and optimize MC programs to the needs of their participants [7, 45, 46]. Our review is novel in its presentation of the current landscape of MC program evaluations, which may generate knowledge, expose gaps, and establish useful practices in conducting these evaluations toward achievement of specific programmatic goals [19]. While we identified several common, overarching themes among extant MC program evaluations to date, we also found considerable heterogeneity in the methodology, goals, and results. Given that a standardized program evaluation does not exist in this field, both common and heterogeneous aspects of these evaluations present potential strengths and weaknesses that may guide future program evaluation design and implementation.

Among the common themes we presented, there are aspects that we believe are strengths to the evaluation process and should be considered and standardized for future evaluations to allow for interstate comparisons. For example, assessing the process of navigating MC authorization and product access among program participants can provide experiential feedback that serves as an indicator of program effectiveness [47, 48]. Similarly, asking participants about their experience can highlight areas of concern that may not be recognized by other means, such as perception of safety from arrest or inequities in access among different patient populations.

Some aspects of evaluation heterogeneity across states may also be beneficial in the design of MC program evaluation. Because every state has developed a unique program, the specific needs or challenges of those programs require customized approaches to evaluation [12‒14]. For example, varying methodologies to garner participant feedback were used; some used surveys, while others used semi-structured interviews or focus groups. Interviews allow participants to speak freely and in-depth about particular topics, such as the certification process, educational sources, experiences in the dispensaries, and barriers to accessing MC [49, 50]. While these topics could be included in surveys, an interview approach provided richer data in areas of interest, such as in New Mexico for discovering reasons that patients use cannabis for both recreational and medical reasons simultaneously, which more effectively informed the state about MC use within the program. In Pennsylvania, an in-depth focus group approach enabled participant dialogue to elucidate specific barriers to MC access. Alternatively, using surveys can provide experiential data representative of large numbers of participants on specific issues, such as in Maryland, where they achieved their programmatic goal of examining patterns of use through a survey administered to thousands of participants [51, 52]. Thus, states may benefit from selecting methodologies that are tailored to best achieve the unique goals of their MC programs.

Despite the advantages of heterogeneity to evaluate MC program evaluations individually, we found that there are also apparent tradeoffs of an individualized approach to generalizability and comparison of results across states. First, sampling methods and sample sizes varied significantly, with differences that preclude interstate comparisons. For example, Maryland and New Mexico had comparable numbers (100,000–150,000) of program enrollees, and both used a survey design to assess participant experience; however, >13,000 (8.4%) were surveyed in Maryland, while only 102 (0.09%) were surveyed in New Mexico. Sampling methods likewise varied, including differences in sampling strategy, from sending online email to recruitment inside a dispensary, and sampling frame, such as only participants versus participants and providers. This potentially created sampling biases between groups and/or excluded populations for comparison. Second, while there are many validated survey instruments to assess MC use and participant experience (e.g., CAMS [53, 54], PGIC [55, 56], DFAQ-CU [57], MACESS [58], ASSIST [59, 60]), these survey instruments were not consistently used, making comparison and aggregation of responses to survey questions difficult across evaluations. Greater instrument consistency might enable better interstate comparisons of survey data. Third, affiliations and backgrounds of partnering evaluators differed, with an apparent difference in the depth and quality of the evaluations. Methods used in evaluations performed by non-state-affiliated groups tended to better align with MC program goals and objectives, as well as sampling methods.

Across evaluations, we found certain topics that are likely to have universal applicability to MC programs but were not included consistently. For example, assessment of illicit and/or nonmedical cannabis use has implications for participants in all state MC programs [61, 62], yet very few evaluations to date have addressed whether participants also purchased and used illicit and/or nonmedical cannabis. Nonmedical cannabis has not been tested for safety or contaminants, and its use may confound intended therapeutic effects of MC or create issues of safety for MC program participants [63, 64]. As another related example, assessment of various risk factors related to cannabis use has broad applicability and could benefit all MC programs. These assessments help determine prevalence of issues such as children accessing of MC products [24, 28], driving under the influence [65], cannabis misuse [30, 31], and/or risk for cannabis use disorder [66, 67]. Broadening the scope of the evaluations to include assessment of other aspects of the MC program, such as including surveys for medical professionals, dispensaries, and laboratories for objective testing of MC products, would strengthen any evaluation.

We note several limitations with this review. First, the number of evaluations that met criteria for our review was small, limiting generalizability of our conclusions. Nevertheless, we believe our results are representative of the scientific and gray literature and state-sponsored online material, based on our comprehensive search strategy in these areas. Second, we acknowledge that our analysis and conclusions are based on expert opinion, rather than evidence-based guidelines, as such guidelines are currently lacking in MC program evaluation. Over time, we expect that as evidence grows, such guidelines will be developed. Third, heterogeneity in evaluation format, reporting, and publication outlet (e.g., website vs. peer-reviewed literature) made some comparisons and synthesis difficult. However, this tradeoff allowed for a broader search strategy with more inclusive results. Lastly, we recognize that excluding internal agency reporting may have excluded relevant information regarding MC programs, but we chose to exclude any reports that had no element of evaluation by examination of the data, remarks on their impact on the program, and appraisal of relevance to the program goals.

MC program evaluation is critical for states to assess program effectiveness and identify actionable changes to policy and practice that will increase effectiveness and improve outcomes. Although many states have MC programs, we found only seven independently conducted evaluations completed to date, with both common themes and heterogeneity across these evaluations providing insight into a robust evaluation process. Accordingly, the following recommendations may be helpful to MC program officials, policy makers, providers, and participants: (1) prioritize partnership with an experienced third party independent of and unaffiliated with the state MC program to design and complete the evaluation; (2) define goals clearly to align the evaluation with distinct objectives of the state MC program; (3) assess issues universally applicable, regardless of program, including use of nonmedical cannabis, MC misuse, driving under the influence, treatment access, and safe use practices; (4) utilize standardized evaluation instrument practices and comparison cohorts where feasible; and (5) tailor methods to minimize bias and address specific areas of inquiry aligned with program goals and objectives.

The authors would like to thank Lydia Howes at the Spencer S. Eccles Health Library for her assistance in collecting literature sources and her guidance with our search strategy.

The authors have no conflicts of interest to declare.

The funds to support this review were allocated by the Utah Department of Health, under contract #236204 titled “Utah Medical Cannabis: A Prospective Cohort Evaluation” with Dr. Gerald Cochran as Principal Investigator for the time period of December 1, 2023, through March 31, 2025. The funder requested that an assessment of existing evaluations be conducted. The funder had no role in the design, data collection, data analysis, and reporting of this study. The content of this narrative review does not reflect the views of the funder.

L.F., G.C., C.R., M.A.I., C.J.H., A.J.G., and A.T.K. all contributed to the conception and design of the project. L.F., C.R., and C.J.H. conducted preliminary search strategies and reviewed abstracts and full papers. A.T.K. and G.C. supervised the search strategy and finalized decisions on inclusion. L.F., G.C., C.R., M.A.I., A.J.G., and A.T.K. assisted in the search strategy and analysis of the evaluations. L.F. wrote the first draft. C.R. assisted in writing and editing. M.A.I., G.C., A.J.G., and A.T.K. assisted in structuring and analysis of results, writing, and editing. All authors provided critical feedback.

The data that support the findings of this study are openly available in PubMed, APA PsychINFO (EBSCOhost), Medline, Embase, Scopus, Google Scholar, Google, and the individual state medical cannabis program websites.