Treatment of Patients with Multiple Myeloma: An Overview of Systematic Reviews

In the United States, cancer is a major public health problem and the second most common cause of death, with 1 in 4 deaths attributed to this disease [1]. Ten percent of all cancers are hematological malignancies, and among these malignancies, multiple myeloma (MM) accounts for 14% of new cases each year [1].

The last decade has witnessed major progress in the management of MM. The median survival of 3 years in the 1960s to mid-1990s has significantly and meaningfully improved to about 5 years from the late 1990s to today [2]. To a large extent, this improvement in survival has been possible through the introduction of novel therapies by rigorous testing in randomized controlled trials (RCTs) [2]. Well-designed RCTs generate the most reliable evidence about any particular treatment intervention of interest to health care professionals and their patients. They are considered the ‘gold standard’ in testing comparative effects of competing interventions [3,4,5]. However, individual studies rarely provide definitive answers related to whether a given intervention is more beneficial than harmful [6]. More reliable answers can be obtained by performing research synthesis of the entire body of evidence (‘totality of evidence’) on a given topic through the methodology of a systematic review.

A systematic review is a scientific investigation that focuses on a specific question and uses explicit, pre-planned scientific methods to identify, select, assess and synthesize all relevant studies on a specific topic [7]. Systematic reviews of the totality of evidence on benefits and harms should form the basis for rational medical decision-making [8,9]. Accordingly, the objective of this overview is to summarize all existing systematic reviews on MM treatments.

We searched the Clinical Queries section of the PubMed database to identify systematic reviews using MeSH terms for MM. In addition, the Cochrane database of systematic reviews was searched to identify systematic reviews that may have been missed by the PubMed search. The search was performed from its inception to November 2009.

All systematic reviews of phase III RCTs assessing the effect of treatments on MM were eligible for inclusion in this overview. Systematic reviews of observational studies and other study designs were excluded.

All citations and relevant abstracts were reviewed by two independent reviewers (S.G. and A.K.) for their inclusion in the overview. If the decision on inclusion was not possible from the abstract, a full text review was undertaken to reach a decision. Any disagreement was resolved by consensus.

Data extraction was performed using a data extraction form. Data on patients, intervention, control and outcomes (PICO) were extracted from each systematic review. In case a meta-analysis was performed as part of the systematic review process, quantitative data on outcome were extracted for each PICO question. If there were multiple systematic reviews addressing the same PICO question, we extracted summary data from the most complete and recent systematic review. We also evaluated the methodological quality of the systematic reviews using the AMSTAR [10] assessment tool.

The initial search identified 277 citations. Upon further evaluation, 11 studies qualified as systematic reviews [11,12,13,14,15,16,17,18,19,20,33] and met the inclusion criteria. The flow diagram illustrating the selection process is depicted in figure 1. Of the 11 systematic reviews, 10 addressed one unique clinical question across different stages of treatment and performed and reported a meta-analysis on outcomes of interest [11,12,13,14,15,16,17,18,19,20,33]. One systematic review performed in 2003 by our group summarized the evidence from all RCTs, and systematic reviews and meta-analyses of RCTs related to key clinical questions in MM management [21]. Key questions addressed by the systematic reviews and a summary of results to the questions are presented here according to the treatment sequence of the disease (tables 1, 2, 3, 4, 5, 6, 7, 8).

Question 1: Early versus Deferred Treatment for Early-Stage MM

Benefits. We identified one systematic review [16] of 3 RCTs enrolling 262 patients with stage I disease comparing early versus deferred treatment for early-stage MM (table 1). The review found that early treatment significantly reduced myeloma progression [odds ratio (OR) 0.16, 95% confidence interval (CI) 0.09–0.29]. However, there was no significant difference between early and deferred treatment in survival (OR 1.11, 95% CI 0.67–1.84), response rate (OR 0.63, 95% CI 0.33–1.23) or the likelihood of vertebral compressions (OR 0.18, 95% CI 0.02–1.59).

Harms. The systematic review also found that early treatment with melphalan plus prednisolone was associated with a statistically non-significant increase in acute leukemia compared with deferred treatment (OR 3.20, 95% CI 0.55–18.73).

Comments. The authors of this systematic review also noted that, to achieve a difference of 10% reduction in mortality between the two regimens, 800 people would have to be enrolled in the study. However, this meta-analysis with 262 people had power of only 51%, which means that the chance that findings are false negatives is 49%.

This systematic review fulfilled 9 of 11 criteria of the AMSTAR instrument. Of note, preliminary results from a recent RCT (n = 40) evaluating the effects of early versus deferred treatment with the novel agent lenalidomide in combination with dexamethasone in smoldering myeloma also showed that early treatment is associated with significantly reduced progression of disease with no effect on survival [22].

Question 2: Combination Chemotherapy versus Melphalan plus Prednisone as Treatment for MM

Benefits. We found 2 systematic reviews [11,15] evaluating the efficacy of various combination chemotherapies versus melphalan plus prednisone (MP) for MM treatment. The first systematic review by Gregory et al. [15] was published in 1992 and included published data from 18 RCTs comprising 3,814 patients. The second systematic review by the Myeloma Trialists’ Collaborative Group published in 1998 included data from 30 RCTs involving 6,633 patients [11]. The most current systematic review [11] included individual-patient data (IPD) from 20 RCTs involving 4,930 patients and published data from 7 RCTs (1,703 patients). Since the systematic review by the Myeloma Trialists’ Collaborative Group [11] was the most updated and included all RCTs previously presented by Gregory et al. [15], the results from this review are summarized here (table 2).

The pooled results showed no difference in survival with combination chemotherapy versus MP. ORs were 0.98 (95% CI 0.92–1.04) for survival with IPD and 1.03 with published data (95% CI 0.83–1.25). The summary OR, including IPD and published data, was 0.98 (95% CI 0.926–1.05). The response rate was significantly better with combination chemotherapy versus MP (60.2% with combination chemotherapy vs. 53.2% with MP; p < 0.00001).

Harms. Neither of the systematic reviews reported data on morbidities or treatment-related mortality.

Comments. Although this review may be considered of historical interest at this time, we have included it here because it was one of the most influential research syntheses in the field of myeloma. The use of combination chemotherapy dramatically ceased after publication of this systematic review.

The systematic review by the Myeloma Trialists’ Collaborative Group [11] fulfilled 6 of 11 criteria of the AMSTAR instrument.

Question 3: High-Dose Therapy with Single Autologous Hematopoietic Stem Cell Transplant versus Chemotherapy for Newly Diagnosed MM Patients

We found 2 systematic reviews assessing the efficacy of high-dose chemotherapy with single autologous hematopoietic stem cell transplant (AHCT) versus conventional chemotherapy (tables 3, 4) [18,19]. The first systematic review by Levy et al. [19] performed an IPD meta-analysis using data from 3 RCTs and reported a statistically non-significant difference in survival between single AHCT and conventional chemotherapy (hazard ratio, HR, 0.89, 95% CI 0.743–1.083). The second systematic review by Koreth et al. [18] included 10 RCTs enrolling 2,411 patients primarily with stage II or III disease. The pooled results showed a statistically non-significant difference in survival between high-dose chemotherapy plus stem cell rescue (HR 0.92, 95% CI 0.74–1.13, p = 0.40) versus chemotherapy. For the outcome of progression-free survival, there was a statistically significant benefit associated with high-dose chemotherapy plus stem cell rescue (HR 0.75, 95% CI 0.59–0.96, p = 0.02).

Both systematic reviews reported a statistically significant increase in treatment-related mortality associated with single AHCT versus conventional chemotherapy. The systematic review by Levy et al. [19] reported an HR of 5.27 (95% CI 1.43–19.4). Koreth et al. [18] also found an increased treatment-related mortality associated with high-dose chemotherapy plus stem cell rescue (OR 3.01, 95% CI 1.64–5.50, p < 0.01).

Comments. The systematic review by Levy et al. [19], involving IPD meta-analysis, as well as the review by Koreth et al. [18] failed to demonstrate any significant difference in survival for single AHCT compared with conventional chemotherapy. In the systematic review by Koreth et al. [18], of 10 included RCTs, data were available for 9 RCTs for the main outcomes of overall and progression-free survival (1 RCT was published in abstract form, thereby precluding accurate data extraction). Data on adverse events were available from all 10 RCTs. All RCTs included in the systematic review were well designed and of high quality. However, information on power analysis was reported for all but 3 RCTs. This systematic review also explored the effect of dose (high vs. intermediate dose), different conditioning regimens, timing of transplant (early vs. late) and status of the disease at transplant (chemo-responsive disease vs. not) and found essentially the same effect in all subgroup analyses.

The systematic review by Levy et al. [19] fulfilled 6 of 11 possible criteria of the AMSTAR instrument, and the systematic review by Koreth et al. [18] fulfilled 9 of 11 criteria.

Question 4: Tandem versus Single AHCT as First-Line Therapy for Newly Diagnosed Patients

Benefits. We identified one systematic review including 5 RCTs enrolling 1,608 patients (table 5) [20,33]. The pooled results from this systematic review showed a statistically non-significant difference between tandem AHCT and single AHCT for the outcome of overall survival (HR 0.89, 95% CI 0.76–1.04; p = 0.16). The event-free survival was significantly better with the use of tandem AHCT (HR 0.79, 95% CI 0.70–0.89; p < 0.0001). The response rate was also significantly better with tandem AHCT [risk ratio (RR) 0.78, 95% CI 0.66–0.92; p = 0.003].

Harms. The pooled results of data from 4 RCTs (1,381 patients) showed a statistically non-significantly higher risk for treatment-related mortality (RR 1.65, 95% CI 0.99–2.75) associated with the use of tandem AHCT.

Comments. The role of tandem AHCT as routine up-front treatment is uncertain. Most centers use it in patients with poor prognostic features and inadequate response to induction treatment. However, this practice is based on the results from a single RCT [23].

This systematic review was well performed and fulfilled 10 of 11 criteria of the AMSTAR instrument.

Question 5: Role of Thalidomide in Previously Untreated MM Patients

Benefits. We found 1 systematic review of 13 RCTs enrolling 4,144 patients that assessed the efficacy of thalidomide as induction therapy (9 RCTs), maintenance therapy (3 RCTs), and combined induction and maintenance therapy (1 RCT; table 6) [17]. Addition of thalidomide to the induction regimen of MP or dexamethasone-based regimens resulted in a statistically significant improvement in overall survival (4 RCTs, 1,540 patients; HR 0.67, 95% CI 0.56–0.81; p < 0.0001). The use of thalidomide as maintenance therapy was associated with borderline statistically significant improvement in overall survival (4 RCTs, 1,703 patients; HR 0.61, 95% CI 0.37–1.01; p = 0.056). However, there was a statistically significant heterogeneity among the pooled trials assessing the efficacy of thalidomide as maintenance regimen. The authors of this systematic review attributed the heterogeneity to the RCT by Barlogie et al. [31], as this study had uniquely used thalidomide in the induction as well as the maintenance phase of treatment. Removing the RCT by Barlogie et al., the analysis resulted in a statistically significant benefit for survival (HR 0.49, 95% CI 0.32–0.74; p = 0.0007). In addition, the RCT by Abdelkefi et al. [32] was retracted. However, removing their study from the analysis did not change results (HR 0.46, 95% CI 0.32–0.66; p < 0.0001). Response rate was significantly better with thalidomide in RCTs that planned or allowed transplant (4 RCTs, 1,537 patients; RR 1.39, 95% CI 1.23–1.57; p < 0.00001) as well as in RCTs that did not involve transplant (5 RCTs, 1,392 patients; RR 1.60, 95% CI 1.33–1.94; p < 0.00001).

Harms. The systematic review by Hicks et al. [17] also reported an increased frequency of grade III/IV adverse events associated with the use of thalidomide across all included studies. However, data on grade II/IV adverse events were not extractable from the meta-analysis. Pooled results showed a statistically significant risk for venous thromboembolism associated with thalidomide use in induction (10 RCTs, 3,109 patients; RR 2.56, 95% CI 1.88–3.49; p < 0.00001) as well as maintenance RCTs (3 RCTs, 1,491 patients; RR 1.95, 95% CI 1.15–3.30; p = 0.01).

Comments. Thalidomide appears to improve the overall survival of patients with newly diagnosed myeloma when used as induction and maintenance therapy. However, its use is also associated with a high risk of adverse events, such as venous thromboembolism, leading to frequent therapy discontinuation. If a patient is administered thalidomide, prophylactic treatment against venous thromboembolism is mandatory.

This systematic review was well performed and fulfilled 10 of 11 criteria of the AMSTAR instrument.

Question 6: Interferon-α for MM Treatment

Benefits. We found 2 systematic reviews addressing the efficacy of interferon-α (IFNα) for MM treatment (table 7) [12,14]. The systematic review by Fritz and Ludwig [14] utilized published data and included 30 RCTs enrolling 3,948 patients, and the systematic review by the Myeloma Trialists’ Collaborative Group [12] was based on IPD from 24 RCTs enrolling 4,012 patients.

Induction Treatment. The systematic review by Fritz and Ludwig [14] evaluated 17 RCTs enrolling 2,333 patients and reported a statistically significant difference for response rate (6.6% higher with IFNα versus control; p < 0.002), and prolongation of relapse-free (4.8 months; p < 0.01) and overall survival (3.1 month; p < 0.01) favoring IFNα versus control.

The systematic review by the Myeloma Trialists’ Collaborative Group [12] performed an IPD meta-analysis using data from 12 RCTs involving 2,469 patients. Response rates (57.5% with IFNα vs. 53.1% with controls; p = 0.01) and progression-free survival (OR 0.781, 95% CI 0.683–0.893; p = 0.0003) were significantly better with IFNα versus control. However, there was no difference in overall survival between IFNα and control.

Maintenance Treatment. The systematic review by Fritz and Ludwig [14] evaluated 13 RCTs enrolling 1,615 patients and reported a statistically significant difference for relapse-free (4.4-month prolongation; p < 0.01) and overall survival (7.0-month prolongation; p < 0.01) with IFNα versus control.

The IPD meta-analysis [12] evaluated 12 RCTs enrolling 1,543 patients and also reported a statistically significant difference for progression-free (OR 0.66, 95% CI 0.587–0.742; p < 0.00001) and overall survival (OR 0.875, 95% CI 0.772–0.992; p = 0.04) with IFNα versus control.

Harms. Neither systematic review assessed or reported data on harms.

Comments. Overall results combining data from the induction and maintenance phases of studies showed a statistically significant benefit with IFN use for the outcomes of progression-free (OR 0.71, 95% CI 0.65–0.77) and overall survival (OR 0.90, 95% CI 0.837–0.976) [12]. This was also confirmed in the meta-analysis by Fritz and Ludwig [14]. Their systematic review also included a cost-effectiveness estimation: the costs of 1-year survival were USD 42,482.28 during the induction phase and USD 18,968.16 during the maintenance phase of treatment [14]. Although none of the systematic reviews reported data on harms, IFN is currently not recommended for use in clinical practice due to an association with increased risk for adverse events.

Both systematic reviews fulfilled only 4 of 11 criteria of the AMSTAR instrument.

Question 7: Role of Bisphosphonates in MM

Benefits. We identified 1 systematic review of 17 RCTs enrolling 3,010 patients (table 8) [13,24,25]. In comparison to placebo/no treatment, the pooled analysis demonstrated a beneficial effect of bisphosphonates on prevention of pathological vertebral fractures (7 RCTs, 1,116 patients; RR 0.74, 95% CI 0.62–0.89; p = 0.001), skeletal-related events (6 RCTs, 1,334 patients; RR 0.81, 95% CI 0.72–0.92; p = 0.001) and amelioration of pain (8 RCTs, 1,281 patients; RR 0.75, 95%; CI 0.60–0.95, p = 0.01). There was no significant effect of bisphosphonates on survival, progression-free survival, hypercalcemia or the reduction of non-vertebral fractures.

Harms. There were no significant adverse effects associated with the administration of bisphosphonates. Only two RCTs reported data on osteonecrosis of the jaw (ONJ). The systematic review also included observational studies for assessment of adverse events. Data from 7 observational trials evaluating 1,068 MM patients for ONJ suggest that ONJ may be a common event (range: 0–51%). Because ONJ was only sporadically reported in RCTs, the results from observational studies may be an overestimate due to their non-controlled design.

Comments. A statistically significant heterogeneity among included trials for the outcome of survival and pain was noted. The heterogeneity for the outcome of pain was explained by variation in the scales used to measure pain. The heterogeneity for overall survival was attributed to 1 RCT with unrealistic treatment effects (‘an outlier effect’). This systematic review also performed an indirect meta-analysis to assess the potential superiority of one type of bisphosphonates over another. The results of indirect meta-analysis showed no difference among any specific type of bisphosphonates.

This systematic review fulfilled all 11 criteria of the AMSTAR instrument.

In 2003, we performed a systematic review of RCTs in MM that examined the effects of various treatment modalities on outcome in patients with MM and addressed 21 specific clinical questions related to the management of this disease [21]. Evidence for effectiveness of competing treatment alternatives were assessed along with the critical appraisal of the available body of evidence. This systematic review concluded that the overall quality of the body of evidence for myeloma management was poor. Many trials were done with small sample sizes, the majority of studies had inadequate allocation concealment, and few were analyzed according to the intention-to-treat principle. This systematic review concluded that the quality of total evidence supporting treatment recommendations in myeloma is modest at best and has an ample scope for improvement. Reassessment of the quality of evidence in 2008 [2] indicated that the quality of individual trials, including sample size, had significantly improved since 2003.

In 2003, our systematic review of the available body of evidence in myeloma found that the two most important therapeutic advances in the management of myeloma related to the introduction of high-dose chemotherapy, which appeared to be superior to conventional chemotherapy, and bisphosphonates, which decrease the probability of pathological vertebral fractures [26]. This overview, which in fact represents an update of the 2003 review, indicates that high-dose chemotherapy may be associated with superior event-free survival but not overall survival. The conclusions related to the effects of bisphosphonates remain the same. Other important findings that have emerged from our overview of systematic reviews include the following: early treatment does not offer therapeutic benefit [16], thalidomide is associated with improved survival as maintenance or induction treatment when added to standard MP therapy [17] in non-transplant patients, and tandem AHCT does not prolong survival when compared to single AHCT [20,33].

Unfortunately, our current overview failed to identify a research synthesis on the effects of novel agents, such as the proteosome inhibitor bortezomib or the immunomodulatory agent lenalidomide, which are associated with undeniable progress in the management of myeloma [2]. These agents, either alone or combined with other agents, have been credited with vast improvements in the outcome of patients with myeloma. Research synthesis of the available evidence on the relative effects of treatments based on these novel agents has not been performed to date. This is rather unfortunate because without consideration of the total body of evidence on benefits and harms of treatments, both physicians and patients remain confused when they are presented with numerous individual reports in the literature, particularly when these reports generate conflicting results, as is often the case. Systematic review of the entire body of evidence on a given health topic represents the rational platform for informed decision-making [27].

The results of our overview show that, despite the importance of systematic reviews and meta-analysis in making informed decisions, there are only 11 systematic reviews in the field of MM [11,12,13,14,15,16,17,18,19,20,21]. A majority of these systematic reviews (n = 9) and meta-analyses utilized published data [12,13,14,15,16,17,18,20,21,33], and 2 [12,19] were based on IPD, which are considered the ‘gold standard’ for assessing the effects of health care interventions.

Our overview provides the impetus for two lines of future research endeavors: (1) the urgent need to perform research synthesis on the effects of novel agents in the management of myeloma and (2) the improvement in standards for conducting and reporting systematic reviews themselves.

Our overview found a remarkable variation in the quality of reporting a systematic review (table 9). Two systematic reviews fulfilled all 11 criteria of the AMSTAR instrument [13,17], 1 fulfilled 10 [20,33] and 2 fulfilled 9 of 11 criteria [16,18]. The remaining systematic reviews fulfilled 6 (n = 2) [19] and 4 (n = 3) [11,14,15] criteria, respectively. However, it is important to note that the AMSTAR instrument was first published in 2007 [10], and its impact on reporting of systematic reviews remains to be seen. Thus, it may be incorrect to judge the quality of systematic reviews performed before the advent of the AMSTAR instrument. Nevertheless, it is important to emphasize that systematic reviews on MM performed under the auspices of the Cochrane Collaboration [28] fulfilled all AMSTAR criteria even before the AMSTAR instrument was published. Therefore, we believe there is a scope for improvement in the quality of systematic reviews. We should also note that the quality of reporting versus the actual quality of the systematic review cannot often be ascertained with the exception of systematic reviews published by the Cochrane Collaboration. Only the Cochrane Collaboration makes all protocols available in the public domain, thus permitting the comparison of what was planned to what was performed. The findings of this overview should encourage policy makers to create a central registry of systematic review protocols, similar to clinical trials, and a peer review process that will help to improve the quality of conducting and reporting systematic reviews.

The major problem for current practice is that systematic reviews on the effects of novel agents are not available; hence, practitioners, as well as guideline developers, are left to pick and choose among hundreds of individual reports to guide their treatment recommendations. Nevertheless, even when systematic reviews do exist, evidence from systematic reviews on MM is not incorporated into clinical practice. For example, despite the availability of well-performed systematic reviews on several key questions in myeloma, the evidence is not cited in the guidelines of the National Comprehensive Cancer Center Network [29], which are widely used by practicing oncologists. There are 2 systematic reviews comparing the efficacy of high-dose therapy plus single AHCT versus chemotherapy [18,19]. One systematic review was based on published data [18] and another used IPD [19]. Both concluded that there is no survival benefit with high-dose therapy plus single AHCT versus chemotherapy. Similarly, data from another systematic review of RCTs comparing single versus tandem AHCT showed no survival benefit with tandem AHCT [20,33]. However, none of these well-performed systematic reviews are referenced in the myeloma guidelines of the National Comprehensive Cancer Network. It is worrisome that the dissemination of evidence lags behind research efforts. However, these findings are in line with previous analyses on dissemination of research, which concluded that ‘despite the considerable amount of money spent on clinical research, relatively little attention is paid to ensuring that the findings of research are implemented in routine clinical practice’ [30]. With the current emphasis on comparative effectiveness research [6], we hope that more clinical decisions will be supported by synthesized evidence based on the totality of research rather than relying on unacceptable practice of making decisions based on selective citations of evidence.

Apart from the Cochrane systematic reviews, which are updated on a regular basis as new RCTs are completed and published, most of the other systematic reviews are not updated regularly; thus, evidence from earlier systematic reviews might be irrelevant to current practice. It appears very unlikely that findings would change in a significant manner as new RCTs are performed; nevertheless, for a systematic review to be relevant, it needs to be continuously updated.

In summary, the results from our overview show that, in spite of the recognition of systematic reviews and meta-analysis as being critical to decision-making, few systematic reviews address the treatment options in MM. Some of these reviews, such as the comparison between combination chemotherapy and MP, or perhaps even the use of IFN, are not relevant to current practice. Nevertheless, even when data from a systematic review are available, they do not appear to be incorporated in treatment guidelines. The most worrisome problem is that research synthesis of the widely promulgated novel agents in the management of myeloma is yet to be performed.