Background: The alliance between therapists and patients is one of the most investigated factors of psychotherapy and a robust predictor of treatment outcome in face-to-face interventions. The implementation of internet-based interventions for psychological disorders has become more and more relevant over the past years, particularly in European countries. Regarding the relevance of the alliance for therapy outcome in internet-based interventions, mixed results are reported. The present correlative meta-analysis systematically summarizes the relation between alliance and therapy outcome in internet-based interventions for psychological disorders. Methods: Based on a systematic search, we detected 18 internet-based studies with 23 independent samples (1,177 participants). Results: The omnibus revealed a moderate effect of r = 0.252, p < 0.001, 95% CI [0.19, 0.31]. The analyses of the subscales indicated that the task component (r = 0.28) was slightly more predictive than the bond component (rbond = 0.16; QM(3) = 26.85, p < 0.001). None of the investigated moderators indicated significance. Discussion: This meta-analysis underlines the assumption that the alliance is a robust facilitative factor across mental health treatments.

Hintergrund: Die therapeutische Beziehung zwischen Patient*in und Therapeut*in (Allianz) in herkömmlichen Face-to-Face-Interventionen ist einer der am meisten untersuchten Faktoren der Psychotherapie und ein robuster Prädiktor des Therapieerfolgs. Zahlreiche Forschungs­ergebnisse weisen bei internetbasierten Ansätzen, die einen Kontakt mit Fachleuten beinhalten, auf gute bis sehr gute Behandlungseffekte hin. Bezüglich der Bedeutung der Allianz für den Therapieerfolg internetbasierter Interventionen werden jedoch uneinheitliche Ergebnisse diskutiert. Ziel der vorliegenden Metaanalyse ist, den Zusammenhang zwischen Allianz und Therapieerfolg internetbasierter Interventionen systematisch zu untersuchen. Material und Methode: Durch die systematische Literaturrecherche konnten 18 Studien zu internetba­sierten Interventionen mit 23 unabhängigen Stichproben und 1’177 Teilnehmenden eingeschlossen werden. Ergebnisse: Die gewichtete durchschnittliche Gesamt-Effektstärke betrug r = 0,252, p < 0,001, 95% CI [0,19, 0,31], was einen moderaten Zusammenhang mit einer Vari­anzaufklärung von ungefähr 6% hinsichtlich des The­rapieerfolgs darstellt. Dabei war die Übereinstimmung zwischen Patient*in und Therapeut*in bezüglich der Aufgaben innerhalb der Behandlung (rtask = 0,28) leicht prädiktiver für den Therapieerfolg als die emotionale Bin­dung zum*zur Therapeut*in (rbond = 0,16; QM(3) = 26,85, p < 0,001). Die Effektstärken zeigten sich homogen und es wurden keine signifikanten Moderator-Variablen identifiziert. Diskussionen und Schlussfolgerung: Auch bei internetbasierten Interventionen kommt der Allianz eine wichtige Rolle in Bezug auf den Therapieerfolg zu.

SchlüsselwörterArbeitsallianz, Internetbasierte Therapie, Allgemeine Wirkfaktoren

The collaborative quality of the therapist-client relationship (alliance) is worldwide one of the most investigated factors of therapy process [Flückiger et al., 2018b]. In 1979, Bordin proposed a transdiagnostic, pantheoretical concept of the working alliance, which consists of the following three interconnected components: (a) agreements on the therapeutic goals (goal), (b) consensus on the tasks that make up therapy (tasks), (c) a bond between the client and the therapist (bond). Out of the big amount of alliance measure instruments, the most used alliance measure is the Working Alliance Inventory (WAI) [Flü­ckiger et al., 2018b]. The overall weighted average effect size of r = 0.28 for the alliance-outcome association indicates the alliance as a very robust predictor of treatment outcome and supports the pantheoretical conception of the alliance empirically [Norcross and Lambert, 2011; Flückiger et al., 2012; Norcross and Lambert, 2018].

Even though effective psychotherapies exist, many people with psychological disorders remain untreated [Kohn et al., 2004]. Reasons for this so-called treatment gap may be, among others, a perceived stigmatization, financial occasions, a lack of availability of services, or geographical accessibility [Mechanic, 2007]. These obstacles can be overcome in part by modern information and communication media. The use of internet-based interventions for psychological disorders became more and more important over the past years [Andersson, 2016; Titov et al., 2018]. In internet-based interventions the internet can be exclusively used either as an information medium in so-called unguided self-help or as a communication medium in so-called internet-based psychotherapies [Berger, 2015]. The most investigated type of internet-based interventions are guided self-help approaches, in which a combination of the internet as an information and communication medium is used. In this type of internet-based interventions the patients receive regular therapist online contact in addition to the web-based self-help program. Regarding the efficacy of internet-based interventions for psychological problems and disorders, findings, especially for guided self-help approaches, indicate medium to large effects which are equivalent to face-to-face interventions [Barak et al., 2008; Wagner and Maercker, 2011; Klasen et al., 2013; Andersson et al., 2014; Wagner et al., 2014; Berger, 2015; Carlbring et al., 2018]. Furthermore, several systematic reviews and meta-analyses indicate a superiority of guided self-help over unguided self-help regarding efficacy [Palmqvist et al., 2007; Spek et al., 2007; Baumeister et al., 2014]. Research on so-called blended treatments, where online and face-to-face interventions are combined, is still scarce [Kleiboer et al., 2016].

Considering the increasing implementation of internet-based interventions in the treatment of psychological disorders and the relevance attributed to alliance in general, the question arises how these two aspects conceptually and empirically are related to each other and what relevance is attached to alliance in internet-based interventions for psychological disorders [Cook and Doyle, 2002; Knaevelsrud and Maercker, 2006; Sucala et al., 2012; Berger, 2017]. A current meta-analysis indicated an alliance-outcome correlation that was quite similar to that found for face-to-face psychotherapy (r = 0.27) [Flückiger et al., 2018b]. Berger [2017] points out the heterogeneous significance levels of alliance-outcome correlations in guided self-help approaches, resulting in significant alliance-outcome correlations [Bergman Nord­gren et al., 2013; Knaevelsrud et al., 2015], nonsignificant alliance-outcome correlations [Knaevelsrud and Maer­cker, 2007; Preschl et al., 2011; Andersson et al., 2012; Jasper et al., 2014] as well as mixed results [Berger et al., 2014]. Based on findings indicating a positive relationship between the amount of therapeutic contact and the outcome of therapy [Palmqvist et al., 2007; Spek et al., 2007; Baumeister et al., 2014], a moderating influence of the amount of therapeutic contact on the alliance-outcome correlation seems plausible. Beyond that, it is assumed that a face-to-face contact is conducive to the building of a therapeutic relationship and hence could moderate the relationship between alliance and outcome. Furthermore, findings suggest mixed results concerning predictors of therapy outcome such as the influence of the age of patients on the possibility to benefit from an internet-based intervention [Barak et al., 2008; Karyotaki et al., 2018]. “Specificity of outcome” and time of the alliance assessment turned out to be significant moderators of the alliance-outcome relation in face-to-face psychotherapies [Flückiger et al., 2018b]. The relation between alliance and outcome was higher when the outcome measure was not disorder-specific but a general symptom measure and when the alliance was measured later in the therapy process. It is assumed that these moderating influences also play a role in internet-based interventions.

From a conceptual angle, the differentiation in the alliance-subscales seems to be of particular interest. Especially the question to what extent the quality of the bond-subscale is relevant for success of internet-based interventions arises [Berger, 2017]. Descriptively (based on nonsystematic reviews) the subscales seem to be quite heterogeneous, while the alliance on the subscale level also appears to relate positively to the outcome in most cases. Higher relations are hypothesized for the task and goal subscales compared to the bond subscale [Knae­velsrud and Maercker, 2006; Berger et al., 2014; Knae­velsrud et al., 2015; Meyer et al., 2015; Berger, 2017], which raises the issue to what extent the alliance concept can be adapted conceptually to internet-based interventions [Berger, 2017; Pihlaja et al., 2017]. How far the alliance subscales in internet-based interventions predict therapy outcome cannot be appraised systematically due to the limited number of studies and the descriptive presentation of the reviews [Berger, 2017]. That is exactly the point where the present meta-analytic investigation starts by addressing the following questions.

  1. Given that the alliance is positively associated with therapy outcome [Flückiger et al., 2018b], we pose the question to what extent this alliance-outcome correlation in internet-based interventions is moderated by the amount of therapeutic contact and other potential moderators. The following potential moderators are examined: (1) amount of therapeutic contact; (2) occurrence of a face-to-face contact; (3) average age of participants; (4) treatment duration; (5) specificity of outcome; (6) time of the alliance assessment.

  2. Is there a significant difference between the effect sizes of the alliance subscales (goal, task, bond)?

The methodological conception of the present meta-analysis was conducted according to the MAP-24 checklist [Flückiger et al., 2018a].

Literature Search and Selection of Studies

The literature search was based on an already conducted literature search of Flückiger et al. [2018b]. The flowchart in Figure 1 provides an overview of the literature search and selection of studies with reasons for exclusion of manuscripts. The criteria for inclusion of a study in this meta-analysis were: (1) the author referred to the therapy process variable as helping alliance, working alliance, or therapeutic alliance; (2) the data reported allow for extracting or calculating a measure for the relation between alliance and outcome; (3) the study includes an alliance measure during treatment; (4) the participants were adults; (5) the manuscript was written in English, German, Italian, or French; (6) the study includes the investigation of an electronic or internet-based intervention. Criteria for exclusion of a study were (1) a nonclinical sample and (2) a qualitative study or less than 5 participants. Overall, resulting from the literature search there were 18 studies with 23 independent samples and 1,177 participants included in the meta-analysis. In addition, we included the only study with an adolescent sample we know, with an average age of 12.1 years. Ta­ble 1 summarizes the studies contained in the current meta-analysis.

Table 1.

Study characteristics of the independent samples included in the meta-analysis

 Study characteristics of the independent samples included in the meta-analysis
 Study characteristics of the independent samples included in the meta-analysis
Fig. 1.

Flowchart of the literature search and selection of studies. AOC, alliance-outcome correlation.

Fig. 1.

Flowchart of the literature search and selection of studies. AOC, alliance-outcome correlation.

Close modal

Coding of the Included Studies

The measure for the relation between alliance and outcome extracted from the included studies contained alliance-outcome correlations, so that the effect sizes underlying the correlative meta-analysis were correlation coefficients. Positive correlations were defined as the relation between high alliance values and better treatment outcome. If the relation between alliance and outcome was reported using a regression analysis, these values were converted into correlation coefficients. If the data reported for the alliance-outcome relation contained both correlation and regression measures, correlation measures were preferred for the subsequent analysis. If exclusively standardized beta values were reported (e.g., adjusting for confounding variables), these were integrated in the meta-analysis. The statistical parameters were requested from the author if not mentioned exactly in the report [Berger et al., 2014].

The following moderators were coded: (1) The amount of therapeutic contact during the intervention was classified into three categories according to the categorization of Berger [2015]. (a) The first category consisted of interventions that only use the internet to provide information in so-called internet-based unguided self-help programs. These programs do not require any contact with a clinician during treatment, therapeutic contact at the most exists during the diagnostic procedure, wherefore this category represents the one with the least amount of therapeutic contact. (b) To the second category the so-called internet-based guided self-help approaches, in which the internet is used both as an information and as a communication medium, were allocated. Despite the regular therapist online contact, compared to face-to-face interventions the amount of therapeutic contact is less. (c) The third category consisted of so-called internet-based psychotherapies that used the internet exclusively as a communication medium. The amount of therapeutic contact is approximately equivalent to the one in face-to-face interventions. The classification in categories of the studies was based on the self-reported description of the authors. (2) The occurrence of a face-to-face contact was coded as a dichotomous variable (face-to-face contact vs. no face-to-face contact) and dichotomizes the scale outlined above. Any face-to-face contact with a clinician or support personnel which took place in the context of the study was classified as a face-to-face contact in this connection. (3) The average age of participants was coded on the basis of the descriptive data of the original studies. (4) The reported treatment duration was coded in weeks. (5) The moderator variable “specificity of outcome” refers to whether the therapy outcome was assessed with a disorder-specific measure [Flückiger et al., 2012, 2018b]. This variable was therefore coded as a dichotomous variable (disorder-specific outcome measure vs. other outcome measure). (6) The time of alliance assessment was classified into four categories (early, mid, late, and average). The early time of alliance assessment was defined as the first third, the mid time of assessment as the middle third, and the late time of assessment as the last third of the duration of intervention. The fourth category contained alliance values which were reported as average values of different assessment points. A random sample of the coding (20%) was coded by a second rater to evaluate the interrater reliability, resulting in an accordance of 96%. In case of ambiguous descriptions, a consensus decision was made (independent of the effect size coding).

Statistical Analysis

The separate steps of the statistical analysis comprise the coding of the reported effect sizes, the aggregation of multiple within-study effect sizes, computation of the summary effect, examination for potential publication bias, the moderator analysis, and the analysis of the WAI subscales. All analyses were conducted using the statistical software environment R. For calculations the package MAc [Del Re and Hoyt, 2010] was used; graphs, the trim and fill method, and the multilevel linear model were conducted using the metafor package [Viechtbauer, 2010].

Aggregation. The existence of multiple effect sizes within one independent sample leads to dependencies in the data. To account for these dependencies among the outcomes, the multiple alliance-outcome correlation effect sizes within one independent sample were aggregated, deriving a single effect size per independent sample for integration in the analysis. For the following analyses the correlation coefficients were converted to Fisher’s z values [Del Re and Flückiger, 2016].

Computation of the Summary Effect. Using the z-converted effect sizes, a summary effect as weighted average of the individual effect sizes of the independent samples was calculated. Regarding the distribution of the individual effect sizes, it is assumed that between-study variance is not due only to sampling error, wherefore a random-effects model was selected for calculating the summary effect. In a visual manner, the effect sizes and heterogeneity estimates are depicted with a forest plot [Viechtbauer, 2010].

Examination for Publication Bias. The examination for a potential publication bias was conducted using the trim and fill method in combination with a funnel plot [Viechtbauer, 2010].

Moderator Analysis. For analyzing categorical moderator variables, the macat function of the MAc package was used [Del Re and Hoyt, 2010]. The analysis of continuous moderator variables was conducted using the mareg function of the same package. Some independent samples contained several categories of the variables “specificity of outcome” and “time of alliance assessment,” so that a random selection procedure was conducted in these cases. To rule out confounds among moderator variables, Spearman correlations were calculated [Del Re and Hoyt, 2010].

Analysis of the WAI Subscales. Out of the 23 independent samples, the ones which reported the alliance-outcome correlation separately for the subscales of the WAI as alliance measure were included in the separate analysis of the WAI subscales (bond, task, goal). To examine the summary effects of the separate subscales for significant differences, a meta-analysis via multilevel linear model was undertaken (level 1: separate subscales; level 2: independent samples). Regarding the specification of the variance structure within studies, an unstructured variance-covariance matrix is assumed. Restricted maximum likelihood was selected as estimation procedure.

Table 1 summarizes the independent samples with the characteristics of each study included in the current meta-analysis. All included studies belonged to cognitive behavior therapy (CBT) as the therapy approach, and in 20 out of 23 independent samples the WAI, respectively WAI-S or a revised version of the WAI, was used as the alliance measure instrument. Fourteen out of the 18 studies (78%) were conducted in Europe.

Summary Effect

The overall weighted average effect size was r = 0.252, p < 0.001, 95% CI [0.19, 0.31], k = 23. Thus, the overall alliance-outcome correlation accounted for about 6% of the variability of treatment outcomes. The forest plot in Figure 2 depicts the effect size estimates of the independent samples together with the summary effect1. The assessment of heterogeneity of the effect sizes was not significant (Q(22) = 25.10, p = 0.29; I2 = 12.35). For the examination of a potential publication bias the funnel plot in combination with the trim and fill method was used. The resulting funnel plot is depicted in Figure 3 [Viechtbauer, 2010]. The examination of publication bias resulted in no studies to be augmented to the observed data so that the funnel plot is more symmetric.

Fig. 2.

Forest plot with effect sizes (in r) of the single independent samples and the summary effect (RE Model), each with 95% confidence interval for the alliance-outcome association.

Fig. 2.

Forest plot with effect sizes (in r) of the single independent samples and the summary effect (RE Model), each with 95% confidence interval for the alliance-outcome association.

Close modal
Fig. 3.

Funnel plot of the alliance-outcome relation with the trim and fill method, including all effect sizes (k = 23). The funnel plot depicts the effect sizes on the x axis and the standard error on the y axis for every independent sample. An asymmetrical plot indicates a publication bias.

Fig. 3.

Funnel plot of the alliance-outcome relation with the trim and fill method, including all effect sizes (k = 23). The funnel plot depicts the effect sizes on the x axis and the standard error on the y axis for every independent sample. An asymmetrical plot indicates a publication bias.

Close modal

Moderator Analysis

In Table 2 the 23 independent samples with corresponding characteristics in the investigated moderator variables are listed. The Spearman correlations with Bonferroni correction with p ≥ 0.43 indicated no significant confounds among moderator variables.

Table 2.

Study characteristics of the independent samples regarding the moderator variables

 Study characteristics of the independent samples regarding the moderator variables
 Study characteristics of the independent samples regarding the moderator variables

None of the investigated categorical moderator variables significantly accounted for variability of the effect sizes (amount of therapeutic contact Qb(2) = 2.08, p = 0.35; occurrence of a face-to-face contact Qb(1) = 1.62, p = 0.20; specificity of outcome Qb(1) = 0.21, p = 0.64; time of alliance assessment Qb(3) = 4.43, p = 0.11). The effect sizes of the separate categories of the categorical moderator variables with corresponding 95% confidence intervals and p values are summarized in Table 3.

Table 3.

Summary of the investigated categorical moderators

 Summary of the investigated categorical moderators
 Summary of the investigated categorical moderators

The examination of both continuous moderator variables (average age of participants, treatment duration) did not yield significant findings either. Neither the av­erage age of participants as moderator variable with Qm(1) = 1.80, p = 0.18 and an unstandardized regression coefficient estimate of r = 0.004 nor the treatment duration with Qm(1) = 0.02, p = 0.89 and an unstandardized regression coefficient estimate of r = 0.001 significantly accounted for variability of the effect sizes.

Analysis of the WAI Subscales

The investigation of the relation between the separate WAI subscales and therapy outcome was based on 9 studies with 13 independent samples and a total of 517 participants. The examination of significant differences between the summary effects of the separate subscales based on the multilevel linear model resulted in significant differences between the heterogeneity values of the separate subscales (τ2bond = 0.003, τ2task = 0.028, τ2goal = 0.023, QE(36) = 62.41, p = 0.004, k = 39). Furthermore, a high meta-analytical correlation between the subscales was found (ρ = 1). Regarding the differences of the alliance-outcome correlations among the separate WAI subscales, the results indicated a significant difference between the alliance-outcome correlation of the bond subscale (rbond = 0.16) and the task subscale (rtask = 0.28) with QM(3) = 26.85, p < 0.001. The alliance-outcome correlation of the goal subscale was rgoal = 0.22. Figure 4 depicts a separate forest plot for each WAI subscale.

Fig. 4.

Forest plots separate for the single WAI subscales (bond, task, goal). Depicted are the effect sizes (in r) of the single independent samples and the summary effect (RE Model), each with 95% confidence interval for the alliance-outcome association.

Fig. 4.

Forest plots separate for the single WAI subscales (bond, task, goal). Depicted are the effect sizes (in r) of the single independent samples and the summary effect (RE Model), each with 95% confidence interval for the alliance-outcome association.

Close modal

The relevance of the alliance for therapy outcome in internet-based interventions is debated controversially in the research literature. Based on the current correlative meta-analysis, it can be inferred that also in internet-based interventions alliance plays a decisive role for therapy outcome. The consensus on the tasks that make up therapy (tasks) as one component of the alliance seems to correlate slightly higher with therapy outcome than the bond between the client and the therapist (bond). Regarding the variability of effect sizes, an unexpectedly high homogeneity of the effect sizes was found. The conducted moderator analysis did not account for significant variance of the (small) residual heterogeneity.

The meta-analysis resulted in alliance-outcome correlations that differed significantly and distinctly from zero, both for the overall summary effect and for the separate subscales. This meets the assumption of Sucala et al. [2012] about a positive relation between alliance and therapy outcome in internet-based interventions. For calculating the summary effect a random-effects model was selected. Based on this rather conservative estimate, it can be assumed that additional studies on alliance-outcome relations in internet-based interventions would yield equivalent correlative relations [Del Re and Flückiger, 2016]. The alliance in internet-based interventions plays a role for therapy outcome comparable to that of face-to-face interventions. This contradicts the assumption of for example Andersson et al. [2012] that alliance as a predictor of treatment outcome is less important in internet-based CBT interventions than in face-to-face interventions.

While in face-to-face communication several communication channels and sensory modalities can be used, the online communication in internet-based interventions is mostly reduced to the text-based communication channel [Berger, 2015]. However, this reduction of the communication channels does not have to result inevitably in communication depletion, but seems to be able to be compensated or even to implicate several benefits for the therapeutic process [Berger, 2015]. To consider these specific characteristics of media communication in internet-based interventions, theories of computer-mediated communication can be consulted. It is discussed for example that the lack of nonverbal communication can be compensated by the clients through medium-immanent means by adapting the communication patterns to the medium [Murphy and Mitchell, 1998; Fenichel et al., 2002]. In his hyperpersonal model, Walther [1996] brings forward the argument that the absence of physical presence of the counterpart implicates that we let the other person arise in our mind’s eye. Given that the counterpart can be created according to one’s own ideas and requirements, the evoked pictures are often idealized and perceived as particularly pleasant [Walther, 1996; Brunner, 2009; Berger, 2015]. Another effect that could play a role in online communication is the online disinhibition effect [Suler, 2004]. Suler postulates that absent face-to-face cues in internet-based communication can lead to a higher level of openness, loosened behavior, and less experience of shame. Therefore, it is assumed that the quality of the alliance and the relationship concepts behind internet-based communication may differ qualitative-conceptually, even though the underlying items seems to be judgable over different settings. Interpersonal items for assessing the quality of the relationship (e.g., “I feel that my therapist appreciates me,” “My therapist and I respect each other”) can be rated by clients also in indirect communication anyway [Berger, 2017].

The high homogeneity of the effect sizes (I2 = 12%) found in the meta-analysis contradicts the assumption of mixed results regarding the alliance-outcome relations in internet-based interventions of Berger [2017]. Heterogeneity of effect sizes between studies can generally be due to statistical, methodological, or clinical factors [Higgins and Green, 2011]. The number of integrated studies with k = 23 is moderate, but not small, whereby the explanation of a small number of studies as the reason for the high homogeneity is not applicable [Villar et al., 2001]. According to Del Re and Flückiger [2016], variability of effect sizes between studies is usually due to differences in methods and sample characteristics of the studies included in a meta-analysis. The homogeneity of the effects in the present meta-analysis was potentially favored by high similarities of the basic conditions between the included studies. Seventy-eight percent of the included studies for example were conducted in Europe, preferentially within the framework of a CBT. The possibility to do the internet-based intervention at home in a familiar environment could implicate a sense of freedom, anonymity, and being yourself [Gerhards et al., 2011]. This, for the persons concerned, likewise familiar environment may have affected the homogeneity of the effects as well.

A moderator analysis serves to determine potential moderator variables that might account for some of the variability of effect sizes [Del Re and Flückiger, 2016]. Though there was not significant heterogeneity of the effect sizes between the included studies to explain, a moderator analysis was conducted for explorative purposes to explain the residual variability.

The investigation of the amount of therapeutic contact as moderator variable examined the assumption whether the predictive effect of the alliance on treatment outcome rises with increasing amount of therapeutic contact. The separate correlations of the three categories of the moderator variable do not seem to conform to the assumptions (Table 3). The fact that the three categories consist of a different number of independent samples deriving from the included studies calls for caution interpreting this result. Wagner and Maercker [2011] point out a potential disorder-specific effect by emphasizing the amount of therapeutic contact as a special influencing factor in treating a depression with an internet-based intervention. The relevance of the alliance for therapy outcome could hence differ among different kinds of disorder. A differential effect concerning client groups should also be considered – for some client groups the alliance could be of special importance regarding the treatment outcome [Berger, 2017]. There are indications for the existence of a differential effect regarding the amount of therapeutic contact as moderator variable for the relation between alliance and treatment outcome [Berger et al., 2009, 2011; Berger, 2017], even though so far it has not been possible to confirm such interaction effects meta-analytically in the literature [Wampold et al., 2018].

The investigation of the occurrence of a face-to-face contact as moderator variable examined the assumption that the occurrence of a face-to-face contact in the context of internet-based intervention has a positive effect on the relation between alliance and outcome. This analysis yielded no significant moderation, but showed a tendency consistent with the assumption (Table 3). Regarding this variable, it should be taken into consideration that if a face-to-face contact in the context of internet-based intervention occurs, the alliance is potentially assessed on the basis of this contact.

Another moderator variable examined was the average age of participants. Mixed results exist regarding the influence of age on treatment outcome [Barak et al., 2008; Karyotaki et al., 2018]. In the current meta-analysis, no moderating effect of the average age of participants on the alliance-outcome relation was found.

Investigation of treatment duration as a moderator variable did not yield significant results either. Concerning treatment duration, one should consider that most interventions were short. The average treatment duration of the independent samples was approximately 10 weeks.

In the alliance meta-analysis of Flückiger et al. [2018b] the specificity of outcome – i.e., whether the therapy outcome was assessed with a disorder-specific measure – turned out to be a significant moderator variable in face-to-face interventions. The relation between alliance and outcome was higher when the outcome measure was not disorder-specific, but a general symptom measure [Flü­ckiger et al., 2018b]. The current meta-analysis concerning internet-based interventions, however, did not yield a significant moderation by this variable. A reason for this nonsignificant result could be too low power of the data.

The time of the alliance assessment also turned out to be a significant moderator variable of the relation between alliance and outcome in face-to-face interventions. The alliance-outcome relation was higher when the alliance was measured later in the therapy process [Flückiger et al., 2018b]. In the present meta-analysis concerning the alliance-outcome relation in internet-based interventions, this variable did not represent a significant moderator variable. However, most interventions were short, so that the differences among the time of the alliance assessments were quite small. In addition, it should be taken into account that the category of late alliance assessment was not included in the analysis due to the random selection procedure conducted.

Regarding the analysis of the WAI subscales, recent studies hypothesize that the agreements on the therapeutic goals and the consensus on the tasks that make up therapy show higher predictability for treatment outcome of guided self-help approaches than the bond between the client and the therapist [Knaevelsrud and ­Maercker, 2006; Berger et al., 2014; Knaevelsrud et al., 2015; Meyer et al., 2015; Berger, 2017]. The results of the multilevel linear model conducted in the present meta-analysis confirm this hypothesis, while all correlation coefficients differ from zero (r > 0.15).

There are two main limitations of this meta-analysis: (1) The generalizability of the treatment setting. Most of the research derives from a university and research context. The findings regarding the efficacy of some natu­ralistic studies show promise [Andersson and Hedman, 2013], but further investigation is needed to resolve what relevance is attributed to alliance for treatment outcome in routine clinical practice. The homogeneity of the cultural context and the focus on CBT as a therapeutic approach also restrict the generalizability of the results. Therefore, our findings predominantly apply to internet-based CBT interventions in European countries. (2) Expectations of the clients. Based on the high proportion of the above-mentioned randomized controlled trials deriving from a context close to research and new media, it should be considered in the future how far a possibly more skeptical attitude towards internet-based interventions may affect the building of the alliance and the alliance-outcome prediction [Boettcher et al., 2013].

The relevance of the alliance for treatment outcome in internet-based interventions for psychological disorders found in this meta-analysis implicates that when developing internet-based interventions, attention should be attached to alliance – both regarding the education of therapists and the monitoring of the progress of therapy.

Since particularly unguided self-help approaches go along with high dropout rates [Karyotaki et al., 2015], future research should also examine how far the dropout rate of an internet-based intervention is associated with the quality of the therapist-client relationship and how far it affects the alliance-outcome relation. In terms of a multiperspective assessment, future studies should also include therapist- and observer-based ratings in addition to client-based ratings for alliance assessment [Berger, 2017]. Furthermore, it should be considered how far alliance in internet-based interventions as a dyadic concept can be captured with established alliance measures developed for face-to-face interventions. This leads to the predominant demand that new, specific alliance measures for internet-based interventions be developed in order to assess potentially specific characteristics of internet-based relationships [Knaevelsrud et al., 2004; Berger, 2017].

In conclusion, it seems that common factors of the therapy process such as alliance can be established also in internet-based interventions and play an important role for treatment outcome. In consideration of the expectable increasing relevance of internet-based interventions and the current state of research, diverse relationship aspects may be integrated into the investigation of internet-based interventions [Norcross and Lambert, 2018].

There are no ethical conflicts regarding this work.

The authors declare that there is no conflict of interest.

1.
Anderson
RE
,
Spence
SH
,
Donovan
CL
,
March
S
,
Prosser
S
,
Kenardy
J
.
Working alliance in online cognitive behavior therapy for anxiety disorders in youth: comparison with clinic delivery and its role in predicting outcome
.
J Med Internet Res
.
2012
Jun
;
14
(
3
):
e88
88
.
[PubMed]
1438-8871
2.
Andersson
E
,
Ljótsson
B
,
Hedman
E
,
Enander
J
,
Kaldo
V
,
Andersson
G
, et al
.
Predictors and moderators of Internet-based cognitive behavior therapy for obsessive–compulsive disorder: results from a randomized trial
.
J Obsessive Compuls Relat Disord
.
2015
;
4
:
1
7
. 2211-3649
3.
Andersson
G
.
Internet-Delivered Psychological Treatments
.
Annu Rev Clin Psychol
.
2016
;
12
(
1
):
157
79
.
[PubMed]
1548-5943
4.
Andersson
G
,
Cuijpers
P
,
Carlbring
P
,
Riper
H
,
Hedman
E
.
Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis
.
World Psychiatry
.
2014
Oct
;
13
(
3
):
288
95
.
[PubMed]
1723-8617
5.
Andersson
G
,
Hedman
E
.
Effectiveness of Guided Internet-Based Cognitive Behavior Therapy in Regular Clinical Settings
.
Verhaltenstherapie
.
2013
;
23
(
3
):
140
8
. 1016-6262
6.
Andersson
G
,
Paxling
B
,
Wiwe
M
,
Vernmark
K
,
Felix
CB
,
Lundborg
L
, et al
.
Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder
.
Behav Res Ther
.
2012
Sep
;
50
(
9
):
544
50
.
[PubMed]
0005-7967
7.
Barak
A
,
Hen
L
,
Boniel-Nissim
M
,
Shapira
N
.
A Comprehensive Review and a Meta-Analysis of the Effectiveness of Internet-Based Psychotherapeutic Interventions
.
J Technol Hum Serv
.
2008
;
26
(
2-4
):
109
60
. 1522-8835
8.
Baumeister
H
,
Reichler
L
,
Munzinger
M
,
Lin
J
.
The impact of guidance on Internet-based mental health interventions— A systematic review
.
Internet Interv
.
2014
;
1
(
4
):
205
15
. 2214-7829
9.
Berger
T
.
Internetbasierte Interventionen bei psychischen Störungen
.
Göttingen
:
Hogrefe
;
2015
.
10.
Berger
T
.
The therapeutic alliance in internet interventions: A narrative review and suggestions for future research
.
Psychother Res
.
2017
Sep
;
27
(
5
):
511
24
.
[PubMed]
1050-3307
11.
Berger
T
,
Boettcher
J
,
Caspar
F
.
Internet-based guided self-help for several anxiety disorders: a randomized controlled trial comparing a tailored with a standardized disorder-specific approach
.
Psychotherapy (Chic)
.
2014
Jun
;
51
(
2
):
207
19
.
[PubMed]
1939-1536
12.
Berger
T
,
Caspar
F
,
Richardson
R
,
Kneubühler
B
,
Sutter
D
,
Andersson
G
.
Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help
.
Behav Res Ther
.
2011
Mar
;
49
(
3
):
158
69
.
[PubMed]
0005-7967
13.
Berger
T
,
Hohl
E
,
Caspar
F
.
Internet-based treatment for social phobia: a randomized controlled trial
.
J Clin Psychol
.
2009
Oct
;
65
(
10
):
1021
35
.
[PubMed]
0021-9762
14.
Bergman Nordgren
L
,
Carlbring
P
,
Linna
E
,
Andersson
G
.
Role of the working alliance on treatment outcome in tailored internet-based cognitive behavioural therapy for anxiety disorders: randomized controlled pilot trial
.
JMIR Res Protoc
.
2013
Jan
;
2
(
1
):
e4
.
[PubMed]
1929-0748
15.
Boettcher
J
,
Renneberg
B
,
Berger
T
.
Patient expectations in internet-based self-help for social anxiety
.
Cogn Behav Ther
.
2013
;
42
(
3
):
203
14
.
[PubMed]
1650-6073
16.
Brunner
A
. Theoretische Grundlagen der Online-Beratung. In:
Kühne
S
,
Hintenberger
G
, editors
.
Handbuch Online-Beratung
.
Göttingen
:
Vandenhoeck & Ruprecht
;
2009
. pp.
27
46
.
17.
Carlbring
P
,
Andersson
G
,
Cuijpers
P
,
Riper
H
,
Hedman-Lagerlöf
E
.
Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis
.
Cogn Behav Ther
.
2018
Jan
;
47
(
1
):
1
18
.
[PubMed]
1650-6073
18.
Cook
JE
,
Doyle
C
.
Working alliance in online therapy as compared to face-to-face therapy: preliminary results
.
Cyberpsychol Behav
.
2002
Apr
;
5
(
2
):
95
105
.
[PubMed]
1094-9313
19.
Del Re
AC
,
Flückiger
C
. (
2016
). Meta-analysis. In J. C. Norcross, G. R. VandenBos, & D. K. Freedheim (Eds.), APA Handbook of Clinical Psychology: Vol. 2. Theory and Research (pp. 479-491). Washington, DC: American Psychological Association.
20.
Del Re
AC
,
Hoyt
WT
. (
2010
). MAc: Meta-Analysis with Correlations. R package version 1.1. https://cran.r-project.org/web/packages/MAc/index.html
21.
Dölemeyer
R
,
Klinitzke
G
,
Steinig
J
,
Wagner
B
,
Kersting
A
.
[Working alliance in internet-based therapy for binge eating disorder]
.
Psychiatr Prax
.
2013
Sep
;
40
(
6
):
321
6
.
[PubMed]
1439-0876
22.
Fenichel
M
,
Suler
J
,
Barak
A
,
Zelvin
E
,
Jones
G
,
Munro
K
, et al
.
Myths and realities of online clinical work
.
Cyberpsychol Behav
.
2002
Oct
;
5
(
5
):
481
97
.
[PubMed]
1094-9313
23.
Flückiger
C
,
Del Re
AC
,
Barth
J
,
Hoyt
WT
,
Levitt
H
,
Munder
T
, et al
.
Considerations of how to conduct meta-analyses in psychological interventions
.
Psychother Res
.
2018
May
;
28
(
3
):
329
32
.
[PubMed]
1050-3307
24.
Flückiger
C
,
Del Re
AC
,
Wampold
BE
,
Horvath
AO
.
The alliance in adult psychotherapy: A meta-analytic synthesis
.
Psychotherapy (Chic)
.
2018
Dec
;
55
(
4
):
316
40
.
[PubMed]
1939-1536
25.
Flückiger
C
,
Del Re
AC
,
Wampold
BE
,
Symonds
D
,
Horvath
AO
.
How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis
.
J Couns Psychol
.
2012
Jan
;
59
(
1
):
10
7
.
[PubMed]
0022-0167
26.
Gerhards
SA
,
Abma
TA
,
Arntz
A
,
de Graaf
LE
,
Evers
SM
,
Huibers
MJ
, et al
.
Improving adherence and effectiveness of computerised cognitive behavioural therapy without support for depression: a qualitative study on patient experiences
.
J Affect Disord
.
2011
Mar
;
129
(
1-3
):
117
25
.
[PubMed]
0165-0327
27.
Greene
CJ
,
Morland
LA
,
Macdonald
A
,
Frueh
BC
,
Grubbs
KM
,
Rosen
CS
.
How does tele-mental health affect group therapy process? Secondary analysis of a noninferiority trial
.
J Consult Clin Psychol
.
2010
Oct
;
78
(
5
):
746
50
.
[PubMed]
0022-006X
28.
Hedman
E
,
Andersson
E
,
Lekander
M
,
Ljótsson
B
.
Predictors in Internet-delivered cognitive behavior therapy and behavioral stress management for severe health anxiety
.
Behav Res Ther
.
2015
Jan
;
64
:
49
55
.
[PubMed]
0005-7967
29.
Higgins
JP
,
Green
S
, editors
.
Cochrane Handbook for Systematic Reviews of Interventions
.
Volume 4
.
John Wiley & Sons
;
2011
.
30.
Jasper
K
,
Weise
C
,
Conrad
I
,
Andersson
G
,
Hiller
W
,
Kleinstäuber
M
.
The working alliance in a randomized controlled trial comparing Internet-based self-help and face-to-face-cognitive behavior therapy for chronic tinnitus
.
Internet Interv
.
2014
;
1
(
2
):
49
57
. 2214-7829
31.
Karyotaki
E
,
Ebert
DD
,
Donkin
L
,
Riper
H
,
Twisk
J
,
Burger
S
, et al
.
Do guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis
.
Clin Psychol Rev
.
2018
Jul
;
63
:
80
92
.
[PubMed]
0272-7358
32.
Karyotaki
E
,
Kleiboer
A
,
Smit
F
,
Turner
DT
,
Pastor
AM
,
Andersson
G
, et al
.
Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis
.
Psychol Med
.
2015
Oct
;
45
(
13
):
2717
26
.
[PubMed]
0033-2917
33.
Kiluk
BD
,
Serafini
K
,
Frankforter
T
,
Nich
C
,
Carroll
KM
.
Only connect: the working alliance in computer-based cognitive behavioral therapy
.
Behav Res Ther
.
2014
Dec
;
63
:
139
46
.
[PubMed]
0005-7967
34.
Klasen
M
,
Knaevelsrud
C
,
Böttche
M
.
[The therapeutic alliance in internet-based therapy procedures: an overview]
.
Nervenarzt
.
2013
Jul
;
84
(
7
):
823
31
.
[PubMed]
0028-2804
35.
Kleiboer
A
,
Smit
J
,
Bosmans
J
,
Ruwaard
J
,
Andersson
G
,
Topooco
N
, et al
.
European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries
.
Trials
.
2016
Aug
;
17
(
1
):
387
.
[PubMed]
1745-6215
36.
Knaevelsrud
C
,
Brand
J
,
Lange
A
,
Ruwaard
J
,
Wagner
B
.
Web-based psychotherapy for posttraumatic stress disorder in war-traumatized Arab patients: randomized controlled trial
.
J Med Internet Res
.
2015
Mar
;
17
(
3
):
e71
.
[PubMed]
1438-8871
37.
Knaevelsrud
C
,
Jager
B
,
Maercker
A
.
Internet-Psychotherapie: Wirksamkeit und Besonderheiten der therapeutischen Beziehung
.
Verhaltenstherapie
.
2004
;
14
(
3
):
174
83
. 1016-6262
38.
Knaevelsrud
C
,
Maercker
A
.
Does the quality of the working alliance predict treatment outcome in online psychotherapy for traumatized patients
.
J Med Internet Res
.
2006
Dec
;
8
(
4
):
e31
.
[PubMed]
1438-8871
39.
Knaevelsrud
C
,
Maercker
A
.
Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial
.
BMC Psychiatry
.
2007
Apr
;
7
(
1
):
13
.
[PubMed]
1471-244X
40.
Kohn
R
,
Saxena
S
,
Levav
I
,
Saraceno
B
.
The treatment gap in mental health care
.
Bull World Health Organ
.
2004
Nov
;
82
(
11
):
858
66
.
[PubMed]
0042-9686
41.
Mechanic
D
.
Barriers to help-seeking, detection, and adequate treatment for anxiety and mood disorders: implications for health care policy
.
J Clin Psychiatry
.
2007
;
68
Suppl 2
:
20
6
.
[PubMed]
1555-2101
42.
Meyer
B
,
Bierbrodt
J
,
Schröder
J
,
Berger
T
,
Beevers
CG
,
Weiss
M
, et al
.
Effects of an Internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial
.
Internet Interv
.
2015
;
2
(
1
):
48
59
. 2214-7829
43.
Mulligan
J
,
Haddock
G
,
Hartley
S
,
Davies
J
,
Sharp
T
,
Kelly
J
, et al
.
An exploration of the therapeutic alliance within a telephone-based cognitive behaviour therapy for individuals with experience of psychosis
.
Psychol Psychother
.
2014
Dec
;
87
(
4
):
393
410
.
[PubMed]
1476-0835
44.
Murphy
LJ
,
Mitchell
DL
.
When writing helps to heal: e-mail as therapy
.
Br J Guid Counc
.
1998
;
26
(
1
):
21
32
. 0306-9885
45.
Norcross
JC
,
Lambert
MJ
.
Psychotherapy relationships that work II
.
Psychotherapy (Chic)
.
2011
Mar
;
48
(
1
):
4
8
.
[PubMed]
1939-1536
46.
Norcross
JC
,
Lambert
MJ
.
Psychotherapy relationships that work III
.
Psychotherapy (Chic)
.
2018
Dec
;
55
(
4
):
303
15
.
[PubMed]
1939-1536
47.
Ormrod
JA
,
Kennedy
L
,
Scott
J
,
Cavanagh
K
.
Computerised cognitive behavioural therapy in an adult mental health service: a pilot study of outcomes and alliance
.
Cogn Behav Ther
.
2010
;
39
(
3
):
188
92
.
[PubMed]
1650-6073
48.
Palmqvist
B
,
Carlbring
P
,
Andersson
G
.
Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost
.
Expert Rev Pharmacoecon Outcomes Res
.
2007
Jun
;
7
(
3
):
291
7
.
[PubMed]
1473-7167
49.
Pihlaja
S
,
Stenberg
JH
,
Joutsenniemi
K
,
Mehik
H
,
Ritola
V
,
Joffe
G
.
Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders - A systematic review
.
Internet Interv
.
2017
Nov
;
11
:
1
10
.
[PubMed]
2214-7829
50.
Preschl
B
,
Maercker
A
,
Wagner
B
.
The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression
.
BMC Psychiatry
.
2011
Dec
;
11
(
1
):
189
189
.
[PubMed]
1471-244X
51.
Richards
D
,
Timulak
L
,
Hevey
D
.
A comparison of two online cognitive-behavioural interventions for symptoms of depression in a student population: the role of therapist responsiveness
.
Couns Psychother Res
.
2013
;
13
(
3
):
184
93
. 1473-3145
52.
Scherer
S
,
Alder
J
,
Gaab
J
,
Berger
T
,
Ihde
K
,
Urech
C
.
Patient satisfaction and psychological well-being after internet-based cognitive behavioral stress management (IB-CBSM) for women with preterm labor: A randomized controlled trial
.
J Psychosom Res
.
2016
Jan
;
80
:
37
43
.
[PubMed]
0022-3999
53.
Simpson SG, Reid CL. Therapeutic alliance in videoconferencing psychotherapy: a review. Aust J Rural Health. 2014 Dec;22(6):280–99.
54.
Spek
V
,
Cuijpers
P
,
Nyklícek
I
,
Riper
H
,
Keyzer
J
,
Pop
V
.
Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis
.
Psychol Med
.
2007
Mar
;
37
(
3
):
319
28
.
[PubMed]
0033-2917
55.
Sucala
M
,
Schnur
JB
,
Constantino
MJ
,
Miller
SJ
,
Brackman
EH
,
Montgomery
GH
.
The therapeutic relationship in e-therapy for mental health: a systematic review
.
J Med Internet Res
.
2012
Aug
;
14
(
4
):
e110
.
[PubMed]
1438-8871
56.
Suler
J
.
The online disinhibition effect
.
Cyberpsychol Behav
.
2004
Jun
;
7
(
3
):
321
6
.
[PubMed]
1094-9313
57.
Titov
N
,
Dear
B
,
Nielssen
O
,
Staples
L
,
Hadjistavropoulos
H
,
Nugent
M
, et al
.
ICBT in routine care: A descriptive analysis of successful clinics in five countries
.
Internet Interv
.
2018
Jul
;
13
:
108
15
.
[PubMed]
2214-7829
58.
Viechtbauer
W
.
Conducting Meta-Analyses in R with the metafor Package
.
J Stat Softw
.
2010
;
36
(
3
):
1
48
. 1548-7660
59.
Villar
J
,
Mackey
ME
,
Carroli
G
,
Donner
A
.
Meta-analyses in systematic reviews of randomized controlled trials in perinatal medicine: comparison of fixed and random effects models
.
Stat Med
.
2001
Dec
;
20
(
23
):
3635
47
.
[PubMed]
0277-6715
60.
Wagner
B
,
Brand
J
,
Schulz
W
,
Knaevelsrud
C
.
Online working alliance predicts treatment outcome for posttraumatic stress symptoms in Arab war-traumatized patients
.
Depress Anxiety
.
2012
Jul
;
29
(
7
):
646
51
.
[PubMed]
1091-4269
61.
Wagner
B
,
Horn
AB
,
Maercker
A
.
Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial
.
J Affect Disord
.
2014
Jan
;
152-154
:
113
21
.
[PubMed]
0165-0327
62.
Wagner
B
,
Maercker
A
.
Psychotherapie im Internet - Wirksamkeit und Anwendungsbereiche
.
Psychotherapeutenjournal
.
2011
;
1
:
34
43
.
63.
Walther
JB
.
Computer-Mediated Communication: Impersonal, Interpersonal, and Hyperarousal Interaction
.
Communic Res
.
1996
;
23
(
1
):
3
43
. 0093-6502
64.
Wampold
,
Imel
, Flückinger.
2018
.

Footnotes

1

As the forest plot was built using the metafor package [Viechtbauer, 2010] and the included statistical models differed slightly, the summary effect with r = 0.27, 95% CI [0.20, 0.34] is descriptively somewhat higher than the more conservative estimate outlined above.