The Alliance-Outcome Relation in Internet-Based Interventions for Psychological Disorders: A Correlational Meta-Analysis

The collaborative quality of the therapist-client relationship (alliance) is worldwide one of the most investigated factors of therapy process [Flückiger et al., 2018b]. In 1979, Bordin proposed a transdiagnostic, pantheoretical concept of the working alliance, which consists of the following three interconnected components: (a) agreements on the therapeutic goals (goal), (b) consensus on the tasks that make up therapy (tasks), (c) a bond between the client and the therapist (bond). Out of the big amount of alliance measure instruments, the most used alliance measure is the Working Alliance Inventory (WAI) [Flü­ckiger et al., 2018b]. The overall weighted average effect size of r = 0.28 for the alliance-outcome association indicates the alliance as a very robust predictor of treatment outcome and supports the pantheoretical conception of the alliance empirically [Norcross and Lambert, 2011; Flückiger et al., 2012; Norcross and Lambert, 2018].

Even though effective psychotherapies exist, many people with psychological disorders remain untreated [Kohn et al., 2004]. Reasons for this so-called treatment gap may be, among others, a perceived stigmatization, financial occasions, a lack of availability of services, or geographical accessibility [Mechanic, 2007]. These obstacles can be overcome in part by modern information and communication media. The use of internet-based interventions for psychological disorders became more and more important over the past years [Andersson, 2016; Titov et al., 2018]. In internet-based interventions the internet can be exclusively used either as an information medium in so-called unguided self-help or as a communication medium in so-called internet-based psychotherapies [Berger, 2015]. The most investigated type of internet-based interventions are guided self-help approaches, in which a combination of the internet as an information and communication medium is used. In this type of internet-based interventions the patients receive regular therapist online contact in addition to the web-based self-help program. Regarding the efficacy of internet-based interventions for psychological problems and disorders, findings, especially for guided self-help approaches, indicate medium to large effects which are equivalent to face-to-face interventions [Barak et al., 2008; Wagner and Maercker, 2011; Klasen et al., 2013; Andersson et al., 2014; Wagner et al., 2014; Berger, 2015; Carlbring et al., 2018]. Furthermore, several systematic reviews and meta-analyses indicate a superiority of guided self-help over unguided self-help regarding efficacy [Palmqvist et al., 2007; Spek et al., 2007; Baumeister et al., 2014]. Research on so-called blended treatments, where online and face-to-face interventions are combined, is still scarce [Kleiboer et al., 2016].

Considering the increasing implementation of internet-based interventions in the treatment of psychological disorders and the relevance attributed to alliance in general, the question arises how these two aspects conceptually and empirically are related to each other and what relevance is attached to alliance in internet-based interventions for psychological disorders [Cook and Doyle, 2002; Knaevelsrud and Maercker, 2006; Sucala et al., 2012; Berger, 2017]. A current meta-analysis indicated an alliance-outcome correlation that was quite similar to that found for face-to-face psychotherapy (r = 0.27) [Flückiger et al., 2018b]. Berger [2017] points out the heterogeneous significance levels of alliance-outcome correlations in guided self-help approaches, resulting in significant alliance-outcome correlations [Bergman Nord­gren et al., 2013; Knaevelsrud et al., 2015], nonsignificant alliance-outcome correlations [Knaevelsrud and Maer­cker, 2007; Preschl et al., 2011; Andersson et al., 2012; Jasper et al., 2014] as well as mixed results [Berger et al., 2014]. Based on findings indicating a positive relationship between the amount of therapeutic contact and the outcome of therapy [Palmqvist et al., 2007; Spek et al., 2007; Baumeister et al., 2014], a moderating influence of the amount of therapeutic contact on the alliance-outcome correlation seems plausible. Beyond that, it is assumed that a face-to-face contact is conducive to the building of a therapeutic relationship and hence could moderate the relationship between alliance and outcome. Furthermore, findings suggest mixed results concerning predictors of therapy outcome such as the influence of the age of patients on the possibility to benefit from an internet-based intervention [Barak et al., 2008; Karyotaki et al., 2018]. “Specificity of outcome” and time of the alliance assessment turned out to be significant moderators of the alliance-outcome relation in face-to-face psychotherapies [Flückiger et al., 2018b]. The relation between alliance and outcome was higher when the outcome measure was not disorder-specific but a general symptom measure and when the alliance was measured later in the therapy process. It is assumed that these moderating influences also play a role in internet-based interventions.

From a conceptual angle, the differentiation in the alliance-subscales seems to be of particular interest. Especially the question to what extent the quality of the bond-subscale is relevant for success of internet-based interventions arises [Berger, 2017]. Descriptively (based on nonsystematic reviews) the subscales seem to be quite heterogeneous, while the alliance on the subscale level also appears to relate positively to the outcome in most cases. Higher relations are hypothesized for the task and goal subscales compared to the bond subscale [Knae­velsrud and Maercker, 2006; Berger et al., 2014; Knae­velsrud et al., 2015; Meyer et al., 2015; Berger, 2017], which raises the issue to what extent the alliance concept can be adapted conceptually to internet-based interventions [Berger, 2017; Pihlaja et al., 2017]. How far the alliance subscales in internet-based interventions predict therapy outcome cannot be appraised systematically due to the limited number of studies and the descriptive presentation of the reviews [Berger, 2017]. That is exactly the point where the present meta-analytic investigation starts by addressing the following questions.

1. Given that the alliance is positively associated with therapy outcome [Flückiger et al., 2018b], we pose the question to what extent this alliance-outcome correlation in internet-based interventions is moderated by the amount of therapeutic contact and other potential moderators. The following potential moderators are examined: (1) amount of therapeutic contact; (2) occurrence of a face-to-face contact; (3) average age of participants; (4) treatment duration; (5) specificity of outcome; (6) time of the alliance assessment.

2. Is there a significant difference between the effect sizes of the alliance subscales (goal, task, bond)?

The methodological conception of the present meta-analysis was conducted according to the MAP-24 checklist [Flückiger et al., 2018a].

The literature search was based on an already conducted literature search of Flückiger et al. [2018b]. The flowchart in Figure 1 provides an overview of the literature search and selection of studies with reasons for exclusion of manuscripts. The criteria for inclusion of a study in this meta-analysis were: (1) the author referred to the therapy process variable as helping alliance, working alliance, or therapeutic alliance; (2) the data reported allow for extracting or calculating a measure for the relation between alliance and outcome; (3) the study includes an alliance measure during treatment; (4) the participants were adults; (5) the manuscript was written in English, German, Italian, or French; (6) the study includes the investigation of an electronic or internet-based intervention. Criteria for exclusion of a study were (1) a nonclinical sample and (2) a qualitative study or less than 5 participants. Overall, resulting from the literature search there were 18 studies with 23 independent samples and 1,177 participants included in the meta-analysis. In addition, we included the only study with an adolescent sample we know, with an average age of 12.1 years. Ta­ble 1 summarizes the studies contained in the current meta-analysis.

The measure for the relation between alliance and outcome extracted from the included studies contained alliance-outcome correlations, so that the effect sizes underlying the correlative meta-analysis were correlation coefficients. Positive correlations were defined as the relation between high alliance values and better treatment outcome. If the relation between alliance and outcome was reported using a regression analysis, these values were converted into correlation coefficients. If the data reported for the alliance-outcome relation contained both correlation and regression measures, correlation measures were preferred for the subsequent analysis. If exclusively standardized beta values were reported (e.g., adjusting for confounding variables), these were integrated in the meta-analysis. The statistical parameters were requested from the author if not mentioned exactly in the report [Berger et al., 2014].

The following moderators were coded: (1) The amount of therapeutic contact during the intervention was classified into three categories according to the categorization of Berger [2015]. (a) The first category consisted of interventions that only use the internet to provide information in so-called internet-based unguided self-help programs. These programs do not require any contact with a clinician during treatment, therapeutic contact at the most exists during the diagnostic procedure, wherefore this category represents the one with the least amount of therapeutic contact. (b) To the second category the so-called internet-based guided self-help approaches, in which the internet is used both as an information and as a communication medium, were allocated. Despite the regular therapist online contact, compared to face-to-face interventions the amount of therapeutic contact is less. (c) The third category consisted of so-called internet-based psychotherapies that used the internet exclusively as a communication medium. The amount of therapeutic contact is approximately equivalent to the one in face-to-face interventions. The classification in categories of the studies was based on the self-reported description of the authors. (2) The occurrence of a face-to-face contact was coded as a dichotomous variable (face-to-face contact vs. no face-to-face contact) and dichotomizes the scale outlined above. Any face-to-face contact with a clinician or support personnel which took place in the context of the study was classified as a face-to-face contact in this connection. (3) The average age of participants was coded on the basis of the descriptive data of the original studies. (4) The reported treatment duration was coded in weeks. (5) The moderator variable “specificity of outcome” refers to whether the therapy outcome was assessed with a disorder-specific measure [Flückiger et al., 2012, 2018b]. This variable was therefore coded as a dichotomous variable (disorder-specific outcome measure vs. other outcome measure). (6) The time of alliance assessment was classified into four categories (early, mid, late, and average). The early time of alliance assessment was defined as the first third, the mid time of assessment as the middle third, and the late time of assessment as the last third of the duration of intervention. The fourth category contained alliance values which were reported as average values of different assessment points. A random sample of the coding (20%) was coded by a second rater to evaluate the interrater reliability, resulting in an accordance of 96%. In case of ambiguous descriptions, a consensus decision was made (independent of the effect size coding).

The separate steps of the statistical analysis comprise the coding of the reported effect sizes, the aggregation of multiple within-study effect sizes, computation of the summary effect, examination for potential publication bias, the moderator analysis, and the analysis of the WAI subscales. All analyses were conducted using the statistical software environment R. For calculations the package MAc [Del Re and Hoyt, 2010] was used; graphs, the trim and fill method, and the multilevel linear model were conducted using the metafor package [Viechtbauer, 2010].

Aggregation. The existence of multiple effect sizes within one independent sample leads to dependencies in the data. To account for these dependencies among the outcomes, the multiple alliance-outcome correlation effect sizes within one independent sample were aggregated, deriving a single effect size per independent sample for integration in the analysis. For the following analyses the correlation coefficients were converted to Fisher’s z values [Del Re and Flückiger, 2016].

Computation of the Summary Effect. Using the z-converted effect sizes, a summary effect as weighted average of the individual effect sizes of the independent samples was calculated. Regarding the distribution of the individual effect sizes, it is assumed that between-study variance is not due only to sampling error, wherefore a random-effects model was selected for calculating the summary effect. In a visual manner, the effect sizes and heterogeneity estimates are depicted with a forest plot [Viechtbauer, 2010].

Examination for Publication Bias. The examination for a potential publication bias was conducted using the trim and fill method in combination with a funnel plot [Viechtbauer, 2010].

Moderator Analysis. For analyzing categorical moderator variables, the macat function of the MAc package was used [Del Re and Hoyt, 2010]. The analysis of continuous moderator variables was conducted using the mareg function of the same package. Some independent samples contained several categories of the variables “specificity of outcome” and “time of alliance assessment,” so that a random selection procedure was conducted in these cases. To rule out confounds among moderator variables, Spearman correlations were calculated [Del Re and Hoyt, 2010].

Analysis of the WAI Subscales. Out of the 23 independent samples, the ones which reported the alliance-outcome correlation separately for the subscales of the WAI as alliance measure were included in the separate analysis of the WAI subscales (bond, task, goal). To examine the summary effects of the separate subscales for significant differences, a meta-analysis via multilevel linear model was undertaken (level 1: separate subscales; level 2: independent samples). Regarding the specification of the variance structure within studies, an unstructured variance-covariance matrix is assumed. Restricted maximum likelihood was selected as estimation procedure.

Table 1 summarizes the independent samples with the characteristics of each study included in the current meta-analysis. All included studies belonged to cognitive behavior therapy (CBT) as the therapy approach, and in 20 out of 23 independent samples the WAI, respectively WAI-S or a revised version of the WAI, was used as the alliance measure instrument. Fourteen out of the 18 studies (78%) were conducted in Europe.

The overall weighted average effect size was r = 0.252, p < 0.001, 95% CI [0.19, 0.31], k = 23. Thus, the overall alliance-outcome correlation accounted for about 6% of the variability of treatment outcomes. The forest plot in Figure 2 depicts the effect size estimates of the independent samples together with the summary effect¹. The assessment of heterogeneity of the effect sizes was not significant (Q(22) = 25.10, p = 0.29; I² = 12.35). For the examination of a potential publication bias the funnel plot in combination with the trim and fill method was used. The resulting funnel plot is depicted in Figure 3 [Viechtbauer, 2010]. The examination of publication bias resulted in no studies to be augmented to the observed data so that the funnel plot is more symmetric.

In Table 2 the 23 independent samples with corresponding characteristics in the investigated moderator variables are listed. The Spearman correlations with Bonferroni correction with p ≥ 0.43 indicated no significant confounds among moderator variables.

None of the investigated categorical moderator variables significantly accounted for variability of the effect sizes (amount of therapeutic contact Qb(2) = 2.08, p = 0.35; occurrence of a face-to-face contact Qb(1) = 1.62, p = 0.20; specificity of outcome Qb(1) = 0.21, p = 0.64; time of alliance assessment Qb(3) = 4.43, p = 0.11). The effect sizes of the separate categories of the categorical moderator variables with corresponding 95% confidence intervals and p values are summarized in Table 3.

The examination of both continuous moderator variables (average age of participants, treatment duration) did not yield significant findings either. Neither the av­erage age of participants as moderator variable with Qm(1) = 1.80, p = 0.18 and an unstandardized regression coefficient estimate of r = 0.004 nor the treatment duration with Qm(1) = 0.02, p = 0.89 and an unstandardized regression coefficient estimate of r = 0.001 significantly accounted for variability of the effect sizes.

The investigation of the relation between the separate WAI subscales and therapy outcome was based on 9 studies with 13 independent samples and a total of 517 participants. The examination of significant differences between the summary effects of the separate subscales based on the multilevel linear model resulted in significant differences between the heterogeneity values of the separate subscales (τ²_bond = 0.003, τ²_task = 0.028, τ²_goal = 0.023, QE(36) = 62.41, p = 0.004, k = 39). Furthermore, a high meta-analytical correlation between the subscales was found (ρ = 1). Regarding the differences of the alliance-outcome correlations among the separate WAI subscales, the results indicated a significant difference between the alliance-outcome correlation of the bond subscale (r_bond = 0.16) and the task subscale (r_task = 0.28) with QM(3) = 26.85, p < 0.001. The alliance-outcome correlation of the goal subscale was r_goal = 0.22. Figure 4 depicts a separate forest plot for each WAI subscale.

The relevance of the alliance for therapy outcome in internet-based interventions is debated controversially in the research literature. Based on the current correlative meta-analysis, it can be inferred that also in internet-based interventions alliance plays a decisive role for therapy outcome. The consensus on the tasks that make up therapy (tasks) as one component of the alliance seems to correlate slightly higher with therapy outcome than the bond between the client and the therapist (bond). Regarding the variability of effect sizes, an unexpectedly high homogeneity of the effect sizes was found. The conducted moderator analysis did not account for significant variance of the (small) residual heterogeneity.

The meta-analysis resulted in alliance-outcome correlations that differed significantly and distinctly from zero, both for the overall summary effect and for the separate subscales. This meets the assumption of Sucala et al. [2012] about a positive relation between alliance and therapy outcome in internet-based interventions. For calculating the summary effect a random-effects model was selected. Based on this rather conservative estimate, it can be assumed that additional studies on alliance-outcome relations in internet-based interventions would yield equivalent correlative relations [Del Re and Flückiger, 2016]. The alliance in internet-based interventions plays a role for therapy outcome comparable to that of face-to-face interventions. This contradicts the assumption of for example Andersson et al. [2012] that alliance as a predictor of treatment outcome is less important in internet-based CBT interventions than in face-to-face interventions.

While in face-to-face communication several communication channels and sensory modalities can be used, the online communication in internet-based interventions is mostly reduced to the text-based communication channel [Berger, 2015]. However, this reduction of the communication channels does not have to result inevitably in communication depletion, but seems to be able to be compensated or even to implicate several benefits for the therapeutic process [Berger, 2015]. To consider these specific characteristics of media communication in internet-based interventions, theories of computer-mediated communication can be consulted. It is discussed for example that the lack of nonverbal communication can be compensated by the clients through medium-immanent means by adapting the communication patterns to the medium [Murphy and Mitchell, 1998; Fenichel et al., 2002]. In his hyperpersonal model, Walther [1996] brings forward the argument that the absence of physical presence of the counterpart implicates that we let the other person arise in our mind’s eye. Given that the counterpart can be created according to one’s own ideas and requirements, the evoked pictures are often idealized and perceived as particularly pleasant [Walther, 1996; Brunner, 2009; Berger, 2015]. Another effect that could play a role in online communication is the online disinhibition effect [Suler, 2004]. Suler postulates that absent face-to-face cues in internet-based communication can lead to a higher level of openness, loosened behavior, and less experience of shame. Therefore, it is assumed that the quality of the alliance and the relationship concepts behind internet-based communication may differ qualitative-conceptually, even though the underlying items seems to be judgable over different settings. Interpersonal items for assessing the quality of the relationship (e.g., “I feel that my therapist appreciates me,” “My therapist and I respect each other”) can be rated by clients also in indirect communication anyway [Berger, 2017].

The high homogeneity of the effect sizes (I² = 12%) found in the meta-analysis contradicts the assumption of mixed results regarding the alliance-outcome relations in internet-based interventions of Berger [2017]. Heterogeneity of effect sizes between studies can generally be due to statistical, methodological, or clinical factors [Higgins and Green, 2011]. The number of integrated studies with k = 23 is moderate, but not small, whereby the explanation of a small number of studies as the reason for the high homogeneity is not applicable [Villar et al., 2001]. According to Del Re and Flückiger [2016], variability of effect sizes between studies is usually due to differences in methods and sample characteristics of the studies included in a meta-analysis. The homogeneity of the effects in the present meta-analysis was potentially favored by high similarities of the basic conditions between the included studies. Seventy-eight percent of the included studies for example were conducted in Europe, preferentially within the framework of a CBT. The possibility to do the internet-based intervention at home in a familiar environment could implicate a sense of freedom, anonymity, and being yourself [Gerhards et al., 2011]. This, for the persons concerned, likewise familiar environment may have affected the homogeneity of the effects as well.

A moderator analysis serves to determine potential moderator variables that might account for some of the variability of effect sizes [Del Re and Flückiger, 2016]. Though there was not significant heterogeneity of the effect sizes between the included studies to explain, a moderator analysis was conducted for explorative purposes to explain the residual variability.

The investigation of the amount of therapeutic contact as moderator variable examined the assumption whether the predictive effect of the alliance on treatment outcome rises with increasing amount of therapeutic contact. The separate correlations of the three categories of the moderator variable do not seem to conform to the assumptions (Table 3). The fact that the three categories consist of a different number of independent samples deriving from the included studies calls for caution interpreting this result. Wagner and Maercker [2011] point out a potential disorder-specific effect by emphasizing the amount of therapeutic contact as a special influencing factor in treating a depression with an internet-based intervention. The relevance of the alliance for therapy outcome could hence differ among different kinds of disorder. A differential effect concerning client groups should also be considered – for some client groups the alliance could be of special importance regarding the treatment outcome [Berger, 2017]. There are indications for the existence of a differential effect regarding the amount of therapeutic contact as moderator variable for the relation between alliance and treatment outcome [Berger et al., 2009, 2011; Berger, 2017], even though so far it has not been possible to confirm such interaction effects meta-analytically in the literature [Wampold et al., 2018].

The investigation of the occurrence of a face-to-face contact as moderator variable examined the assumption that the occurrence of a face-to-face contact in the context of internet-based intervention has a positive effect on the relation between alliance and outcome. This analysis yielded no significant moderation, but showed a tendency consistent with the assumption (Table 3). Regarding this variable, it should be taken into consideration that if a face-to-face contact in the context of internet-based intervention occurs, the alliance is potentially assessed on the basis of this contact.

Another moderator variable examined was the average age of participants. Mixed results exist regarding the influence of age on treatment outcome [Barak et al., 2008; Karyotaki et al., 2018]. In the current meta-analysis, no moderating effect of the average age of participants on the alliance-outcome relation was found.

Investigation of treatment duration as a moderator variable did not yield significant results either. Concerning treatment duration, one should consider that most interventions were short. The average treatment duration of the independent samples was approximately 10 weeks.

In the alliance meta-analysis of Flückiger et al. [2018b] the specificity of outcome – i.e., whether the therapy outcome was assessed with a disorder-specific measure – turned out to be a significant moderator variable in face-to-face interventions. The relation between alliance and outcome was higher when the outcome measure was not disorder-specific, but a general symptom measure [Flü­ckiger et al., 2018b]. The current meta-analysis concerning internet-based interventions, however, did not yield a significant moderation by this variable. A reason for this nonsignificant result could be too low power of the data.

The time of the alliance assessment also turned out to be a significant moderator variable of the relation between alliance and outcome in face-to-face interventions. The alliance-outcome relation was higher when the alliance was measured later in the therapy process [Flückiger et al., 2018b]. In the present meta-analysis concerning the alliance-outcome relation in internet-based interventions, this variable did not represent a significant moderator variable. However, most interventions were short, so that the differences among the time of the alliance assessments were quite small. In addition, it should be taken into account that the category of late alliance assessment was not included in the analysis due to the random selection procedure conducted.

Regarding the analysis of the WAI subscales, recent studies hypothesize that the agreements on the therapeutic goals and the consensus on the tasks that make up therapy show higher predictability for treatment outcome of guided self-help approaches than the bond between the client and the therapist [Knaevelsrud and ­Maercker, 2006; Berger et al., 2014; Knaevelsrud et al., 2015; Meyer et al., 2015; Berger, 2017]. The results of the multilevel linear model conducted in the present meta-analysis confirm this hypothesis, while all correlation coefficients differ from zero (r > 0.15).

There are two main limitations of this meta-analysis: (1) The generalizability of the treatment setting. Most of the research derives from a university and research context. The findings regarding the efficacy of some natu­ralistic studies show promise [Andersson and Hedman, 2013], but further investigation is needed to resolve what relevance is attributed to alliance for treatment outcome in routine clinical practice. The homogeneity of the cultural context and the focus on CBT as a therapeutic approach also restrict the generalizability of the results. Therefore, our findings predominantly apply to internet-based CBT interventions in European countries. (2) Expectations of the clients. Based on the high proportion of the above-mentioned randomized controlled trials deriving from a context close to research and new media, it should be considered in the future how far a possibly more skeptical attitude towards internet-based interventions may affect the building of the alliance and the alliance-outcome prediction [Boettcher et al., 2013].

The relevance of the alliance for treatment outcome in internet-based interventions for psychological disorders found in this meta-analysis implicates that when developing internet-based interventions, attention should be attached to alliance – both regarding the education of therapists and the monitoring of the progress of therapy.

Since particularly unguided self-help approaches go along with high dropout rates [Karyotaki et al., 2015], future research should also examine how far the dropout rate of an internet-based intervention is associated with the quality of the therapist-client relationship and how far it affects the alliance-outcome relation. In terms of a multiperspective assessment, future studies should also include therapist- and observer-based ratings in addition to client-based ratings for alliance assessment [Berger, 2017]. Furthermore, it should be considered how far alliance in internet-based interventions as a dyadic concept can be captured with established alliance measures developed for face-to-face interventions. This leads to the predominant demand that new, specific alliance measures for internet-based interventions be developed in order to assess potentially specific characteristics of internet-based relationships [Knaevelsrud et al., 2004; Berger, 2017].

In conclusion, it seems that common factors of the therapy process such as alliance can be established also in internet-based interventions and play an important role for treatment outcome. In consideration of the expectable increasing relevance of internet-based interventions and the current state of research, diverse relationship aspects may be integrated into the investigation of internet-based interventions [Norcross and Lambert, 2018].

There are no ethical conflicts regarding this work.

The authors declare that there is no conflict of interest.