Abstract
Introduction: Despite growing calls for the explicit application of theory when designing behaviour change interventions, limited empirical evidence exists regarding the effectiveness of these methods compared to non-theoretical approaches. A cluster randomized controlled trial (Hide and Seek Project – HaSP) tested two implementation approaches for improving hereditary cancer referral practices with one key distinction: implementation strategies were designed based explicitly on psychological theory or based on stakeholder intuition. This study presents the detailed methods and resources used to facilitate this comparison, whilst examining the strategies generated through both approaches. Methods: Across seven Australian hospitals, clinical stakeholders attended focus groups to co-design site-specific strategies for improving Lynch syndrome referral. Co-design methods differed according to trial arm. Implementation strategy content was examined, with intuitively derived strategies retrospectively coded to determine theoretical alignment. Results: Fifty-one strategies were proposed across all sites (theory-based arm = 32, intuition-based arm = 19). Overall, nine behaviour change technique (BCT) categories were used on 77 occasions. In the theory-based trial arm, eight BCT categories were identified on 53 occasions; and five BCT categories on 24 occasions in the intuition-based arm. BCT categories were largely similar across both arms. After retrospectively coding intuitively derived strategies, 42% contained mechanistic links, thereby demonstrating theoretical alignment. Conclusion: Methods facilitated robust comparison of theoretical and intuitive approaches to implementation strategy design. Recognizing the known benefits of theory for enhancing scientific learning, applying these methods on a larger scale may provide definitive evidence about the comparative effectiveness of theoretical approaches.
Introduction
In dynamic and unpredictable hospital environments, the behaviours of healthcare professionals can be influenced by a wide range of factors, highlighting the need for strategic approaches to drive improvement efforts. In the healthcare setting, implementation interventions (interventions to support implementation of evidence-based practices) aim to improve the adoption of evidence-based practices by modifying clinical practice behaviours [1]. However, the success of implementation interventions to date has been highly variable [2, 3], potentially due to the limited application of systematic approaches to inform their design [1].
Recognizing the complexity of human behaviour, there have been numerous calls for the explicit and systematic application of theory to inform the design of behaviour change interventions [1, 4]. “Target behaviours” are behaviours that could bring about the desired improvement outcome [5]. Implementation strategies are methods used to “enhance the adoption, implementation, and sustainability of a clinical program or practice” [6]. Theory can be used to inform interventions by identifying the constructs that are causally linked to the target behaviour under consideration, thereby enabling the design of targeted implementation strategies [7]. By identifying the constructs to be targeted in a strategy, theory can provide a means for selecting component techniques that have empirical evidence of effectiveness [8].
Behaviour change techniques (BCTs) are the “active components” of a strategy hypothesized to change behaviour, whilst mechanisms of action refer to the processes through which BCTs produce their effects [9]. The BCT Taxonomy version 1 [10] consolidates a large number of published intervention components designed to alter causal processes influencing behaviour (e.g., the “active ingredients” of an implementation strategy). The Theoretical Domains Framework (TDF) can facilitate the identification of behaviour change determinants which can then inform the design of implementation strategies that employ BCTs with known mechanistic links, potentially enhancing strategy effectiveness [11]. The Theory and Techniques Tool [12] is an interactive online resource consolidating links between BCTs and their mechanisms of action, based on a triangulation of evidence from a synthesis of published implementation studies [13], and a large expert consensus study [14].
In addition to informing implementation strategy design, explicitly stating the underlying theory can facilitate the accumulation of evidence across a range of contexts, populations and behaviours [7]. This also enables testing and refinement of the theory itself, thereby advancing the science of implementation more broadly [8]. Together, this enhances the generalizability and replicability of interventions so as to avoid wasted time and resources on “trial and error” approaches.
Despite the potential benefits of theory in both implementation strategy design and reporting, evidence from systematic reviews demonstrates limited use of theory in the planning or selection of implementation strategies, and poor articulation of the hypothesised causal mechanisms of change [15, 16]. In healthcare settings, clinicians can often find the application of theory to be “abstract, intimidating and irrelevant,” and thus tend to develop implementation strategies pragmatically based on tacit, experiential knowledge [4, 17]. Such strategies are not necessarily less effective – in fact, clinicians are well placed to leverage their expert understanding of a problem to develop innovative and contextually relevant solutions to address it [18]. Even when theories are not explicitly used in their design or reporting, these “intuitively derived” strategies may contain techniques that align with existing theory [18, 19]. Working in reverse, it is possible to identify implicitly enacted BCTs that are mechanistically linked with a particular domain or construct being targeted for change [20]. Whilst explicit use of theory in implementation strategy design serves to enhance learning opportunities, no studies to date have directly assessed their effectiveness when compared with non-theoretical approaches.
Lynch syndrome is an autosomal dominant hereditary cancer predisposition conferring an increased risk of colorectal, gynaecological and other cancer types [21]. Tumour-based testing of colorectal cancers (CRCs) (through mismatch repair immunohistochemistry and/or microsatellite instability testing) offers the ability to detect at-risk patients who warrant referral to specialist genetic services for germline genetic testing to establish a Lynch syndrome diagnosis [22]. Diagnosing Lynch syndrome can have long-term health benefits for the both proband and their at-risk relatives, with identified carriers having access to risk management strategies (such as colonoscopic surveillance, risk-reducing surgery, and aspirin prophylaxis) proven to reduce cancer incidence and mortality [23‒26].
Despite these health benefits, current evidence demonstrates uptake of the recommended Lynch syndrome tumour testing and referral pathway is suboptimal, signifying missed opportunities for cancer prevention and early detection [27‒29]. To address this issue, the Hide and Seek Project (HaSP) was a randomized controlled trial (RCT), which used a structured implementation approach to improve detection of Lynch syndrome across seven large Australian hospital networks [30]. The HaSP trial compared two different methods for implementation strategy design – one explicitly informed by psychological theory (using BCTs to address TDF barrier domains; “theory-based” trial arm) and one informed by clinician intuition and tacit knowledge without the explicit use of theory (“intuition-based” trial arm). Whilst several frameworks offer structured pathways for implementation strategy design (e.g., Consolidated framework for Implementation Research [31], Expert Recommendations for Implementing Change [32]), use of the TDF and BCTs in the HaSP trial allowed for a detailed examination of how behaviour change theory could inform and improve implementation efforts in healthcare settings.
These methods establish a robust framework for directly and comprehensively comparing the relative effectiveness of theoretically and intuitively driven approaches to the design of behaviour change implementation interventions. In an RCT setting, this can generate empirical evidence as to which approaches may be more effective for achieving behaviour change, particularly when applied on a larger scale and across different clinical contexts. The primary aim of this study is to present, in detail, the methods and resources used in the HaSP trial to facilitate such comparison, empowering other researchers to replicate and extend this work.
Furthermore, examining the implementation strategies arising from these two distinct approaches provides an initial exploration into the role and significance of theory in implementation strategy design. Interpreted through the lens of HaSP trial outcomes, this study examines the contents of implementation strategies designed through an explicitly theory-driven approach versus a clinician-led intuitively driven approach, assessing their alignment with behaviour change theory.
Methods
Context – Overview of the Hide and Seek Project
This study is part of a larger implementation trial aimed at improving Lynch syndrome tumour testing and referral practices in Australian hospitals. HaSP was a cluster RCT testing two structured implementation approaches, differentiated only by the explicit use of theory in the design of implementation strategies for improving Lynch syndrome-related molecular tumour testing and risk-appropriate referral practices for CRC patients in seven large Australian hospital networks (clustered by state). Each hospital network was randomly assigned to either the intuition-based (n = 3; hospitals 3, 4, 5) or theory-based (n = 4; hospitals 1, 2, 6, 7) implementation approach. A detailed rationale, protocol, and associated published works are available elsewhere [30, 33‒36].
Briefly, at each hospital network, a locally employed healthcare professional (e.g., nurse, genetic counsellor) was appointed and trained as an “Implementation Lead” to coordinate the implementation approach. Implementation Leads oversaw the following phases over a 2-year period: (i) baseline audits of Lynch syndrome tumour testing and genetics services referrals among CRC patients, (ii) formation of multidisciplinary “Implementation Teams,” (iii) process mapping to facilitate identification of target behaviours to achieve practice change [36], (iv) identification and confirmation of barriers to change, (v) generation of implementation strategies, (vi) support of staff to implement strategies, and (vii) evaluation of implementation strategy effectiveness using audit and process evaluation data to assess practice and culture change. Clinical data were extracted to assess the difference in change between the two arms, comparing pretrial and implementation/post-implementation patient outcomes [37]. The primary outcome measure was the proportion of patients with risk-appropriate completion of the Lynch syndrome tumour testing and referral pathway. The current study focuses on the phases: (iv) identification and confirmation of barriers to change and (v) generation of implementation strategies.
Participants and Recruitment
Implementation Leads used snowball methods to recruit a multidisciplinary team of stakeholders involved in the Lynch syndrome referral pathway to take part in two focus groups. Potential participants were identified in consultation with site Principal Investigators, based on their existing knowledge of internal Lynch syndrome stakeholder networks. Implementation Leads were instructed to seek representation from key departments known to be involved in the Lynch syndrome testing and referral pathway, including pathology, surgery, oncology, nursing, and genetics. Potential participants were invited to participate in study focus groups via email, with a request to forward the invitation to other eligible stakeholders. Although efforts were made to coordinate single multidisciplinary focus groups at each site, multiple focus groups or individual interviews took place at some sites due to scheduling conflicts. Participants were informed that focus groups would be audio-recorded and transcribed for analysis, and written consent was obtained prior. In some sites, one or more focus groups were conducted using a videoconferencing platform.
Implementation Strategy Design
Methods used to design strategies differed according to trial arm allocation and are summarized in Figure 1. Two separate research analysis teams (theory-based and intuition-based) were assigned to oversee the implementation strategy design process, with findings concealed between the two analysis teams to avoid contamination.
Theoretically Informed Implementation Strategies
To facilitate identification of barriers surrounding Lynch syndrome tumour testing and referral practices, Implementation Leads distributed the Influences on Patient Safety Behaviours Questionnaire (IPSBQ) [38] to stakeholders involved in the Lynch syndrome referral pathway at their site. A snowball approach was used, with stakeholders encouraged to distribute the questionnaire to the wider staff network involved in the CRC care pathway.
The IPSBQ is a validated questionnaire measuring psychosocial and environmental barriers to performing clinical practice target behaviours [38] and is theoretically underpinned by the TDF domains [39]. The IPSBQ has 35-items (each corresponding to a TDF domain), takes less than 5 min to complete, and has a modifiable end-statement enabling application across different settings and target behaviours [38]. In the context of the HaSP trial, the two key target behaviours were: (1) to ensure all CRC samples undergo mismatch repair immunohistochemistry and BRAF V600E* testing to inform Lynch syndrome risk assessment, and (2) to use Lynch syndrome risk assessment information, including molecular results, for all CRC patients to inform your decision to discuss and make a referral to genetics services. Respondents indicate agreement or disagreement with the statements on a five-point Likert scale (1 = strongly agree to 5 = strongly disagree). The validated version of the IPSBQ [38] was based on the 12 domains of the original version of the TDF [39]. For the purpose of the HaSP trial, the questionnaire was adapted to include new items corresponding to additional domains in Cane and colleagues’ refined version of the TDF (i.e., reinforcement, optimism, intentions, goals) [40]. An example IPSBQ is provided in Supplementary File 1 (for all online suppl. material, see https://doi.org/10.1159/000540612). Respondents each completed separate questionnaires for each target behaviour identified at their site. Implementation Leads were encouraged to seek a large multidisciplinary sample (approx. 20–40) to complete the questionnaire to ensure accurate identification of barriers.
Following completion of the questionnaire, focus groups were held at each site to further explore prominent barriers identified from the questionnaire results. Focus groups were facilitated primarily by the Implementation Lead, with support from an external researcher. Results from the IPSBQ were presented to participants to facilitate a theory-guided discussion about barriers to Lynch syndrome referral. Participants were provided with an overview of the TDF and rationale for use in the development of targeted implementation strategies informed by behaviour change theory. In line with the previous work [38], the top four TDF domain barriers were presented, and participants were asked to discuss and rank them in order of perceived importance. Participants also had the opportunity to suggest other pertinent barriers not represented in the questionnaire data. The external facilitator had experience in conducting TDF-guided focus groups and supported the Implementation Lead by prompting in-depth exploration of the TDF domains. Consensus barriers were determined through a group ranking exercise of the barriers presented through the questionnaire data, as well as any new barriers that emerged during the focus group. Resources used to support these activities are provided in online supplementary File 2.
Using the principles of implementation mapping, a suite of implementation strategies was designed to target consensus barriers identified [41]. The TDF-mapped barrier domains informed the selection of BCTs (from the BCT Taxonomy version 1; BCTTv1 [10]) with evidence-based mechanistic links, using the Theory and Techniques Tool [42]. Selected BCTs were operationalized as implementation strategies to address described Lynch syndrome referral barriers. Multiple BCTs could be encompassed within a single implementation strategy, and multiple strategies could be used to address a single barrier. The theory-based research team led the design of the initial draft intervention package, though implementation strategies proposed by stakeholders in previous HaSP phases were included if they demonstrated mechanistic alignment with the targeted barrier. This sought to achieve a “middle ground” in meeting the needs and preferences of stakeholders (in line with a co-design approach), whilst still allowing testing of the theoretical components distinguishing the two HaSP trial arm approaches.
Intuitively Derived Implementation Strategies
For sites in the intuition-based trial arm, barrier identification and implementation strategy design was guided by stakeholder intuition and tacit knowledge. Stakeholders in this arm also completed the IPSBQ, but solely for process evaluation purposes. Questionnaire results were not shared with participants and did not inform strategy design [29]. Implementation Leads had received no prior training in the TDF and were blinded to its use in the context of the HaSP trial [43].
To identify Lynch syndrome tumour testing and referral barriers, focus group participants were asked to describe what they believed were the key barriers to performing each target behaviour, based purely on their experiences and tacit knowledge (without reference to the TDF or any other theoretical framework). The external facilitator was unfamiliar with the TDF but had previous experience in facilitating program evaluation focus groups and supported Implementation Leads in the intuition-based arm to explore intuitively reported barriers in greater depth. Consensus barriers were determined through a group-ranking exercise of intuitively suggested barriers.
Implementation strategies suggested in previous focus groups were compiled by the “intuition-based” research team and mapped to lay-barriers. Implementation Leads had the opportunity to propose additional implementation strategies based on their understanding of Lynch syndrome referral barriers. No theoretical frameworks were used to guide implementation strategy design.
Implementation Strategy Selection and Operationalization
A similar approach was used across both trial arms to select strategies to take forward for operationalization and formal implementation. Draft intervention packages were provided to Implementation Leads for face validity review and potential refinement, after which focus groups were held across each site to finalize the intervention package. Site Principal Investigators were encouraged to attend the focus group to ensure leadership support for the finalized strategies. The study Chief Investigator (NT) also attended the review focus groups across all sites, to ensure that implementation strategies were within the scope of the HaSP study aims.
Prior to the focus group, an online survey was developed by the research team to guide the selection process by facilitating assessment of the appropriateness and feasibility of proposed implementation strategies. Survey items were modified from the APEASE criteria (assessing Affordability, Practicality, Effectiveness and cost-effectiveness, Acceptability, Side-effects/safety, and Equity [5]) to be more accessible to clinical stakeholders and more relevant to the context of the HaSP trial. Survey items included both tick-box and free-text responses. An example survey is provided in online supplementary File 2 for reference.
During the focus group, attendees were provided with the draft intervention package for discussion and review. The survey was used to guide focus group discussion, with participants asked to discuss each implementation strategy according to the APEASE criteria items. Attendees had the opportunity to suggest amendments to proposed implementation strategies or propose new implementation strategies to better address key barriers. In the theory-based trial arm sites, new or revised strategies were pursued only if mechanistic alignment was maintained. Those in the theory-based sites were also provided with a resource demonstrating BCTs mapped to corresponding TDF barrier domains. Following the focus group, participants were asked to individually complete the online survey, which was also distributed to implementation team members who were unable to attend the focus group.
Selection of implementation strategies during the focus group did not guarantee implementation in phase 6, but rather highlighted strategies to be taken forward in terms of planning and operationalization (prior to pursuing a formal proposal for departmental and hospital management approvals). Following the focus group, an additional meeting was held with the Implementation Lead, site Principal Investigator, the HaSP research team, a health economist, and a Lynch syndrome consumer representative. The purpose of this meeting was to review the selected implementation strategies and discuss logistics of formal implementation, including potential roadblocks, necessary approvals, stakeholder buy-in, and cost implications. This meeting was not recorded or transcribed for analysis and therefore not presented in the results.
Analysis
Barrier Focus Group Analysis
A content analysis approach was used to confirm selected barriers and identify any additional barriers. In the theory-based trial arm, barrier focus group transcripts were analysed using the TDF as a guiding framework. No framework was used for the intuition-based trial arm. Two separate analysis teams were allocated for the two trial arms: researchers analysing intuition-based focus group transcripts had no prior practical experience with the TDF, whilst researchers analysing the theory-based focus group transcripts had prior experience with TDF-guided qualitative analysis. Coding was performed by one researcher and reviewed by members of the research team. Ten percent of transcripts (i.e., at least one focus group for each trial arm) were independently double coded to ensure agreement. Findings were concealed between the two analysis teams to avoid contamination.
Intervention Focus Group Analysis
When analysing focus groups from the intervention development phase, both trial arms again used a content analysis approach to identify implementation strategies that were positively rated across all APEASE criteria components. These were cross-referenced against survey results to ensure agreement.
Mechanisms of Action and Role of Theory
To examine mechanisms of action and the role of theory in intervention design, strategies in the intuition-based trial arm were retrospectively mapped against theory. Intuitively derived implementation strategies were coded against the BCTTv1. Where BCTs were identified, the Theory and Techniques Tool was used to assess evidence of mechanistic alignment in addressing the barriers retrospectively mapped to TDF domains from the previous phase. This retrospective mapping exercise was used to determine the extent to which intuitively derived strategies align with behaviour change theory. For the theory-based trial arm sites, TDF domains were prospectively used to inform the selection of BCTs. Implementation strategies, component BCTs, and targeted mechanisms of action were tabulated to explicitly demonstrate underlying theory and enable comparison between sites.
Results
Demographics
Across all seven sites, there were 71 attendees (excluding the Implementation Lead facilitators) together for both focus group 1 and focus group 2. On average, there were five Lynch syndrome stakeholders per focus group. The most frequent clinical specialty participant groups were colorectal surgeons (N = 22), followed by genetics staff (including geneticists and genetic counsellors; N = 14) and medical oncologists (N = 12). Table 1 provides a summary of the participant breakdown for focus groups held at each site. The response rate could not be calculated due to the snowball methods used.
Site . | H1 . | H2 . | H3 . | H4 . | H5 . | H6 . | H7 . | Total . | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
focus group (FG) . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | |
Discipline | |||||||||||||||
Colorectal surgery | 4 | 3 | 0 | 0 | 1 | 2 | 1 | 1 | 4 | 3 | 1 | 2 | 0 | 0 | 22 |
Pathology | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 2 | 1 | 1 | 2 | 11 |
Genetics | 1 | 0 | 2 | 2 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 14 |
Medical oncology | 1 | 0 | 1 | 2 | 0 | 0 | 2 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 12 |
Nursing | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 8 |
Hospital administration | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1 | 0 | 0 | 4 |
Total | 8 | 4 | 4 | 6 | 3 | 4 | 5 | 5 | 5 | 5 | 8 | 6 | 4 | 4 | 71 |
Site . | H1 . | H2 . | H3 . | H4 . | H5 . | H6 . | H7 . | Total . | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
focus group (FG) . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | FG1 . | FG2 . | |
Discipline | |||||||||||||||
Colorectal surgery | 4 | 3 | 0 | 0 | 1 | 2 | 1 | 1 | 4 | 3 | 1 | 2 | 0 | 0 | 22 |
Pathology | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 2 | 1 | 1 | 2 | 11 |
Genetics | 1 | 0 | 2 | 2 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 1 | 14 |
Medical oncology | 1 | 0 | 1 | 2 | 0 | 0 | 2 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 12 |
Nursing | 0 | 1 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 8 |
Hospital administration | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 1 | 0 | 0 | 4 |
Total | 8 | 4 | 4 | 6 | 3 | 4 | 5 | 5 | 5 | 5 | 8 | 6 | 4 | 4 | 71 |
Design and Implementation of Implementation Strategies
Across the seven sites examined, a total of 51 implementation strategies were proposed (H1 = 7 strategies; H2 = 10; H3 = 5; H4 = 9; H5 = 5; H6 = 9; H7 = 6), with an average of 7.3 strategies per site (8.0 for theory sites, 6.3 for intuition-based sites). Of these, 14 (27%) were successfully implemented upon completion of the trial (H1 = 1 strategies; H2 = 3; H3 = 1; H4 = 6; H5 = 0; H6 = 1; H7 = 2), averaging 2 implementation strategies per site (1.75 for theory sites, 2.3 for intuition-based sites).
Online supplementary File 3 provides the full list of implementation strategies across all four theory sites (H1, H2, H6, H7), whereby BCTs were explicitly selected to drive implementation strategy design based on TDF barrier domains. Table 2 provides a single site example of the range of strategies designed through theory-informed methods. Strategies that were selected and formally implemented in phase 6 are represented in bold. For intuition-based trial arm sites, BCTs were retrospectively mapped to intuitively derived implementation strategies, as depicted in Table 3 using site H4 as an example. The full list of intuitively derived strategies designed for intuition-based sites (H3, H4, H5) is provided in online supplementary File 4.
Target behaviour . | Barrier description . | TDF domain . | Theory-guided strategy . | BCTs . | Mechanistic link (MoA:BCT) . |
---|---|---|---|---|---|
Ensure all CRC samples undergo MMR IHC and BRAF V600Ea testing to inform Lynch syndrome risk assessment | Lack of system for reflex BRAF testing for samples showing loss of MLH1/PMS2 | Environmental context and resources | 1. Facilitate a meeting with key stakeholders (e.g., pathologists, surgeons, oncologists) to develop a process for reflex BRAF testing | 3.2 Social support (practical) | Link (ECR:3.2) |
2. For samples with loss of MLH1/PMS2, institute an automated prompt reminding pathologists to automatically perform BRAF testing | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | |||||
3. Design a poster for display in the pathology department reminding pathologists of the need to perform BRAF testing | 12.5 Adding objects to the environment | Link (ECR:12.5) | |||
7.1 Prompts/cues | Link (ECR:7.1) | ||||
Lack of a follow-up system for outstanding IHC tests | Environmental context and resources | 4. Oncologists/surgeons to send request to pathology for reflex MMR if it is not done, highlighting that this needs to happen for patient referral (e.g., a template could be designed for this to hit the right social support tone) | 3.2 Social support (practical) | Link (ECR:3.2) | |
11.3 Conserving mental resources | |||||
5. Pathology to develop a reminder system (e.g., checklist) to help prompt pathologists to perform IHC testing on all CRC samples | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | |||||
Use Lynch syndrome risk assessment information, including molecular results, for all CRC patients to inform your decision to discuss and make a referral to genetics services | Variability in the reporting of MMR IHC results leads to interpretation difficulties among clinicians | Environmental context and resources; memory, attention, and decision processes | 6. Develop standardized language for the reporting of MMR/BRAF results for use by pathology staff | 3.2 Social support (practical) | Link (ECR:3.2) |
11.3 Conserving mental resources | Link (MADP:11.3) | ||||
Clinicians rely on input from genetic staff at the MDT and may not feel confident making decisions about Lynch syndrome referral independently | Skills; beliefs about capabilities | 7. Provide an educational resource to guide clinicians on how to interpret the reporting of MMR results | 4.1 Instruction on how to perform the behaviour | Link (Sk:4.1) | |
Link (BaCa:4.1) | |||||
Clinicians tend to overlook Lynch syndrome risk indicators for patients who are older, have metastatic disease, or have no family history | Knowledge; skills | 8. Conduct educational training to upskill clinicians in Lynch syndrome risk assessment, highlighting the need and value of referral even if patients do not meet traditional criteria (e.g., Amsterdam) | 5.1 Information about health consequences | Link (Kn:5.1) | |
4.1 Instruction on how to perform the behaviour | Link (Sk:4.1) | ||||
Link (BaCa:4.1) | |||||
Treatment priorities can get in the way of surgeons/oncologists discussing or initiating a genetic referral with their patient | Motivation and goals; environmental context and resources | 9. Clinicians to set a goal to ensure referral is discussed and documented with eligible patients within an agreed upon timeframe in the treatment trajectory | 1.3 Goal setting (outcome) | Link (Go:1.3) | |
10. Institute a reminder system to prompt discussion of referral with eligible patients within the above timeframe | 7.1 Prompts/cues | Link (Mo:1.3) | |||
3.2 Social support (practical) | Link (ECR:7.1) | ||||
11.3 Conserving mental resources | Link (ECR:3.2) |
Target behaviour . | Barrier description . | TDF domain . | Theory-guided strategy . | BCTs . | Mechanistic link (MoA:BCT) . |
---|---|---|---|---|---|
Ensure all CRC samples undergo MMR IHC and BRAF V600Ea testing to inform Lynch syndrome risk assessment | Lack of system for reflex BRAF testing for samples showing loss of MLH1/PMS2 | Environmental context and resources | 1. Facilitate a meeting with key stakeholders (e.g., pathologists, surgeons, oncologists) to develop a process for reflex BRAF testing | 3.2 Social support (practical) | Link (ECR:3.2) |
2. For samples with loss of MLH1/PMS2, institute an automated prompt reminding pathologists to automatically perform BRAF testing | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | |||||
3. Design a poster for display in the pathology department reminding pathologists of the need to perform BRAF testing | 12.5 Adding objects to the environment | Link (ECR:12.5) | |||
7.1 Prompts/cues | Link (ECR:7.1) | ||||
Lack of a follow-up system for outstanding IHC tests | Environmental context and resources | 4. Oncologists/surgeons to send request to pathology for reflex MMR if it is not done, highlighting that this needs to happen for patient referral (e.g., a template could be designed for this to hit the right social support tone) | 3.2 Social support (practical) | Link (ECR:3.2) | |
11.3 Conserving mental resources | |||||
5. Pathology to develop a reminder system (e.g., checklist) to help prompt pathologists to perform IHC testing on all CRC samples | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | |||||
Use Lynch syndrome risk assessment information, including molecular results, for all CRC patients to inform your decision to discuss and make a referral to genetics services | Variability in the reporting of MMR IHC results leads to interpretation difficulties among clinicians | Environmental context and resources; memory, attention, and decision processes | 6. Develop standardized language for the reporting of MMR/BRAF results for use by pathology staff | 3.2 Social support (practical) | Link (ECR:3.2) |
11.3 Conserving mental resources | Link (MADP:11.3) | ||||
Clinicians rely on input from genetic staff at the MDT and may not feel confident making decisions about Lynch syndrome referral independently | Skills; beliefs about capabilities | 7. Provide an educational resource to guide clinicians on how to interpret the reporting of MMR results | 4.1 Instruction on how to perform the behaviour | Link (Sk:4.1) | |
Link (BaCa:4.1) | |||||
Clinicians tend to overlook Lynch syndrome risk indicators for patients who are older, have metastatic disease, or have no family history | Knowledge; skills | 8. Conduct educational training to upskill clinicians in Lynch syndrome risk assessment, highlighting the need and value of referral even if patients do not meet traditional criteria (e.g., Amsterdam) | 5.1 Information about health consequences | Link (Kn:5.1) | |
4.1 Instruction on how to perform the behaviour | Link (Sk:4.1) | ||||
Link (BaCa:4.1) | |||||
Treatment priorities can get in the way of surgeons/oncologists discussing or initiating a genetic referral with their patient | Motivation and goals; environmental context and resources | 9. Clinicians to set a goal to ensure referral is discussed and documented with eligible patients within an agreed upon timeframe in the treatment trajectory | 1.3 Goal setting (outcome) | Link (Go:1.3) | |
10. Institute a reminder system to prompt discussion of referral with eligible patients within the above timeframe | 7.1 Prompts/cues | Link (Mo:1.3) | |||
3.2 Social support (practical) | Link (ECR:7.1) | ||||
11.3 Conserving mental resources | Link (ECR:3.2) |
Bolded strategies are those that were selected for implementation in Phase 6.
BaCa, beliefs about capabilities; ECR, environmental context and resources; Go, goals; IHC, immunohistochemistry; Kn, knowledge; LS, Lynch syndrome; MADP, memory, attention, and decision processes; MDT, multidisciplinary team meeting; MMR, mismatch repair; Mo, motivation; Sk, skills.
aNote: only links for the MoA being targeted in the strategy design are displayed.
Target behaviour . | Barrier description . | TDF domain (retrospectively mapped) . | Intuitively derived implementation strategy . | BCTs (retrospectively mapped) . | Theoretical alignment (MoA:BCT) . |
---|---|---|---|---|---|
Ensure all CRC samples undergo MMR IHC and BRAF V600Ea testing to inform Lynch syndrome risk assessment | Lack of responsibility or ownership of the MMR IHC testing process (who/when should chase missed pathology results) | Social/professional role and identify | 1. Ensure requests for the MMR test are made “routine” and part of standard practice | N/A (none identified) | N/A |
2. Ensure form contains information on status of MMR IHC provided to pathology after surgery | 4.1 Instruction on how to perform the behaviour | Non-link (SPRI:4.1) | |||
3. Ensure clear identification of pending samples via an MDT agenda item, prompting later review | 7.1 Prompts/cues | Non-link (SPRI:7.1) | |||
11.3 Conserving mental resources | Non-link (SPRI:11.3) | ||||
4. Deliver an in service to promote the concept of ensuring that all pathology requests included MMR | 4.1 Instruction on how to perform the behaviour | Non-link (SPRI:4.1) | |||
Lack of time and resource (due to increasing caseloads) make it difficult for non-pathology staff to follow-up missing pathology tests | Environmental context and resources | 5. MDT nurse or coordinator to keep list of patients with outstanding IHC/BRAF results and update when results are available | 3.2 Social support (practical) | Link (ECR:3.2) | |
Use Lynch syndrome risk assessment information, including molecular results, for all CRC patients to inform your decision to discuss and make a referral to genetics services | Referrals may be lost due to lack of reliable and streamlined administrative processes | Environmental context and resources | 6. Implement clear process for referrals which includes confirmation of referral receipts, education to MDT provided on new referral process | 4.1 Instruction on how to perform the behaviour | Link (ECR:7.1) |
7. Ensure presence of genetic counsellor at MDT meetings | 3.2 Social support (practical) | Link (ECR:3.2) | |||
12.2 Restructuring the social environment | Link (ECR:12.2) | ||||
8. Implement regular reminders to adhere to consistent reporting of referrals | 7.1 Prompts/cues | Link (ECR:7.1) | |||
4.1 Instruction on how to perform the behaviour | No evidence (ECR:4.1) | ||||
9. Implement a flag or tick box system at the bottom of reports | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | Inconclusive (ECR:11.3) |
Target behaviour . | Barrier description . | TDF domain (retrospectively mapped) . | Intuitively derived implementation strategy . | BCTs (retrospectively mapped) . | Theoretical alignment (MoA:BCT) . |
---|---|---|---|---|---|
Ensure all CRC samples undergo MMR IHC and BRAF V600Ea testing to inform Lynch syndrome risk assessment | Lack of responsibility or ownership of the MMR IHC testing process (who/when should chase missed pathology results) | Social/professional role and identify | 1. Ensure requests for the MMR test are made “routine” and part of standard practice | N/A (none identified) | N/A |
2. Ensure form contains information on status of MMR IHC provided to pathology after surgery | 4.1 Instruction on how to perform the behaviour | Non-link (SPRI:4.1) | |||
3. Ensure clear identification of pending samples via an MDT agenda item, prompting later review | 7.1 Prompts/cues | Non-link (SPRI:7.1) | |||
11.3 Conserving mental resources | Non-link (SPRI:11.3) | ||||
4. Deliver an in service to promote the concept of ensuring that all pathology requests included MMR | 4.1 Instruction on how to perform the behaviour | Non-link (SPRI:4.1) | |||
Lack of time and resource (due to increasing caseloads) make it difficult for non-pathology staff to follow-up missing pathology tests | Environmental context and resources | 5. MDT nurse or coordinator to keep list of patients with outstanding IHC/BRAF results and update when results are available | 3.2 Social support (practical) | Link (ECR:3.2) | |
Use Lynch syndrome risk assessment information, including molecular results, for all CRC patients to inform your decision to discuss and make a referral to genetics services | Referrals may be lost due to lack of reliable and streamlined administrative processes | Environmental context and resources | 6. Implement clear process for referrals which includes confirmation of referral receipts, education to MDT provided on new referral process | 4.1 Instruction on how to perform the behaviour | Link (ECR:7.1) |
7. Ensure presence of genetic counsellor at MDT meetings | 3.2 Social support (practical) | Link (ECR:3.2) | |||
12.2 Restructuring the social environment | Link (ECR:12.2) | ||||
8. Implement regular reminders to adhere to consistent reporting of referrals | 7.1 Prompts/cues | Link (ECR:7.1) | |||
4.1 Instruction on how to perform the behaviour | No evidence (ECR:4.1) | ||||
9. Implement a flag or tick box system at the bottom of reports | 7.1 Prompts/cues | Link (ECR:7.1) | |||
11.3 Conserving mental resources | Inconclusive (ECR:11.3) |
Bolded strategies are those that were selected for implementation in Phase 6.
CRC, colorectal cancer; ECR, environmental context and resources; IHC, immunohistochemistry; MDT, multidisciplinary team; MMR, mismatch repair; SPRI, social, professional role, and identity.
Representation of BCTs in Implementation Strategies
Across all seven sites combined, analysis of the 51 proposed implementation strategies identified a total of 9 categories of BCTs, which were used on 77 occasions (“occasions” referring to any instance a BCT was identified in an implementation strategy, noting that each implementation strategy often contained multiple BCTs). In the 32 implementation strategies developed in the theory trial arm sites, 8 categories of BCTs were identified on 53 occasions. In the 19 implementation strategies developed in the intuition-based trial arm sites, 5 categories of BCTs were identified on 24 occasions. In three of the intuition-based implementation strategies, BCTs were unable to be identified due to lack of description in behavioural terms.
The most frequently occurring BCTs were “social support – practical” (used on 21/77 occasions), followed by “prompts and cues” (20/77 occasions), “conserving mental resources” (16/77 occasions), and “instruction on how to perform the behaviour” (12/77 occasions). Among the theory trial arm sites, the most frequently observed BCTs were “social support – practical” (used on 15/53 occasions), “prompts and cues” (13/53 occasions), and “conserve mental resources” (12/53 occasions). Among the intuition-based trial arm sites, the most frequently observed BCTs were “prompts and cues” (used on 7/24 occasions) followed by “instruction on how to perform the behaviour” (6/24 occasions) and social support – practical (6/24 occasions).
Assessment of Mechanistic Links
After applying the Theory and Techniques Tool [13], 8/19 (42%) intuition-based implementation strategies contained mechanistic links previously demonstrated to be significant (i.e., theoretical alignment agreed upon by expert consensus AND associations in the intervention literature synthesis). All 32 implementation strategies in the theory arm contained evidence of mechanistic links (noting that mechanistic links explicitly drove selection of BCTs).
Across all seven sites, the most frequently cited BCTs with evidence of mechanistic links were: social support – practical (to address TDF domain: “environmental context and resources”) used on 17 occasions; prompts and cues (TDF domain = “environmental context and resources”) used on 16 occasions; and instruction on how to perform the behaviour (TDF domain = “skills”) used on 6 occasions. Among the theory trial arm sites, the most frequently cited BCTs with evidence of mechanistic links were: social support – practical (to address TDF domain: “environmental context and resources”) used on 14 occasions; prompts and cues (TDF domain = “environmental context and resources”) used on 12 occasions; conserving mental resources (TDF domain: “memory, attention and decision processes”) used on 5 occasions; and instruction on how to perform the behaviour (TDF domain = “skills”) used on 5 occasions. Among the intuition-based sites, the most frequently cited BCTs with evidence of mechanistic links were: prompts and cues (TDF domain = “environmental context and resources”) used on 4 occasions; social support – practical (TDF domain: “environmental context and resources”) used on 3 occasions.
Discussion
The HaSP trial was uniquely designed to determine whether explicit use of theory in implementation strategy design leads to more effective implementation, compared to approaches that instead rely on clinician intuition and tacit knowledge. Outcomes from the HaSP trial (reported in full elsewhere [37]) provide some suggestion that theory-based approaches to implementation strategy design may better support clinical practice changes for improving Lynch syndrome detection, with a significant reduction in the proportion of patients at high risk of Lynch syndrome without a record of genetics referral in the theory-based arm, compared to the intuition-based arm. The interpretive value of the trial outcomes was limited by the rarity of Lynch syndrome and strong correlation of patient-level outcomes within hospitals, with the authors highlighting the need for large-scale trials to confirm results and mechanisms for effective strategies. However, we demonstrate here the methods necessary for achieving such a robust comparison between these different approaches. The demonstrated methodologies not only contribute to the scientific understanding of implementation strategy effectiveness but also serve as a practical guide for researchers seeking to navigate the complexities of comparing (or accounting for) theory-driven and intuitive approaches in diverse clinical settings.
Beyond simply comparing effectiveness in terms of HaSP trial clinical outcomes, the current study sought to explore the types of implementation strategies that were generated through both implementation approaches. Our findings demonstrate both similarities and differences between strategies designed explicitly using theory, and those designed based on clinician intuition and tacit knowledge. There was a significant overlap in the overarching strategies between the two trial arms. For example, strategies common to both groups included staff training opportunities, standardized language for pathology reporting, clinical data review, and reminder systems. A theory-guided approach yielded a wider range of strategies (and BCTs) to select from, including some not previously reported in the literature (e.g., goal setting). However, the most widely used categories of BCTs and mechanistic links were largely similar across the two trial arm sites.
In line with findings from recent studies [18, 19], some strategies developed in the intuition-based trial arm sites contained evidence of mechanistic links, thus aligning with behaviour change theory (albeit retrospectively). Across both groups, strategies tended to have a practical focus (e.g., addressing skills and environmental barriers), perhaps reflecting the nature of the Lynch syndrome referral practices and behaviours being targeted for change. It may be that (at least in the first instance) these practical strategies provide relatively simple, yet effective approaches for addressing key issues present in this clinical context. Indeed, findings from a recent systemic review provide evidence for the effectiveness of similar strategies for overcoming genetic referral barriers [15]. It is possible that theory may offer greater benefit for informing strategies to address more complex barriers, for example, emotions and belief systems [18].
Whilst a previous meta-analysis found that more explicit use of theory produced greater intervention effects [44], some researchers have previously encountered challenges applying behavioural theory in implementation strategy design, finding the use of such methods to be “constraining” and “inflexible” [45]. As noted by Davidoff et al. (2015), theory should be considered “a resource, not a shackle,” and pragmatic co-design approaches that encourage healthcare professional input should be considered. Whilst a comprehensive process evaluation is currently underway to thoroughly examine the advantages and limitations of the methods employed in the HaSP trial [30], the methods and resources presented in online supplementary File 2 offer potential solutions to some of these challenges. For example, the use of the adapted APEASE criteria and survey-based approach to prioritize strategies balances the potential limitations of theory by ensuring that strategies are relevant to the local context and needs of stakeholders. Furthermore, analysis of information gained from previous HaSP trial phases provided an opportunity to document strategies spontaneously suggested by clinician stakeholders during process mapping and barrier discussions. Retrospectively aligning these intuitive strategies with an underlying theoretical framework provided a streamlined and pragmatic means of harnessing the expertise of both clinicians and implementation science researchers.
Explicit use of theory also appeared to provide greater behavioural specificity in the description of implementation strategies. For a number of implementation strategies in the intuition-based trial arm, BCTs were unable to be retrospectively discerned. This is not to say that BCTs were not present, but rather that the description of strategies lacked detail and/or clarity. This resulted in confusion among the clinical teams, and consequently, some of these strategies were rejected during the review stage. This finding highlights a need for training and tools to support clinicians in designing strategies that are behaviourally specified in terms of the individual(s) involved what they need to do, when and where they need to do it, with whom, and how often [11].
There have been overwhelming calls for better reporting of implementation strategies in the implementation science literature [46‒48], requiring precise specification of the behaviours to be changed, the BCTs to be used, modes of delivery, and characteristics of the target population [49]. To facilitate replication across other contexts, others must be able to clearly discern the active ingredients of an implementation intervention and the proposed causal pathways through which they produce the intended effects. Specifying these pathways for the implementation strategies developed across both trial arms in this study will inform site-specific logic models reported in a later HaSP process evaluation. These logic models can serve as a valuable tool in synthesizing complexity and providing a visual representation of the conceptual relationships between multiple variables, and the causal mechanisms by which they produce their effects [49].
It is worth noting that although a significant number of implementation strategies were proposed in this study, only a small percentage (27%) were selected for formal implementation by the site-based implementation teams. The appropriateness and effectiveness of implementation strategies can vary at different stages of implementation, and unanticipated challenges, such as feasibility and acceptability issues, may arise during real-world implementation [50]. In this regard, the use of the adapted APEASE criteria provided a valuable framework for clinicians to carefully assess the feasibility of each implementation strategy and prioritize efforts towards those that are feasible and most likely to be effective in the given context. Considering the dynamic nature of hospital environments, implementation strategies that were rejected during the HaSP trial may still hold value for implementation at a later stage when the context evolves. Additionally, consolidating all the proposed strategies (together with their component BCTs and targeted mechanisms of action) within this study can serve as a valuable resource for other clinicians and researchers. Strategies rejected in one context may prove highly effective in another, highlighting the potential transferability and adaptability of these approaches.
A key strength of this study is the detail in which the different methods used to design implementation strategies have been described, providing detailed resources that can support and accelerate future research translation and implementation research. Furthermore, we have developed a comprehensive suite of implementation strategies mapped to a wide range of barriers, with explicit specification of underlying behaviour change theory. Interpretation of these strategies in the context of an RCT, together with in-depth process evaluation data, will facilitate further understanding of implementation effectiveness sand the impact of contextual moderators. Within these methods, opportunities exist to incorporate and test a recently proposed algorithm, which articulates a structured approach to standardize hypothesized behavioural pathways for both theory-driven and intuitive implementation strategies [20].
This study is not without limitations. Different Implementation Leads facilitated the focus groups at each site. Whilst all received the same training and structured focus group guides, it is possible that the depth of information gained from the exercises (and therefore the strategies that were developed) may have been determined by the skillset of the Implementation Lead, despite the provision of facilitator support from an external researcher. Furthermore, some sites were unable to achieve multidisciplinary representation across all key clinical specialties involved in the Lynch syndrome referral pathway. In such instances, barriers may not have been accurately captured and/or strategies may have lacked relevance. Finally, trial activities were significantly impacted by the COVID-19 global pandemic, which may have affected site capacity to design and implement strategies.
Conclusions
Whilst large-scale studies are needed to determine whether an explicitly theory-driven approach leads to the design of more effective interventions, our findings demonstrate that prospective application of theory may provide structure and enhance behavioural specificity in the description of implementation strategies, thus better guiding the planning and selection process. Furthermore, we have demonstrated robust methods to enable comparison of the effectiveness of these approaches across a range of settings. Whilst intuitively derived strategies may retrospectively align with underlying theory, it is important that intended mechanisms of action be made explicit to enhance replicability and advance the science of implementation.
Acknowledgments
We sincerely thank all Implementation Leads for facilitating focus groups and all stakeholders who generously offered their time and expertise to participate in the focus groups and contribute to intervention strategy design and/or review. We are also grateful to site Principal Investigators for their assistance with focus group recruitment.
Statement of Ethics
Ethical approval was granted by the Royal Prince Alfred Hospital Human Research Ethics Committee (reference HREC/17/RPAH/542). The research was undertaken in compliance with the Australian Code for the Responsible Conduct of Research and National Statement on Ethical Conduct in Human Research. Informed written consent was obtained from all study participants. Participant identifiers have been removed, so the persons described are not identifiable.
Conflict of Interest Statement
The authors declare no competing conflicts of interest.
Funding Sources
This study was funded in part by Cancer Institute NSW (2017/CDF005) and Cancer Australia (1123924).
Author Contributions
A.M. and J.S.: conceptualization, formal analysis, investigation, methodology, and writing. P.C. and G.T.: data curation, formal analysis, investigation, and writing. E.K.: data curation, formal analysis, and investigation. E.H.: data curation, formal analysis, investigation, and methodology. D.D.: formal analysis, investigation, methodology, and writing. N.T: funding acquisition, conceptualization, formal analysis, investigation, methodology, supervision, and writing. All authors read and approved the final manuscript.
Data Availability Statement
The data are not publicly available due to privacy or ethical restrictions. Further information is available in the supplementary material or by contacting the corresponding author.