Methods Article for a Study Protocol: Refining Stroke Prediction in Atrial Fibrillation Patients in an Ethnically Diverse Population – A Study Protocol Open Access

Atrial fibrillation (AF) is a leading cause of ischaemic stroke (IS) worldwide, and the burden is expected to rise due to an ageing population and trends in cardiovascular risk factors [1]. IS resulting from AF is associated with greater disability and death [2]. In one retrospective cohort study of 739,695 New Zealanders, AF was found to affect approximately 1.7% of the population, highlighting its substantial health burden [3]. Anticoagulants are the primary strategy for IS prevention, reducing the risk by approximately two-thirds compared with a 22% risk reduction with aspirin [4]. However, these drugs increase the risk of intracranial and systemic haemorrhage, necessitating careful patient selection to ensure a favourable risk-benefit balance.

Despite the rising prevalence of AF, associated IS rates have declined. We conducted a Poisson regression analysis of five studies, encompassing over 800,000 patients with AF from Canada, the USA, Finland, and Italy, which reported incident IS rates in patients with AF between 1992 and 2021 [5‒10]. Our analysis demonstrated a significant 2% relative annual reduction in IS incidence (p < 0.001). The model showed little variance, indicating that geographical location did not significantly affect this trend. This decline was significantly associated with increased anticoagulant use (p < 0.05). On the contrary, one Finnish study of 144,879 oral anticoagulant (OAC)-naive patients reported stable IS rates from 2007 to 2017 (56 to 52 per 1,000 person-years, p = 0.7) [11].

CHA₂DS₂-VASc (congestive heart failure/left ventricular dysfunction, hypertension, age ≥75 [doubled], diabetes, IS/TIA/thromboembolism [doubled] – vascular disease [eg, IHD, aortic plaque], age 65–74, and sex category [female]) is the most common risk stratification tool used to guide anticoagulation decisions for IS prevention. More complex models exist, such as the Global Anticoagulant Registry in the FIELD-Atrial Fibrillation (GARFIELD-AF) and biomarker-based scores such as the Anticoagulation and Risk Factors in Atrial Fibrillation (ATRIA) score [12, 13]. Although female sex was previously associated with a 20–30% higher IS risk, recent studies have found no significant difference between sexes [14]. As a result of this and other studies, the European Society of Cardiology now recommends the use of CHA₂DS₂-VA, omitting female sex as an independent risk factor [15, 16]. The consensus is that IS risk assessment in AF should balance simplicity and practicality against calibration to ensure clinical utility. In various studies, CHA₂DS₂-VASc has a demonstrated c-statistic of 0.6–0.67, indicating moderate discriminative ability [12, 17].

Risk scores such as CHA₂DS₂-VASc have been validated predominantly in European cohorts [15, 17]. However, risk outcomes vary substantially across studies due to methodological differences, geographic location, and participant ethnicity [18]. Traditional vascular risk factors, such as hypertension, diabetes, and obesity, non-traditional conditions such as rheumatic heart disease, and disparities in healthcare significantly impact patient outcomes. Based on data from ARCOS IV, stratified by ethnicity, IS is associated with AF in 32.1% of Europeans, 30.4% of Māori, and 23% of Pacific peoples [19]. As a result of these factors, Māori and Pacific Islanders experience stroke approximately 10 years earlier than New Zealand Europeans (60 vs. 71 years, p < 0.001) [20]. There is evidence to suggest that refining CHA₂DS₂-VASc according to ethnicity improves risk prediction [18].

Few studies have attempted to validate CHA₂DS₂-VASc in New Zealand. The PREDICT-CVD registry was established to investigate cardiovascular disease risk and its management in primary care. In one study assessing AF associated stroke (haemorrhagic and IS) at 1 year, CHA₂DS₂-VASc was found to overestimate the stroke risk in all ethnicities except Māori [21]. This may be attributable to the study’s methodology, which included a lower risk cohort (patients <75 years old) and a high proportion (∼50%) of patients on antithrombotic therapy. These findings underscore the need for further research to refine stroke risk prediction in New Zealand populations.

In a New Zealand primary care-based cohort study, 39.5% of high-risk AF patients (i.e., CHA₂DS₂-VASc ≥2) did not have a prescription for OAC [3]. International data suggest lower rates of OAC prescribing in females, a trend also observed in New Zealand females and the elderly, despite evidence showing higher adherence in these groups [22, 23].

A New Zealand-based retrospective study of 52,413 patients found that Māori and Pacific ethnicity, rural residence, older age (≥75 years), and poor medication adherence (≤75% days covered) were associated with thromboembolism while on OAC, with an ischaemic and unspecified stroke rate of 4.9% [24]. Globally, up to 30% of AF patients are non-adherent to OAC, a major risk factor for IS and death [25, 26]. Another critical issue is inappropriate dosing, particularly underdosing, which occurs in 7–40% of AF patients [27]. Underdosing has been linked to increased all-cause mortality without reducing bleeding risk [28].

Given the lack of reliable assays and routine measurements, the use of coagulation assays to determine medication compliance is not considered useful. Instead, adherence is typically measured using the proportion of days covered (PDC) and, for warfarin users, time in therapeutic range (TTR). PDC is preferred for those on direct oral anticoagulants (DOACs, e.g., dabigatran and rivaroxaban) as it accounts for early refills and is calculated by dividing the total days covered by medication within a given timeframe by the number of days in that timeframe. TTR represents the proportion of international normalised ratio values within the therapeutic range, preferably determined using Rosendaal’s linear interpolation method. This method estimates the time spent within the target international normalised ratio range by interpolating between observed values. Literature and guidelines suggest an optimal TTR of ≥65–70% [29, 30]. In the RE-LY trial, which compared dabigatran and warfarin for IS prevention in AF, dabigatran was superior overall; however, subgroup analyses showed similar IS risk reduction in warfarin-treated patients with high TTR [31]. A clinically meaningful OAC adherence threshold has yet to be determined in AF [26].

In the RE-LY trial, only 17.1% of participants weighed more than 100 kg [32]. Case reports have described breakthrough IS in obese patients on dabigatran despite therapeutic dosing (but with subtherapeutic plasma drug levels) [32, 33]. However, two retrospective cohort studies have demonstrated efficacy across the body mass index (BMI) spectrum [34, 35]. Given that New Zealand has the third highest adult obesity rate among Organisation for Economic Co-operation and Development (OECD) countries, we aim to assess whether these findings hold true in this population [36].

DOACs inhibit thrombus formation but increase the risk of systemic bleeding. Proton pump inhibitors (PPIs) are commonly co-prescribed to reduce gastrointestinal bleeding risk, though their benefit remains uncertain [37]. Dabigatran, unlike factor Xa inhibitors, requires an acidic environment for optimal absorption [38]. Studies have shown that PPIs reduce peak levels, shown in Figure 1, and trough dabigatran plasma levels, shown in Figure 2 [39‒42]. This trend was seen with several PPIs, suggesting a class-wide effect. While subgroup analyses from the RE-LY trial found no signal of harm with PPI co-prescription, case reports have raised concerns of an increased IS risk [32, 33, 38]. However, no large real-world studies have assessed this association. This interaction appears to be restricted to dabigatran, and no such association has been demonstrated with the anti-Xa inhibitors (e.g., rivaroxaban and apixaban).

• 1.

  To measure the regional prevalence of AF in hospitalised IS patients overall and stratified by age, sex and ethnicity using data from the fifth Auckland Regional Community Stroke Study (ARCOS V, 1st September 2020–31st August 2021). These findings will be compared with ARCOS IV (2011–2012) to assess trends in disease burden.

• 2.

  To evaluate the performance of CHA₂DS₂-VA and CHA₂DS₂-VASc in New Zealand in predicting IS/TIA risk.

  • -

    Sub-aim 2: to assess whether refining CHA₂DS₂-VASc by:

    • Addition of an ethnicity-based adjustment for Māori and/or Pacific Islanders (i.e., CHA₂DS-VASc E, model 1), or

    • Widening the age criterion (A = 50–74 years, 1 point, model 2), improves model calibration and discrimination for IS/TIA in patients with AF.

• 3.

  To determine rates and associations with anticoagulant failure (defined as TIA or IS on OAC) in patients with AF.

ARCOS is a five-decade observational study focused on stroke/TIA in Auckland. It is internationally recognised as an ideal registry containing comprehensive data on patient demographics, comorbidities, and outcomes. The target population covers the well-defined geographical area of the Auckland region. Complete case ascertainment is ensured through multiple overlapping sources of information on all new hospitalised or non-hospitalised cases (e.g., private hospitals, general practice, and coroner/autopsy reports). All residents meeting the inclusion criteria are included, even if their stroke occurs outside the Auckland region, while those whose strokes or TIAs occur in Auckland but reside outside the region are excluded. Controls will be selected from the same population as ARCOS patients, sourced from the National Minimum Dataset (NMD), a database of hospital discharge codes which is collated and curated by Manatū Hauora (the New Zealand Ministry of Health).

A well-known rule of thumb is to have 10 events per predictor item. With CHA₂DS₂-VASc, this would indicate a minimum of 8 (predictors) × 10 events = 80 events in the sample under study. With a modification of the CHA₂DS₂-VASc by ethnicity, a further 10 events would be required. The validity of this traditional method, however, has been debated.

Based on model performance metrics from sample estimation, assuming the AUC is 0.70 in the study population, to achieve the precision level of +/−0.05 of the AUC estimate [43] with 95% confidence level, we will require 113 IS/TIA cases, and using a 1:4 ratio, 452 patients who did not have IS/TIA for the same period (controls). To develop ethnicity-specific models for Māori and Pacific peoples, based on preliminary data assuming AUC for Māori and Pacific peoples is 0.76 and 0.66, respectively, will require 85 Māori and 128 Pacific AF IS/TIA cases who experienced an event within 1 year. Using a ratio of 1:2 for Māori and Pacific peoples, we will require 170 and 256 controls. Extrapolating from this, we will require a total sample of 1,493 controls and 374 cases with IS/TIA. Figure 3 illustrates a flow chart of patient selection.

Patient demographics, including self-identified ethnicity will be recorded (i.e., European, Māori, Pacific peoples and “others”). Patient-level data on co-morbidities, e.g., AF, diabetes, hypertension, cardiac failure, peripheral vascular disease, pulmonary embolism, and ischaemic heart disease (components of CHA₂DS₂-VASc), will be obtained from the ARCOS V registry and the Auckland-wide clinical portal [43]. The definitions of predictors and outcomes are provided in Table 1.

Exclusion criteria: we will exclude patients who develop AF during hospitalisation following heart surgery as this may reflect temporary post-surgical complications, as well as patients with valvular AF, defined as the presence of a metallic prosthetic valve, or moderate to severe mitral stenosis on echocardiogram. Data will be collected to calculate BMI and to track prescriptions of antithrombotic medications and PPIs during 2020–21. This will facilitate the calculation of medication adherence (PDC & TTR) and the exploration of potential medication interactions. We will also investigate the aetiology of IS in patients with treatment failure and explore associations.

Continuous variables will be summarised as means (standard deviations) for normally distributed data or medians (interquartile range) for non-normally distributed data. Categorical data will be presented as frequencies and percentages.

Differences will be compared using t test/analysis of variance for normally distributed continuous data, and Wilcoxon rank-sum/Kruskal-Wallis test for non-normally distributed continuous data. Chi-squared or Fisher’s exact tests will be used to test categorical variables.

We will calculate the CHA₂DS₂-VASc risk score for all patients on the August 31st, 2020, in order to use logistic regression analysis to evaluate associations with binary outcomes (IS or TIA) at 1 year, ending August 31st, 2021.

Time-to-event data from date of AF diagnosis will be analysed using Cox regression; patients will be censored should the outcome of IS or TIA occur or at the end of the study period (August 31st, 2021). An interaction test in regression modelling will be conducted to assess whether the effect of one independent variable on the IS risk changes depending on the level of another independent variable, essentially revealing if there is a “combined” effect between variables that goes beyond their individual impacts. This will allow for a more nuanced understanding of the relationships within the data. Interaction tests will also be conducted to assess the effect of PPIs on the efficacy of dabigatran. A Hosmer and Lemeshow goodness-of-fit test will be performed to assess model calibration, though it only provides an overall measure and is highly dependent on sample size [48]. A two-sided p value of <0.05 will be considered statistically significant.

A nested case-control study will be conducted, which is a well-established method for developing and validating multivariable risk prediction models. The variable coefficients in the existing CHA₂DS₂-VASc model will be examined to confirm the associated weightings. The relative weights for the new predictor(s) in the multivariable prediction model will be determined by the coefficients.

Following this, we will estimate the model’s predictive performance, including its calibration, discrimination and net reclassification index (NRI). Internal validation will be performed to adjust for potential overfitting [48]. C-statistics of the developed models will be compared using the De Long test.

Two important reporting guidelines will be utilised, addressing key aspects such as pre-specifying and accurately defining predictors, and employing methods to reduce bias, such as collecting data from multiple sources.

• 1.

  TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) statement, a guideline specifically designed for the reporting of studies developing or validating a multivariable prediction model [49]. This is a guide for reporting research and does not prescribe how to perform validation studies; further, this is not a quality assessment tool to gauge the quality of a model.

• 2.

  PROBAST (Prediction model Risk Of Bias Assessment Tool) assesses both the risk of bias and concerns regarding the applicability of studies evaluating (developing, validating, or updating) multivariable diagnostic or prognostic prediction models [50]. It helps identify shortcomings in the study design, conduct or analysis which may lead to bias of a model’s predictive performance or render the model inadequate for addressing the research question.

We do not anticipate a significant problem with missing data since data for this study are routinely collected in clinical care. Therefore, we plan to perform a complete case analysis. However, if substantial amounts of missing data are encountered, the missingness is deemed to be missing completely at random (e.g., administrative errors without any relationship to the data) or at random (e.g., related to other variables). We will address this using multiple imputation based on known literature and pattern of “missingness.”

A major limitation of this study is its observational nature, which may result in inaccurate estimates due to unmeasured or unknown confounders that could influence the outcomes. Additionally, data for the control group will be collected retrospectively, which can introduce challenges such as incomplete data. To enhance data quality, we will merge multiple databases. These factors may affect the robustness of the findings and limit the ability to establish causal relationships. Furthermore, while efforts will be made to account for known confounders, the presence of unmeasured variables remains a significant concern, particularly given the diverse population and comorbidities involved. Finally, the generalisability of the study results may be restricted to the New Zealand/Auckland population, necessitating further validation in other regions or populations to assess the broader applicability of the findings.

Our goal is: (1) to assess the temporal changes in AF prevalence in IS in Auckland and determine if it follows global trends, and (2) to assess the validity of the CHA₂DS_2-VA and CHA₂DS₂-VASc risk score within the New Zealand context and evaluate whether refinements improve model performance. Finally, we aimed to investigate associations with treatment failure, including interactions between PPIs and dabigatran, which may suggest that transitioning to alternative OACs is preferable.

This study has been approved by the Health and Disability Ethics Committee (HDEC) (ref.: 2023 AM 9094) and Auckland University of Technology Ethics Committee (24/4). This study has been granted an exemption from obtaining informed consent, granted by Auckland University of Technology Ethics Committee (AUTEC) (24/4).

K.M.M. received a Health Research Council of New Zealand grant as part of his doctoral studies for this research. The other authors did not receive any financial support for the research, authorship, and/or publication of this article. H.W. reports grant support paid to the institution and fees for serving on Steering Committees of the ODYSSEY trial from Sanofi and Regeneron Pharmaceuticals, the ISCHAEMIA and MINT study from the National Institutes of Health, the Librexia and AF and ACS studies from Janssen Research and Development LLC and the MK0616 Study from Merck Sharp & Dohme Ltd. V.F. is the editor and R.K. is the associate editor of the journal Neuroepidemiology. This manuscript submission was managed by a guest editor.

This work was supported by the Health Research Council, New Zealand. The funder had no role in the design, data collection, data analysis, and reporting of this study (HRC Grant No. 20-680).

K.M.M. designed the study and drafted this manuscript. V.F., H.W., I.Z., and R.K. reviewed the manuscript and provided valuable insights. R.K assisted with obtaining approvals for this study. Everyone listed has agreed to authorship.

Data from this study will be made available on reasonable request.