Artificial Intelligence-Assisted Software Significantly Decreases All Workflow Metrics for Large Vessel Occlusion Transfer Patients, within a Large Spoke and Hub System

Endovascular thrombectomy (EVT) is standard of care for patients with large vessel occlusion (LVO) [1]. However, the efficacy of EVT is time dependent [2, 3], and delays in treatment may render patients ineligible for EVT. The most substantial delays occur in patients requiring transfer to an EVT-capable facility, and some studies suggest that up to 41% of transfers for EVT are futile [4]. Stroke systems of care have evolved rapidly during the last decade, with many health systems adopting a spoke and hub model, to promote timely and efficient diagnosis and treatment of LVO stroke [5].

Artificial intelligence (AI)-driven software is an emerging diagnostic modality, powered to function as a clinical adjunct in diagnosis, team communication, and early treatment of LVO stroke [5, 6]. Viz LVO was authorized by the Food and Drug Administration in 2018 to assist with LVO detection in computed tomography angiography (CTA). The high sensitivity and negative predictive value of this AI-driven software might help decrease the rate of underdiagnosed patients who would benefit from EVT [7]. The application includes features that enable early team notification and communication between the stroke team members, and early diagnosis and team notification may decrease workflow time [7, 8].

There are limited data on the workflow impact such tools may have in large spoke and hub health systems. The authors hypothesized that early diagnosis and notification by Viz LVO may significantly decrease workflow metrics for suspected/confirmed LVO cases requiring transfer from primary stroke centers (PSCs) utilizing Viz LVO compared to PSCs without.

The Mount Sinai Health System (MSHS) consists of 3 comprehensive stroke centers (CSCs) and 7 PSCs. Patients with suspected or confirmed (by CTA) LVO stroke are transferred from PSCs to CSCs within our system. In addition, PSCs outside of the MSHS may transfer these cases to our CSCs. Each spoke center within or outside of MSHS acts independently and has the same availability and access to EMS transfer services. They are all geographically located within the NYC metropolitan area and follow the same transfer protocols.

Since September 2019, Viz LVO has been implemented in all MSHS facilities (PCSs and CSCs); however, PSCs outside our system lack this AI-driven tool. Viz LVO is an FDA-cleared AI-powered software that provides computer-assisted triage of suspected LVOs on CTA scans. Viz LVO is trained to identify LVOs in the supraclinoid internal carotid artery (ophthalmic, choroidal, and communicating segments) and the M1 (horizontal part) of the MCA. However, it does not assess the extracranial circulation, the posterior circulation, or the infraclinoid internal carotid artery [7]. In instances where a partial or complete occlusion is suspected, or when a vessel's caliber is less than the reference threshold, an LVO is suspected, and an alert is automatically sent to the stroke team [8]. For every CTA scan that is processed by Viz, a positive or negative LVO notification is provided, rather than the exact location of the occlusion.

For the purposes of this study, our institutional stroke database was reviewed in order to identify all suspected/confirmed LVO patients transferred from PSCs within and outside of our healthcare system from January 2020 to December 2021. Data collected included age, gender, ethnicity, race, rates of intravenous thrombolysis and mechanical thrombectomy, baseline modified Rankin Scale (mRS) score, presenting National Institutes of Health Stroke Scale (NIHSS), and initial Alberta Stroke Program Early CT Score (ASPECTS). Primary outcomes included peripheral arrival to peripheral CTA, transfer time, and all available time metrics from peripheral CTA.

The “Viz-transfers” group includes all LVO transfers from PSCs within our system (3 spoke hospitals), while the “Non-Viz-transfers” group (control group) is comprised of all LVO transfers from PSCs that are MSHS-affiliated but belong outside of our system (4 spoke hospitals). Spokes within MSHS are empowered with Viz, while spokes outside MSHS are not Viz-empowered. For non-MSHS spokes, interventional neuroradiology (INR) team notification time after CTA depends on how fast radiology and stroke teams diagnose the LVO. For MSHS spokes, post-CTA INR team notification is instantaneous when an LVO is suspected by Viz. To minimize confounding, contemporaneous LVO transfers within and outside the MSHS were compared. Patients that were placed on an “LVO watch” due to mild symptoms were excluded. Patients with missing time metrics were also excluded. This study was approved by our local IRB with waiver of informed consent.

“Viz transfers” and “Non-Viz transfers” were compared. Categorical variables were described with frequencies and percentages. For continuous variables, the Shapiro-Wilk Normality Test was conducted to assess normality. We reported means and standard deviations for normally distributed variables, whereas medians and interquartile ranges (IQRs) were reported for non-normally distributed variables. A χ² test was conducted to compare categorical variables between groups. The independent two-sample t test was conducted in order to compare normally distributed time and outcome metrics between the two groups. The Mann-Whitney U/Wilcoxon rank-sum test was conducted to compare non-normally distributed time and outcome metrics between the two groups. A probability value (p value) of <0.05 was used to determine statistical significance throughout the study. All statistical analyses were performed using R software version 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

In total, 78 patients were identified that required transfer for suspected/confirmed LVO. The median age of the total study sample was 70 (IQR 17.8), with just over half female (42; 53.8%). Overall, 38 (48.7%) patients were transferred from PSCs that had Viz LVO and 40 (51.2%) from PSCs that did not. The baseline characteristics were comparable between groups (Table 1). Within the Viz-transfers group, median age was 70 (IQR 21.8) and 55.2% were female, and within the non-Viz-transfers group median age was 71.5 (IQR 17.3) and 52.5% were female. Intravenous thrombolysis was administered to 6 (15.8%) patients in the Viz-transfers group and to 12 (30%) in the non-Viz-transfers group (p = 0.22). Mechanical thrombectomy was conducted at a similar frequency in both groups (27 [71.1] vs. 26 [65]). Baseline mRS score, mean presenting NIHSS, and median initial ASPECTS were comparable between the two groups.

All hospital workflow metrics after CTA acquisition were significantly shorter in the Viz-transfers group compared to the non-Viz-transfers group (Table 2). Median peripheral arrival to peripheral CTA and transfer times were comparable between groups (20.5 [24.3] vs. 32 [45], p = 0.280, and 23 [18] vs. 26 [13.5], p = 0.763, respectively). However, median peripheral CTA to INR team notification was 12 (16.8) min versus 58 (59.5) min, p < 0.001, median peripheral CTA to transfer call was 26.5 (29.3) versus 54 (55), p = 0.001, median peripheral CTA to transfer activation was 32 (29) versus 64 (65), p < 0.001, and median peripheral CTA to peripheral departure was 91.5 (37) versus 122.5 (68.5), p < 0.001. Median peripheral arrival to peripheral departure was 116.5 (75.5) versus 169 (126.8), p = 0.002, and median peripheral arrival to central arrival was 145 (62.5) versus 207 (97.8), p < 0.001. In addition, median peripheral CTA to central hospital emergency department arrival was 111.5 (54.5) versus 158 (66.5), p < 0.001, median peripheral CTA to angiography suite arrival was 121 (41) versus 207 (92.5), p < 0.001, median peripheral CTA to arterial puncture (AP) was 146 (53) versus 234 (99.8), p < 0.001, and median peripheral CTA to recanalization was 198 (25) versus 253.5 (86), p < 0.001. Selected time metrics are depicted in Figure 1.

This study demonstrates the impact AI-assisted software can have on workflow metrics in acute LVO stroke treatment. Among patients with suspected/confirmed LVO transferred from PSCs to CSCs within our health system, patients treated at centers utilizing Viz LVO software had shorter times to INR team notification, transfer call, transfer activation, and peripheral arrival to departure. Despite comparable transfer times, these patients also had had shorter peripheral arrival to central arrival; peripheral CTA to central hospital emergency department arrival, angiography suite arrival, AP, and recanalization times. Importantly, there were no differences in time from peripheral arrival to peripheral CTA or transfer time between the two groups, indicating similar workflow speed in both Viz and non-Viz spokes. In addition, similar transfer times indicate that both kinds of spokes geographically lie within the same driving distances from our spoke centers. These data support the use of AI-assisted software in maximizing the efficiency of transfers in modern acute stroke systems of care.

While acute stroke systems of care are evolving rapidly, transfers play a significant role. By some estimates, only 56% of people in the USA live within 60 min of endovascular centers. Even with modeling of optimal CSC access, only 63.1% of the population would be within 60 min of a CSC by ground transportation [9]. Among more than 37,000 Get With the Guidelines patients treated with thrombectomy between January 2012 and December 2017, 42.9% were transferred to the hospital that treated with thrombectomy [10]. Unfortunately, transfers are associated with costs to patients and systems of care. Multiple studies have demonstrated delays to treatment and poorer outcomes among transferred patients treated with thrombectomy [11, 12]. In the MR CLEAN trial, transferred patients had an adjusted delay of 57 min and were 8.5% less likely to achieve functional independence than directly presenting patients [13]. For every 15 min saved from onset to recanalization, 34 patients per 1,000 may have an improved disability outcome [14].

AI-assisted software is one tool to streamline acute stroke transfer processes. Its potential application is broad, from assessment of ASPECTS score to identification of salvageable tissue with perfusion imaging and LVO detection [7, 15]. Some AI software, including Viz LVO, includes tools for rapid communication as well. We have previously demonstrated the high sensitivity and specificity of AI software in detecting LVO [7].

Prior studies have compared acute stroke metrics prior to and following Viz implementation. In our preliminary analysis of 26 patients post-Viz compared to 29 patients pre-Viz, we demonstrated shorter door to neuroendovascular team notification (25.0 min [IQR = 12.0] vs. 40.0 min [IQR = 61.0], p = 0.01) [16]. Another analysis of 43 LVO patients transferred from PSC to CSC before and after Viz adoption demonstrated a 22.5 min reduction in median CTA at PSC to arrival at CSC time [5]. A study of patients with ELVO detected by Viz LVO software that included a subset of 27 patients with AI-identified ELVO demonstrated shorter CTA to first notification time, door to AP, and CTA to AP compared to usual care LVO [6]. In this analysis, we compared contemporaneous cohorts, focusing specifically on patients requiring transfers for acute stroke care, and included a more comprehensive set of acute stroke systems of care metrics that capture the transfer activation process and pre-groin puncture workflow.

Our study has several limitations, including the retrospective nature and the small sample. In this study, we could only capture hospitals within the MSHS. Even though peripheral arrival to peripheral CTA did not differ among groups, MSHS spokes workflow times might be shorter compared to non-MSHS spokes. Notably, IVT was administered in fewer patients of the VIZ-transfers group, which may have affected certain workflow times, in favor of the Viz-transfers group. However, with these data, we could account for other differences in systems of care that may impact times of treatment between PSCs that use VIZ LVO and PSCs that do not. Additionally, we do not know how other AI-assisted software systems perform or the impact of AI-assisted software on outcomes. Of course, future research efforts will have a multicenter, multi-software nature, with power enough to detect small effect sizes in terms of radiologic, hospital, and clinical outcomes.

Within the spoke and hub system of Mount Sinai, Viz LVO significantly decreased workflow metrics following CTA acquisition for patients who were transferred from PSCs with Viz versus PSCs without Viz to CSCs, for a suspected/confirmed LVO. The results of this work highlight the potential of such diagnostic adjuncts in facilitating early diagnosis and treatment of emergent LVO stroke cases.

This study protocol was reviewed and approved with a waiver of informed consent, due to the retrospective nature of the study, by the Program for the Protection of Human Subjects, before the commencement of the study (STUDY-19-00956-CR001).

Laura Stein: research funding from the American Heart Association. Johanna Fifi: personal Fees – Penumbra, Stryker, Microvention, and Cerenovus. Ownership interest: Imperative Care and Cerebrotech. Grant: Viz.

Laura Stein is supported by American Heart Association (Grant #857015/Stein/2021). Johanna Fifi has received research funding from Viz.ai.

Stavros Matsoukas: design, data acquisition and handling, statistical analysis, initial draft, and critical review of the manuscript. Laura Stein: initial draft and critical review of the manuscript. Johanna Fifi: conceptualization, study design, supervision, and critical review of the manuscript. All authors reviewed and approved the final manuscript.

All data gathered and analyzed during this study are included in this article. Further inquiries may be directed to the senior author.