Preparing for Reality: A Randomized Trial on Immersive Virtual Reality for Bronchoscopy Training

Flexible bronchoscopy is an essential diagnostic procedure in pulmonary medicine, and it is crucial to have a systematic approach to visualize all bronchial segments to ensure diagnostic accuracy. Although the procedure is relatively safe, overlooking life-threatening diseases can have serious consequences [1‒3]. The US and the UK guidelines recommend doing 100 bronchoscopies under supervision before performing the procedure independently [1, 4]. However, that number may not be enough to ensure that the bronchoscopist is sufficiently prepared; simulation-based training and competency assessment could increase the quality of diagnostic bronchoscopies [2, 3].

Simulation-based training is increasingly used in medical procedures training because it allows novices to practice skills in a patient-safe environment [5, 6]. For the most part, simulation-based training solely focuses on training technical skills in isolation. However, during real procedures, the doctor is exposed to distractions, disturbances, and unforeseen events, which can impact performance and the quality of the procedure [7, 8]. Immersive virtual reality (iVR) is a new modality in simulation-based training, where doctors can train in more realistic scenarios preparing them for, e.g., distractions during procedures. Simulation-based training can shorten the operating time of novice surgeons when they begin in the OR, and iVR has also been found to shorten the operating time, in some studies, even more than traditional simulation-based training [9, 10].

Furthermore, studying training in a standardized and patient-safe iVR environment can provide us with valuable information on which and how much distractions disturb participants and whether distractions affect the quality of a procedure. The objective of this study was to explore whether including bronchoscopy training in an iVR environment with distractions compared to only using conventional simulation-based training would make the doctor better at coping with distractions and improve the quality of the bronchoscopy procedure when tested in a simulated scenario with distractions.

The trial was a single-center randomized superiority trial, following the CONSORT statement (Fig. 1) [11, 12] at Copenhagen Academy for Medical Education and Simulation (CAMES). The trial was exempt from ethical approval by the Regional Committee on Health Research Ethics (H-21010842) and was registered on clinicaltrials.gov (NCT05078762).

Participants were junior doctors in their first year of training with a medical license. They were recruited from contacting various hospital departments in the Capital Region of Denmark and Facebook announcements in groups for junior doctors. Participants were included if they had no prior experience in trials involving bronchoscopy training or with independent bronchoscopy. Participants had to provide informed consent and be able to speak Danish on a conversational level. Informed consent was obtained after written and oral information and before inclusion.

The participants first answered a short demographics questionnaire and watched a 30-min video explaining the theory behind and demonstrating a diagnostic bronchoscopy. They then practiced for 30 min on a conventional bronchoscopy simulator with assistance from the principal investigator (AGA). Afterward, the participants were randomized to train for either 30 min on the bronchoscopy simulator while in the iVR environment in the training scenarios with distractions or 30 min on the same bronchoscopy simulator without iVR.

Finally, all participants had to undergo a test in the iVR environment where they were exposed to various distractions different from the ones used in the training scenarios. Both groups got 5 min to get familiar with the iVR environment before performing the bronchoscopy. In addition, a 5-min ECG was obtained before training began, at the end of each training session, and during the test. After the test, all participants answered the Surg-TLX questionnaire and a questionnaire previously used in iVR studies [13] on nausea, vertigo, or physical uneasiness.

Three expert bronchoscopists were interviewed to single out common distractions during bronchoscopies. These distractions were divided into passive and active distractions and were included in seven virtual scenarios (see online suppl. material; for all online suppl. material, see www.karger.com/doi/10.1159/000528319).

The scenarios were filmed with a 360-degree video camera (QooCam 8K, KanDao Technology Co., Ltd, Shenzhen, China) at Bispebjerg Hospital, Copenhagen, Denmark, in an actual OR with actors with a medical background (Fig. 2). The virtual OR was created in the Cinclus360 Software (www.cinclus360.dk) developed by CAMES. The seven virtual scenarios consist of three calm scenarios, three scenarios with varying degrees of distractions and variations for training, and one test scenario (see online suppl. material).

The outcomes were the quality of bronchoscopy based on two previously validated and reliable assessment tools for assessing technical skills [14, 15] and cognitive load measures [16‒18]. The primary-outcome measures were a score based on the structured progress score; the systematic visualization and structured passage through the bronchial tree measured in points from 0 to 18 [14], and procedure time from passing the vocal cords to retraction of the endoscope in seconds.

The secondary outcomes were diagnostic completeness; the fraction in percent (%) of visited bronchial segments, where each observed segment contributed to 5.6 percent as there were 18 possible segments to visualize (segments one and two on the left side were grouped together); and Motor Bronchoscopy Skill Score: an objective and automatic composite score based on lower arm movement, measured with an Inertial Measurement Unit, and electromyography findings of hand and finger movement [15]. The tertiary outcome was cognitive load during the training and testing of the participants. This was assessed with heart rate variability (HRV). HRV was measured using the LF/HF ratio, the fraction between the low frequency (ms²) and high-frequency heart rates, measured as R-R intervals on an electrocardiogram for 5 min [17]. In addition, the participants were asked to complete the Surg-TLX [18] questionnaire to assess cognitive load. Subjective symptoms of motion sickness were assessed as an explorative outcome using the Motion Sickness Assessment Questionnaire [19] on a point scale from 1 to 9.

Participants were randomized for the intervention group or the control group 1:1 using the Web-based service Sealed Envelope^TM (Sealed Envelope^TM, London, UK). The participants were stratified by sex (man/woman) as this has been seen to influence skill acquisition during the early part of the learning curve [20].

Due to the nature of the study, neither the participants nor the principal investigator could be blinded to the intervention. The statistical analysis was performed blinded by the last author.

The bronchoscopy simulator consisted of an endoscopy tower with a flexible bronchoscope (EVIS Exera II and Q180 flexible bronchoscope, Olympus, Japan) and a phantom with a three-dimensional bronchial tree (CLA Broncho Boy, CLA, Coburg, Germany). To create the iVR environment, the bronchoscopy simulation was combined with HTC VIVE Pro Eye (HTC Corporation, Taiwan) with an additional computer handling the VIVE Pro Eye and playback of 360° videos of the endoscopy room and a capture card (Epiphan SDI2USB3.0, Epiphan Systems Inc., Canada and USA) to get the image output from the endoscopy rack to the computer.

The participants’ hand movements were monitored with MetaMotionR (Mbientlab Inc., USA and Korea) that measures lower arm movements through a 6-axis accelerometer + gyroscope inserted in a wristband. HRV (LF/HF ratio) was measured on an electrocardiogram (Faros 180° sensor [Mega Electronics Ltd., Kuopio, Finland] in 5-min intervals [21] and analyzed in Kubios HRV Premium version 3.5.0 (Biosignal Analysis and Medical Group, Kuopio, Finland). The setup is demonstrated in Figures 3and 4.

Data were analyzed with SPSS version 27 (IBM, Armonk, NY, USA). Data were analyzed using Student’s t test if they were normally distributed; otherwise, the Mann-Whitney test was used.

The sample size estimation was based on the procedure time (from passing the vocal cords to retraction of the scope). It was based on the assumption that the intervention group would be faster than the control group, where a minimum of 33% reduction in procedure time was deemed relevant. The control group was expected to complete the procedure in 372 s, and the intervention group was expected to complete the procedure in 280 s. The standard deviation (SD) was 80 s in both groups [14]. Using a two-sided significance level of 0.05 with alpha set at 5% and a power of 90%, a sample size of a minimum of 17 participants in each group was required.

Thirty-six participants were included, and 34 completed the trial. One participant withdrew before training, and one did not complete the intervention. The intervention group consisted of 8 women and 9 men with a mean age of 30 (SD 3.1) years. The control group consisted of 9 women and 8 men with a mean age of 32.2 (SD 4.5) years.

The intervention group had a significantly higher score in diagnostic completeness (p = 0.029) and structured progress (p = 0.032). For diagnostic completeness, the results mean that the intervention group was less likely to miss any segments, whereas more participants in the control group missed one or more lung segments (100 i.q.r. 100–100 vs. 94 i.q.r. 89–100). In numbers, 13 out of 17 participants in the intervention group observed all segments, while only 6 out of 17 participants in the control group saw all segments. The procedure time was slightly faster for the iVR group, though not statistically significant (367 s SD 149 vs. 445 SD 219, p = 0.06) (Table 1).

There was a slight tendency for the HRV to be lower in the control group (median 5.76, i.q.r. 3.77–9.06) than in the iVR group (median 4.12, i.q.r. 2.68–6.27). This was however not significant (p = 0.25). In addition, there was no significant difference in total Surg-TLX points between the two groups (p = 0.77).

The motion sickness scores were low in both groups, with the maximum point given on a scale from 1 to 9 being four and the highest median score being two. There were however four areas where the control group scored themselves higher after the test (Table 2).

We found that the iVR group was more structured during their test and had higher diagnostic completeness than the control group. While not significant, the iVR group performed the procedure faster as well. A structured approach to ensure diagnostic completeness is fundamental when doing a diagnostic bronchoscopy to ensure that all segments are visualized. While both groups scored high on diagnostic completeness, it is very important not to miss any segments during a bronchoscopy as there will be a risk of missing relevant pathology [22]. This shows that iVR has the potential as an educational tool to reduce the gap between traditional simulation environments and the real world.

Both the intervention group and the control group had a high structured progress score and diagnostic completeness. In an earlier study by Cold et al. [14], examining the structured progress score, the mean score for novices with no prior training was 5, while the mean scores were 12 for the control group and 16 for the intervention group in our study. However, the procedure time varied from person to person, and only three participants got a max structured progress score, pointing toward bronchoscopy being challenging to get perfect. There was no training time on the bronchoscopy simulator before the test in the study by Cold et al. [14], indicating that the training time in our study made a big difference.

Two studies that investigated first-person-view iVR as an introduction before a procedure also found better performance in the iVR group compared to the control group [23, 24]. In a study by Ros et al. [23], the intervention group completed a lumbar puncture more efficiently and faster than the control group on a mannequin. There was no change in generalized knowledge of lumbar puncture. However, this study did not use iVR in the training of the procedure but only in the theoretical introduction, and the test was done on a mannequin without iVR. Chao et al. [24] investigated performance and cognitive load in medical students who likewise had an introduction to history-taking and physical examination with either first-person-view iVR or a 2D video. The intervention group similarly performed the physical examination on an outpatient better compared to the control group, although there was no difference in history-taking. Interestingly the intervention group had a higher cognitive load as well, measured with the Cognitive Load Component Questionnaire.

While neither the self-reported cognitive load nor the HRV showed a significant difference in cognitive load between the two groups, there was a clear trend for the HRV measured as the LF/HF ratio to be higher in the iVR group, suggesting that they were exposed to a higher cognitive load than the control group. Existing research suggests that cognitive load is higher during iVR training. A study by Sankaranarayanan et al. [25] showed that a higher cognitive load during training for a bimanual surgical task resulted in better performance under similar real-world conditions. As with our study, the point of raising the cognitive load was to address the fact that trainees face a higher cognitive load under real-world circumstances compared to the learning environment.

We found that novices who trained with iVR performed a diagnostic bronchoscopy better in a test where they were exposed to common distractions and stressors compared to novices who had not trained with iVR. Frederiksen et al. [13] examined cognitive load and performance while performing a simulated laparoscopic salpingectomy with or without iVR with distractions. They found that the iVR group performed the procedure worse than the control group and that the iVR group had a higher cognitive load measured by increased reaction time [13]. Though cognitive load was measured differently, the study showed a much higher difference in cognitive load; the intervention group had a cognitive load 7.9% higher than conventional VR. This is consistent with our finding that the iVR group had a higher cognitive load; however, Frederiksen et al. [13] did not allow the participant to train before being tested.

Increased cognitive load in the real world and iVR can result in overload, leading to worse performance. In more simple procedures, these cognitive challenges could help prepare the users for reality while training for the procedure, while in more demanding procedures, this could lead to cognitive overload and worsen their performance and training output. A review by Han et al. [26] advises that more simple tasks could be better for iVR training. It might be that diagnostic bronchoscopy is a more straightforward procedure than a laparoscopic procedure with complications and, therefore, might be better suited for iVR simulation.

This study is one of the first randomized controlled studies to compare simulation training in iVR with conventional simulation-based training. The introduction material and videos were completely standardized for an equal starting point for the participants. The setup was realistic, with 4K quality videos filmed in a real bronchoscopy setting with realistic distractions defined by three expert bronchoscopists. The screen showing the ongoing bronchoscopy was of the same output and quality as the conventional setup.

The assessment methods for bronchoscopy performance and cognitive load have previously been validated and tested. However, only one investigator scored the performance and was not blinded to group allocation.

The intervention group both trained in the conventional setup without iVR and with iVR. This was to familiarize both groups with the procedure before introducing iVR. This also meant that the intervention group had a blended experience in contrast to the possibility of completely splitting the two groups into a control and intervention group.

Using an iVR setup adds an additional cost to simulation-based mannequin training. It requires a computer with the hardware to run the iVR software and a head-mounted VR device. Compared to the cost of full-scale simulations, the cost of the abovementioned simulation is a one-time cost and is relatively inexpensive compared with the cost of bronchoscopy simulators and the rest of the equipment used.

In our study, the participants had the disadvantage of not being able to see their hands while doing the test in iVR, and the intervention group had practiced this way for 30 min. In contrast, the control group only had a short time to familiarize themselves with this different way of training.

Both groups trained on the same mannequin with the same lung tree and were tested on the same mannequin. This was to make the only variable the iVR setup itself. However, this could limit the clinical transfer as there are many anatomical variations in the real world. Additionally, other different interventions could have been included in the training and test to make it harder, and make it more realistic, as there are many instances in the real world where it would be necessary to, e.g., perform a biopsy or a bronchial lavage during the bronchoscopy; however, this was not possible in our setup. The participants did not perform bronchoscopy on real patients, which would be the most accurate assessment of the intervention. In the future, it would be very interesting to investigate the effect of iVR training on real procedures and further develop our understanding of how to use iVR by investigate different procedures and different types of distractions.

Immersive VR has the potential to change simulation-based training, by bridging the gap between traditional learning and the real world by increasing the realism of the simulation-based environment. In the future, it could be possible to replay the experience to reflect on performance, to include helping texts or slow down the experience for novices, and include adverse events for more experienced doctors. With the technology becoming more advanced, so will the implications for simulation-based education.

Training with iVR and being exposed to distractions increases the quality of bronchoscopy compared with conventional simulation-based training when tested in a simulated environment. Participants might tend to be more mentally stressed if they have trained in an iVR environment before, thus inducing a higher cognitive load, but this does not translate into worse performance, and this indicates that a higher cognitive load is not necessarily a hindrance to learning, depending on the procedure.

The authors would like to express their gratitude to Helle Frost Andreassen for help in finding a filming location and to the Department of Pulmonary Medicine at Bispebjerg Hospital to make premises available. We thank Anna Forsbjerg, Emilie Vester Engel, Lea O-Reilly, Marie Dyrehave, and Mikkel Ravn Dyhr for their help in filming. We would also like to thank Jens Jerichau Clausen for photographing the iVR and bronchoscopy setup and Benedikte Hartvigsen for demonstrating the setup.

All subjects have given their written informed consent. The trial was a single-center randomized superiority trial, following the CONSORT statement (Fig. 1) [11] at Copenhagen Academy for Medical Education and Simulation (CAMES). The study protocol was deemed exempt from ethical approval by the Regional Committee on Health Research Ethics (H-21010842) and was registered on clinicaltrials.gov (NCT05078762).

The authors have no conflicts of interest to declare.

The project was funded by the InnoExplorer grant from the Innovation Fund Denmark.

Annarita Ghosh Andersen had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Annarita Ghosh Andersen, Morten Bo Søndergaard Svendsen, and Flemming Bjerrum were responsible for the data analysis. Annarita Ghosh Andersen, Laila Rahmoui, Tor-Salve Dalsgaard, Morten Bo Søndergaard Svendsen, Paul Frost Clementsen, Lars Konge, and Flemming Bjerrum contributed substantially to the study design and writing of the manuscript.

All data generated or analyzed during this study are included in this article and its online supplementary material. Further enquiries can be directed to the corresponding author.