Background: Flexible bronchoscopy should be performed with a correct posture and a straight scope to optimize bronchoscopy performance and at the same time minimize the risk of work-related injuries and endoscope damage. Objectives: We aimed to test whether an automatic motion analysis system could be used to explore if there is a correlation in scope movements and the operator's level of experience. Our hypothesis was that experienced bronchoscopists move less and keep the flexible scope straighter than less-experienced bronchoscopists while performing procedures. Methods: Eleven novices, 9 intermediates and 9 experienced bronchoscopy operators performed 3 procedures each on a bronchoscopy simulator. The Microsoft Kinect system was used to automatically measure the total deviation of the scope from a perfectly straight, vertical line. Results: The low-cost motion analysis system could measure the accumulated deviation of the scope precisely during the procedure. The deviations were greatest for the novices and smallest for the most experienced operators for all 3 procedures (p = 0.01, p = 0.01 and p = 0.04, respectively). The total deviation from the straight scope correlated negatively with the performance on the simulator (virtual-reality simulator score; p < 0.001). Conclusion: The motion analysis system could discriminate between different levels of experience. Automatic feedback on correct movements during self-directed training on simulators might help new bronchoscopists learn how to handle the bronchoscope like an expert.

Flexible bronchoscopy is an essential procedure in the diagnosis and treatment of lung diseases. It is often associated with some patient discomfort but is considered safe because of its low complication rate. Serious complications are hypoxemia, cardiac arrhythmias, myocardial infarction, respiratory depression, cardiorespiratory arrest, severe bleeding and pneumothorax, and these occur in <1% of the cases [1]. The mortality rate is also reported to be very low (0.02%) [2]. However, it has been found that patients undergoing bronchoscopy performed by novice bronchoscopists have an increased rate of complications [3], particularly during the first trimester of the bronchoscopist's training [4]. Simulation-based training has been proposed as a way to developing health professionals' knowledge and skills, whilst protecting patients from unnecessary risk. Virtual-reality simulators have been demonstrated to be effective as a training tool to improve technical skills, even when compared with conventional training methods [5,6], as have training tools using dexterity [7].

Bronchoscopy simulators allow trainees to practice without patients being at risk for complications and, furthermore, the simulators provide the trainees with performance metrics as useful feedback. Self-learning on simulators may save valuable time for specialists, although it is important to acknowledge that learning bronchoscopy is a unique and complex process that involves trainee competence, trainer efficiency, teacher-learner interactions and the setting in which the procedure is performed. Flexible bronchoscopy should be performed with a correct stance and a straight scope in order to optimize scope movement and to minimize the risk of both work-related injuries and damage to the scope. At present, the virtual-reality simulators are not able to provide feedback on posture and equipment handling, and therefore the trainees may risk learning wrong techniques during unsupervised training.

Only small movements of the physician's left hand are necessary to move the tip of the bronchoscope within the patient's airways - the physician should not move his entire body in his eagerness to visualize certain segments, as this may lead to poorer bronchoscopy quality. Kinematics is the field of classical mechanics which describes the movement of points, bodies (objects) and systems of bodies (groups of objects) without consideration of the causes of motion. Within the gaming industry, in particular, recent progress has made motion analysis easily accessible at limited costs. Other studies have shown the importance of role-modeling in teaching bronchoscopy, i.e. imitating the instructor [8].

Applying these analyses to bronchoscopy training could provide important additional information to the trainees striving to perform the procedure like the experts. We hypothesized that less-experienced physicians perform more burdensome and redundant movements than experienced physicians [9,10]. The bronchoscopist's physical strain is therefore increased and the efficiency of the procedure is decreased.

The aims of the study were to test whether an automatic motion analysis system could be used to track the movements of the scope and, furthermore, to explore if there is a correlation in scope movements and the operator's experience level. Our hypothesis is that more-experienced bronchoscopists move less and keep the flexible scope straighter while performing procedures, in comparison to less-experienced bronchoscopists.

Test System

The test setup consisted of a GI-BRONCH Mentor flexible bronchoscopy simulator (Simbionix Inc., Cleveland, Ohio, USA; cost: approx. USD 100,000) and a Kinect™ motion analysis system (Microsoft Inc., Redmond, Wash., USA; cost: approx. USD 200). The Kinect system was based upon the Xbox 360 gaming system and consists of external sensors capable of detecting movements. The Kinect was mounted on the wall to ensure that the placement of the motion system in relation to the simulator was unchanged throughout the test period.

Participants

The participants were divided in 3 groups based on their level of experience in performing flexible bronchoscopy. The novice group included students or physicians with no prior experience in bronchoscopy, the intermediate group included physicians who had performed between 5 and 100 bronchoscopies and the experienced group included physicians with experience in >500 bronchoscopies. None of the novices had ever tried the simulator, but 2 intermediates and 2 experienced operators had some experience in simulated bronchoscopies.

Procedures

The purpose of the study, including the parameters considered, was hidden to all participants in order to not influence their performance. Before the first test, the novices received a basic explanation of how a bronchoscope works. Each participant performed 3 tasks on the simulator, where they had to follow a blue ball through a virtual maze with twists and turns simulating a bronchial tree. Knowledge of anatomy was not required, as the pathway did not reflect the true anatomy. The simulator presented exactly the same set of 3 predefined tracks for every participant.

Outcome Parameters

The Kinect system was programmed to automatically measure the deviation of the scope hand from a perfectly straight, vertical line arising from the nose of the simulated patient (fig. 1) in a 3-dimensional manner. The Kinect system produces a map by the reflections of the person performing the procedure, performing an in-depth analysis within a range of 1.2-3.5 m. Deviations on the map from the vertical line were measured in centimeters at 30 Hz and added up to a total deviation score. The experiment was based on a number of different metrics, such as height above the horizontal line and distance between hands, but statistical analysis showed only the distance of the scope hand from the vertical line. Based on the movement of the tip of the bronchoscope in the virtual-reality simulator, a performance metric was provided after each procedure. This score is preprogrammed by the manufacturer and consists of parameters such as the number of scope collisions with the virtual wall and the duration of the procedure.

Fig. 1

Schematic of the motion analysis score (1 frame). The vertical line originates from the input orifice of the simulator. The horizontal line shows the distance that the scope hand deviates from the optimal position. The total deviation from straight is calculated by summating this distance for each recorded frame.

Fig. 1

Schematic of the motion analysis score (1 frame). The vertical line originates from the input orifice of the simulator. The horizontal line shows the distance that the scope hand deviates from the optimal position. The total deviation from straight is calculated by summating this distance for each recorded frame.

Close modal

Study Design

This was a prospective, comparative study.

Statistical Analysis

The correlation between the virtual-reality simulator score and the total deviation from straight scope measured by the Kinect system was studied by calculating Pearson's r. The performance of the 3 groups (novices, intermediates and experienced operators) was compared using one-tailed, one-way analysis of variance (ANOVA). This was conducted for each test (the first, second and third) and for both simulator score and for total deviation from the straight scope. An one-tailed test was used as it was assumed that novices did not show movements superior to those of the more experienced bronchoscopists. Multiple linear regressions were made for simulator score and total deviation from the straight scope to test the significance of group and test number as predictors. Statistical analysis was performed using a statistical software package (PASW v19.0; SPSS Inc., Chicago, Ill., USA). Differences were considered to be statistically significant when the p value was <0.05.

Twenty-nine participants were enrolled in the study: 11 novices, 9 intermediates and 9 experienced operators. All participants completed the study.

The total deviation from the straight scope measured by the Kinect system correlated negatively with the virtual-reality simulator score, i.e. a higher deviation = a lower score (Pearson's r = -0.67, p < 0.001; fig. 2).

Fig. 2

The correlation between virtual-reality simulator score and total deviation from straight scope measured automatically by the Kinect system. ‘Simulator score' is an arbitrary score between 0 and 100 that was calculated based on the movements of the scope inside the simulator. ‘Total deviation from straight' is the total distance in centimeters that the scope hand deviated from the optimal position during the procedure (see fig. 1). Pearson's r = -0.67, p < 0.001.

Fig. 2

The correlation between virtual-reality simulator score and total deviation from straight scope measured automatically by the Kinect system. ‘Simulator score' is an arbitrary score between 0 and 100 that was calculated based on the movements of the scope inside the simulator. ‘Total deviation from straight' is the total distance in centimeters that the scope hand deviated from the optimal position during the procedure (see fig. 1). Pearson's r = -0.67, p < 0.001.

Close modal

ANOVA only found significant differences in performances measured by the virtual-reality simulator in the first test (p = 0.01; fig. 3) whereas performances measured by the Kinect system were significantly different in all 3 tests (p = 0.01, p = 0.02 and p = 0.04, respectively; fig. 4).

Fig. 3

The performances of the 3 groups in the 3 tests measured by the virtual-reality simulator. ‘Simulator score' is an arbitrary score between 0 and 100 that was calculated based on the movements of the scope inside the simulator.

Fig. 3

The performances of the 3 groups in the 3 tests measured by the virtual-reality simulator. ‘Simulator score' is an arbitrary score between 0 and 100 that was calculated based on the movements of the scope inside the simulator.

Close modal
Fig. 4

The performances of the 3 groups in the 3 tests measured automatically by the Kinect system. ‘Total deviation from straight' is the total distance in centimeters that the scope hand deviated from the optimal position during the procedure (see fig. 1).

Fig. 4

The performances of the 3 groups in the 3 tests measured automatically by the Kinect system. ‘Total deviation from straight' is the total distance in centimeters that the scope hand deviated from the optimal position during the procedure (see fig. 1).

Close modal

Multiple linear regression showed that the test number (first, second or third, respectively) was the strongest predictor for outcome measured by the virtual-reality simulator (B = 4.1 and B = 5.3 for group and test number, respectively). In the Kinect system, however, the group (i.e. novices, intermediates or experienced operators) was the strongest predictor for outcome (B = -81.8 for group and B = -45.0 for test number).

Tables 1 and 2 show the data of the virtual reality simulator score and the Kinect score (total deviation from the straight line), respectively.

Table 1

The performance of the 3 groups in the 3 tests as measured by the virtual-reality simulator

The performance of the 3 groups in the 3 tests as measured by the virtual-reality simulator
The performance of the 3 groups in the 3 tests as measured by the virtual-reality simulator
Table 2

The performance of the 3 groups in the 3 tests as measured automatically by the Kinect system (total deviation from the straight scope in centimeters)

The performance of the 3 groups in the 3 tests as measured automatically by the Kinect system (total deviation from the straight scope in centimeters)
The performance of the 3 groups in the 3 tests as measured automatically by the Kinect system (total deviation from the straight scope in centimeters)

This is the first study to use an automatic tracking device to monitor the movements of bronchoscopists and to analyze the correlation of the movements of the bronchoscope and the operator's level of experience. It was found that the deviation of the scope from the straight line correlated negatively with the virtual-reality simulator score (p < 0.001). This could indicate that the correct, deliberate movements of an experienced bronchoscopist transform into an efficient and gentle procedure where the scope is moved quickly and kept in the center of the lumen without collisions with the bronchial wall. This coincides with the theory of Fitts and Posner [10] that people who are skilled at a specific task will limit unnecessary movements.

Each group handled the flexible scope differently (p < 0.05 in all 3 tests). The largest mean deviation from the straight line was observed in the novice group while the smallest was observed in the experienced group, showing that the experienced bronchoscopists used smaller, less strenuous movements and kept the scope straighter. This was consistent in all 3 tests, although the virtual-reality simulator score found statistically significant differences between the 3 groups only in the first test. The multiple linear regression model showed that the novices were quickly able to ‘cheat' the virtual-reality simulator and to achieve a high score (test number was a stronger predictor for a high simulator score than group). This finding underpins the importance of exploring the validity of simulator metrics before using them to assess competency [11,12,13].

In sports like tennis or golf, athletes [14] try to simulate the movements of the experts. We believe that applying this principle to flexible bronchoscopy could make teaching more effective and faster. The Kinect system may be a valuable tool to help trainees adapt their movements during the procedure and, thus, provide an added benefit to directed self-learning, which is considered an efficient way of practising [15]. At present, the accuracy of the Kinect sensor [16,17] only allows for analysis of stance/pose features. More complex calculations would require a higher precision, but further research in similar motion analysis tools could be of great interest to the applicability of motion-based assessment in bronchoscopy.

Performing flexible bronchoscopy with a bend scope, especially using acute angles, increases the ‘wear-and-tear' of the bronchoscope and probably also the repair costs. Lunn et al. [18] were able to reduce repair costs by 84% after introducing an educational program into their interventional pulmonology program. It should be investigated whether using the Kinect system to teach trainees to keep the scope straight could result in cost savings.

Although previous guidelines [19,20] required a minimum of 100 procedures to achieve competency, recent guidelines [1] state that stipulating a specific number of procedures is not meaningful, as these minimum numbers are not evidence-based [1,19,20]. Instead, the important finding of when a trainee is ready to perform procedures without supervision should be based on objective assessments of competences and credible pass-fail standards [21,22]. We advocate that the equally important decision of when a trainee is allowed to start practising bronchoscopy on patients should also be based on objective measurement of skills in a simulated environment [11]. Combined automated analyses, such as kinematics analyses and other objective measurements, have the potential to replace subjective assessments by experienced bronchoscopists. A synchronized video-based motion tracking was found to be an instant, objective, valid and reliable mode of assessment of laparoscopic performance in the operating room [23,24]. There were significant differences for motion tracking parameters between experts and novices as well as in the time taken for the procedure (Calot's triangle dissection in the number of movements).

However, it is important to acknowledge the fact that the complex art of performing flexible bronchoscopy cannot be summarized by relatively simple parameters that describe the movements of the scope and the operator. According to the framework of competence in the study by Miller [25], physicians have to demonstrate skills on 4 different levels, i.e. ‘knows', ‘knows how', ‘shows how' and ‘does'. The necessary theoretical knowledge of the first 2 levels includes the knowledge of anatomy and the indications, contraindications, risks, benefits and methods of the procedure. This is best tested using theoretical tests with solid evidence of validity like in a recently published test of knowledge (with multiple-choice questions) in endosonography [26]. The metrics presented in this paper relate to the third level where trainees demonstrate technical competence in a simulated environment. Other important aspects such as selecting patients, administering drugs, interpreting imaging and providing information to patients and relatives need to be addressed before a trainee is allowed to function independently in a bronchoscopy unit. Even the best bronchoscopy training program, including theoretical tests and simulation-based certification, should be followed by supervised clinical performance. That being stated, this tool could provide an automatic assessment of physicians performing bronchoscopies that could give a more diverse baseline for the continuous assessment of the bronchoscopy skills of physicians.

Our study is the first to explore motion analysis for flexible bronchoscopy and has several limitations. The sample size was relatively small and all data collection was done in a simulated environment. Furthermore, the actual effect of correct movement on scope damage, physical strain on the bronchoscopist, and ultimately patient care, has to be explored in future studies.

The motion analysis system could discriminate between bronchoscopists with different levels of experience. Automatic feedback on correct movements during self-directed training on simulators might help new bronchoscopists learn how to handle the bronchoscope like an expert.

We thank all participating consultants, trainees, and medical students for their kind participation. This work was supported by Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark.

The authors declare that they have no competing interests.

1.
Du Rand IA, Blaikley J, Booton R, Chaudhuri N, Gupta V, Khalid S, Mandal S, Martin J, Mills J, Navani N, Rahman NM, Wrightson JM, Munavvar M; the British Thoracic Society Bronchoscopy Guideline Group: British Thoracic Society guideline for diagnostic flexible bronchoscopy in adults. Thorax 2013;68:i1-i44.
2.
Facciolongo N, Patelli M, Gasparini S, et al: Incidence of complications in bronchoscopy. Multicentre prospective study of 20,986 bronchoscopies. Monaldi Arch Chest Dis 2009;71:8-14.
3.
Stather DR, MacEachern P, Chee A, Dumoulin E, Tremblay A: Trainee impact on procedural complications: an analysis of 967 consecutive flexible bronchoscopy procedures in an interventional pulmonology practice. Respiration 2013;85:422-428.
4.
Ouellette DR: The safety of bronchoscopy in a pulmonary fellowship program. Chest 2006;130:1185-1190.
5.
Wahidi MM, Silvestri GA, Coakley RD, Ferguson JS, Shepherd RW, Moses L, Conforti J, Que LG, Anstrom KJ, McGuire F, Colt H, Downie GH: A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows. Chest 2010;137:1040-1049.
6.
Ost D, DeRosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC: Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med 2001;164:2248-2255.
7.
Marsland CP, Robinson BJ, Chitty CH, Guy BJ: Acquisition and maintenance of endoscopic skills: developing an endoscopic dexterity training system for anesthesiologists. J Clin Anesth 2002;14:615-619.
8.
Bjerrum AS, Hilberg O, van Gog T, Charles P, Eika B: Effects of modelling examples in complex procedural skills training: a randomised study. Med Educ 2013;47:888-898.
9.
Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A: Relationship between skill and outcome in the laboratory-based model. Surgery 2002;131:318-323.
10.
Fitts PM, Posner MI: Human Performance. Oxford, Brooks/Cole, 1967.
11.
Konge L, Arendrup H, von Buchwald C, Ringsted C: Using performance in multiple simulated scenarios to assess bronchoscopy skills. Respiration 2011;81:483-490.
12.
Konge L, Annema J, Clementsen P, Minddal V, Vilmann P, Ringsted C: Using virtual-reality simulation to assess performance in endobronchial ultrasound. Respiration 2013;86:59-65.
13.
Cesanek P, Uchal M, Uranues S, Patruno J, Gogal C, Kimmel S, Bergamaschi R: Do hybrid simulator-generated metrics correlate with content-valid outcome measures? Surg Endosc 2008;22:2178-2183.
14.
Guadagnoli M, Holcomb W, Davis M: The efficacy of video feedback for learning golf swing. J Sport Sci 2002;20:615-622.
15.
Brydges R, Nair P, Ma I, Shanks D, Hatala R: Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ 2012;46:648-656.
16.
Mobini A, Behzadipour S, Saadat Foumani M: Accuracy of Kinect's skeleton tracking for upper body rehabilitation applications. Disabil Rehabil Assist Technol 2014;9:344-352.
17.
Obdrzalek S, Kurillo G, Ofli F, Bajcsy R, Seto E, Jimison H, and Pavel M: Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. Conf Proc IEEE Eng Med Biol Soc 2012;2012:1188-1193.
18.
Lunn W, Garland R, Gryniuk L, Smith L, Feller-Kopman D, Ernst A: Reducing maintenance and repair costs in an interventional pulmonology program. Chest 2005;127:1382-1387.
19.
Ernst A, Silvestri GA, Johnstone D; American College of Chest Physicians: Interventional pulmonary procedures: guidelines from the American College of Chest Physicians. Chest 2003;123:1693-1717.
20.
British Thoracic Society Bronchoscopy Guidelines Committee, a Subcommittee of Standards of Care Committee of British Thoracic Society: British Thoracic Society guidelines on diagnostic flexible bronchoscopy. Thorax 2001;56(suppl 1):i1-i21.
21.
Konge L, Larsen KR, Clementsen P, Arendrup H, von Buchwald C, Ringsted C: Reliable and valid assessment of clinical bronchoscopy performance. Respiration 2012;83:53-60.
22.
Konge L, Clementsen P, Larsen KR, Arendrup H, Buchwald C, Ringsted C: Establishing pass/fail criteria for bronchoscopy performance. Respiration 2012;83:140-146.
23.
Aggarwal R, Dosis A, Bello F, Darzi A: Motion tracking systems for assessment of surgical skill. Surg Endosc 2007;21:339 (Letter to the editor).
24.
Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A: An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 2007;245:992-999.
25.
Miller GE: The assessment of clinical skills/competence/performance. Acad Med 1990;65(9 suppl):S63-S67.
26.
Savran MM, Clementsen PF, Annema JT, Minddal V, Larsen KR, Park YS, Konge L: Development and validation of a theoretical test in endosonography for pulmonary diseases. Respiration 2014;88:67-73.
Copyright / Drug Dosage / Disclaimer
Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.