Background: Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists. Objectives: The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion. Methods: Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS). Results: Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson’s correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson’s correlation, r = −0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups’ method, a passing score of −0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures. Conclusions: MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

1.
Du Rand
IA
,
Barber
PV
,
Goldring
J
,
Lewis
RA
,
Mandal
S
,
Munavvar
M
, et al
British Thoracic Society guideline for advanced diagnostic and therapeutic flexible bronchoscopy in adults
.
Thorax
.
2011
;
66
(
Suppl 3
):
iii1
21
. .
2.
Hsu
LH
,
Liu
CC
,
Ko
JS
.
Education and experience improve the performance of transbronchial needle aspiration: a learning curve at a cancer center
.
Chest
.
2004
;
125
(
2
):
532
40
. .
3.
Ouellette
DR
.
The safety of bronchoscopy in a pulmonary fellowship program
.
Chest
.
2006
;
130
(
4
):
1185
90
. .
4.
Mitsumune
T
,
Senoh
E
,
Adachi
M
.
Prediction of patient discomfort during fibreoptic bronchoscopy
.
Respirology
.
2005
;
10
(
1
):
92
6
. .
5.
Stather
DR
,
MacEachern
P
,
Chee
A
,
Dumoulin
E
,
Tremblay
A
.
Trainee impact on procedural complications: an analysis of 967 consecutive flexible bronchoscopy procedures in an interventional pulmonology practice
.
Respiration
.
2013
;
85
(
5
):
422
8
. .
6.
Naur
TMH
,
Nilsson
PM
,
Pietersen
PI
,
Clementsen
PF
,
Konge
L
.
Simulation-based training in flexible bronchoscopy and endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA): a systematic review
.
Respiration
.
2017
;
93
(
5
):
355
62
. .
7.
Ost
D
,
DeRosiers
A
,
Britt
EJ
,
Fein
AM
,
Lesser
ML
,
Mehta
AC
.
Assessment of a bronchoscopy simulator
.
Am J Respir Crit Care Med
.
2001
;
164
(
12
):
2248
55
. .
8.
Siddaiah-Subramanya
M
,
Smith
S
,
Lonie
J
.
Mastery learning: how is it helpful? an analytical review
.
Adv Med Educ Pract
.
2017
;
8
:
269
75
. .
9.
McGaghie
WC
,
Harris
IB
.
Learning theory foundations of simulation-based mastery learning
.
Simul Healthc
.
2018
;
13
(
3S Suppl 1
):
S15
s20
. .
10.
Konge
L
,
Clementsen
P
,
Larsen
KR
,
Arendrup
H
,
Buchwald
C
,
Ringsted
C
.
Establishing pass/fail criteria for bronchoscopy performance
.
Respiration
.
2012
;
83
(
2
):
140
6
.
11.
Colella
S
,
Søndergaard Svendsen
MB
,
Konge
L
,
Svendsen
LB
,
Sivapalan
P
,
Clementsen
P
.
Assessment of competence in simulated flexible bronchoscopy using motion analysis
.
Respiration
.
2015
;
89
(
2
):
155
61
. .
12.
Bjerrum
AS
,
Hilberg
O
,
van Gog
T
,
Charles
P
,
Eika
B
.
Effects of modelling examples in complex procedural skills training: a randomised study
.
Med Educ
.
2013
;
47
(
9
):
888
98
. .
13.
Datta
V
,
Mandalia
M
,
Mackay
S
,
Chang
A
,
Cheshire
N
,
Darzi
A
.
Relationship between skill and outcome in the laboratory-based model
.
Surgery
.
2002
;
131
(
3
):
318
23
. .
14.
Fitts
PM
,
Posner
MI
.
Human performance
.
Oxford, UK
:
Brooks/Cole
;
1967
.
15.
Kutafina
E
,
Laukamp
D
,
Bettermann
R
,
Schroeder
U
,
Jonas
SM
.
Wearable sensors for elearning of manual tasks: using forearm EMG in hand hygiene training
.
Sensors
.
2016
;
16
(
8
):
1221
. .
16.
Sathiyanarayanan
M
,
Rajan
S
, editors.
MYO armband for physiotherapy healthcare: a case study using gesture recognition application
.
2016 8th International Conference on Communication Systems and Networks (COMSNETS)
;
2016 Jan 5–10
;
2016
.
17.
Ryser
F
,
Butzer
T
,
Held
JP
,
Lambercy
O
,
Gassert
R
.
Fully embedded myoelectric control for a wearable robotic hand orthosis
.
IEEE Int Conf Rehabil Robot
.
2017
;
2017
:
615
21
. .
18.
Lin
P
,
Chen
HY
, editors.
Design and implement of a rehabilitation system with surface electromyography technology
.
2018 IEEE International Conference on Applied System Invention (ICASI); 2018 Apr 13–17
;
2018
.
19.
Kowalewski
KF
,
Garrow
CR
,
Schmidt
MW
,
Benner
L
,
Müller-Stich
BP
,
Nickel
F
.
Sensor-based machine learning for workflow detection and as key to detect expert level in laparoscopic suturing and knot-tying
.
Surg Endosc
.
2019 Nov
;
33
(
11
):
3732
40
.
20.
Messick
S
.
Validity
. In:
Linn
RL
, editor.
Educational measurement
. 3rd ed.
New York, NY
:
American Council on Education and Macmillan
;
1989
. p.
13
104
.
21.
Downing
S
,
Yudkowsky
R
.
Assessment in health professions education
.
New York, NY
:
Routledge
;
2009
.
22.
Borgersen
NJ
,
Naur
TMH
,
Sørensen
SMD
,
Bjerrum
F
,
Konge
L
,
Subhi
Y
, et al
Gathering validity evidence for surgical simulation: a systematic review
.
Ann Surg
.
2018
;
267
(
6
):
1063
8
. .
23.
Clementsen
P
,
Nayahangan
L
,
Konge
L
.
Bronchoscopy basisbook
.
Copenhagen Academy for Medical Education and Simulation
;
2015
.
24.
Colt
HG
,
Crawford
SW
,
Galbraith
O
 3rd
.
Virtual reality bronchoscopy simulation: a revolution in procedural training
.
Chest
.
2001
;
120
(
4
):
1333
9
. .
25.
Konge
L
,
Arendrup
H
,
von Buchwald
C
,
Ringsted
C
.
Using performance in multiple simulated scenarios to assess bronchoscopy skills
.
Respiration
.
2011
;
81
(
6
):
483
90
. .
26.
Jørgensen
M
,
Konge
L
,
Subhi
Y
.
Contrasting groups’ standard setting for consequences analysis in validity studies: reporting considerations
.
Adv Simul
.
2018
;
3
(
1
):
5
.
27.
Jones
E
,
Oliphant
T
,
Peterson
P
.
SciPy: Open Source Scientific Tools for Python
.
2001
.
Online;. Available from: http://www.scipy.org/ Accessed 2019 May 12
.
28.
Hunter
JD
.
Matplotlib: a 2D graphics environment
.
Comput Sci Eng
.
2007
;
9
(
3
):
90
5
. .
29.
Walt
S
,
Colbert
SC
,
Varoquaux
G
.
The NumPy array: a structure for efficient numerical computation
.
Comput Sci Eng
.
2011
;
13
(
2
):
22
30
.
30.
Gilbert
CR
,
Thiboutot
J
,
Mallow
C
,
Chen
A
,
Pastis
NJ
,
Argento
AC
, et al
Assessment of ergonomic strain and positioning during bronchoscopic procedures: a feasibility study
.
J Bronchology Interv Pulmonol
.
2020 Jan
;
27
(
1
):
58
67
.
31.
Mendez
I
,
Hansen
BW
,
Grabow
CM
,
Smedegaard
EJL
,
Skogberg
NB
,
Uth
XJ
, et al
Evaluation of the Myo armband for the classification of hand motions
.
IEEE Int Conf Rehabil Robot
.
2017
;
2017
:
1211
4
. .
32.
Vilmann
AS
,
Norsk
D
,
Svendsen
MBS
,
Reinhold
R
,
Svendsen
LB
,
Park
YS
, et al
Computerized feedback during colonoscopy training leads to improved performance: a randomized trial
.
Gastrointest Endosc
.
2018
;
88
(
5
):
869
76
. .
33.
Datta
V
,
Mackay
S
,
Mandalia
M
,
Darzi
A
.
The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model
.
J Am Coll Surg
.
2001
;
193
(
5
):
479
85
. .
34.
Buescher
JF
,
Mehdorn
AS
,
Neumann
PA
,
Becker
F
,
Eichelmann
AK
,
Pankratius
U
, et al
Effect of continuous motion parameter feedback on laparoscopic simulation training: a prospective randomized controlled trial on skill acquisition and retention
.
J Surg Educ
.
2018 Mar–Apr
;
75
(
2
):
516
26
. .
35.
Svendsen
MB
,
Preisler
L
,
Hillingsoe
JG
,
Svendsen
LB
,
Konge
L
.
Using motion capture to assess colonoscopy experience level
.
World J Gastrointest Endosc
.
2014
;
6
(
5
):
193
9
. .
36.
Arnold
SH
,
Svendsen
MB
,
Konge
L
,
Svendsen
LB
,
Preisler
L
.
Three-dimensional motion tracking correlates with skill level in upper gastrointestinal endoscopy
.
Endoscopy
.
2015
;
47
(
9
):
825
8
. .
37.
Nerup
N
,
Preisler
L
,
Svendsen
MB
,
Svendsen
LB
,
Konge
L
.
Assessment of colonoscopy by use of magnetic endoscopic imaging: design and validation of an automated tool
.
Gastrointest Endosc
.
2015
;
81
(
3
):
548
54
. .
38.
McGaghie
WC
,
Issenberg
SB
,
Cohen
ER
,
Barsuk
JH
,
Wayne
DB
.
Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations
.
Acad Med
.
2011
;
86
(
11
):
e8
9
. .
39.
Konge
L
,
Svendsen
MB
,
Preisler
L
,
Svendsen
LB
,
Park
YS
.
Combining different methods improves assessment of competence in colonoscopy
.
Scand J Gastroenterol
.
2017
;
52
(
5
):
601
5
. .
40.
Cook
DA
,
Hatala
R
.
Got power? a systematic review of sample size adequacy in health professions education research
.
Adv Health Sci Educ Theory Pract
.
2015
;
20
(
1
):
73
83
. .
41.
Wahidi
MM
,
Silvestri
GA
,
Coakley
RD
,
Ferguson
JS
,
Shepherd
RW
,
Moses
L
, et al
A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows
.
Chest
.
2010
;
137
(
5
):
1040
9
. .
Copyright / Drug Dosage / Disclaimer
Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.
You do not currently have access to this content.