Reliability and Validity of a Smartphone-Paired Pulse Oximeter for Screening of Critical Congenital Heart Defects in Newborns

There is overwhelming evidence that pulse oximetry screening increases early detection of critical congenital heart defects (CCHD) in newborns [1,2,3,4,5]. Consequently, many programs and groups around the world have recommended and adopted this type of screening [4,6,7,8,9,10]. However, barriers to widespread implementation of pulse oximetry screening of CCHD include increasing trends of out-of-hospital births, cost of implementation (equipment and staff costs), and management of test positives (neonatal admissions and echocardiography) [11].

When selecting a pulse oximeter for CCHD screening, it is recommended that the monitor should be approved for use in neonates, be motion tolerant, and be able to read through low perfusion [10,12]. Therefore, hospital-grade pulse oximeters are used in all screening programs [2,5,6,7,8,9,13,14]. Nevertheless, the price of such devices is a limitation for the extension of the CCHD pulse oximetry screening to home births and low-resource settings. In recent years, several manufacturers have developed low-cost pulse oximeters devices that can be paired to a smartphone or a tablet [15,16,17]. The inherent computing power of the mobile phone or tablet and its everyday availability offer the opportunity to create a low-cost stand-alone device which may extend pulse oximetry from hospital into nonhospital settings [16,17]. However, to the best of our knowledge, none of these devices has been tested in the setting of CCHD screening. In this investigator-initiated study, we compared the performance in CCHD screening of a smartphone-paired pulse oximeter (Masimo iSpO₂-Rx™) and a hospital-grade pulse oximeter (Masimo Radical-7™) from the same manufacturer.

The study was approved by the local ethics committee and informed consent of the parents was obtained. All healthy term (>37 weeks) infants born at the Maastricht University Medical Center (MUMC) were potentially eligible for the study.

The equipment and supplies used in the study were acquired at market price by the purchasing department of MUMC. Pulse oximetry screening of CCHD was introduced in MUMC in January 2016 as part of the routine care of healthy newborns. The screening is performed by trained nurses using the Masimo Radical-7 (Masimo Corp., Irvine, CA, USA; price in the Netherlands: EUR 1,800) equipped with disposable Masimo LNCS sensors. Screening is performed 12-24 h after birth and involves preductal (right-hand) and postductal (either foot) measurements. Screenings before 12 h or after 24 h are also allowed. Whenever possible, it is attempted to screen infants awake, quiet, and without pacifier, bottle, or concurrent feeding.

The infants included in the study were subjected to a second screening with a Masimo iSpO₂-Rx pulse oximeter (price in the Netherlands: EUR 300) equipped with a reusable Masimo M-LNCS YI sensor and connected to a tablet (Apple iPad mini). The iSpO₂-Rx is a CE-marked, dual-wave pulse oximeter equipped with signal extraction technology (SET™) that, according to the manufacturer, provides reliable readings even in low-perfusion states and with patient movements [18,19]. The companion software application (app) provides the oxygen saturation, the pulse rate, and the perfusion index [17,19]. The screening with the iSpO₂-Rx took place either before or after the routine screening with the Radical-7 and was performed by one of the investigators (M.J.H., I.A.C., or E.V.). The Radical-7 and the iSpO₂-Rx screening teams proceeded independently and were unaware of each other's results.

A screening result was labeled as “fail” if any oxygen saturation measure was ≤89%. Any screening that was ≥95% in either extremity with ≤3% absolute difference in oxygen saturation between the upper and lower extremity was considered a “pass” result, and screening was ended. When saturation in both extremities was ≥90% but ≤94% or one of the saturations was ≥95 but there was a ≥4% absolute difference in oxygen saturation between the right hand and foot, the screening result was labeled as “repeat” and repeated after 1 h. Two consecutive “repeat” results were considered as a positive screening. For the purpose of the study, only the result of the first measurement was taken into account.

In order to evaluate the performance of the iSpO₂-Rx pulse oximeter in detecting low oxygen saturations, a group of 12 infants admitted to the neonatal intensive care unit (NICU) of the MUMC and showing a stable saturation <95% were also included in the study. These infants were continuously monitored with an IntelliVue MP70 patient monitor (Philips Medical Systems, The Netherlands) equipped with a Masimo SET™ OEM board pulse oximeter. This pulse oximeter was maintained in one foot and the iSpO₂-Rx was connected to the other foot. Oxygen saturation was simultaneously measured in both feet during 10 min and recorded every minute.

Results are expressed as counts and percentages, median and interquartile range (IQR, Q1-Q3), or means and standard deviation (SD). For agreement between the two pulse oximeters, a Bland-Altman analysis was applied calculating bias as the mean difference between both pulse oximeters and limits of agreement (bias ±1.96 SD) as the range in which 95% of the differences between the two measurements are expected to lie [20,21]. Sample size was determined based on Bland's recommendation of using a sample size of 200 subjects to accurately estimate the limits of agreement between 2 devices or methods of measurement [20].

Duplicated screening with the Radical-7 and the iSpO₂-Rx was performed in 201 term infants (51% female) born between July 12, 2016 and February 7, 2017. Screening with the Radical-7 took place at a median time of 21.2 h (IQR 16.4-24.6) and screening with the iSpO₂-Rx took place at a median time of 22.2 h (IQR 18.0-27.0). The median difference between the measurements with the Radical-7 and the iSpO₂-Rx was 1.2 h (IQR 0.3-6.1).

For the preductal oxygen saturation, the mean bias (Radical-7 minus iSpO₂-Rx) was -0.08 (SD 1.76) and the lower and upper limits of agreement (bias ± 1.96 SD) were -3.52 and 3.36, respectively (Fig. 1). For the postductal oxygen saturation, the mean bias (Radical-7 minus iSpO₂-Rx) was -0.11 (SD 1.68) and the lower and upper limits of agreement (bias ± 1.96 SD) were -3.41 and 3.18, respectively (Fig. 2).

In 196 of the 201 infants (97.6%), the screening was labeled as “pass” with both devices after the first measurement. In 5 cases the results of the first measurement were discrepant between the two pulse oximeters. Two cases were labeled as “repeat” with the Radical-7 (preductal-postductal saturations: 94-94 and 92-94) but as “pass” with the iSpO₂-Rx (preductal-postductal saturations: 94-96 and 96-99, respectively). Two cases were labeled as “pass” with the Radical-7 (preductal-postductal saturations: 98-97 and 98-99), but as “repeat” with the iSpO₂-Rx (preductal-postductal saturations: 97-93 and 100-94, respectively). Finally, in one case the infant failed the screening with the Radical-7 (preductal-postductal saturation: 91-88) but the result with the iSpO₂-Rx was labeled as “repeat” (preductal-postductal saturation: 92-90). This infant showed a mild respiratory distress and was immediately admitted to the neonatal unit where saturations around 90% were confirmed. No heart murmur or other abnormalities were detected by physical examination. Treatment with antibiotics was started under the diagnosis of suspected sepsis. Saturations and the clinical condition of the infant improved in the following hours and echocardiography was not considered necessary.

As mentioned in the methods section, in 12 NICU patients (8 with bronchopulmonary dysplasia, 2 with pulmonary hypertension, 1 with hypoplastic left ventricle and 1 with tetralogy of Fallot) under continuous monitoring with a Masimo SET pulse oximeter and showing postductal (foot) saturations <95%, we performed a simultaneous measurement of the saturation with the iSpO₂-Rx in the contralateral foot. Ten measurements per infant were recorded showing a mean bias (Masimo SET minus iSpO₂-Rx) of 0.01 (SD 1.74) with lower and upper limits of agreement (bias ± 1.96 SD) of -3.42 and 3.43, respectively (Fig. 3). Visual inspection of the Bland-Altman scatterplots (Fig. 1, 2, 3) revealed no distinct patterns or funneling that would indicate important heteroscedasticity (i.e., larger variability for higher saturation values) [22].

We examined the performance of the Masimo iSpO₂-Rx, a smartphone-paired pulse oximeter, in the setting of CCHD screening in 201 newborn infants. The iSpO₂-Rx demonstrated a high degree of agreement with the Masimo Radical-7, a hospital-grade pulse oximeter. Moreover, the iSpO₂-Rx provided reliable measurements of saturations below 95% in a group of NICU patients. We did not observe any systematic bias and the differences between the two pulse oximeters can be attributed to the inherent bias that all saturation monitors have when they function according to their specifications [18]. Unfortunately, we can only speculate on this point because we did not perform repeated measurements with any of the two devices in order to estimate the inherent bias in our setting. Nevertheless, it should be noted that current industry data report neonatal-based pulse oximetry algorithms to have a precision (SD of the bias) that can be as low as 2% on some monitors but as high as 5% on others [18]. Interestingly, the SD of the bias that we observed between the Radical-7 and the iSpO₂-Rx was less than 2%. Therefore, altogether, our data suggest that CCHD screening with the iSpO₂-Rx is feasible and accurate.

In the Netherlands, pulse oximetry screening of CCHD has not been included in the universal screening program, but an implementation performing trial aiming to screen 20,000 infants in the Amsterdam-Haarlem-Leiden region is currently underway [13]. As analyzed by Narayen et al. [13,14], the perinatal health care system in the Netherlands is unique due to its high incidence of home births and early discharge after uncomplicated deliveries. In total, 33% of all low-risk deliveries are supervised by a community midwife, of which 55% occur at home and 45% at a birthing facility or policlinic [13,14]. Therefore, it has been estimated that implementation of a universal pulse oximetry screening of CCHD in the Netherlands would require the provision of pulse oximeters and adequate training to all 1,800 community midwives [13]. A reliable smartphone-paired pulse oximeter would offer a reduction of the cost and, simultaneously, an improvement in the quality of the screening. The latter can be achieved through the development of smartphone apps that guide the user in performing and interpreting the pulse oximetry screening.

Earlier studies have shown the feasibility of home-birth screening of CCHD with hospital-grade portable pulse oximeters [13,14,23]. Besides the Netherlands, rates of planned out-of-hospital birth (i.e., births intended to occur at home or at a freestanding birth center) have increased in high- and middle-income countries [24,25,26]. In addition, there is an increasing tendency toward early discharge after uncomplicated deliveries in hospital [13]. This may increase the necessity of performing domiciliary CCHD screening in high- and middle-income countries and good quality smartphone pulse oximeters could be applied for these settings.

CCHD screening has less priority in low-income countries because treatment options are limited [27]. However, since CCHD screening is based on the detection of hypoxemia, infants with a positive screen may have another potentially serious and treatable condition, such as neonatal sepsis [1,2,3,4,5]. Simple procedures for identifying infants with infection that need referral for treatment are of major public health importance [28]. Providing a more extensive access to pulse oximetry in the first hours of life may reduce neonatal mortality and morbidity by early detection of hypoxemia in conditions such as sepsis, pneumonia, or other infections [2,27].

As claimed by several CCHD-screening advocates, the question is not whether we should implement an inexpensive, quick, painless screen with the potential to save lives, but how it should be implemented [4,29]. Smartphones are powerful tools that offer both computational and communication opportunities which can be leveraged for the benefit of health care [17,30]. Our study suggests that one of these benefits could be the extension of the screening of CCHD, as well as other forms of clinically undetectable hypoxemia, to a larger neonatal population.

The authors declare no conflicts of interest.