Comparisons of Handheld Retinal Imaging with Optical Coherence Tomography for the Identification of Macular Pathology in Patients with Diabetes

Diabetic retinopathy (DR) remains one of the leading causes of blindness among working-age adults worldwide and may remain asymptomatic until late in the disease process [1, 2]. Adherence to recommended screening and follow-ups remain limited especially in low- to middle-income countries and marginalized communities, leading to productive years lost from significant visual impairment [3]. Efforts to extend the reach of DR screening programs to improve early identification of vision-threatening disease include handheld retinal imaging cameras and population-based retinal screening programs [4‒6].

Handheld retinal imaging cameras are relatively inexpensive and highly portable devices that have the potential to significantly expand teleophthalmology initiatives, allowing a much broader population to be evaluated. These compact cameras can be more widely deployed in underserved areas and can be utilized by primary care physicians, allied health personnel, and nonmedical staff with proper training [7]. However, it is essential to evaluate if these devices can accurately identify vision-threatening macular diseases if DR screening programs will rely on these instruments. Macular diseases such as diabetic macular edema (DME) primarily affecting central vision are the leading cause of preventable vision loss in the diabetes population [8]. Upon detection of DME, evaluation by an ophthalmologist with appropriate and timely initiation of therapy is critical for the long-term preservation of vision. Delays in treating visually threatening disease may result in suboptimal outcomes and long-term morbidity. Therefore, it is necessary to evaluate handheld devices in terms of sensitivity and specificity to detect macular pathology before they can be implemented in large-scale screening programs.

Spectral domain optical coherence tomography (SDOCT) surveillance clinics have recently been established in the UK [9, 10]. These serve as intermediaries to refine referrals between screening and specialist centers for patients with suspected DME, as false-positive rates remain high. High false positives lead to excessive nonmedically indicated referrals, translating to an unnecessary healthcare burden on society. Moreover, commercial interest in developing low-cost handheld SDOCT, and combined handheld retinal imagers with SDOCT capability, are already present [11]. DR screening working groups need to establish if current clinical data support the development and use of these new technologies, or the formation of additional SDOCT surveillance clinics, to check if these healthcare models prove more cost-effective than relying on fundus photos alone. While SDOCT is expected to be more sensitive and specific in detecting macular pathology, majority of screening programs still do not have OCT capability especially in developing countries. Most still rely on fundus photographs or clinical exam, but no prior study measures these metrics for handheld retinal imagers to check if they can perform at a satisfactory level for community screening. In this study, three commercially available handheld retinal imaging devices were compared with gold standard SDOCT imaging for the diagnosis of macular pathology, to evaluate their suitability for use in large-scale DR screening programs.

A single-site, prospective, clinic-based, comparative instrument validation study evaluated detection of macular pathology among mydriatic 40°–60° macula-centered images taken with 3 handheld retinal imaging devices (shown in Fig. 1). The devices included the Optomed Aurora IQ (Optomed, Oulu, Finland), Optomed SmartScope PRO (Optomed, Oulu, Finland), and RetinaVue 700 Imager (Hillrom, Chicago, IL, USA), which were compared with the Cirrus 6000 SDOCT (Carl Zeiss Meditec Inc., Dublin, CA) macular scans. Standard 7-field Early Treatment Diabetic Retinopathy Study (ETDRS) fundus photos were also taken. Images were taken using a standardized protocol after pupillary dilation during the same visit. The sequence of imaging was based on device availability and was random. The study design was consistent with the tenets of the Declaration of Helsinki and approved by the Institutional Review Board of The Medical City, Philippines. Written informed consent was obtained from all participants. Written informed consent was obtained from all patients before study participation, and the conduct of the study complied with the Health Insurance Portability and Accountability Act of the USA and the Data Privacy Act of the Philippines.

Patient eligibility was determined from a medical record review of the most recently diagnosed clinical level of DR severity. Study participants were selected to ensure the distribution of various severity levels of DR, ranging from no DR to high-risk proliferative DR (PDR). Patients were eligible for the study if they met all the following inclusion criteria: age ≥18 years, diagnosis of type 1 or 2 diabetes mellitus as defined by the American Diabetes Association, willingness to sit through photography and imaging sessions, and willingness to sign the institutionally approved informed consent specifically designed for this study. Patients were excluded from the study if they had no history of diabetes, had a history of a condition in either eye that might prevent pupil dilation, were using eye drops (mydriatic or miotic) that would alter pupil size or reactivity, had a history of panretinal laser photocoagulation, or had media opacities precluding adequate imaging of the retina.

Images were graded for macular pathology by 2 certified retina specialists (C.M.J. and A.R.) viewed on a standard Joslin Vision Network reading station optimized for viewing clarity. Disagreements on grading were resolved by adjudication through consensus with a senior retina specialist (P.S.S.). The grading of DR severity was performed at a centralized retinal center based on the International Classification of Diabetic Retinopathy (ICDR) [12]. The agreement of DR severity assessment between handheld retinal imaging and stereoscopic 7-field ETDRS standard photography has been published elsewhere [13‒16]. The assessment of the presence and severity of DME on fundus photography was based on ETDRS standards for the detection of DME. Noncentral involved DME (ciDME) was defined as retinal thickening not involving the central 100 μm subfield zone at the fovea, and ciDME was defined as retinal thickening involving the central 100 μm subfield zone [17]. Only monocular fundus photographs were acquired with the handheld devices, and grading used surrogate markers of thickening to evaluate the presence of DME as outlined in the National Health and Nutrition Examination Survey (NHANES) Digital Grading Protocol [18‒21]. These surrogate markers include focal color changes at the macula, slight loss of the normal transparency of the retina, evidence of prior focal/grid laser, rings of organized hard exudate, a deviation of the normal pathway of the retinal blood vessels, or lipids at the foveal center. Epiretinal membranes (ERMs) were noted when a crinkled cellophane appearance at the macula was observed, which may be correlated with vascular distortion or dragging of the macula in more advanced cases. Other pathologies noted were retinal detachment and localized pigment epithelial detachments.

SDOCT macular cube scans centered on the fovea were evaluated for DME-based central subfield thickness (CST) values greater than or equal to sex-matched eligibility criteria from the Diabetic Retinopathy Clinical Research (DRCR) network DME trials [22‒24], and for the presence of intraretinal and/or subretinal hyporeflective spaces. The presence of ERM was defined as a continuous hyperreflectivity in the inner surface of the retina, which may be associated with inner retinal striae, distortions of the foveal contour, lamellar holes, and intraretinal fluid [25]. The full spectrum of mild to severe ERM was noted, and at least 3 consecutive sections of the macular cube should demonstrate the continuous hyperreflective signal in the inner retinal surface, to distinguish it from normal posterior hyaloid reflectivity [25].

Agreement between the assessments of macular images from different handheld devices compared to SDOCT was assessed by calculating sensitivity, specificity, positive predictive value, and negative predictive value. All statistical analyses were performed using SAS version 9.4 (AS Inc., Cary, NC).

A total of 225 eyes from 116 patients were enrolled. The demographics and distribution of DR severity by ETDRS photography and DME by SDOCT are shown in Table 1. The severity by ETDRS 7-field stereoscopic photography grading was no DR 75 (33.3%), mild non-PDR 46 (20.4%), moderate 32 (14.2%), severe 26 (11.6%), proliferative 46 (20.4%), and ungradable for DR 0%; no DME 187 (83.1%), non-ciDME 11 (4.9%), ciDME 27 (12%), and ungradable for DME 0%. Gradable images by SDOCT (N = 217, 96.4%) showed no DME in 170 (75.6%), non-ciDME in 22 (9.8%), and ciDME in 25 (11.1%). ERMs were the second most common pathology, present in 8% of eyes. The ungradable rate for images (poor visualization in >50% of the macula) was Aurora: 0.9%, SmartScope: 4.4%, and RetinaVue 700: 6.2%. The sensitivity and specificity of the different handheld imaging devices compared to SDOCT for macular pathology are presented in Table 2. For DME, sensitivity and specificity were similar across devices (0.5–0.64 and 0.93–0.97, respectively). For nondiabetic macular pathology (ERM, pigment epithelial detachment, traction retinal detachment) across all devices, sensitivity was low to moderate (0.2–0.5), but specificity was high (0.93–1.00). Figure 1 shows a comparative montage of retinal images taken with the 3 handheld devices in a patient with PDR and ciDME. Figure 2 is a representative montage of false-negative and false-positive diagnoses for DME using the SmartScope. Figure 3 is a representative montage of false-positive and false-negative diagnoses for ERM using the Aurora and RetinaVue 700.

For subgroup analysis, among eyes with 20/40 vision or worse (N = 60), sensitivity and specificity (0.63–0.68 and 0.90–0.94, respectively) were comparable to the main cohort of eyes (Table 3). In eyes with CST ≥400 μm (N = 12), all eyes identified to have DME on handheld devices were true positives with a resulting specificity of 1.0. In eyes that reached DRCR network thresholds for referable CST values (305 μm for males and 320 μm for females, N = 28), there was an increase in the sensitivity to detect any DME (0.73–0.80) compared to the main cohort (0.55–0.64). The specificity in this subgroup remained moderate however, from 0.71 to 0.86.

Monocular fundus photography grading using NHANES definitions for DME suggests that among the three tested handheld cameras, there was low sensitivity for detecting DME (0.5–0.64) but high specificity (0.93–0.97) when compared with OCT. The performance of the handheld imaging as compared to OCT imaging in eyes with 20/40 or worse vision is comparable to the entire cohort. The performance of handheld imaging appears to improve in eyes with increasing retinal thickness. The sensitivity for eyes reaching DRCR network OCT CST thresholds was higher than with the main cohort (0.73–0.80 vs. 0.55–0.64). The sensitivity for non-DME pathology is substantially lower, with sensitivity to detect ERMs, traction retinal detachment, and pigment epithelial detachments between 0.2 and 0.5 across devices. For medical devices used to detect sight-threatening DR, the English National Health Service recommends a minimum sensitivity of 80% and a minimum specificity of 95% [26]. While these standards were developed for DR severity and not for macular pathology, there is a considerable disparity between these recommendations and handheld imaging results. While it is expected that SDOCT will perform better than fundus photos, this study shows the magnitude of discrepancy between the two devices. The results emphasize the need to reevaluate current standards of care, specifically the sole reliance on fundus photography for most DR screening programs. While handheld cameras have the potential to significantly expand ophthalmic care, sensitivity for sight-threatening macular pathology needs to be adequate to allow accurate triage to specialized levels of care. This is especially relevant as they are being considered for use in high-volume, low-resource settings where subsequent patient reevaluations may be few and far between. Given the lower sensitivity of handheld devices for macular pathology, in the absence of SDOCT a lower referral threshold is necessary whenever there is evidence of subtle macular findings. Where possible, integrating SDOCT into DR screening programs is recommended to optimize the triage of patients to appropriate levels of care.

At present, it is not cost-effective to include SDOCT for each patient especially in developing countries due to the prohibitive prices of these machines, along with the extended visit duration and staffing demands of this protocol. Unfortunately, alternative methods to screen for DME using clinical exam are time-consuming, resource-intensive, and lack sensitivity versus SDOCT [27]. Recognizing the lack of sensitivity with current screening options, once suspected patients with DME on fundus photography are identified, using SDOCT for refining the referral for specialist care is appropriate. Patients with positive findings should be scheduled for close follow-up, while patients without disease on SDOCT can be maintained on routine screening. Similar to practices in the UK, the data support the establishment of SDOCT clinics which serve as intermediaries between screening centers and specialist care, to more effectively triage patients who need further evaluation and treatment.

Another alternative strategy is the development of low-cost handheld SDOCT, with the capability to incorporate fundus cameras in the future [11]. The development of this novel technology is driven by the increased availability of miniaturized optics, along with the recognition of the clinical need. This approach may be more appropriate in low-resource settings where establishing intermediary SDOCT clinics is not economically feasible. In a pilot study by Song et al. [11] the images captured by the low-cost OCT were adequate for clinical diagnostics, with a unit cost of $5,037, a significant discount from commercially available tabletop units.

This study reports high specificity among all types of macular pathology for all devices, ranging from 0.93 to 1.00. These are unlike data from other groups, which have reported deficiencies in diagnostic specificity leading to excessive nonmedically indicated referrals. In a study by Wong et al. [9] among 352 patients recruited in a large-scale Hong Kong DR screening program, the specificity to detect DME from monocular fundus photos compared to SDOCT criteria was 0.13, leading to a false-positive rate of 0.87. Differences in methodology for their study included a certified optometrist grading DME from fundus photos, blinding OCT graders to CST values, and desktop-based fundus photography machines. Economic analysis of these excessive referrals to ophthalmology showed that for every 1,000 patient referrals for DME, only 13.4% (134) or less might require treatment, with an associated cost of each consult at HK$ 650 (excess cost of HK$ 562,900/1,000 patients). Apart from the financial burden on the health system, this also leads to healthcare workforce overburdening, with decreased schedule availability for patients that need treatment. The specificity for DME detection in our study is substantially higher than in previous studies. It may be due to certified retinal imager graders experienced in handheld retinal image capture performing the imaging and retina specialists with formal certification in DR/DME grading for macular image evaluation.

A comparison of OCT with traditional tabletop cameras to detect DME was performed and previously published [28]. In the study by Bressler et al. [22], they used color fundus photographs taken with a 30° tabletop camera (Zeiss FF4; Carl Zeiss Meditec) or a 60° camera (Canon CF-60 DSi; Canon, Inc.) and compared against OCT. In this cohort, 58.2% and 18.0% of eyes without DME on OCT were diagnosed with DME on monocular fundus photographs using MESA and NHANES definitions, respectively, and 47.0% and 10.3% with CSME, respectively. Among eyes with DME on OCT, 26.9% and 32.7% were not diagnosed with either DME or CSME on monocular fundus photographs using MESA and NHANES definitions, respectively. In addition, this study was part of a series of evaluations of handheld retinal imaging [29, 30]. Comparing 4 models of handheld imaging devices (iNview [Volk Optical Inc.], RetinaVue 700 [Welch Allyn], SmartScope [Optomed Ltd.], and Aurora [Optomed Ltd.]) against gold standard ETDRS 7-field photography (Visucam; Carl Zeiss Meditec, Inc.) to detect DME, the weighted kappa was 0.76–0.83 and 0.77–0.91 with a nonmydriatic and mydriatic protocol, respectively.

The second most identified pathology was ERM, found in 8% of eyes via SDOCT, and sensitivity for their detection was low at 0.2–0.47 across all devices. The full spectrum from mild to severe ERM was included in this count. ERMs have been associated with diabetes mellitus and DR from previous studies, making it important to screen for this pathology [31‒34]. Detection and monitoring of ERMs are important in preserving visual function, as progression leads to chronic foveal distortion and intraretinal fluid, which clinically manifests as metamorphopsia and blurred vision. In a study by Delyfer et al. [25] of 624 patients recruited from a multisite population-based study, sensitivity to detect ERMs of different severity stages via fundus photos was 0.1–0.45, while specificity was 0.93–0.94 compared to SDOCT. This is similar to our data showing the inadequacy of handheld cameras as a screening tool for ERM.

Despite these limitations, handheld cameras still offer clear advantages as a tool for DR screening programs. A limitation of large desktop fundus cameras is their cumbersome and stationary nature, which may be challenging to use with hospitalized or uncooperative patients. Portable cameras are more easily positioned and can be brought to the bedside, increasing their clinical utility and providing imaging to patients with mobility concerns. The cost of acquiring a handheld retinal imaging device is significantly lower, compared to traditional desktop imaging devices easily costing from $20,000 to $50,000 [35]. Data from low-to-middle income countries in 2014 show that only 10% of secondary level hospitals could take fundus images, partly due to the prohibitive purchasing and maintenance costs of these devices preventing widespread distribution [36]. More widespread distribution of fundus photography capabilities to primary care physician clinics or other allied health facilities may accompany teleophthalmology initiatives, as images can be distributed in a store and forward manner to be read by certified fundus photography graders. Nonophthalmologists, allied health personnel, and nonclinical staff may undergo training to properly use these portable devices, increasing the capability to acquire photos while maintaining image quality [7, 37‒39]. Having portable devices widely available in combination with proper technical training can greatly accelerate the reach and scope of screening initiatives, as retinopathy evaluation can be integrated into the same primary care visit, removing the additional step of scheduling a dedicated DR screening consult.

The strengths of this study include the standardized evaluation of all retinal images by Joslin Vision Network certified retina graders, and the use of standardized data collection forms designed specifically to evaluate DR outcomes. Limitations of this study include handheld fundus photography acquired by certified retinal imager-graders which led to relatively low ungradable rates and good image quality. Less experienced photographers may not achieve similar image quality results, which is important to account for when using these cameras in large-scale screening programs. Their deployment in primary care, rural, and underserved areas must be accompanied by formal training programs so that even nonclinical personnel can use them reliably in the community.

In conclusion, compared to SDOCT, handheld macular imaging attains high specificity but low sensitivity in identifying macular pathology. Without stereopsis, 36–50% of eyes without DME on photos have DME on SDOCT, and 3–7% of eyes with DME on handheld imaging will have no DME on SDOCT. Additionally, 53–80% of eyes with macular ERM are missed without SDOCT imaging. This suggests the use of lower thresholds for referral of macular disease when handheld devices are used and the importance of SDOCT evaluation for patients suspected to have macular pathology leading to a more refined referral approach. Future research looking at cost-effectiveness models for DR screening using handheld imaging and SDOCT in low-resource settings can further clarify the economic impact expected with the widespread use of these devices.

The authors thank Lloyd Paul Aiello, MD PhD and Jennifer K. Sun, MD MPH (Beetham Eye Institute, Joslin Diabetes Center, Department of Ophthalmology, Harvard Medical School) for their valuable contributions to the review, discussion, and editing of the scientific content of the paper.

This study protocol was reviewed and approved by the Institutional Review Board of the Medical City, Metro Manila, Philippines with approval number GCS OVS 2019-033. Written informed consent was obtained from all participants.

C.M.P.J., R.P.S., A.K.R., L.A.C.A., G.P.A., and A.V.S.: no financial relationship to disclose. T. P.: personal fees – Novartis, Bayer, Roche, Heidelberg, and Optos, outside of the submitted work; and financial support – Optomed. P.S.S.: financial support – Optomed, Hillrom, and Optos, outside of the submitted work.

Research is funded jointly by the UK Medical Research Council (58 Victoria Embankment, London EC4Y 0DS, UK) and the Philippine Council for Health Research and Development (Saliksik Building, DOST Compound, Gen. Santos Ave, Bicutan, Taguig City 1631, Philippines) through the Newton-Agham Grant. Grant Title: The UK – Philippines Remote Retinal Evaluation Collaboration in Health: Diabetic Retinopathy (REACH-DR) Project (Project Reference: MR/R025630/1). The funding agencies have no role in the design or conduct of this research. Nonfinancial research support was received from Optomed Plc (Oulu, Finland) and Hillrom Inc. (Chicago, IL, USA) for the temporary loan of handheld devices to the Philippine Eye Research Institute.

Dr. Paolo S. Silva had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Tunde Peto and Paolo S. Silva. Acquisition, analysis, or interpretation of data: Cris Martin P. Jacoba, Recivall P. Salongcay, Abdulrahman K. Rageh, Lizzie Anne C. Aquino, Glenn P. Alog, Aileen V. Saunar, Tunde Peto, and Paolo S. Silva. Drafting of the manuscript: Cris Martin P. Jacoba, Recivall P. Salongcay, and Paolo S. Silva. Critical revision of the manuscript for important intellectual content: Cris Martin P. Jacoba, Recivall P. Salongcay, Abdulrahman K. Rageh, Lizzie Anne C. Aquino, Glenn P. Alog, Aileen V. Saunar, Tunde Peto, and Paolo S. Silva. Statistical analysis: Cris Martin P. Jacoba and Paolo S. Silva. Obtained funding: Tunde Peto and Paolo S. Silva. Administrative, technical, or material support: Cris Martin P. Jacoba, Recivall P. Salongcay, Lizzie Anne C. Aquino, Tunde Peto, and Paolo S. Silva. Study supervision: Recivall P. Salongcay, Tunde Peto, and Paolo S. Silva.

Abdulrahman K. Rageh was affiliated with Joslin Diabetes Center during the conduct of the study and is now at the Duke Eye Center.

The data that support the findings of this study are not publicly available due to containing information that could compromise the privacy of research participants, but are available from the corresponding author (P.S.S.) upon reasonable request.