Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies

Digital health technologies have been defined as “a system that uses computing platforms, connectivity, software, and sensors for healthcare and related uses [1].” Using technologies to collect data created and recorded by participants, known as “person-generated health data,” has gained considerable interest and traction in clinical trials. These tools can collect patient data, enabling objective and frequent monitoring of physiology, behavior, and function compared to snapshot in-clinic assessments. In addition, the growth in digital health technology use in the general population has enabled the bring your own “device” (BYOD¹) model. While this approach previously focused on using individuals’ smartphones to capture electronic patient-reported outcome data, there is a growing interest in expanding the BYOD model to include personal digital health technologies.

The digital health technologies with the greatest potential to be amenable to BYOD studies are fitness trackers and smartwatches [2]. With intuitive and easy-to-use interfaces, embedded multi-modal sensors can derive various physiological measures, including physical activity, sleep, and vital sign data (e.g., heart rate, heart rate variability, pulse oximetry) [3-6]. Smartphones have increasing utility as digital health technologies with inbuilt sensors and technology such as accelerometers, global positioning system sensors, microphones, cameras, gyroscopes, and magnetometers. These sensors are used as a source of data for passive monitoring and to deliver functional assessments to study participants via mobile applications [7, 8].

The benefits of BYOD studies are multifarious; the approach allows participants to use their own technologies, leading to better compliance [9, 10] and, potentially, less chance of introducing biases, such as the Hawthorne effect, from monitoring technologies [11]. For participants, there is a familiarity with and access to the technology they use in their daily lives [12, 13]. Sponsors can design patient-centric studies with lower costs and burden on study sites [12-14]. The BYOD model potentially expands participation in clinical trials for populations with limited access to clinical facilities, e.g., older adults, people with disabilities, or living in remote locations [15]. Conversely, limiting eligibility to those with good health and digital literacy, internet connection, and the latest technologies may bias results with data not representative of the target population [15]. BYOD models may not be appropriate in all circumstances; the technologies required to generate study endpoints may not be readily available to the study populations. A model requiring provisioned devices for subsets of participants who do not possess the required technology may be optimal [15, 16]. A comparison between BYOD and provisioned device options is presented in Table 1.

The BYOD approach is feasible in non-interventional and interventional studies using observational, randomized controlled trials, pragmatic (practical) clinical trial designs, and real-world evidence studies (see Appendix 1, available online at www.karger.com/doi/10.1159/000525080, for study definitions, best practices, and checklists). Many observational and postmarketing research questions can be addressed with BYOD to collect data with minimal disruptions to daily life. For example, BYOD models have been successfully deployed in surveillance studies where sensor data from fitness trackers and smartwatches generate data on Flu and COVID-19 infections from 1.3 million [17], 200,000 [18], and 30,000 [19] individuals. These studies follow a typical BYOD approach where participants download a study-specific application to their smartphones to capture patient-reported outcomes, diagnostic test results, and data from connected technologies [13]. BYOD studies incorporating digital health technologies are beginning to emerge in interventional clinical trials [20-22]. Figure 1 showcases examples of different BYOD configurations utilized in clinical studies.

This paper provides considerations for designing and deploying a BYOD model to capture data for clinical studies. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; and (5) statistical considerations to meet regulatory requirements (Fig. 2). This paper is framed using the Agency for Health Research and Quality guidelines for real-world evidence study design [23]. These guidelines have been broadly adopted and selected as an overarching guide to develop our approach to support sponsors and researchers in designing BYOD studies across diverse patient populations and therapeutic areas. Such considerations are intended for a broad audience, including academic research, drug developers, and patient advocacy organizations.

Understanding the needs, concerns, and impact of the BYOD approach on stakeholders can improve the quality and efficiency of the study. Therefore, the proposed steps in the overarching approach are as follows; (1) stakeholder mapping to identify both internal and external stakeholders, (2) stakeholder engagement, a bidirectional interaction to gain understanding, and (3) stakeholder management to facilitate the smooth operationalization of BYOD in the clinical trial.

A cross-functional team approach is required, including, but not limited to, representatives from the following groups: data management, medical affairs, biostatistics, data science and data engineering, clinical operations, regulatory affairs and safety (Table 2). Consultation is essential to understand the potential impact of BYOD on the different work-streams [24].

Early engagement with participants, caregivers, vendors, site personnel, and other external stakeholders (Table 3) will maximize value and minimize the burden during study development. These are novel study designs; therefore, clear communication, education, support, and training are essential for a successful study to ensure that the sites and investigators are appropriately trained in the study objectives and the considered technology. Participant materials need to be informative, addressing any participants’ concerns with respect to data privacy, mobile data costs, and the consenting process. Gauging the digital literacy of potential study participants during these initial activities will allow for planning and developing appropriate training supports (e.g., videos, a support line) to mitigate differences among participants in how they engage with the technology. Developing a partnership approach with the technology vendor is important so technical issues and mitigation strategies can be jointly developed.

The first set of questions are: Are digital health technologies appropriate to address the study objectives and research questions? Does a BYOD study design minimally provide equivalent data to traditional approaches, and add value, providing insights not derived via standard approaches?

Study teams creating the evidence dossier to support specific digital health technology and digital endpoints should consider the data context: data generated by participants’ technology contrasts with a provisioned approach where the team has complete control of the technology. Study applications should capture information about the data source, i.e., technology specifications (e.g., model, version, manufacturing date), to inform the data analysis and interpretation. Study teams must define the minimum technological requirements necessary to generate the required digital endpoints, such as the operating system, the model, firmware, and data storage requirements. Once minimum requirements are established, study teams may further restrict the technology type for a more targeted study or proceed with more flexibility and specify analysis sets [16].

In any protocol, the eligibility criteria define the population of interest [23]. When establishing eligibility criteria for BYOD studies, the potential limitation of BYOD specifications needs to be considered [19].

Eligibility criteria considerations affect the generalizability of the results, may bias the collected sample, and impact inclusion and diversity in clinical studies. Ownership of a specific technology or internet connection, as part of the eligibility criteria, may bias the study cohort towards individuals of higher socioeconomic strata and deter the participation of those with limited access to technology and the internet [25]. Participants may not feel comfortable with the sponsor accessing study data from their technology where their personal information is stored [26]. Technology should never limit study participation. Mitigation strategies that provide these participants with stand-alone technology or internet service promote broader inclusivity [15]. The Critical Path Institute’s electronic patient-reported outcome Consortium recommends provisioning devices for participants who do not own the technology required for the study [27]. This helps address gaps due to low socioeconomic status.

BYOD studies could potentially limit participation to those who own digital health technologies; therefore, ensuring that the study sample represents the population of interest is essential. A recruitment plan, even for BYOD studies heavily reliant on technology, should incorporate a variety of methods such as broadcast and clinic advertisements and targeted outreach. Recruiting solely through social media can exclude participants with no or limited access to the internet, such as those living in rural areas and with low socioeconomic status [28, 29]. Participants may own the required technology to enroll in the study (e.g., a smartphone) and may be willing to participate but lack other resources such as reliable internet connection and research awareness. Such practices nurture the digital divide and limit the ability to understand the needs of historically underrepresented communities.

The consent process in BYOD studies includes training that may require face-to-face interaction to explain the study’s aim, scope, and risks [30]. In studies deemed as “low risk” (e.g., observational, non-interventional studies collecting nonidentifiable information), online consent could be used [31], depending on local regulations. Procedures must ensure participant and technology user verification (e.g., identity and meeting eligibility criteria), mitigating data privacy and security risks [32]. The process for withdrawing consent and disengaging their technology from the study needs to be straightforward. Study software installation also needs to be simple, responsive, and intuitive. Technologies must comply with local regulations, e.g., the Health Insurance Portability and Accountability Act or the General Data Protection Regulation. Local institutional review boards should be consulted and study protocols approved as necessary [30].

Communications need to be tailored for health and technical literacy. Consent materials should be explicit and appropriate so participants can understand their responsibilities, the study data collected by their technology, the method and duration of data storage, the data usage, privacy and sharing, and how it aligns with the research goals. Participants need to be aware of the importance of not changing their technology during the study without alerting the study team and have information related to technical support should their technology malfunction or have issues with the study application.

Despite the convenience and potential cost savings of a BYOD model, the same approaches used for outcome selection in traditional clinical trials should be applied [23]. Sponsors should select measures that are meaningful to the population being studied [33-35]. The United States Food and Drug Administration (FDA) recognizes that the clinical meaningfulness of digital outcomes can impact regulatory decision-making [36]. It is essential to consider the multiple stakeholders (as in Tables 2, 3), study scope and objectives, and the intended application of the outcomes [23].

Endpoints need to be reliable and accurate, measure the treatment effects, validated against an appropriate reference standard, and assess the population under consideration [37-40]. A fundamental challenge of BYOD models is ensuring the equivalence of digital endpoints captured or derived from different technologies. As discussed below, study teams should rely on published literature or conduct methodology studies in the population of interest to establish differences in endpoints derived from data collected by the different technologies under consideration. Estimating measurement errors and data reliability should also be addressed in the analysis (Table 4).

Teams should consider the proportion of study participants who may have access to the proposed technology and develop plans to provision technologies to those who have not. The selected technology needs to be “fit for purpose” [41]. Digital health technologies can be used to collect digital measures directly, such as heart rate, [42] and to derive novel endpoints such as resting tremor [43] or scratch and sleep quantification [44]. The selection process is impacted by the sampled data quality, such as the signal-to-noise ratio that impacts the downstream derivation of the endpoints [45], the battery life, which can affect compliance, and the ability to deploy the study app.

Digital health technology selection must consider the analytical and clinical validation of the endpoints, using frameworks such as the V3 framework, encompassing device verification, analytical and clinical validation [38], security practices, data rights and governance, utility, usability, and their economic feasibility [46]. The EVIDENCE (EValuatIng connecteD sENsor teChnologiEs) checklist can be used to support technology performance evaluation [47].

Pretrial feasibility studies to evaluate suitability, establish measurement errors, and test equivalence of different technologies under consideration may be needed [38]. Feasibility studies can be expensive and time-consuming. Secondary data sources such as vendor quality documents and peer-reviewed literature validating the technologies and establishing data accuracy can be an alternative solution [48]. Published studies and publicly available datasets, preferably combining data from multiple digital health technologies, can be leveraged to compare technologies [46]. Identifying systematic measurement errors and data limitations is crucial for interpreting BYOD study data (e.g., different wristbands can impact the sensor-skin interface and, subsequently, the derived endpoints) [49]. It also informs endpoint-selection and estimates the expected data variability, which helps estimate sample size and develop the analysis plan.

Deliberation is required regarding the data capture capabilities of the technology, including the (a) type of data output (e.g., will the device provide raw data, epoch level data, summary data, or endpoint level data?); (b) the data ingestion and transfer (e.g., is this enabled via WiFi or Bluetooth, does it need a dedicated mobile hub?); (c) data storage capacity (e.g., data generation based on the chosen sampling rate, data storage before data transfer initiation, and memory characteristics); (d) data security: appropriate and up-to-date cyber security processes and procedures [46]; (e) system validation – this should include computer system validation according to international best practice [50].

BYOD models have the advantage that participants carry their technology for personal use and regularly interact with it – this can be leveraged to engage participants and provide information and motivation alongside data capture. Different strategies can be deployed to maintain compliance, e.g., alerts via the study application, automated reminders, or direct text messages and phone calls from the study team. Compliance reminders can be incorporated as part of the study intervention to manage adherence to a medication or therapy as successfully deployed across several therapeutic areas, including cardiovascular [51] and respiratory disease [52]. Ongoing data monitoring strategies based on study-specific compliance algorithms can be used to alert study teams to noncompliant participants and help optimize interaction with participants and technical support. As in any study, compliance thresholds, e.g., a “valid” day consists of 10 h of wear time every 24 h [53], must be outlined a priori in the protocol and the statistical analysis plan [54].

Technology and user-related issues may affect data quality in a BYOD study. Connectivity can affect data quality and cause errors in the capture and synchronization of the data [15]. Participants may lose or change their DHTs, or the software may be upgraded during the trial, adding to potential variations in data quality. Incorrect usage (e.g., charging, updating, or non-wear/use) adds to the challenges of adequate data capture and data loss [15]. It is therefore essential to implement an automated and centralized data monitoring system [50].

Data Transmission. In contrast to studies with provisioned technology where data capture falls under the auspices of the study teams who provide sim cards, data plans, and mobile hubs where appropriate, BYOD studies rely on the participants’ own connectivity. Internet access required for data transfer may be problematic: participants may have limited data allowance, restricted by their data plan. International travel during the study could impose roaming charges or disabled data plans. Consideration should be given to contingency plans such as data uploads configured to use WiFi as the primary preference to reduce participant costs and mitigate data loss. Reimbursement plans should consider expenses associated with data transmission [26].

Data Heterogeneity. This can arise unexpectedly during the study and must be addressed during data processing and analysis. Causes of data heterogeneity include (1) changes in the software such as upgrades to the operating system and internal signal processing algorithms; (2) variations in the digital health technology: participants may be wearing different versions of the same technology allowed within eligibility criteria that differ in size and wearability characteristics (e.g., different wristbands) which may impact the accuracy of sensors and battery life; and (3) participants may change technologies during the study due to loss, malfunction or for personal reasons. This information needs to be captured by the study application. The study team should determine whether it is appropriate to incorporate this data in the analysis based on their definition of a valid dataset prior to study start as outlined in the statistical analysis plan.

Data Privacy and Security. DHTs may store personally identifiable information and personal health information. These data need to be safeguarded in accordance with local data protection requirements. Study vendors (Table 3) need to provide evidence of how the software (e.g., firmware, cloud, and study apps) influences the functionality, and the data capture is adequately protected and up to date, ensuring the data is secure [46]. The Digital Medicine Society Playbook outlines key data privacy and security considerations [55]. The Clinical Trials Transformation Initiative provides additional resources that outline approaches for securing data, including encryption, automatic backup, and user authentication [56]. Risk management plans should address data security breaches and potential interference of study tools with other applications on the participant’s technology [50]. Before the study starts, investigators should use this information to define how the data will be transmitted, secured, evaluated for completeness, and establish analysis rules to address data heterogeneity [15].

Although participants continue to have access to data routinely available from their technology, investigators and sponsors may choose to share study data with participants for transparency or because participants prefer it [57]. If this is deemed appropriate, the study team needs to determine which of the study-specific data (e.g., newly derived digital endpoints) can be shared, including the frequency, timing (e.g., during study participation or at the end of the study), and the mode of communication. The Clinical Trials Transformation Initiative has suggested a decision tree for this purpose [58].

Confounding variables influence results and impact the interpretation of data and study findings. Thus, these variables must be collected and accounted for in the analyses. Specific to BYOD, technology ownership duration (in months/years), and usual wear time (hours/day) can help interpret compliance and other potential sources of bias [59]. Participants’ digital literacy may be especially relevant in studies involving active assessment (such as conducting performance tasks via an app) instead of passive monitoring of activity with wearables [60].

Obtaining contextual data (i.e., relevant background information) can further assist in analyzing and interpreting the more heterogeneous BYOD data, e.g., when measuring physical activity, it is helpful to know what may impact participants’ daily activity patterns. Contextual variables that can be obtained with minimum burden to participants include employment status (full-time employment vs. retired), periods of vacation time (documented in a participant diary), weekend/weekday, hospitalization, and if their location is known, weather and seasonality data [61]. Concerns for data and personal privacy remain of utmost importance – the collection of contextual data can be made optional to participants and requires the approval of the local ethics board.

Estimating the number of participants required to address the scientific objectives of the study is an essential part of any study design [23]. The study size rationale varies by study type, and corresponding checklists guide sample sizing (Appendix 1). To address the added variability in BYOD studies, one can predetermine the minimum sample size required for subsets of the data to address specific hypotheses (e.g., based on demographics or expected compliance, by technology) in addition to the overall total sample size [48].

Sample size estimation should also account for the expected attrition and noncompliance in technology usage [62], which can significantly impact study size and feasibility of BYOD approaches. Strategies to mitigate dropout rates include: (1) including the participants’ perspective in the design of digital tools to optimize usability benefit; (2) incorporating comprehensive participant and site training; (3) having clear audio-visual instructions for every interaction; and (4) offering 24/7 technical support systems to participants and caregivers [63].

The statistical analysis plan must include (1) detailed information on the derivation and analysis of the digital endpoints; (2) the thresholds and methods used to establish minimally clinically important differences [64, 65]; (3) multiple hypothesis testing; and methodologies to assess the accuracy, sensitivity, and specificity of any predictive models built on DHT data [48]. Specific BYOD data considerations are outlined in Table 4.

Today, a large percentage of the global population possesses the technology to generate health data – 85% of Americans now own a smartphone, and almost one in five regularly uses a fitness tracker [66, 67]. This presents an opportunity to use these technologies to objectively quantify human physiology, behavior, and function in the real world. How we harvest this data with the rigor required for clinical studies requires careful considerations and planning.

The BYOD model provides certain advantages over conventional studies that deploy provisioned devices, including familiarity among trial participants with their own technology such that the technology itself does not function as an intervention and the reduced burden of carrying additional devices. However, its widespread use is hampered by a lack of commonly accepted methodologies describing critical success factors and an evolving regulatory landscape. Adopting the Agency for Health Research and Quality guidelines [23], we provide the following considerations on five key aspects of the design and deployment of BYOD studies for clinical research.

The input from a variety of stakeholders is key to successful technology selection and implementation. We identified internal (cross-functional team members responsible for study design, protocol development, and execution) and external (participants, caregivers, service providers, site personnel) stakeholders needed to provide input and facilitate the operationalization of BYOD models. Participants should be consulted during the study design phase when selecting outcomes, endpoints, and technologies to appropriately address their needs and preferences [68]. Focus groups [69] allow participants to test technologies before deployment and provide an opportunity to collect valuable feedback on the study design and usability of accompanying technology (e.g., study applications). This facilitates the early identification and mitigation of design and operational issues. End-of-study questionnaires that evaluate the participants’ experience in the trial can be leveraged to optimize future studies [70].

The technology of choice should be appropriate to address study objectives and research questions. The eligibility criteria should include technological requirements to generate the data stipulated in the study objectives. The generalizability of the results should be considered very carefully as it may be impacted by the preselection of participants with access to selected technologies. There can be potential inbuilt bias if recruitment is restricted to the latest technology model. Recruitment and consenting should include various options to engage broad socioeconomic strata in compliance with local regulations. Consideration should be given to digital literacy, health conditions, and education level of participants.

Digital health technology data collected from more diverse populations can reduce bias in pharmaceutical research by making clinical trials accessible to communities that are distant from traditional clinical sites [71]. However, the access to the internet and technology is limited in underrepresented communities [72]. Mitigation strategies, such as provisioned technologies and connectivity enabling approaches, ensure that BYOD studies do not exacerbate the digital divide to the detriment of participants [15]. Transparency regarding study methodology, design, and data limitations is crucial in improving BYOD strategies and advancing all-inclusive research and development.

BYOD study outcomes and endpoint-selection are governed by the same principles as traditional clinical studies, including selecting appropriate digital health technologies to generate reliable, accurate, and clinically meaningful endpoints. In addition, the technology manufacturer’s “end of life” or technology obsolescence strategy needs to be considered to ensure that the technology selected for the study does not restrict the inclusion of particular socioeconomic groups or regions where newer models may not be readily available, or older versions are no longer compatible.

The context of use is a crucial consideration because technologies are not universally optimized and some have been shown to not work as well in certain groups. E.g., the photoplethysmography sensors used to measure oxygen saturation and respiration rate are less reliable on skin types with high pigmentation [73]. Similarly, the accuracy of some wrist-worn devices to accurately measure levels of physical activity in older adults and those reliant on mobility aids have been questioned [74]. Such examples highlight the need to ensure the technology being considered has been validated in the population under consideration.

The BYOD approach needs to account for the specific challenges of data collection, including data transmission, which requires increased cooperation of participants; data heterogeneity as multiple technologies may be used along with unplanned software upgrades; and protection of personally identifiable information. As in conventional studies, participant compliance needs to be monitored with solutions to intervene if compliance falls below a defined threshold. Teams should determine appropriate strategies for sharing study data with participants.

The BYOD study’s statistical analysis plan should assess potential biases in the study population, variability introduced by deploying more than one type of technology, variability arising from a mixture of BYOD and provisioned technologies, and approaches to account for missing data and conduct fit-for-purpose sensitivity analyses.

In conclusion, this article aims to provide considerations for study design and technical, operational, and statistical considerations to successfully implement BYOD models in clinical research. Questions remain about the feasibility of the BYOD approach, e.g., does it provide the same insights as a provisioned device approach? Can this approach be fully operationalized in a pivotal global study? While the body of evidence does not yet exist, the field of digital health research is rapidly evolving, driven in part by the growing interest in decentralized and hybrid trials. This is exemplified by the declaration by the FDA in its 2021 draft guidance that sponsors should consider the appropriateness of participants’ own technology to collect data [30]. We anticipate that future studies will showcase examples of BYOD deployment that help refine the concepts outlined in this paper, document key learnings, and include additional considerations stemming from emerging regulatory guidelines.

This publication is a result of collaborative research performed under the auspices of the Digital Medicine Society (DiMe). Special thanks to Jennifer Goldsack, Isaac Rodriguez-Chavez, Benjamin Vandendriessche, Elizabeth L. Kunkoski, and Amir Lahav for their detailed manuscript reviews.

Dr. Charmaine Demanuele and Pirinka Georgiev are employees and shareholders of Pfizer Inc. Dr. Elena Izmailova is an employee of Koneksa Health and may own company stock. Dr. Kelly H. Zou is an employee and shareholder of Viatris Inc. Ms. Marie McCarthy is an employee of Novartis Ireland Ltd. and may own company stock. Dr. Cynthia Lokker, Dr. Emre Sezgin, Dr. Krishna Jhaveri, and Mrs. Cindy Geoghegan have no conflicts of interest to declare. The views expressed are the authors’ own and do not necessarily represent those of their employers.

None of the Authors received funding in the preparation of data or the manuscript.

Charmaine Demanuele, Pirinka Georgiev, Elena Izmailova, Kelly H. Zou, Marie McCarthy, Cynthia Lokker, Emre Sezgin, Krishna Jhaveri, and Cindy Geoghegan substantially contributed to the conception of this manuscript and played a role in drafting and revising the article. All authors approved the final version for publication.