The Biggest Struggle: Navigating Trust and Uncertainty in Genetic Variant Interpretation

The price of genomic sequencing is quickly declining, increasing the accessibility of these tests and their promise to improve health outcomes. Recently, researchers found that genomic sequencing out-performed targeted gene sequencing tests in diagnosing infants with a suspected genetic disorder, identifying a molecular diagnostic variant nearly 50% of the time [1]. To improve clinical outcomes, professional societies like the American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) have sought to issue more standardized guidelines that clarify how these tests should be interpreted in patient care [2‒4]. Yet despite these advances, discordance in genetic variant identification and classification between genetic testing laboratories continues to complicate the wide-scale use of this testing. The above study, for example, found that when two different laboratories detected the same variant, their classification of that variant differed 43% of the time [1]. Analyses of variant classifications in ClinVar suggest that of the 11% of variants with multiple submitters, classifications vary 17% of the time [5]. Indeed, as researchers from the Clinical Sequencing Evidence-Generating Research (CSER) Consortium have identified, continued discordance in genetic variant interpretation remains a critical place for improvement in clinical genomics [6].

In addition to ongoing concerns about how laboratories classify variants, a crucial question remains: how might this discordance have a real impact on the clinical management of patients? While some of these discrepancies may not impact clinical care, a study of variant interpretation discordance for Li-Fraumeni Syndrome suggested that 11% of families may receive discordant classifications leading to clinically important differences in medical management [7]. Additionally, clinicians may question or disagree with laboratory findings based on other data about patients that laboratories may not receive. Understanding how discordance between laboratories in variant identification and classification, and discordance between laboratories and clinicians, may impact patient care is particularly important as genomic sequencing becomes available to medical specialties outside of clinical genetics. Can clinicians without thorough knowledge of variant interpretation in laboratories integrate this kind of testing into their practice if discordance remains a serious concern? If clinicians are unaware of potential discordance issues, might this lead them to be overconfident in interpreting any given laboratory report?

To begin answering these questions, we draw upon 40 semi-structured interviews conducted with clinicians including clinical geneticists, genetic counselors, primary care providers, and cardiologists. Our approach focuses on genetic testing as a sociotechnical system, which suggests that the ways new technologies are designed and used cannot be understood without accounting for social context and relationships [8]. We build on a related concept of the “certainty trough,” first developed by sociologist Donald MacKenzie, which provides a theoretical lens to assess how expertise in a technological system like genomic sequencing impacts confidence and trust in others using and interpreting those systems [9]. One might expect that the people with the most intimate knowledge of a technology would have the highest trust and confidence in it, and those with the lowest levels of knowledge and expertise would report the least confidence and trust. Instead, MacKenzie finds the opposite, and we suggest that this theoretical insight can help explain discrepancies between the way clinicians with and without genetics expertise interpret and trust results obtained from genetics laboratories.

Our analysis draws from 40 semi-structured interviews with clinicians involved in the ordering and interpreting of genetic tests, including 20 clinical geneticists or genetic counselors with specific training in genetics and 20 cardiologists or primary care providers without specific genetics training. The data presented here are part of a larger study on uncertainty, evidence, and actionability in clinical genomics. We did not design the study to assess how clinicians view and respond to discrepancies in variant classification and interpretation – these insights instead emerged in the data collection and analysis process, as is common in qualitative research. Thus, we consider this research preliminary and hypothesis-generating, and we are unable to determine whether thematic saturation was reached in our sample on the specific topics of clinician trust in genetic laboratories or laboratory discordance. All participants in our sample were asked and answered questions about these topics. Participants were recruited by email through listservs or snowball sampling. Because this is a small qualitative study, we did not intend to enroll a fully representative sample but still sought demographic diversity. All participants were based in the USA, and most came from large academic medical centers. The study was determined to meet IRB exemption criteria by the IRB at the University of Pennsylvania in August 2020.

Our analysis explores similarities and differences between clinicians with particular genetics expertise versus clinicians without particular genetics expertise. We analyzed transcripts and field notes from interviews using the principles of situational analysis, a modified approach to grounded theory [10]. Two authors (K.O. and D.M.A.) completed coding and thematic analysis using NVivo for Mac 1.7.1, and any coding discrepancies were resolved via discussion. Data analysis took place between January and September 2022. More comprehensive details about the methods for this study can be found in the Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist in the Supplementary Materials, which also includes our interview guide.

A main finding of our analysis is that clinicians with genetics expertise report significant frustration with and uncertainty about laboratory reports for whole genome or exome tests. As genetic tests get larger and are carried out by a greater number of laboratories, there are more opportunities for discordance to arise. Clinicians with specific expertise in genetics expressed concern that despite efforts to standardize reporting criteria across institutions, results can still vary:

Faced with uncertainty over the accuracy of results, these clinicians draw on their own knowledge of genetics in treating patients. While clinicians do not dismiss laboratory results entirely, they nonetheless will sometimes provide alternative interpretations of findings to patients:

These clinicians reported frustration with multiple forms of discrepancies with laboratories – concerns about different laboratories’ ability to detect a variant (analytic validity), laboratories’ appropriate classification of that variant (classification accuracy), and clinician disagreements with laboratory classifications due to their additional knowledge about a patient’s clinical presentation and family history. All of these factors influenced their trust in laboratory reporting, and these concerns were often discussed together. For clinical geneticists and genetic counselors, this uncertainty has significant implications for clinical practice. One genetic counselor interviewed for this study described it as “probably the biggest struggle that we have currently in the field,” noting that:

In general, clinicians with genetics expertise were more likely to raise doubts or second-guess the validity of results in comparison to clinicians without genetics expertise.

Given the declining price and expanding availability of genomic sequencing, clinical genetics is rapidly entering medical settings populated by nonexperts. Therefore, this study also interviewed cardiologists and primary care providers who are involved in ordering genetic tests but who do not have specific genetics expertise. In contrast to clinical geneticists and genetic counselors, these providers expressed a higher level of trust in laboratory results. Rather than questioning the results reported by genetics laboratories, providers described them as “thorough”:

While several factors could explain why these clinicians are more likely to trust laboratory results, one reason that they provided was lack of genetics knowledge:

Clinicians whose specialties are not dependent on specific knowledge of genetics and who have not spent significant time understanding genetic technology may not possess the necessary expertise to further analyze these results. Given that genetic counselors report time constraints as a deterrent to giving patients alternative explanations, in the absence of specific knowledge, nonexperts are more likely to trust what laboratories report to them [11]. We did not identify any key differences between groups when broken down by demographic categories such as gender or race, although differences may be more apparent in a larger sample size.

The discrepancy between the level of trust invested in laboratory results by clinicians with and without genetics expertise is likely to have consequences for patient care. One interview participant described a situation in which questionable laboratory results led a patient to seek potentially unnecessary care elsewhere:

The clinician’s training in both genetics and cardiology, combined with additional genetic analysis and evaluation of the patient’s family members, led them to the conclusion that there was insufficient evidence to support the need for a defibrillator for the patient’s sister. However, providers without genetics expertise often treat these results with less uncertainty, and in this case the patient wound up being treated by another care provider:

In situations such as this, whether a clinician interprets laboratory results with certainty or not has significant consequences for patient care and clinical outcomes. Patients may also receive different diagnoses and treatments depending on access to specific providers.

The concept of the “certainty trough” implies that, contrary to expectation, the people with the most knowledge about a technology, which in this case includes clinical geneticists and genetic counselors, end up reporting more uncertainty about the quality of the data and knowledge they are producing [9]. In comparison, people committed to genomics as an avenue for improving health outcomes, but who are users, rather than producers of knowledge, report the highest certainty in the quality of that data. In the case presented here, this is primary care providers and cardiologists – providers who order genetic tests but who do not necessarily have genetics expertise.

More specifically, our preliminary analysis suggests that clinicians with genetics expertise expressed frustration with many of their interactions with commercial genetics laboratories and spent significant time assessing laboratory quality regarding accuracy and validity, structure and content of laboratory reports, and billing requirements [11]. Even when clinicians found laboratories they liked, they often reassessed or reclassified variants based on their own research. This is despite efforts to standardize variant classification and reporting from organizations like the ACMG [2]. The impact of this effort from clinical geneticists and genetic counselors is unclear – while these clinicians may play an important role in resolving discrepancies between laboratories or understanding when laboratories may be missing important patient or family history details, the benefits or risks of clinicians making their own variant interpretations that differ from laboratory interpretations are unknown. In many cases, we would expect that laboratory geneticists are better positioned and have better training to interpret and classify variants, and clinician disagreements may introduce unnecessary or inaccurate complications.

As clinical genetics continues to move outside of expert spaces into places like primary care offices, the field faces the task of how to appropriately understand the content and context of laboratory results. Because participants in our sample with expertise in genetics tended to view laboratory results with much more scrutiny and skepticism than providers without that expertise, we expect that this will have an impact on the way patients understand their results, and their resulting care. Additional research would be necessary to confirm this finding. There are a number of ways the field of clinical genetics can address the problems that result from the “certainty trough:” First, genetics researchers and practitioners can try to further standardize how laboratories classify variants and report results, as is already occurring on ClinGen Variant Curation Expert Panels [12‒14]. Despite existing guidelines, clinicians are still reporting significant differences in practice. Health systems could also standardize how clinicians are taught to understand these results. Or, due to differences in clinician confidence in genetic testing results, health systems could ensure that patients with more complicated genetic results get the expert care that they may need to truly make sense of their results. Given the complex sociotechnical nature of genetic testing, changes to all of these practices are likely required to provide more effective patient care.

The related study protocols were reviewed and approved by the Institutional Review Board at the University of Pennsylvania and determined to meet criteria for IRB review exemption authorized by 45 CFR 46.104, category #2 (Decision Reference Numbers 829895 and 843839). The IRB reviewed and approved our verbal informed consent procedure, and verbal consent was obtained from participants to participate in the study via an IRB-approved verbal consent script. The authors attest that the research included in this report was conducted in a manner consistent with the principles of research ethics, such as those described in the Declaration of Helsinki and/or the Belmont Report.

The authors have no conflicts of interest to declare.

The research reported in this manuscript is supported by the National Human Genome Research Institute (NHGRI) at the National Institutes of Health (NIH) (R00HG010905). The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Z.G.: data analysis, investigation, writing – original draft, review, and editing; D.M.A.: data curation and analysis, investigation, and writing – review and editing; K.O.: conceptualization, methodology, data curation and analysis, investigation, writing – review and editing, supervision, and funding acquisition.

Due to the sensitive and potentially identifiable nature of the interview transcripts reported in this study, our research data is not publicly available. Redacted transcripts and other study materials may be made available upon request. Further inquiries can be directed to the corresponding author.