To Win the Battle, First Know Your Enemy: Error Rates in Immunohematology External Quality Assessment Results

The Chinese wisdom of “to win the battle, first know your enemy” can also be applied in the context of an immunohematology laboratory or transfusion service. The enemy – incorrect results – is often unknown regarding type and incidence, and the battle can only be won with appropriate preventive measures. Unless discovered by a lucky coincidence, errors without clinical consequence may go undetected. In many instances, incorrect immunohematology laboratory results can have a negative impact on patient safety. Consequences can range from a simple delay in result reporting, to eliciting irregular antibodies in patients, to serious and even fatal hemolytic transfusion reactions. It may negatively impact pregnancy monitoring by withholding anti-D IgG prophylaxis that would markedly increase the probability of hemolytic disease of the fetus and newborn (HDFN). Other fields of medicine are also particularly affected by erroneous immunohematology results, such as hematopoietic stem cell transplantation, ABO-incompatible renal transplantation, passenger lymphocyte syndrome, autoimmune hemolytic anemias, the pre- and postnatal management of HDFN, and therapeutic apheresis, just to name a few. Estimation of error probability is based on the actual number of errors within a number of examinations performed and may easily be underestimated due to a lack of knowledge regarding the rate of unrecognized errors, resulting in a fatal chain reaction: the overall risk of the testing procedure is underestimated, inappropriate improvement measures are initiated, and finally the risk for the patient persists. Implementation and maintenance of a quality and error management system is the most appropriate weapon to overcome the continuous struggle against patient risk in immunohematology testing.

External quality assessment (EQA) schemes provide an objective judgement of individual analytical laboratories’ results by comparison with predetermined targets [1]. Results from EQA participation are a standard quality indicator for individual laboratory performance and a surrogate marker for analytical performance [2]. EQA schemes summarize results of all participants in general reports, bringing to light even low-frequency errors in laboratory procedures that some participants may not even have thought possible. Furthermore, irregularities and error patterns that may be indicators for limited performance of individual test systems or reagents are revealed. Thus, EQA schemes contribute to postmarket performance follow-up of in vitro diagnostic test systems, their components or batches, as requested by authorities [3].

To apply this to pretransfusion testing, however, more data on error rates from immunohematology EQA schemes than are currently available are needed [4-6]. We therefore analyzed a large number of red cell immunohematology EQA results and investigated error types and incidences, underlying analytical failures, as well as differences in error rates between manual and automated examination methods.

In the red cell immunohematology EQA scheme of ÖQUASTA (Austrian Association for Quality Assurance and Standardization of Medical and Diagnostic Tests), participants performed blood group serology determinations and reported the results and analytical methodology used. Participants received four suspensions of human erythrocytes (designated “patient 1,” “patient 2,” “donor 1,” and “donor 2”) and two samples of human serum that possibly contain irregular antibodies (designated “patient 1” and “patient 2”). Sample pairs are either obtained from 1 donor each, or intended constellations of antigens and antibodies are prepared. Samples were produced by DiaMed AG, Cressier, Switzerland, or Antitoxin GmbH, Bammental, Germany, in accordance with ISO Guide 35 [7]. The task in this EQA scheme is to identify the ABO blood group, Rh (Rhesus), and Kell phenotype of erythrocytes, and perform a direct antiglobulin test (DAT) with “patient” cells. Moreover, screening for irregular antibodies is to be performed in “patient” serum samples, and – in case of a positive result – the irregular antibodies are to be identified. Two “donor” erythrocyte samples are provided to perform cross-match with “patient” serum samples. Participants are instructed to carry out the examinations of EQA samples in the same way as routine examinations and to submit the results either electronically via the web portal or by fax. The immunohematology EQA scheme of ÖQUASTA included three distributions per year through 2015 and four per year after 2015. ABO phenotyping, Rh phenotyping, and antibody screening and identification were included throughout, but from 2015 (75th distribution) Kell phenotyping and serological cross-match, and from 2016 (78th distribution) DAT were additionally included. According to Chapter 7.6 of ISO 13528:2015, the correctness of results submitted by participants is to be assessed by comparison to targets obtained in an expert laboratory [8]. The evaluation of results was carried out by a software tool that compares each individual result with the target assigned to the specific sample and assesses it as matching or mismatching. Due to the small methodological differences of the individual test procedures and the exclusively qualitative (positive/negative) results, results of participants were not grouped according to test systems used. Rather, results of all participants were summarized in a general report and participants’ performance was further assessed in an individual report.

Results submitted from a total of 187 laboratories in 58 distributions in the period from 1999 to 2017 were collected within this scheme. Participants included 112 hospital laboratories, 44 independent laboratories, and 26 transfusion services. On average, about 140 institutions participated in each distribution. Laboratories reported results according to the level of detail that they routinely report for patients. Upon submission, participants reported whether they used manual or automated methods for examination. Automated methods were defined as samples, sera pipetting, and reading of reactions by machines, whereas manual methods were defined as manually pipetting and reading reactions by individuals, both regardless of test systems used. Each result was individually evaluated for its correctness. A result that matched the target value to the level of detail that was reported was considered as correct. Results that did not meet the target were further delineated as either “not determinable” or “incorrect,” depending on whether “not determinable” or a result was submitted by the participant. Each incorrect result was screened for possible sample swap, which was defined as a pair of results in which the incorrect result reported for sample 1 would have been correct for sample 2 and vice versa. Those results were excluded from further analysis. Incorrect results were not corrected when participants reported data entry or submission errors after receiving their individual reports. Results for ABO blood group antigen determinations were not reported by participants, but were deduced from submitted ABO phenotyping results.

For each parameter, the error incidence was determined separately according to the underlying manual or automated examination method. For this purpose, every reported result was evaluated for its correctness on the one hand, and the reported examination method under which it was obtained on the other hand. A comparison of the results of individual device types was not intended for this evaluation. For evaluating the effectiveness of implementation of automated methods, the rates of incorrect results from 45 laboratories that changed from manual to automated methods during the observation period were analyzed. For these comparisons, results obtained with earlier manual and later automated methods were compared for each parameter separately. Since 2015 (distribution 77 in this EQA scheme), Austrian guidelines in blood group serology introduced a double determination with two different test systems for Rh D, C/c, and E/e phenotyping [9]. To evaluate the effects, error rates in Rh phenotyping before and after this change were compared.

Statistical analyses were performed using generalized estimating equations for binomial data, considering the fact that results were clustered per laboratory [10]. Results are provided by means of odds ratios, which in comparison of two groups reflect the odds of producing a wrong result in the first mentioned group with respect to the odds of a wrong result in the second mentioned group. They are based on the frequencies of correct and incorrect results in relation to the total number of results. p values test the hypothesis that odds ratios are equal to 1. With p values <0.05 it is assumed that the odds of producing a wrong result is significantly different between the compared groups.

Participants reported 58,768 results during the observation period from 1999 to 2017. For all incorrect results, 21 cases of possible sample swap were detected (14 affecting ABO phenotyping, 5 Rh phenotyping, and 2 antibody screening; data not shown); those 42 results were excluded from further analysis.From the remaining 58,726 submitted results, 620 (1.06%) did not match the target. Within those, 563 (0.96%) were incorrect and in 57 cases (0.10%) participants could not interpret the results of the examinations with sufficient reliability and reported “not determinable” as the result. A detailed analysis of the results is presented in Table 1. The significant differences in error rates between manual and automated methods are noteworthy (1.04 vs. 0.42%, p < 0.0001). Table 2 shows the results of the analysis of incorrect results in the determination of antigens A, B, C, c, D (including Dweak), E, e, and K. Overall rates of false positive and false negative results are about equal, but D phenotype was reported predominantly false negative (0.79 vs. 0.17%) and antigens A (0.13 vs. 0.05%), e (2.49 vs. 0.18%), and c (1.42 vs. 0.28%) were reported predominantly false positive. The incorrect results in ABO phenotyping were based on 16 incorrect determinations of one (A or B) and five incorrect determinations of both antigens (A and B). In Rh phenotyping, incorrect results were based on 153 incorrect determinations of one antigen, 43 based on two, and 16 based on three or more incorrect determinations (data not shown). Table 3 shows the detailed analysis of incorrect results of DAT, serologic cross-match, antibody screening, and antibody identification. There are no significant differences between false positive and false negative results for DAT, whereas for serologic cross-match and for antibody screening false positive results predominate, and for antibody identification false negative results clearly prevail. In all 11 cross-match results with target “positive” but misinterpretation “negative” by participants, incompatibility was solely due to mismatch in blood group systems other than ABO (data not shown). Of all 352 incorrect results in antibody identification, 196 were based on incorrect determination of one irregular antibody, 45 based on two, and 19 based on three or more (data not shown). The rates of false negative and false positive results in antibody identification varied strongly. Anti-K (6.98%) and anti-E (4.84%) were missed the most, and anti-C (1.06%) and anti-E (0.82%) were reported as false positive the most.

Accumulations of incorrect results were found for the same analyte in several distributions (Fig. 1). Anti-K specificity was missed in distributions 32, 38, and 57 by 16/46 (34.8%), 15/48 (32.3%), and 30/44 (68.2%) participants, respectively. In distribution 65, 33/52 (63.5%) results for antibody identification were incorrect, including 32 (61.5%) results missing Anti-K, and 10 (19.2%) results falsely reporting Anti-C, with both errors partly occurring simultaneously. In distribution 74, 9/45 (20%) results falsely reported Anti-C, in distribution 79, 11/47 (23.4%) missed Anti-E, and in distribution 81, 10/46 (21.3%) results for antibody identification were incorrect for one sample, including 9 (19.6%) results missing Anti-E. No detectable patterns for errors in distributions 32, 34, 56, and 82 were observed. After the introduction of mandatory double testing in 2015, mean error rates for Rh phenotyping significantly decreased from 1.55% in distributions 27–76 to 0.50% in distributions 77–85 (p < 0.0001).

The 187 participants were assigned to one of three groups according to the frequency and analyte reference of reported incorrect results. Group 1 includes 53 laboratories (28%) that did not report a single incorrect result out of 18–505 (mean 212) results. Group 2 includes 37 participants (20%) who reported a maximum of one incorrect result per analyte out of 15–519 (mean 295) results. Group 3 includes 97 participants (52%) that repeatedly reported incorrect results for one or more analytes out of 113–514 (mean 370) results; in this group the highest counts of incorrect results of individual participants were 3 for ABO phenotyping, 14 for Rh phenotyping, and 2 for Kell phenotyping. For DAT and antibody screening, the highest counts for individual participants were 2, for antibody identification were 16, and for serologic cross-match were 3.

To illustrate the comparability of the three groups, laboratories that participated frequently were separately considered. Those 96 participants were identified by the subjective limit of having submitted at least 350 of the maximal 520 results that could be submitted during the study period. In group 1, 14 high-frequency participants (14%) reported 370–520 (mean 413) results, 17 (18%) in group 2 reported 360–519 (mean 411), and 65 (68%) in group 3 reported 355–520 (mean 438) results (Fig. 2). The characteristics of those three groups are not displayed for confidentiality reasons of small groups.

A reduction of the rates of incorrect results was seen after the introduction of automated methods (Table 4). In the results of 45 laboratories that changed from manual to automated methods during the observation period, differences in the rates of incorrect results obtained with manual and automated methods were found (0.74 vs. 0.35%, p = 0.0034).

This analysis of a large number of EQA scheme results showed that all included laboratory examinations have a certain potential for error and incorrect results. Error rates varied amongst the investigated tests, with some being particularly prone to incorrect results.

The overall rate of almost 1% incorrect results appears alarmingly high in a high-risk area such as red cell immunohematology and calls for improvement. The comparatively low error rates in ABO phenotyping may be due to both the confirmation of results by reverse typing, as well as the obligatory double ABO antigen determination with two different sets of test reagents in Austria. Importantly, the introduction of a mandatory double determination in 2015 for Rh C/c and E/e was associated with a clear reduction of error rates from distribution 77 onwards. Hence, extended testing can lead to a reduction in error rates probably not only in phenotyping, but may likely also increase the reliability of antibody identification. Here, the high error rates could be diminished by using a sufficient number of test cell samples. Antibody identification shows a clear predominance of false negative results, i.e., unrecognized but clinically relevant irregular antibodies. However, the high absolute number of false positive results is also disconcerting: once reported, irregular antibodies remain in the patient’s history and are to be considered for transfusion purposes. Such false specificities may thus unnecessarily complicate the selection of blood products for transfusion. Error rates indicate that anti-K, anti-E, and anti-C are relatively often misrecognized or go undetected, which is both surprising and alarming as these antibody specificities are amongst the most frequent in clinical routine. Moreover, these specificities could readily be differentiated by employing even a standard test cell panel. Given the admittedly low error rates for DAT, antibody screening, and cross-match, it is important to realize that false negative results in these tests are much more likely to cause problems than false positive results. False negative results in DAT and antibody screening can lead to wrong diagnoses and thus prevent adequate treatment or delay its onset. For cross-match, false negative results give transfusing physicians the erroneous expectation that the blood transfusion will be well tolerated by the recipient. Of note, the ABO-bedside test that is mandatory in some countries prior to transfusion would not have detected the false negative cross-match results of this study, since incompatibility in any sample combination was due to mismatch in blood group systems other than ABO.

From our data, we cannot explain the predominantly false negative results of D-phenotyping and the predominantly false positive results for antigen A. We also cannot give a conclusive explanation for the predominantly false positive results for antigens c and e.

The 53 participants – and within those especially the 14 high-frequency participants – who did not report a single incorrect result show that accurate work in immunohematology is possible. The proportion of those participants is surprisingly low at 28% of all participants and 14% of high-frequency participants. Hardly better is the proportion of those laboratories which did report one, but no repeated incorrect results for the same analyte (namely 20 and 18%). Those 52% of all participants or 68% of high-frequency participants repeatedly reporting incorrect results for one or more analytes should question their competence in these tests, analyze the root causes of the errors, and determine appropriate corrective actions to avert patient harm. Regular participation in each of the 58 distributions, each with two samples, gives a total of 116 results. In view of this fact, the above-mentioned error counts of some participants are not for the faint of heart: for example, the maximum counts of 11, 14, and 16 incorrect results with regular participation in all 58 distributions would correspond to 9.5, 12.1, and 13.8% incorrect results; with less participation these values would be even higher. It is hard to understand why participants with such high error rates do not stop – or rather, are not stopped from – performing those tests in favor of passing samples on to more competent laboratories.

The result “not determinable” shows that a laboratory has recognized that its analytical competence is insufficient for the individual case and therefore a reliable result cannot be released. The procedures in the laboratory should provide that, in such cases, sample material be transferred to a specialized laboratory for resolution of the problem. Many of the tests with “not determinable” results in fact do not require a particularly high level of competence, but when uncertain, accepting the limits of one’s own abilities is less risky than reporting false results carelessly or in doubt.

Sample and result swap must not be underestimated in medical analytics. EQA schemes are primarily designed to assess performance of analytical procedures, as it is almost impossible for an EQA provider as an objective third party to assess extra-analytical procedures where sample or result swapping can occur. In case of incorrect EQA results, root cause analysis should be conducted with the special consideration that extra-analytical procedures such as sample handling and identification, data transcription, as well as reporting and the transmission of results are likely to be different for EQA samples than for routine samples.

According to our data and as predicted earlier, automating pretransfusion testing was associated with reduced error rates [11]. However, laboratories that switched to automated methods had already shown lower rates of false results for most analytes than the total population of all participants using manual methods. It cannot be ruled out that other factors, such as the introduction of quality systems, contributed to these changes in error rates [12]. It can also be seen that double performing the same test with two different test systems leads to a significant reduction of incorrect results.

Unusually high numbers of errors in individual distributions may indicate failures in analysis systems or in EQA samples used. Error counts in the distributions 32 (concerning identification of irregular antibodies), 38, 57, and 65 call for cause analysis due to the high numbers of missed anti-K, in distributions 65 and 74 for the false positive anti-C, and in distributions 79 and 81 for the missed anti-E. Errors in distributions 32 (concerning Rh), 34, 56, and 82 followed no pattern.

If errors are rarely detected, this does not mean that errors rarely occur. It is a fallacy to estimate error probability for a laboratory test based solely on error frequency found in laboratory routine and to ignore the possibility of additional errors that are left unrecognized. The results presented here describe the incidence and types of incorrect results in pretransfusion testing, and thus allow definition of measurable goals for risk management. EQA scheme participation in combination with stringent error management appears to be an appropriate weapon to start to combat errors that may compromise patient safety.

The authors have no ethical conflicts to disclose.

The authors have no conflicts of interest to declare.

C.B., W.C., and G.F.K. designed the study. C.B. collected data and searched the literature. C.B. and W.C. analyzed and interpreted the data. C.B., W.C., and G.F.K. drafted the manuscript. W.R.M., M.M.M., and G.F.K. revised the paper.