External Quality Control of Cervical Cytopathology: Interlaboratory Variability

The cervical cancer screening programs based on screening tests has proven its efficiency in many countries where such programs are organized and mortality rates have been reduced [1]. However, this screening test must present a good diagnostic accuracy [1,2].

The screening tests show a high percentage of false negatives (FN), with a range of 2-62%. The main causes of the low sensitivity are related to mistakes during the collection of material, the examination of the smear or the interpretation of results [3,4,5]. Programs of quality control in cytopathology are an alternative that attempts to minimize these mistakes [6,7].

Internal quality control is a necessary tool in the routine of laboratories in order to reduce mistakes in the assessment and interpretation of screening tests [8,9]. However, the evaluation of the performance of different laboratories can only be achieved through external quality control [10].

External quality control aims at verifying the technical and diagnostic quality of smears, standardizing results, screening diagnostic difficulties, assessing accordance between source laboratories and the reviewer laboratory, working as a mechanism of diagnostic standardization as well as pointing to continuing education on cytopathology [11].

In Australia, laboratories are subject to regular inspection. There are many requirements ranging from scientific qualifications, ongoing education and limitations on the number of slides screened. Laboratories must also be enrolled in an external quality assurance program. In addition, laboratories reporting cervical cytology must meet specific performance indicators. These are a set of key, quantifiable criteria that allow comparisons of performance of individual laboratories [12].

In Brazil, the laboratories accredited by the Health System to carry out screening tests have to take part in an external quality control via the Unit for External Quality Control (UEQC), a reference laboratory designated by the state.

Overall, the errors in the examination of the smear or the interpretation of results are still a problem faced by laboratories, and it is known that their participation in external quality control programs may be a strategy for ensuring the quality of the screening tests [13,14]. The objective of this study was to evaluate the variability of the results from the screening tests of laboratories with the results from the UEQC, checking the frequency of disagreeing, false-positive (FP), FN or unsatisfactory cases or those with a delay in clinical management and diagnostic agreement.

This is a transversal study carried out by the UEQC at the School of Pharmacy, Federal University of Goiás, Goiânia, Goiás, Brazil, and was approved by the research ethics committee of the institution under protocol No. 117/07.

The study included technical professionals responsible for 14 laboratories accredited by Health System Care, appointed by the Municipal Secretariat of Health of the city of Goiânia. The sample consisted of conventional cervical smears and results selected by the System of Information on Cervical Cancer [11], entering the screening test results of the source laboratories, including all positive, unsatisfactory and at least 10% of the negative smears performed by laboratory per month from January 2007 to December 2008, totaling 10,053 cases analyzed. The laboratories review 10% of negative smears as a method of internal quality control.

The method by which smears were reviewed by the UEQC consists of detailed manual screening by experts of all smears from the laboratories accredited by the Brazilian Health System, which are defined in this study as source laboratories. First, the smears and results were checked, and when a nonconformity was identified (identification of smears not matching the result, broken smears, smears with no result or results without a smear), the laboratory was informed and the material was returned.

After being checked, all the smears and results classified as negative, unsatisfactory or with any alteration (atypias, low- and high-grade squamous intraepithelial lesion, carcinomas and adenocarcinomas) by the source laboratory were sent to the first reviewers.

The smears considered accordant after the first review were approved and considered as a final diagnosis. The smears considered discordant after the first review, were referred to a second review; if there was any consensus between the 2 reviews, then this was considered a final diagnosis. In the case of disagreement between the reviews, the smear was referred to a consensus meeting in which a final diagnosis was defined.

The cases considered to be disagreeing were the ones with a change in clinical procedure, according to the criteria established by the Ministry of Health/National Cancer Institute.

The results were considered FN, FP and a delay in clinical management as follows:

- FN: initial diagnosis of unsatisfactory smears, negative cytology and review classified as a diagnosis of atypical squamous cells of undetermined significance (ASC-US), ASC cannot exclude a high-grade squamous intraepithelial lesion (ASC-H), a low-grade squamous intraepithelial lesion (LSIL), a high-grade squamous intraepithelial lesion (HSIL), HSIL with features suspicious for invasion (HSIL-suspicious of invasion), squamous cell carcinoma, atypical glandular cells not otherwise specified (AGC-NOS), AGC favor neoplasia (AGC-NEO), adenocarcinoma in situ and invasive adenocarcinoma.

- FP: initial diagnosis of ASC-US, ASC-H, LSIL, HSIL, HSIL-suspicious of invasion, squamous cell carcinoma, AGC-NOS, AGC-NEO, adenocarcinoma in situ and invasive adenocarcinoma and the review classified as negative or unsatisfactory.

- A delay in clinical management: an initial diagnosis of ASC-US or LSIL and review classified as a more serious diagnosis of ASC-H, HSIL, HSIL-suspicious of invasion, squamous cell carcinoma, AGC-NOS, AGC-NEO, adenocarcinoma in situ and invasive adenocarcinoma.

- The clinical managements for cervical cytology diagnosis were established in accordance with the Ministry of Health recommendations, as specified below [15].

- Negative for intraepithelial lesion or malignancy: follow the routine cytological screening.

- ASC-US and LSIL: repeat cytology screening tests in 6 months. If changes persist or there is a worse diagnosis, send patient for colposcopy and biopsy.

- ASC-H and HSIL: perform colposcopy and biopsy.

- Squamous cell carcinoma, AGC-NOS, AGC-NEO, adenocarcinoma in situ, invasive adenocarcinoma: perform colposcopy. In case of lesions, perform biopsy. If not, perform conization.

The Bethesda System [16] was used to evaluate the adequacy of the sample and the classification of the cytology result.

The disagreeing results were sent to the source laboratory, which used the UEQC diagnosis, in the case of disagreement with the final diagnosis. When the source laboratory agreed with the final diagnosis, it had to reissue the result, mentioning that the review was confirmed by the UEQC. When the source laboratory disagreed with the final diagnosis, a consensual meeting took place between the source laboratory and the UEQC, in which the final diagnosis was defined.

The data analysis used the application SAS for Windows [17]. The magnitude of agreement among tests performed by source laboratories and by the UEQC was measured by the kappa coefficient, weighted with its respective 95% confidence intervals (CIs), depending on the need to assign different weights to the disagreements. The agreement level was classified as follows: <0: worst agreement, 0-0.2: bad agreements, 0.2-0.4: reasonable agreements, 0.4-0.6: good agreements, 0.6-0.8: very good agreements and 0.8-1.0: excellent [18].

Out of the 10,053 cases analyzed, the source laboratories classified 5.06% as ASC-US, 3.41% as ASC-H, 8.38% as LSIL, 3.76% as HSIL, 0.14% as HSIL-suspicious of invasion, 0.36% as AGC-NOS and 0.31% as AGC-NEO. The UEQC classified 3.97% as ASC-US, 2.79% as ASC-H, 6.67% as LSIL, 6.0% as HSIL, 0.23% as HSIL-suspicious of invasion, 0.11% as AGC-NOS and 0.12% as AGC-NEO (table 1).

Out of the 10,053 cases analyzed, 7.59% were considered disagreeing. It was estimated that 1.1% were estimated to be FN, 0.37% of which were reclassified as ASC-US, 0.22% as LSIL, 0.30% as ASC-H, 0.13% as HSIL, 0.01% as HSIL-suspicious of invasion, 0.07% as AGC and 0.02% as adenocarcinoma in situ. Out of the 2.44% cases considered to have had a delay in clinical management, 0.84% were initially classified as ASC-US, 0.53% were reclassified as ASC-H, 0.31% as HSIL and 0.01% as AGC. Out of the 1.50% initially classified as LSIL, 0.19% were reclassified as ASC-H, 1.38% as HSIL and 0.02% as AGC. The study considered 2.82% of the cases to be FP and 1.24% to be unsatisfactory. The diagnostic agreement between the source laboratories and the UEQC was excellent (kappa = 0.81) (table 1).

The agreement of the sample adequacy was reasonable (kappa = 0.30). Regarding the representation of squamous, glandular and metaplastic epithelia, the agreement was excellent with kappa (0.84, 0.93 and 0.91, respectively; table 2).

It was observed that the diagnostic agreement between the laboratories and the UEQC was considered excellent for four laboratories, very good for eight and good for two (table 3).

The results of this study showed that the agreement between results from screening tests in most source laboratories and the UEQC was very good; however, among the disagreeing results, the FP prevailed, followed by cases with a delay in clinical management.

Among cases considered to be FN, 0.59% would have to repeat the screening test in 6 months and 0.51% would have to undergo colposcopy. In a study by Pereira et al. [14], out of the 67.954 revised screening tests, 1.67% were considered FN.

As a measure to minimize the FN results and ensure their improvement, studies have shown that the rapid review of 100% of results previously classified as negative is an efficient and cost-effective method [19,20].

In addition, 2.44% of the cases reviewed had a delay in clinical management, i.e. all these cases should have been sent immediately for colposcopy. For such disagreeing cases, whether they were FN or had a delay in clinical management, a new result was reissued, locating the patients who initially received inadequate results and sending them for recommended clinical management and proper treatment.

Among cases considered as FP, 1.52% did not go on to repeat the screening tests in six months and 1.30% did not undergo colposcopy. This minimized the loss for both patients and the Health System, because once the lesion had been diagnosed, the patient would be sent to perform the recommended clinical management, and so unnecessary spending was avoided. A study has shown results similar to this one, in which most cases were reclassified as negative, avoiding unnecessary costs in repeating the screening tests and in the follow-up [21].

Moreover, the cytology criteria of the source laboratories and the UEQC were similar in this study with an excellent rate of agreement (92.41%).

These results were consistent with Maeda et al. [22], who found excellent levels of agreement between the source laboratory and reviewer (86.62%), and observed no FN cases, showing the applicability of external quality control in the public health system and meeting the expectations of quality required by the Ministry of Health.

It was observed that the majority of disagreeing cases were related to borderline cytology results (ASC and ASC-H). This interlaboratory and interobserver variability has been found in other studies. Confortini et al. [23] showed a low interlaboratory reproducibility in the category ASC-US (kappa = 0.34). Gatscha et al. [24] showed in their review that, out of 632 smears first diagnosed with atypical squamous, only 200 cases (32%) showed interobserver agreement, which demonstrated the low reproducibility of this diagnostic category. Smith et al. [25], Juskevicius et al. [26] and Stoler and Schifman [27] showed that rates of atypical cells and squamous intraepithelial lesions can be influenced by the rigidity in the adoption of morphological criteria and by the degree of experience in the interpretation of cytological specimens, often without knowing the factors affecting the interobserver reproducibility.

Nevertheless, this study showed that concerning the adequacy of the sample, the agreement was considered reasonable (kappa = 0.30). The majority of disagreeing cases is related to obscuring factors that partially affect the analysis of the smear. Our results were consistent with those of Cocchi et al. [28], who found a low interlaboratory agreement concerning the adequacy of the sample that showed kappa variation between 0.01 and 0.29 and epithelial abnormalities ranging between 0.53 and 0.78.

This study found that 1.24% of smears first classified as negative were reclassified as unsatisfactory. So sample collection was repeated, according to instructions by the Bethesda System and Ministry of Health, avoiding a possible FN [16,25].

External quality control enables the evaluation of laboratories via review of negative, positive and unsatisfactory selected smears, helping to reduce FN and FP results and to improve the accuracy of screening tests.

The results of this study showed the importance of external quality control and serve to support the implementation of continuing education for professionals to establish the standardization of diagnostic criteria, improving the quality of results of cytopathology screening in cancer of the cervix.

I would like to thank the technical professionals of the UEQC for cooperation in the screening.

The authors have no conflicts of interest to declare.