Systematic Review, Meta-Analysis and Randomized Controlled Trials in Cytopathology

Systematic review (SR), meta-analysis (MA) and randomized controlled trials (RCTs) are commonly used in the current practice of evidence-based medicine (EBM) [1,2,3,4,5,6,7]. SR is a comprehensive search to identify and collect all available and relevant primary studies that satisfy a predefined research question and preset inclusion and exclusion criteria. MA is a particular type of SR that focuses on numerical results and uses statistical analyses that combine the results of the retrieved studies to produce an estimate of the overall effect. SR and MA mostly use RCTs as the source of the primary studies. RCTs are planned prospective experimental studies designed to compare the effect of two treatments or measures and minimize bias by the process of randomization and control. SR, MA and RCTs are considered the best available high-quality evidence and therefore are commonly used in clinical medicine [1,2,3,4,6]. However, the utility of these tools in laboratory medicine is still suboptimal [4,6]. Even though they have been in the medical practice for a long time, it seems that pathologists have utilized these tools less than their colleagues in other medical specialties [4,5,6]. Most attempts to standardize pathology guidelines and recommendations were based on either subjective authority views or observational studies and not on SRs and MAs [6].

In this study, we attempted to investigate and compare the frequency and the distribution of SRs, MAs and RCTs in cytopathology compared to other pathology specialties, general clinical medicine and other selected medical specialties.

PubMed, Ovid, EBSCO, CINAHL and the Cochrane Collaboration Library electronic databases were searched for SR, MA and RCT articles in cytopathology and cytology without date restrictions. In addition, all issues of four cytopathology journals (Acta Cytologica, Cytopathology, Diagnostic Cytopathology and Cancer Cytopathology) were searched and screened for relevant titles and abstracts. Eligible articles were retrieved in full text. Manual searching of the reference lists of the full-text articles was performed to retrieve articles not present during the electronic search. The core search string included the following keywords: ‘cytology’, ‘pathology’, ‘cytopathology’, ‘review’, ‘meta-analysis’, ‘randomized controlled trial’, ‘trial’, ‘evidence-based medicine’, ‘evidence-based practice’, ‘pooled analysis’ and ‘Cochrane’. They were linked to ‘cytology’, ‘cytopathology’, ‘FNA’, ‘Pap smear’ and ‘fluid’ using the Boolean operators (AND, OR). We compared the levels of evidence in four cytopathology, histopathology, and clinical laboratory journals as well as four different clinical specialties and four general clinical medicine journals (see table 2). The original papers and articles of four representative journals of each specialty were searched issue by issue over 6 years from 2003 to 2008. The study design of each article was scrutinized through the title, abstract, key words and sometimes the full-text article was retrieved to identify the study design. They were subcategorized into four levels of evidence according to their study designs and tabulated for comparison.

Fifty-three RCTs, 30 SR/MA and 3 Cochrane reviews in cytology were found (table 1) [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53]. The initial database search identified 150 potentially relevant abstracts of which 122 articles were retrieved in full texts. Forty-seven articles were excluded because they were narrative nonsystematic reviews, including personal views, expert opinions, state of the art reviews, timely reviews, invited reviews and committee reviews. Gynecology cytology (GC) utilized RCTs, SRs and MAs more frequently (83%) than nongynecology (NG) and fine needle aspiration (FNA) cytopathology. Both NG and FNA cytopathology research had comparable frequency and distribution of RCTs, SRs and MAs (table 1). Noncytopathologists performed most of the gynecology RCTs [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43]. The noncytopathologists to cytopathologists ratio was 4.3:1. Epidemiologists, gynecologists, public health physicians and clinicians performed the majority (81%) of RCTs, SRs and MAs in GC. Most of the gynecology studies focused on comparisons between liquid-based cytology and conventional Pap smears, human papilloma virus (HPV) testing, rapid prescreen review, and collection devices of cervical smears. The focus of RCTs, SR and MA in FNA practice was on respiratory, thyroid and pancreatic lesions [44,45,46,47,48,49,50,51,52,53]. They also compared FNA with tissue biopsy and frozen section. Exfoliate cytopathology focused on urine and cyst fluid. In contrast to GC, cytopathologists performed RCTs, SRs and MAs in NG and FNA cytology more frequently than clinicians [44,45,46,47,48,49,50,51,52,53].

Cytopathology, histopathology and clinical pathology journals showed a comparable frequency and distribution in their use of SRs, MAs and RCTs (table 2). However, this was lower than clinical specialty and general medicine. Cytopathology journals showed a slightly better percentage of level I and II articles than histopathology journals even though it had a smaller range of publications. Selected specialty medicine journals showed a higher coverage (5.6%) of SR, MA and RCTs than pathology and laboratory medicine (2.2%). However, the Journal of Clinical Pathology and Clinical Chemistry published more level I evidence articles than the Journal of Clinical Immunology, although the latter published fewer articles (table 2). Clinical medicine journals, on the other hand, published a much greater amount (30%) of SRs, MAs and RCTs (table 2). Even though the number of original articles was variable between the journals, level III evidence was utilized in a similar frequency (87%) in cytopathology, histopathology, laboratory medicine and clinical specialty medicine journals. It was, however, lower (61%) in general medicine publications. Level IV evidence was comparable between all medical fields.

EBM is the integration of the best available current evidence in making decisions in the form of guidelines and recommendations for good clinical practice [4,5,6,7,54]. It uses analytical and quantitative methods to evaluate the validity of primary studies. Not all evidence is equal in terms of quality, validity and reliability, however [7]. Therefore, a hierarchy focusing on the effectiveness of research stratifies evidence into levels and grades according to the quality and strength of the study designs [2,6,7]. This is because study designs are not equal in terms of errors and bias. Traditional narrative review articles are helpful in covering broad subject areas and in textbooks. Traditional reviews may include expert opinions, timely reviews and state of the art reviews. Most of these reviews are, however, individual-based. They are based on nonexperimental descriptive studies [2,4,5] and are at risk of selection and inclusion bias and therefore considered of lower quality than RCTs, SRs and MAs in which the risks of selection and bias are minimized [2]. SRs and MAs are at the top of the hierarchy of evidence and therefore are of high-quality evidence [2,6]. SR answers well-formulated questions extracted from more extensively covered and focused areas [2,6]. SR is a rigorous process, usually performed by at least two independent investigators to minimize selection bias (table 3) [2,3]. A more detailed description of the process and the steps of SR and MA can be found in the Cochrane Handbook for Systematic Reviews [2].

This study highlights the significant finding that cytopathology and other pathology specialties are relatively deficient in utilizing high-quality EBM in the form of SR, MA and RCTs compared with clinical medicine (table 2). This begs the question of why cytopathologists and pathologists in general are more reluctant or less prepared than clinicians to use these tools in their own field. Cytopathologists probably put more emphasis on cytohistologic correlation and tissue results as endpoints. In addition, RCTs, SRs and MAs are more difficult, complex, costly and time-consuming tools [2]. These factors may shunt cytopathologists away from performing these studies. Some may argue that cytopathology tests are mostly noninterventional and do not lend themselves to RCTs because diagnostic tests are nonexperimental and probably less suitable for RCTs [4,5,6]. However, the fact that many RCTs were performed by cytopathologists (tables 1, 2) suggests that RCTs are suitable for some diagnostic tests but are not being used more often. In addition, SRs and MAs can be used to validate the more commonly used level III studies in cytopathology if RCTs of these tests are not suitable or possible. A further question to address is why GC utilized RCTs and SRs more commonly than NG and FNA cytology? This probably reflects the greater contribution of the clinicians in gynecology cytology research [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53]. In addition, the introduction of new technology in the processing of cervical cytology became a point of intervention and thus more suitable for RCTs. These gynecology-centered RCTs became the basis for more SRs and MAs in GC. In addition, the multidisciplinary nature of cytopathology and its frequent assessment as part of teamwork attests to its role in clinical medicine, which could explain the greater clinicians’ contribution to RCTs, SRs and MAs in GC.

The deficiency of RCTs, SRs and MAs in cytopathology and other pathology specialties compared with clinical medicine could have future implications for pathologists. Some could argue that physicians might consider pathology a clinical practice that is based on studies usually of a lower level of evidence than other medical specialties because it is essentially an observational nonexperimental science with less utilization of RCTs, SRs and MAs [4,5,6]. This might require reassessment of the clinical effectiveness of certain guideline schemes used by pathologists [4,5,6]. Some suggest that pathologists need to develop a better methodology to evaluate the cost-effectiveness of their diagnostic and prognostic tests [4,5]. It is true that many cytopathology practices have been tested through subsequent cytohistological correlation and that cytopathology has proved itself through the test of time. However, SRs and MAs will further improve the quality of cytopathology research and practice by providing a critical appraisal platform. Others could argue that this study shows that specialist cytopathology and histopathology are doing something rather different from general clinical papers that are more suited to RCTs and therefore SRs and MAs. This is in part because clinical papers generally do not deal with specialist subjects such as the application of new diagnostic tests or molecular markers. Therefore, cytopathology and pathology practices are no less evidence-based practices but are more suited to specific specialty journals rather than general clinical readership journals. This could be a valid argument, but it should not deter cytopathologists from performing SR and MA on level III studies, which usually dominate their practice. In addition, cytopathologists probably do not frame their research questions in ways that can be addressed by RCTs. Research questions should address patient outcomes and not just the strength of cytohistological correlation. For example, do patients diagnosed with one set of diagnostic criteria, immunomarkers or molecular genetic markers do better in terms of morbidity or mortality than patients diagnosed by competing tests? Questions framed in this manner lend themselves to examination by RCTs and therefore provide a pool for SR and MA.

Cytopathologists need to embark on certain steps to improve the practice of EBM in cytopathology. Both RCTs and observational studies can contribute valid evidence, the level of which will be determined by SR and MA. Therefore, observational studies should continue in cytopathology. Well-designed cross-sectional and cohort studies provide high-quality observational studies for SR and MA when RCTs are unsuitable for certain cytopathology tests. Therefore, well-structured abstracts with emphasis on study designs as well as the use of appropriate key words should be standardized in all cytopathology journals for a better and easier retrieval by SR. Lists of papers published in each issue with the author, subject index and both declarative and descriptive titles (including methodology, study size, results, outcomes and key words) should be added at the end of each journal volume. This facilitates manual searching for SR. Studies with good study designs but negative findings should be encouraged for publication in cytopathology journals. This diminishes publication bias and heterogeneity in SR. Cytopathology societies should sponsor multidisciplinary consensus conferences for experts to systematically review the literature. This would promote international collaboration between cytopathologists and clinicians that will harmonize cytopathology practice guidelines and recommendations, and avoid unnecessary repetition of similar SRs. Standard statements and recommendations, for example PRISMA, STARD and QUADAS, for the conduct of good research and improvement of the quality of reporting and pooling of primary observational and diagnostic studies should be adopted [55,56,57]. They can provide practical checklists, evaluation plans, scoring systems, flow diagrams and assessment tools that improve the usefulness of SR and MA for authors, reviewers, editors, readers and decision makers [55,56,57]. Finally, the teaching of the concepts, the methodology and the utility of RCTs, SR and MA should be stressed more during cytopathology residency and fellowship training.

In conclusion, cytopathology and other pathology practices are deficient in utilizing RCTs, SR and MA compared with specialty and general clinical medicine. Cytopathologists and editors of cytopathology journals should consider the potential future implications and repercussions of this fact on their practice. They should be more aware of the benefits of SR and MA in resolving uncertainties in cytopathology and improving the clinical effectiveness and applicability of diagnostic studies. Therefore, steps to improve high-quality cytopathology research in the form of more SRs and MAs are recommended.

The author has no conflicts of interest or financial disclosure to declare.