The Diagnostic Performance of Laboratory Tests of Neurosyphilis: A Systematic Review and Network Meta-Analysis

Syphilis is a multisystem infection caused by the spirochete Treponema pallidum [1]. When it demonstrates symptoms, it may affect almost any organ in the human body, including the central nervous system, skin, bone, and cardiovascular system. Neurosyphilis (NS), however, can occur during any stage of the disease [2]. Moreover, the incidence rate of acquired syphilis has been rising in several countries [3‒6]. The clinical manifestations of NS are diverse, and their characteristics have changed in the era of antibiotics.

Even if there are relevant clinical and radiological features, the differential diagnosis of NS is complicated because it shares many common nervous system syndromes [7]. Meningitis, meningovascular syphilis, tabes dorsalis, and dementia are traditionally described presentations [8]. The performance of diagnostic tools is far from ideal, causing much inconvenience to clinicians and patients [9, 10]. Even though the gold standard for diagnosing NS is not yet available, various laboratory parameters in cerebrospinal fluid (CSF) and blood can assist the diagnosis, with warnings in different patient groups [11]. Detection of nonspecific syphilis antibodies and Treponema pallidum antibody is the main means of NS laboratory diagnosis [12]. The main antiphospholipid antibody tests currently used include rapid plasma reagin (RPR), venereal diseases research laboratory (VDRL), and toluidine red unheated serum test (TRUST) [12, 13].

Furthermore, the fluorescent treponemal antibody absorption (FTA-ABS), treponema pallidum particle agglutination (TPPA), cerebrospinal fluid-enzyme immunoassay (CSF-EIA) chemoattractant, chemokine (C-X-C motif) ligand 13 (CXCL13) tests are also employed in clinical practice [1, 13‒16]. Molecular biology techniques, such as polymerase chain reaction (PCR), have also been used to diagnose NS [17]. The interpretation of these laboratory tests is diverse, depending on the stage of NS. Besides, the diagnostic performance of these modalities tended to be heterogeneous. This study aimed to evaluate and rank these laboratory tests regarding the diagnosis of NS in a diagnostic test accuracy network meta-analysis (DTA-NMA) and to provide evidence-based findings for clinicians and scientists in the field of NS management.

This systematic review and NMA were performed according to the guidelines of the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement [18, 19]. Ethical approval and informed consent were not required because this study did not contain individual patient data, and all data in this meta-analysis were acquired from articles published online. The processes of database search, study screening, data extraction, and quality evaluation were performed by two independent researchers. We resolve our differences through discussion.

Electronic databases, including PubMed, Embase, and the Cochrane Library, were searched from inception to March 28, 2022. “Central Nervous System Syphilis,” “Syphilis, CNS,” “Syphilis, Central Nervous System,” “Neurosyphilis, Asymptomatic,” “Neurosyphilis, Gummatous,” “Paretic Neurosyphilis,” “Neurosyphilis, Secondary,” “Secondary Neurosyphilis,” “Neurosyphilis, Symptomatic,” “Neurosyphilis, Juvenile” and “diagnosis” were keywords used for database search. Only articles in English were considered in the literature search. Furthermore, the reference lists of relevant articles and the enrolled studies were manually screened for potentially eligible records.

The inclusion criteria for this meta-analysis were as follows: (1) studies using laboratory tests to assist the diagnosis of NS regardless of types of modalities; (2) study participants were patients with clinically confirmed NS or suspected of having NS without age limitations, and the sample size of each included study was not limited; (3) the reference standard was reported; (4) the number of study subjects with true positive, false positive, false negative, and true negative results regarding the diagnosis of NS were directly reported or were able to be calculated by other indicators, including sensitivity, specificity, positive predictive value, negative predictive value, and accuracy. The exclusion criteria were as follows: (1) patients who were treated for NS; (2) the aforementioned indicators were unable to be extracted; (3) duplicated publications; (4) conference abstracts, systematic reviews, summary articles, comments, animal studies; (5) articles not in English.

Based on a predefined form, two researchers independently carried out data extraction. Information on the primary author’s name, year of publication, country, study design, age, gender, reference standard and type of laboratory diagnostic modalities, and an absolute number of TP, FP, FN, and TN were extracted. The revised tool for the quality assessment of diagnostic accuracy studies (QUADAS-2) was used to evaluate the quality of each eligible study [20]. The entire scale constituted four domains for the risk of bias, patient selection, index test, reference standard, and flow and timing. Moreover, there were three domains for applicability concerns: patient selection, index test, and reference standard. Each domain was classified into 3 degrees of bias: low-risk, unclear risk, and high-risk bias.

The traditional diagnostic meta-analysis was conducted firstly for studies utilizing different diagnostic modalities. The pooled sensitivity, specificity, positive likelihood ratio (+LR), negative likelihood ratio (−LR), diagnostic odds ratio (DOR), AUC of SROC, and their respective 95% confidence intervals (CIs) were calculated by the Stata version 16.0 software (StataCorp, College Station, TX, USA). An AUC <0.7 represents a poor diagnostic ability. The random effects models were used for the pooled analysis. Heterogeneity among studies included was detected using the I² static; an I² statistic value of >50% was of statistical heterogeneity [21]. Deek’s funnel plot was used to test potential publication bias in studies included [22].

Secondly, a Bayesian NMA was conducted using R studio (4.1.2, R Foundation, Vienna, Austria) to compare the diagnostic performance of different diagnostic modalities. Each analysis was based on noninformative priors for effect sizes and precision. We checked and confirmed convergence and lack of autocorrelation after 2 chains and a 1,000 simulation burn-in phase. Eventually, direct probability statements were derived from an additional 10,000 simulation phase [23]. Posterior estimates of absolute sensitivity, absolute specificity, diagnostic odds ratio, relative sensitivity, relative specificity, and their corresponding 95% credible intervals were calculated by the two-way analysis of variance (ANOVA) model from all the available data [24]. The superiority (S) index was utilized to test the superiority of a diagnostic test. The value of S ranges from 0 to 1. S tends to 1, and S tends to 0, indicating that the increase of the degree of an index test is superior and inferior, respectively. And S tends to 1, suggesting the tests’ performance tends to be equal [25].

A total of 1,271 records were identified through a comprehensive database search. After removing 327 duplicates and 447 ineligible articles, 497 papers were screened by titles and abstracts. Eleven citations (3,115 participants), including 26 studies, were enrolled in the final analysis after a full-text search. The flow of the literature search is shown in Figure 1; the year of publication ranged from 2000 to 2021. Studies were conducted in China, Austria, Portugal, Canada, France, and the USA. Laboratory tests employed to diagnose NS included Albumin quotient, CSF-TPHA, CSF-EIA, CSF-LDH, CSF-WBC, CSF-CXCL13, FTA-ABS, CSF-PCR, RPR, CSF-TPPA, TRUST, and CSF-VDRL, their respective numbers of studies were 1, 1, 1, 1, 1, 2, 1, 3, 2, 4, 2, and 7. The baseline information for each study included is detailed in Table 1. The studies’ quality was judged as high according to the QUADAS-2 criteria. Details of the Cochrane risk of bias assessment of included studies are shown in Figure 2.

Traditional pooled analysis was performed only in studies investigating CSF-TPPA and CSF-VDRL based on the actual number of studies regarding each diagnostic modality. The pooled estimates of sensitivity, specificity, +LR, −LR, DOR, the area under the curve of SROC, and their respective 95% CIs for CSF-TPPA were 0.97 (0.17, 1.00), 0.84 (0.62, 0.95), 6.2 (2.5, 15.6), 0.03 (0, 4.25), 186 (2, 15,818), and 0.93 (0.91, 0.95). As for CSF-VDRL, the pooled sensitivity, specificity, +LR, −LR, DOR, the area under the curve of SROC and their respective 95% CIs were 0.74 (0.59, 0.85), 0.99 (0.93, 1.00), 102.2 (9.0, 1,160.2), 0.26 (0.16, 0.44), 387 (28, 5,372), and 0.94 (0.91, 0.96), respectively (Fig. 3, 4). Deeks’ funnel plot asymmetry test for publication bias yielded p values of 0.051 and 0.518 for CSF-TPPA and CSF-VDRL, respectively (Fig. 5).

As compared to TRUST, CSF-TPHA demonstrated the highest relative sensitivity of 1.35 (0.92, 2.15), and CSF-EIA showed the second highest relative sensitivity (1.30 [0.66, 2.12]). Concerning relative specificity, CSF-VDRL manifested the highest (1.05 [0.91, 1.48]). CSF-TPHA and TRUST both showed the second highest. In addition, CSF-TPHA, TRUST, CSF-VDRL, CSF-EIA, and RPR ranked as the top five laboratory tests with superiority index; their corresponding values of superiority index were 15.98, 6.03, 5.15, 4.14, and 3.50. Posterior estimates and their respective 95% credible intervals, as estimated by the ANOVA model from all the available data, are shown in Table 2.

For decades, many investigations have evaluated the diagnostic performance of different laboratory tests for NS [11, 17, 26‒34]. The results of these studies are heterogeneous. Besides, the number of laboratory modalities compared in every study was limited. Evidence of the superiority of these tests is scarce. DTA-NMA is a novel meta-analysis technique that allows multiple diagnostic tests to be interpreted in a single analysis and compares numerous screening techniques without head-to-head comparisons [35‒37]. It gives access to compare both direct and indirect evidence [38]. NMA shows further development in evidence synthesis, particularly in the decision-making field. It combines data from random comparisons to offer an internally consistent estimation and realizes randomization in the evidence [39]. In this study, we performed a diagnostic test accuracy network meta-analysis by collecting the published evidence to assess and rank the modalities used to diagnose NS.

Results demonstrated that 12 laboratory tests were evaluated in this work, including Albumin quotient, CSF-TPHA, CSF-EIA, CSF-LDH, CSF-WBC, CSF-CXCL13, FTA-ABS, CSF-PCR, RPR, CSF-TPPA, TRUST, and CSF-VDRL. Based on the number of studies about different tests, studies investigating the diagnostic performance of CSF-TPPA and CSF-VDRL were analyzed by traditional meta-analysis. The pooled sensitivity and specificity of CSF-VDRL were 0.74 (0.59, 0.85) and 0.99 (0.93, 1.00), respectively. Current NMA showed similar outcomes of absolute sensitivity and specificity of CSF-VDRL, which indicated that as the only acceptable nontreponemal assay that can be performed on CSF to diagnose NS [40, 41]. Our results align with the conventional clinical experience that CSF-TPHA is highly sensitive, and CSF-VDRL is a better choice for specificity.

As other researchers have described, a negative CSF-TPHA would rule out the possibility of NS, but a positive CSF-TPHA is insufficient to confirm the diagnosis. For CSF-VDRL, clinical practitioners often use it to ascertain the detection of NS. Chow recommends that using a higher titer cut-off greater than 1:320 for the CSF-TPPA may improve the utility of the TPPA as a supporting criterion for clinicians in diagnosing NS [16]. The increased specificity of the higher cut-off value and the higher sensitivity may lead to a single test superior to CSF-VDRL, at least for patients with NS diagnosed for the first time. CSF-TPPA could be utilized if CSF-VDRL is negative in a patient with clinically high suspicion for NS. Results of this NMA suggested that a single test is of unsatisfactory performance in diagnosing NS. Combined methods of different tests remain recommended based on patients’ baseline characteristics and the availability of testing modalities in clinical practice.

Furthermore, this network meta-analysis manifested that CSF-TPHA, TRUST, CSF-VDRL, CSF-EIA, and RPR ranked as the top five superior laboratory tests in detecting NS. Major clinical guidelines recommend that the diagnosis of NS depends on a reactive CSF-VDRL test in combination with high CSF cell counts and an elevated CSF protein [42‒44]. Usually, laboratory tests vary in different medical institutions, so the aforementioned alternative tests would work when CSF-VDRL is not available in some cases. Furthermore, CSF collection through lumbar puncture is an invasive approach, which makes it unacceptable for patients with mental symptoms, lumbar spine diseases, or lumbar dislocation [45, 46]. For these patients, clinical manifestations and results of blood tests may provide a reference to the diagnosis of NS for practitioners in the clinical setting.

In this study, we performed a systematic database search to gather as many studies on the diagnosis of NS using laboratory modalities as possible. Two independent researchers screened citations, extracted data, and assessed the enrolled studies’ quality to minimize subjective bias and error. Both traditional and network meta-analyses were employed for data synthesis and comparison of different tests. The two-way ANOVA model was used to consider the relationship between sensitivity and specificity in each study. Limitations of this work should be acknowledged. First, heterogeneity was regarded as high in the traditional meta-analysis, we planned to perform subgroup analysis based on baseline parameters, but the number of studies was too limited to complete this analysis. Second, reference standards utilized in included studies were heterogeneous. The included studies spanned an large time range during which the diagnostic criteria for NS changed, and studies were designed slightly differently between regions according to the guidelines or consensus of the studied area. Four studies used European guidelines published in different years [42, 47, 48], 2 used CDC guidelines released in 1998 and 2005, respectively [49, 50], and 1 study referred to Chinese policies [51]. Four studies did not specify the guidelines they referred to. Nearly half of the included studies were conducted in China, which may cause potential bias; interpreting the pooled results should be cautious. Third, the minimum titer level in any of the tests included was reported by two studies; more studies are needed to clarify the cut-offs in different laboratory tests for NS. Finally, data on the performance of diagnostic tests among patients with NS and HIV coinfection were not separately extractable. Therefore, pooled data analysis could not be performed to quantitatively assess the diagnostic value of these laboratory modalities in these patients. Despite the limitations of the current meta-analysis, this work provides evidence for researchers and practitioners in the necessity of improving the performance and significance of laboratory tests in diagnosing NS.

In conclusion, laboratory tests such as CSF-TPHA, TRUST, CSF-VDRL, CSF-EIA, and RPR show acceptable performance in the detection of NS, comprehensive diagnostic strategies involving clinical symptoms, disease history, results of combined use of different laboratory modalities are still essential in establishing the diagnosis of NS, their significance in the diagnosis of NS should further be evaluated.

Since this article is a systematic review and does not involve human studies, ethical approval is not required.

The authors declare that they have no competing interests.

This work was supported by the National Key Research and Development Program of China (2016YFC1306300, 2016YFC1306000) and the National Natural Science Foundation of China (81970992, 81571229, 81071015, 30770745).

Junhua Gao and Duyu Ding performed the citation screening, data extraction, quality assessment, and statistical analysis and drafted the manuscript. Wei Zhang and Dongmei Xu made suggestions and revised the manuscript. All authors read and approved the final manuscript. We have consulted a professional statistician, Wan Gang, in our mathematical analysis.