Deep Learning-Based Algorithm for Staging Secondary Caries in Bitewings

While the first choice for detecting caries around restorations, defined as secondary caries in a recent consensus paper [1], is the visual-tactile examination [2, 3], adjunct diagnostic methods can be helpful in some situations [3]. The treatment decision for secondary caries lesions should consider the activity status and severity, determined by the presence of cavitations and lesion depth [4]. Although visual-tactile examination is an accurate method to evaluate the activity status and presence of cavitation, it is not optimal for estimating a lesion’s depth [4]. The radiographic method through bitewings is more valuable for this task [2, 4‒7], although it tends to underestimate the actual mineral loss, especially for initial caries lesions [4].

Some guidelines have proposed integrating the detection and staging of caries lesions by combining visual inspection and radiographic examination to enhance the decision-making process related to caries management [8, 9]. While detecting lesions is the first step in the diagnostic strategy for dental caries, assessing the activity status and the severity of the lesions (staging) is essential for determining the best treatment decision. Nevertheless, the practical application of these frameworks still faces challenges as the process is significantly influenced by the clinician’s expertise and the quality of the radiographic image [5, 7]. To overcome this subjectivity, dental support systems for caries detection based on artificial intelligence (AI) have been proposed as advancement in dental diagnostics [10, 11].

Considering the radiographic method, automated algorithms could provide a less subjective and more accurate diagnosis, adding essential information to the clinical diagnosis and decision-making performed by practitioners. However, challenges encountered in conventional interpretation can also be found in automated detection [11, 12]. These systems identify caries by recognising radiolucent patterns, sharing human difficulty in discerning subtle differences related to dental caries [12]. Nevertheless, the algorithm’s ability to learn from vast datasets enables its continuous refinement, making it increasingly adept at navigating the complexities of dental radiographs.

However, most developments of AI-based algorithms for caries detection are focused on primary caries [11]. Moreover, most have not considered different thresholds for caries detection or lesion staging. We recently developed a CNN-based algorithm with high accuracy in identifying primary and secondary caries [13]. Based on this previous development, we aimed to refine our CNN-based algorithm by improving its ability to detect secondary caries at different thresholds, especially staging the lesions using the radiographic images. We hypothesised that this refinement would provide a more detailed and accurate assessment of caries lesions in varied stages using bitewings, making this algorithm more applicable and valuable in supporting dentists in clinical practice. The main innovation of the present study was the two-stage training employed. First, we focused on strengthening the algorithm’s detection capabilities by incorporating adjacent anatomical structures. Then, we intended to refine the algorithm’s capacity to determine the extent of caries lesions, including progression into dentine. Our ultimate goal was to offer a tool more closely related to decision-making and managing dental caries.

The study adhered to the ethical standards of research, followed the World Medical Association’s Declaration of Helsinki, and adhered to the AI checklist for dental research [14]. Ethical approval for using image data in this research was obtained from the CMO Arnhem-Nijmegen Ethics Committee, under file number CMO 2015-1565. The participants older than 18 signed a written informed consent form to participate in the study. For subjects under the age of 18 years, their parents or legal guardians signed the informed consent. All pertinent project details, including the algorithm’s code, have been made available on the Open Science Framework [15].

In our initial experiment [13], the model demonstrated promising capacities. For clarity, we briefly summarise here: it was able to identify teeth, restorations, residual caries, and primary/secondary caries lesions. For this report, the used dataset comprised bitewing images used in previous research [16]. The bitewings were sourced from a Dutch dental practice-based research network. They were selected among radiograph images from patients who received dental care between January 2015 and January 2017, including all types of restorations. Only caries lesions on occlusal or interproximal surfaces were included. A total of 413 bitewings from 383 patients aged 15 to 88 were analysed (median = 46 years old). The sample included 202 females, 179 males, and 2 with unspecified gender.

The inclusion criteria were images that included amalgam, composite resin, glass ionomer restorations or ceramics; only occlusal or interproximal primary or secondary caries lesions; bitewings with sufficient quality to allow unambiguous identification of restorations and caries lesions, images with high contrast, proper exposure, and no significant motion blur. The exclusion criteria were radiographs with substantial distortions, artefacts, or unsuitable image orientations. Moreover, duplicate images were removed to ensure the uniqueness and integrity of the dataset. Each bitewing was recorded using the available imaging device of the dental practice, converted to JPG format, and anonymised before analysis. The comprehensive application of this dataset and further details have been described elsewhere [13, 16].

For the present study, we meticulously annotated secondary caries on a pixel-by-pixel basis using the Darwin Platform (V7 Labs, London, UK). The annotation process followed a conservative approach; dubious images (i.e., those where consensus was not reached after discussion) were not annotated as dental caries. Building upon this groundwork, we expanded our analysis to include a detailed annotation and labelling of teeth, restorations, and caries lesions. Every tooth was annotated for any existing restoration, adhering to the FDI two-digit numbering system [17] and encompassing fillings, single crowns, and fixed bridges. Caries lesions were classified into residual caries (probably due to selective excavation) and primary or secondary caries. The presence of residual caries was considered when the radiolucency was beneath the restoration but had not reached the margins. Moreover, primary caries lesions were identified to distinguish caries lesions unrelated to restorations. Additionally, for each primary/secondary caries lesion, two critical points were identified to specify the lesion’s severity: the lesion’s entry point at the enamel-dentine junction and the nearest point to the pulp, further refining the staging of the lesion. When the enamel-dentine junction was not directly visible due to restorations, the visible adjacent areas and anatomical landmarks were used to estimate the entry point of the lesion.

The annotation process was collaboratively performed by two PhD students, who are currently developing research on AI-based identification of structures in bitewings (ETC and LC, 5 and 4 years of clinical experience, respectively). They independently evaluated the teeth, restorations, and caries lesions. The agreement between the two annotators was recorded, and in case of discrepancies, they reached a consensus on their findings. Then, caries lesion annotations were further revised by two senior dentists in secondary caries detection and management (FMM and MSC, 28 and 23 years of clinical experience, respectively). Discrepancies among the experts and the first annotators were resolved through discussions in joint sessions, adopting a consensus-based strategy for the classification.

Subsequently, the critical points for each primary/secondary caries lesion were independently determined by a dentist (VHDR, with 5 years of clinical experience) and an expert in caries detection (MSC). In instances of significant discrepancy, the points were thoroughly revised to achieve consensus. In other cases, the points were averaged, ensuring the accuracy of our dataset.

This study utilised the Mask R-CNN architecture with a Swin Transformer backbone [18, 19]. The setup was implemented using MMDetection (version 3.1.0) based on PyTorch 2.1.0 [20, 21]. Mask R-CNN efficiently predicted object masks with corresponding classifications (e.g., primary/secondary caries), with the Swin Transformer initially processing the bitewing image to extract features. Following this, a region proposal network identified bounding boxes around potential objects. The ROI align method standardised the features within each identified region to a uniform size, producing three distinct outputs: a class label, a bounding box, and an object mask. The Swin Transformer leverages self-attention mechanisms within specific windows and shifts these windows into subsequent layers to adeptly model long-range dependencies [19].

As previously mentioned, a two-stage training approach was adopted to optimise the detection of caries lesions. The first stage predicted all the annotated objects and classified them into tooth, filling, crown (including fixed bridges), residual caries, and primary/secondary caries. The second stage focused on detecting primary/secondary caries and estimating a lesion’s severity, ignoring residual caries.

The AdamW optimizer, with a learning rate of 5 ∙ 10⁻⁵ and a weight decay of 0.1, was utilised [22]. The Swin Transformer tiny variant, pre-trained on the COCO dataset [23], was chosen for initial parameters. Training occurred over 24 epochs with a mini-batch size of 4, reducing the learning rate after the 20th epoch. Data augmentations applied horizontal flipping, cropping, and resizing.

The training was structured by dividing all bitewings into ten subsets for tenfold cross-validation, with 80% for training, 10% for validation, and 10% for testing. This configuration was identical for both stages of the training approach and aimed to optimise the model’s parameters on the training set, fine-tune hyperparameters on the validation set, and accomplish performance assessments on the hold-out testing set.

Computations were executed on a workstation with a high-capacity RTX A6000 48GB (NVIDIA Corporation; Santa Clara, CA, USA) with 128GB memory.

Secondary caries lesions were distinguished from primary caries lesions in restored teeth by determining whether they intersected with a restoration annotation. Restored teeth annotated with caries lesions were considered decayed for the reference standard, and restored teeth that were not annotated were classified as sound. Subsequently, reference and detected caries lesions were matched to tooth annotations to determine whether each restored tooth had a secondary caries lesion. The accuracy parameters of the tooth classifications based on the reference annotations were presented by calculating specificity, sensitivity, precision, F1-score, and area under the receiver operating characteristic curve (AUC), considering the mean and standard deviation obtained from tenfold cross-validation.

These values were derived considering two different thresholds: all secondary caries lesions (no caries vs. initial, moderate, and severe lesions) and secondary dentine caries lesions (no caries and initial lesions vs. moderate and severe lesions). For this latter approach, dentine caries lesions were defined as moderate to severe, with a lesion severity score ≥0.33, similar to the approach previously described [24]. The global accuracy values obtained through AUC were numerically compared with the results obtained with the preliminary development of this algorithm [13].

The inter-examiner variability in determining the lesion severity scores was evaluated through Pearson correlation coefficient (PCC) and non-parametric Bland-Altman plots. We also used this approach to evaluate the validity of the lesion severity scores obtained by the examiners’ consensus and the results obtained with the algorithm. Reference caries lesions missed by the model (false negatives) were excluded only from this analysis, whereas the accuracy parameters were calculated considering all secondary caries lesions.

A total of 2,612 restored teeth were present. The annotators presented an excellent agreement in scoring the severity of secondary caries (PCC = 0.908). Bland-Altman and scatter plots showing this agreement are available in the online supplementary material (online suppl. Fig. S2A, B).

Considering the restored teeth included in the sample, 2,225 (85.2%) presented no secondary caries in any stage, 149 (5.7%) had initial lesions, 175 (6.7%) had moderate lesions, and 71 (2.7%) had severe lesions. Restored teeth in our sample showed a prevalence of secondary caries lesions of 14.8% in the radiographs and 9.3% for secondary caries lesions reaching the dentine.

Most results were true negatives. When considering the all-lesions threshold, a higher number of false negatives than false positives was observed (Fig. 1a). Conversely, for detecting dentine secondary caries lesions, the number of false positives was higher than false negatives (Fig. 1b).

The CNN-based algorithm performed well in detecting secondary caries at both thresholds. The AUC for detecting all caries lesions was 0.940 (Fig. 2a), and considering only more advanced caries lesions, the tool reached an AUC of 0.946 (Fig. 2b). Both values were higher than those obtained in the preliminary development of the algorithm for secondary caries [13]. The sensitivity for detecting all secondary caries lesions was about 0.74, which increased to 0.81 when considering only dentine caries lesions. For both thresholds, specificity was around 0.96, and the F1 score was around 0.75 (Table 1).

Regarding the staging of secondary caries lesions, a good agreement between the consensus severity scores and the severity predicted by the algorithm was obtained (PCC = 0.802, Fig. 3b). The Bland-Altman analysis also showed a good agreement, with no systematic difference between the methods (mean difference = −0.024, 95% confidence interval = −0.257–0.308). In the Bland-Altman plot, we can observe a slight increase in the discrepancies for more severe caries lesions. Moreover, we could see a trend for a negative difference for shallower lesions and a positive difference for deeper lesions. The regression line in the plot demonstrates this tendency (Fig. 3a).

Figure 4 illustrates 3 cases where the model accurately detected caries lesions. Notably, in the second case, the model correctly dismissed residual caries in tooth 27. Moreover, in the cases presented in the first and second rows, we can observe the ability of the algorithm to differentiate between secondary and primary caries lesions. The predicted severity scores closely matched the consensus scores, confirming the model’s accuracy in staging caries lesion severity. Conversely, Figure 5 depicts three instances of the model’s wrong predictions. The first case presents a false-negative prediction, as the model misses the caries lesion in tooth 36. In the second case, the model underestimated the area affected by caries in tooth 36, resulting in a lower severity score (1.0 vs. 0.66). Lastly, a false-positive prediction can be seen in case 3, where the model confused a radiolucency due to missing material with a caries lesion. More examples showing the ability of the algorithm to detect lesions with different stages, as well as to distinguish between secondary and primary caries lesions, are presented in the online supplementary material (online suppl. Fig. S3).

The development of AI-based algorithms for caries detection has been extensive and is expected to result in a transformative shift in clinical practice in the coming years. A collaborative approach between practitioners and AI-based tools will provide faster, more reliable, and more accurate diagnostic strategies [11, 25]. However, the vast majority of published studies have focused on primary caries detection [11]. In this study, we refined a previously developed AI-based tool dedicated to detecting secondary caries [13], reaching a notable improvement in the accuracy parameters. Moreover, we aggregated tools for staging the lesions around restorations. These advancements have validated our initial hypothesis, presenting the updated algorithm as a more sophisticated and accurate tool for detecting and staging secondary caries lesions. This system has a more significant potential to be more helpful in supporting practitioners as it is more closely aligned with treatment decision-making in clinical practice.

To our knowledge, one previous study from another research group developed a system that included the assessment of secondary caries, achieving an area under the precision-recall curve of 0.96 [26]. However, the authors did not include sound teeth to calculate the accuracy parameters [26], impairing the comparability to our findings. Considering the accuracy obtained in our study (higher than 0.90 considering both thresholds), our algorithm showed comparable validity with previous studies using algorithms for detecting primary caries in bitewings, which have presented accuracy values varying from 0.88 to 0.95 [11]. This finding reinforces the good performance obtained with the improvement of our algorithm since the detection of secondary caries lesions is more challenging than that of primary caries lesions. Moreover, the automated detection of secondary caries with our tool was more accurate than that performed by dentists [27].

Diagnostic strategies for caries must distinguish the different stages of lesions to aid practitioners in reaching better management alternatives. While initial caries lesions with no cavitation typically require no treatment or non-operative treatment, more advanced lesions usually need operative treatments [28]. Therefore, the researchers involved with automated caries detection tools should consider different thresholds when developing their models. Recent studies have addressed this point for primary lesions. A previous study sub-grouped primary caries into enamel, early dentine, and advanced dentine, demonstrating that dentists supported by AI significantly increased their sensitivity, particularly for enamel caries, without substantially affecting their specificity [29]. In contrast, our previous study focuses on secondary caries detection with a CNN-based algorithm [13]. In the present manuscript, we integrated a continuous outcome measure to assess the severity of secondary caries lesions, improving the method’s accuracy at different thresholds.

Another positive finding is the high correlation between the algorithm’s classification of scores related to caries lesion severity and the ground truth. We used an innovative method to train our model for staging caries lesions, considering measurements manually made by the annotators. In this study, we implemented a consensus approach, engaging specialists to ensure more precise staging annotations. This strategy enhances the training dataset and bolsters the algorithm’s diagnostic accuracy [30]. Achieving a high inter-rater agreement lays a solid foundation for the algorithm’s learning process, minimizing the risk of incorrect diagnoses due to variable training data. Moreover, the variation in training methods across different algorithms highlights the need for transparency in disclosing annotation processes and concordance metrics. Establishing guidelines or frameworks for annotating caries lesions in bitewings could significantly contribute to standardizing these processes, ensuring reliability and enhancing the comparability of AI-supported diagnostic tools.

Another important innovation of our study was the new protocol for determining lesion severity with continuous outcome. A severity score from 0 to 1 was defined as the depth of a caries lesion into dentine based on a lesion segmentation and two critical points (online suppl. Fig. S4). This objective measurement was shown to be reliable with a high inter-annotator agreement. Although we have observed a trend for an overestimation in determining the lesion depth for shallower lesions and an underestimated and less accurate prediction for deeper lesions, these differences were not clinically relevant. They would probably not affect the diagnosis of secondary caries in clinical practice. Caries lesions have been staged in the current clinical practice based on the practitioner’s experience and subjective assessment by assigning an ordinal category, such as initial, moderate, or severe. Introducing objective measurements and continuous outcomes for diagnosis can improve the reliability of diagnosis between dentists.

This achievement is more noteworthy, considering our evaluations focused on secondary caries. Factors related to the opacity of restorative materials, overlapping of restorations and dental surfaces, and possible confounders, such as gaps between restoration and tooth, presence of bond layer, or residual caries, may hamper an accurate diagnosis [2, 10]. By concealing or exaggerating the appearance of dental lesions, these factors significantly complicate the accurate assessment and diagnosis of secondary caries. Our model has shown great potential in distinguishing between actual caries lesions and confounders, notably by its ability to rule out primary caries in restored teeth and even residual caries.

The developed automated tool reached high overall accuracy values, with AUCs higher than 0.9. Moreover, the algorithm presented high specificity values, which is crucial considering the current paradigm for minimally invasive dentistry [4]. Moreover, we presented separate results that considered different thresholds for caries severity. While the performance of a caries detection method in detecting all stages of caries lesions is essential to indicate which surfaces require more attention, accurate detection of lesions reaching dentine is essential for the clinician to decide between a non-operative or operative treatment. In the present study, suitable accuracy parameters were observed for both thresholds. Noteworthy, automated detection algorithms developed with radiographic images should be considered as an aid in clinical practice. Clinicians must keep visual inspection, including evaluation of activity status, as the driving examination for decision-making [5].

Despite the promising results, we should recognise some possible limitations. As previously mentioned, we developed the algorithm using a convenience sample selected from radiographs taken in general dental practices in the Netherlands, with at least one restored tooth and including secondary caries with different stages of severity [13, 16]. In this convenience sample, the prevalence of secondary caries was about 15% and 9%, considering all lesions and only dentine lesions, respectively. The occurrence of this condition is usually retrieved from clinical studies and is largely varied, ranging from 0 to 44% [31]. In an observational study with a representative sample, the prevalence of secondary caries was 3.6% [32]. Therefore, caries lesions in our sample were probably overrepresented, even considering that the images were taken in the dental office setting. Another point is that, although our sample size is relatively limited compared to other studies [11], we could achieve good performance with our automated tool. However, new experiments, such as integrating bitewings from other datasets through a federative learning approach [33], are desirable to improve the algorithm’s validity further.

Moreover, the AI tools reflect the training by a few examiners (annotators), who could have common concepts for caries diagnosis. However, the several steps employed in our training process, involving many examiners with different expertise, can minimise this possible limitation, making our algorithm more accurate than the diagnosis made in the clinical practice. Another limitation is that our model was not yet trained to account for possible confounding factors of secondary caries.

Further studies could improve the algorithm’s accuracy, including threshold adjustments and integrating additional features related to restorations and radiographic caries confounders, such as marginal gaps, restoration overhangs, and cervical burnouts. Furthermore, it is fundamental to test our algorithm across diverse datasets and clinical contexts to assess its generalizability and real-world applicability. The industries interested in this field should have this point in mind, ensuring these AI tools are commercially available only after robust accuracy studies have been conducted by independent research groups and longitudinal research has been performed evaluating the potential benefits for patients.

In this study, we developed an algorithm to support clinicians in detecting secondary caries in bitewings. We incorporated an innovative approach for annotation, training the model to consider the lesion severity as a continuous outcome. This procedure improved the algorithm for detecting and staging the severity of secondary caries lesions.

We acknowledge the valuable support and contributions to this research by Radboud Dental AI Hub.

The ethical approval for using image data in this research was obtained from the CMO Arnhem-Nijmegen Ethics Committee (number CMO 2015-1565). The participants older than 18 signed a written informed consent form to participate in the study. For subjects under the age of 18 years, their parents or legal guardians signed the informed consent.

The authors have no conflicts of interest to declare.

This study received only the support of the author’s institutes and human and facilities resources. N.N., S.V., and T.X. are partially supported by Radboud AI for Health. E.T.C., V.H.D.R., and G.S.L. are partially supported by UFPel, Radboudumc, and Coordenação de Aperfeiçoamento de Pessoal de Ensino Superior (CAPES PRINT #88887.363970/2019-00). M.S.C., L.C., B.A.C.L., and M.-C.H. are partially supported by Radboudumc. F.M.M. is supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, Grant No. 2022/16528-3).

N.N., E.T.C., M.S.C., B.A.C.L., G.S.L., M.-C.H., S.V., and F.M.M. conceptualized the study design. E.T.C., L.C., M.S.C., and V.H.D.R. annotated the images. N.N., T.X., T.F., B.G., K.E.G., and S.V. worked on the model development and analysed the data. F.M.M., E.T.C., N.N., and M.S.C. drafted the original manuscript. All authors revised the manuscript critically for important intellectual content and read and gave the final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Niels van Nistelrooij, Eduardo Trota Chaves, Shankeeth Vinayahalingam and Medeiros Mendes contributed equally to this work.

The protocol of the project and the codes used for the algorithms are publicly available on the Open Science Framework (OSF) platform: https://doi.org/10.17605/OSF.IO/27E4Q.