Does Machine Learning Prediction of Magnetic Resonance Imaging Prostate Imaging Reporting and Data System Correlate with Target Prostate Biopsy Results? Open Access

Prostate cancer (PCa) is the fourth most prevalent cancer globally and the second most frequently diagnosed cancer among the men [1]. The standardized incidence rate of PCa is higher in Western countries than in the Middle Eastern and Arab countries [2]. However, the incidence rate has increased in the Gulf region over the last 10 years [3]. Magnetic resonance imaging (MRI) was initially employed in 1982 for the examination of prostate cancer. With advancements in technology and the integration of functional parameters, the precision of MRI has significantly improved, leading to the widespread adoption of multiparametric MRI for evaluating the prostate cancer risk. The Prostate Imaging Reporting and Data System (PI-RADS) V2 was developed to categorize MRI results on a scale of 1–5, indicating the probability of clinically significant prostate cancer (csPCa) [4]. Numerous studies have confirmed the effectiveness of MRI in diagnosing prostate cancer, demonstrating its high sensitivity and strong negative predictive value in clinically significant conditions [5, 6]. It provides an increasingly dependable visualization of potentially significant prostate cancers, thereby offering advantages in more accurately selecting patients for biopsy [7]. Nonetheless, PI-RADS has certain limitations due to imaging mimics and pitfalls that can be mistaken for prostate cancer, resulting in unavoidable false positives and unnecessary biopsies [8]. Additionally, there is growing interest in integrating radiomics, which extracts quantitative features from medical images, into the multi-omics framework to enhance the understanding of prostate cancer at the biomolecular level. Radiomics can help identify relationships between imaging features and clinical outcomes, utilizing modalities such as MRI, transrectal ultrasound, CT, and PET/CT using tracers such as radiolabeled prostate-specific membrane antigen and 18F-choline [9].

Machine learning (ML) is a branch of AI that focuses on learning by generating the best algorithms that represent the dataset in question or simply investigating the ability of computers to learn or improve their performance based on data [10, 11]. This is one of the most important directions for modeling along with the statistical modeling [12]. ML divides the datasets into two groups, training and validation datasets. Training datasets were used to generate the algorithm, and its validity was tested on the validation dataset to increase the chance of generalizability of the algorithm to other similar datasets. Four learning methods are commonly used under the umbrella of ML: supervised, unsupervised, semi-supervised, and reinforcement learning [10]. The main objective of the current study was to determine whether ML algorithms have the capability of predicting and classifying MRI PI-RADS accurately and whether there is concordance between the PI-RADS scores and the outcome of prostate biopsy.

Our study included all consecutive patients with a PSA level of ≥3.5 ng/mL who underwent prostate biopsy, either through transperineal (TPUS) or transrectal (TRUS) ultrasound guidance, at the two primary hospitals in Riyadh, Saudi Arabia (King Khalid University Hospital and King Faisal Specialist Hospital) between 2019 and 2023.

Prostate multiparametric magnetic resonance imaging (mpMRI) was performed on patients using a Siemens A 3 Tesla magnet Skyra system (Erlangen, Germany) along with an external multichannel body phase array coil. The MRI procedure included the following steps: (i) axial and coronal T2-weighted fast spin echo sequences (TSE, ETL 25) with 3 mm slice thickness, TR/TE: 5,540/107; (ii) axial diffusion-weighted high-resolution sequence, readout segmented EPI (RESOLVE), with 3 mm slices and ADC maps (including quantitative ADC evaluation), TR/TE: 5,250/62; (iii) the axial T1-weighted 3D gradient echo sequence for dynamic contrast-enhanced MRI featured a 3.5 mm slice thickness, 1,922 matrix, TR/TE/FA of 4.9/6.7/150, 10-s temporal resolution, 40 time points, and a bolus injection of 0.1 mm/kg Gd-DOTA iv. The axial T1-weighted fat-suppressed sequence with a gradient echo sequence, 3.5 mm slice thickness, TR/TE: 3.5/1.5, and FOV = 240 mm was utilized for late post-contrast imaging of the pelvis and node.

The Artemis system, developed by Eigen in Grass Valley, CA, USA, along with its software, received an electronic update incorporating the MRI region of interest in accordance with the procedures implemented in the two hospitals. Two to three cores were obtained from the target lesion, depending on its size, before each patient underwent a systematic 12-core procedure. Three radiologists interpreted the MRIs scans in both hospitals using the same protocol, and a senior radiologist at each hospital reviewed and verified the positive results. Data on patients with PSA levels and suspected prostate cancer following MRI and biopsy were retrieved from the records, including age, BMI, PSA level, PSA density, prostate volume, MRI PI-RAD scores, number of lesions, and diameter of the largest lesion.

The Gleason grading system was modified in 2005 because of significant changes [13]. The outcome of the target biopsy was categorized into three categories: biopsy-negative (cancer-free), non-significant PCa (GS <3 + 4), and significant PCa (GS ≥3 + 4).

The PI-RAD scoring system includes five scores (1: very low, 2: low, 3: intermediate, 4: high, and 5: very high). Categories with small sample sizes can lead to unreliable coefficient estimates and violations of model assumptions, making it difficult to detect effects. Therefore, combining them can make the model more robust. For these reasons, a score of “0” was assigned to patients with very low and low PI-RAD scores, while a score of “1” was assigned to those with intermediate, high, or very high PI-RAD scores. This combination is based on data exploration and model diagnostics. The data were split into two sets: one for training and the other one for testing. We allocated 80% of the data to the training set and 20% to the testing set (Fig. 1).

In total, distinct machine learning models were developed, encompassing XGBoost, Logistic Regression, Support Vector Classification (SVC), Decision Tree, Random Forest, Extra Trees, AdaBoost, and Gradient Boosting. The models were constructed using features such as age, body mass index (BMI), prostate volume (as determined by MRI), total PSA (tPSA), PSA density, lesion count identified on MRI, and size of the largest lesion. Hyperparameters for all models were fine-tuned using the Grid Search method, and 5-fold cross-validation was utilized to assess the accuracy of each algorithm. The grid parameters varied across the different models. Additionally, a consistent random seed was used to ensure identical training and validation sets across the various ML algorithms. The models’ performance on the test dataset was evaluated using metrics such as accuracy, sensitivity, specificity, F1-score, and area under the curve (AUC). The model that achieved the highest accuracy and average AUC post-training was deemed to be the best. Python 3.10.12 was used to develop the models.

Cross-tabulation was performed to assess the concordance between PI-RAD scores and prostate biopsy outcomes. A chi-square test was used to evaluate concordance. If the test assumptions were violated, Monte Carlo simulations were applied.

Table 1 shows the performance of the eight models in predicting PI-RAD scores. Random Forest and Extra Trees achieved the best performance compared to the other methods. The accuracies of both models were 97.48% and 96.47%, respectively, on the training data, whereas on the testing data, it was 91.95% for both the models. The AUC was nearly 1 for both models on the training data, while on the testing data, it was 0.9329 for the Random Forest model and 0.9404 for the Extra Trees model. The top five AUC values for predicting the PI-RADS score are shown in Figure 2.

The importance of classifying the outcomes of the two selected models has been presented in Figure 3. For the Random Forest model, PSA, PSA density, and diameter of the largest lesion were the most important features, while the number of lesions, diameter of the largest lesion, and age were the most important features for the Extra Trees model. BMI was the least important feature in both these models.

The concordance between the original PI-RAD scores (1–5) and the target biopsy results is presented in Table 1. For low PI-RAD scores (very low and low), all samples with a very low score had a negative biopsy result, whereas for a low score, 91% of the samples were negative, while 9% indicated non-clinically significant prostate cancer (NCSPc). Regarding high PI-RAD scores (high and very high), 28.3% of the samples with a high score were clinically significant for prostate cancer, and 60.9% of those with a very high score were clinically significant (CSPc). Biopsy-negative results constituted less than two-thirds (61.5%) of the PI-RAD 4 cases and one-third (32.7%) of the PI-RAD 5 cases. Monte Carlo test showed a significant difference between PI-RAD score and target biopsy results (p = 0.000).

Table 2 evaluates the distribution of target biopsy results across the combined categories of MRI PI-RAD scores (0 and 1), both before and after applying a machine learning prediction model using the Random Forest (RF) algorithm. Chi-square values were used to assess statistical significance. In Class 0, csPCa cases increased from 0% to 1.9%, indicating that the model identified previously missed cases. In Class 1, csPCa decreased slightly from 32.1% to 30.6%, but the number of negative cases increased from 60.9% to 62.2%. ML model improved risk stratification by reducing false negatives in low-risk while maintaining accuracy in the high-risk cases.

To manage prostate cancer in men, it is crucial to precisely evaluate the presence of clinically significant lesions, determine the extent of the disease at the time of diagnosis, and assess the risk of future progression. This approach helps prevent unnecessary overtreatment in men with a low risk of progression and avoids undertreatment that could lead to treatment failures, particularly for those choosing active surveillance. Furthermore, there is a growing preference for liquid biopsies over tissue biopsies because of their minimally invasive nature and ability to provide a systemic view of the disease. Liquid biopsies, including the analysis of circulating tumor cells, DNA fragments, and exosomes, provide valuable insights into the molecular landscape of prostate cancer. Notably, urinary liquid biopsy is emerging as a promising tool for detecting prostate cancer, with exosomal analysis of blood, urine, and semen showing significant potential as a source of novel biomarkers [13].

In the present study, Random Forest and Extra Tree models showed excellent capability for prediction and discrimination of MRI PI-RAD scores, as detected from the corresponding accuracy, precision, recall, F1-score, and area under the curve (AUC). PSA density, PSA level, diameter of the largest lesion, and number of lesions were the most significant features that contributed to the prediction and classification models.

Identifying clinically significant cancers at an early stage is crucial because it allows patients to receive timely active treatment, which can be beneficial. In the case of treatment, recent studies suggest that for patients with high-risk non-metastatic prostate cancer, abiraterone combined with prednisolone and androgen deprivation therapy (ADT) should be regarded as the new standard. However, for those starting long-term ADT in a metastatic setting, a combination of enzalutamide and abiraterone is not recommended. Importantly, the addition of abiraterone to ADT has shown clinically significant improvements in survival, with benefits maintained for over 7 years [14]. The integration of mpMRI and the PI-RADS has enhanced the process of selecting patients for biopsy and has enabled the use of MRI-guided sampling techniques to improve risk assessment [15]. The MRI pathway is unique in that men with low-probability MRI results do not undergo biopsy, whereas those with higher-probability results receive only MRI-targeted biopsy (excluding systematic cores). The benefit of this MRI approach is that it decreases the number of men requiring biopsies and reduces the total number of biopsy cores in men with high-probability MRI results, thereby minimizing the overdiagnosis of clinically insignificant conditions [16].

Van Leeuwen et al. [17] created a model incorporating factors such as age, DRE, serum PSA, prostate volume, and PI-RADS v1 to predict the rate of csPCa detection. The initial model, which relied on clinical predictors, achieved an AUROC of 0.80. This figure rose to 0.88, with the inclusion of MRI data (p < 0.001). By setting a threshold of 12.5% for csPCa, 34.3% of prostate biopsies were avoided, whereas 6.1% of csPCa cases were missed. In 2017, Bjurlin et al. [18] created two MRI prediction models for csPCa biopsy. The csPCa detection rate was 33.6%. These models used age, PSAD, and PI-RADS v1 score from the mpMRI report as predictors. The AUROCs for detecting csPCa were 0.91 in men who had not previously undergone a biopsy and 0.86 in those with prior negative biopsies; 42% and 34% of prostate biopsies were avoided in these groups, respectively, with a 5% miss rate for csPCa. Lee et al. [19] developed an MRI prediction model incorporating age, biopsy status, PSAD, and biparametric MRI as predictors of csPCa in Essex, UK, from 2012 to 2015. The clinically significant prostate cancer detection rate was 38.5%. The AUROC of MRI increased from 0.87 to 0.92. At a 30% csPCa risk threshold, 34.6% of prostate biopsies were avoided, and 2.5% of csPCa cases were missed. External validation was not performed.

In recent studies, PSA density and lesion diameter were shown to be the most powerful risk predictors of csPCa compared to other variables [20], whereas the number of lesions was found to be irrelevant in predicting PCa [21]. Sigle et al. [22] concluded that age, prior biopsy results, and PSA Density serve as independent indicators of csPCa in men with uncertain prostate MRI findings. Although recent advancements in mpMRI techniques have significantly enhanced the diagnosis and staging of prostate cancer, the sensitivity and specificity of MRI in detecting significant cancers vary across different studies. The accuracy of MRI in diagnosing prostate cancer has been reported to range from 44% to 87%, with sensitivity and specificity ranging from 58% to 96% and 23%–87%, respectively. This broad range of sensitivity and specificity presents a challenge in clinical settings [23]. In the present study, biopsy was negative in almost all cases with PI-RAD 2 (91%) and 100% of cases with PI-RAD 1; on the other hand, it was negative in 61% and 33% of cases with PI-RAD 4 and 5, respectively. csPCa cases were more frequent in patients with PI-RAD 5. According to Stabile et al. [24], clinically significant cancers were found in 8% of PI-RADS 2 lesions and 15% of PI-RADS 3 lesions, respectively.

The concordance of MRI PI-RADS (low vs. high risk) with target biopsy results was examined before and after the application of machine learning, revealing that ML algorithms refined the risk stratification of MRI Pi-RADS. In Class 0, ML identified a small subset of csPCa cases (1.9%) that were previously misclassified, reducing false negatives and avoiding delayed diagnosis. In Class 1, ML slightly reduced csPCa detection (32.1–30.6%) but maintained high-risk classification accuracy. The lower chi-square post-ML suggests that the model redistributed biopsy outcomes, potentially improving specificity in low-risk cases while preserving sensitivity in high-risk cases. However, the persistence of 62.2% of biopsy-negative cases in Class 1 underscores the need for multimodal risk assessment to address PI-RAD’s false-positive limitations of PI-RAD.

Several studies have confirmed the presence of csPCa using PI-RADS scores, with a primary focus on PI-RADS 3, 4, and 5 lesions. Some studies have explored the possibility of safely bypassing prostate biopsy after conducting a pre-biopsy mpMRI. For instance, a pivotal PROMIS study suggested that pre-biopsy mpMRI can eliminate the need for an initial biopsy in 27% of patients [6]. Furthermore, the PRECISION study showed that the MRI-targeted biopsy group had a higher detection rate of clinically significant cancer than the standard prostate biopsy group (38% vs. 26%) [25]. It is important to consider that avoiding TRUS-biopsy after a normal or unclear mpMRI should be approached with caution as 18.5% of cancer cases were found in this category and 9.8% of those diagnosed with csPCa [26].

Mota et al. [27] recently reported false-positive rates of 29% for PI-279 RADS 4 lesions and 3.7% for RADS 5 lesions in relation to any cancer type. Therefore, patients with negative biopsies after being diagnosed with PI-RAD 4 and 5 should be followed up. Kornienko et al. [28] studied 222 patients with PI-RAD 4 and 5 with whom had a negative biopsy outcome; 80% of these were followed up; of them 48% received a repeat MRI, where 46% were downgraded to PI-RADS2 and (13%) to PI-RADS 3.

There is a concern that radiologists’ reports on prostate MRIs might not match the accuracy found in specialized centers. Consequently, reports from less experienced and less trained new radiologists could impact the effectiveness of the clinical practice [29]. The expertise of the reader is a significant factor which can potentially contribute to variability in the study reports and can also influence patient care. Several elements affect the learning curve for interpreting prostate MRIs, including radiologists' level of expertise, availability of histopathological and urologic feedback during multidisciplinary meetings, and mentoring sessions [30].

Although the ML models demonstrated excellent classification accuracy (91.95%), they may still misclassify the high-risk groups. The high false-positive rate in PI-RAD 4/5 underscores the need for multimodal risk stratification, emphasizing the need for additional parameters to refine clinical decision-making [24, 25].

The data used in the current study were retrieved from records, some details of which were unavailable, such as PSA velocity and free PSA. Another limitation is the non-inclusion of radiomics analysis, which precluded the extraction of quantitative imaging features from medical scans. The retrospective nature of the study and the format of the existing imaging data pose challenges in standardizing protocols for radiomic workflows. Future research should prioritize the integration of radiomics into similar investigations to enhance predictive modeling, improve personalized treatment strategies, and bridge the gaps between imaging and clinical outcomes.

We developed models that could predict and discriminate candidates with suspicious MRI PI-RAD based on pre-biopsy characteristics. Random Forest and Extra Trees showed excellent ability to predict MRI Pi-RAD and discriminate between low- and high-risk scores. However, caution should be exercised as a high percentage of negative biopsy cases have been assigned high PI-RAD scores, which delineate false-positive outcomes of MRI readings. Combining PI-RAD scores with significant parameters, such as PSA density, lesion diameter, and age, in decision curve analysis and utility paradigms would assist physicians in avoiding unnecessary biopsies and overtreatment.

The authors are grateful to the Deanship of Scientific Research, King Saud University, Riyadh, Kingdom of Saudi Arabia, for funding through the Vice Deanship of Scientific Research Chairs, Cancer Research Chair.

This study was conducted in accordance with the Declaration of Helsinki. Approval was granted by the Ethics Committee of the College of Medicine at King Saud University, Riyadh, Saudi Arabia (No: 19/0299/IRB). This research was compliant with the guidelines of human studies and was conducted ethically in accordance with the World Medical Association Declaration of Helsinki.

The authors declare that they have no conflicts of interest.

Deanship of Scientific Research, Vice Deanship of Scientific Research Chairs, King Saud University, Riyadh, KSA.

Mostafa A. Arafa, Karim H. Farhat, Danny M. Rabah and Alaa Mokhtar: Study design; Nesma Lotfy: data curation, analysis, visualization, and implementation of the machine learning models. She is also responsible for writing the results section; Farrukh Kamel Khan, Abdulaziz M. Althunayan, Sultan S. Al-Khateeb, Waleed Al-Taweel, Alaa Mokhtar, Sami Azhari: Data collection; Mostafa A. Arafa, Nesma Lotfy, Karim H. Farhat: drafting and/or critical revision of important scientific content. All authors approved the version to be published.

The data that support the findings of this study are available from the corresponding author upon reasonable request.