Introduction: Well-calibrated models for personalized prognostication of patients with gastrointestinal neuroendocrine neoplasms (GINENs) are limited. This study aimed to develop and validate a machine-learning model to predict the survival of patients with GINENs. Methods: Oblique random survival forest (ORSF) model, Cox proportional hazard risk model, Cox model with least absolute shrinkage and selection operator penalization, CoxBoost, Survival Gradient Boosting Machine, Extreme Gradient Boosting survival regression, DeepHit, DeepSurv, DNNSurv, logistic-hazard model, and PC-hazard model were compared. We further tuned hyperparameters and selected variables for the best-performing ORSF. Then, the final ORSF model was validated. Results: A total of 43,444 patients with GINENs were included. The median (interquartile range) survival time was 53 (19–102) months. The ORSF model performed best, in which age, histology, M stage, tumor size, primary tumor site, sex, tumor number, surgery, lymph nodes removed, N stage, race, and grade were ranked as important variables. However, chemotherapy and radiotherapy were not necessary for the ORSF model. The ORSF model had an overall C index of 0.86 (95% confidence interval, 0.85–0.87). The area under the receiver operation curves at 1, 3, 5, and 10 years were 0.91, 0.89, 0.87, and 0.80, respectively. The decision curve analysis showed superior clinical usefulness of the ORSF model than the American Joint Committee on Cancer Stage. A nomogram and an online tool were given. Conclusion: The machine learning ORSF model could precisely predict the survival of patients with GINENs, with the ability to identify patients at high risk for death and probably guide clinical practice.

1.
Pavel
M
,
Öberg
K
,
Falconi
M
,
Krenning
EP
,
Sundin
A
,
Perren
A
, et al
.
Gastroenteropancreatic neuroendocrine neoplasms: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up
.
Ann Oncol
.
2020
;
31
(
7
):
844
60
.
2.
Shah
MH
,
Goldner
WS
,
Benson
AB
,
Bergsland
E
,
Blaszkowsky
LS
,
Brock
P
, et al
.
Neuroendocrine and adrenal tumors, version 2.2021, NCCN clinical practice guidelines in oncology
.
J Natl Compr Canc Ne
.
2021
;
19
(
7
):
839
68
.
3.
Ito
T
,
Masui
T
,
Komoto
I
,
Doi
R
,
Osamura
RY
,
Sakurai
A
, et al
.
JNETS clinical practice guidelines for gastroenteropancreatic neuroendocrine neoplasms: diagnosis, treatment, and follow-up: a synopsis
.
J Gastroenterol
.
2021
;
56
(
11
):
1033
44
.
4.
Cives
M
,
Strosberg
JR
.
Gastroenteropancreatic neuroendocrine tumors
.
CA Cancer J Clin
.
2018
;
68
(
6
):
471
87
.
5.
Xu
Z
,
Wang
L
,
Dai
S
,
Chen
M
,
Li
F
,
Sun
J
, et al
.
Epidemiologic trends of and factors associated with overall survival for patients with gastroenteropancreatic neuroendocrine tumors in the United States
.
Jama Netw Open
.
2021
;
4
(
9
):
e2124750
.
6.
Dasari
A
,
Shen
C
,
Halperin
D
,
Zhao
B
,
Zhou
S
,
Xu
Y
, et al
.
Trends in the incidence, prevalence, and survival outcomes in patients with neuroendocrine tumors in the United States
.
Jama Oncol
.
2017
;
3
(
10
):
1335
42
.
7.
Rindi
G
,
Klöppel
G
,
Alhman
H
,
Caplin
M
,
Couvelard
A
,
de Herder
WW
, et al
.
TNM staging of foregut (neuro)endocrine tumors: a consensus proposal including a grading system
.
Virchows Arch
.
2006
;
449
(
4
):
395
401
.
8.
Rindi
G
,
Klöppel
G
,
Couvelard
A
,
Komminoth
P
,
Körner
M
,
Lopes
JM
, et al
.
TNM staging of midgut and hindgut (neuro) endocrine tumors: a consensus proposal including a grading system
.
Virchows Arch
.
2007
;
451
(
4
):
757
62
.
9.
Nagtegaal
ID
,
Odze
RD
,
Klimstra
D
,
Paradis
V
,
Rugge
M
,
Schirmacher
P
, et al
.
The 2019 WHO classification of tumours of the digestive system
.
Histopathology
.
2020
;
76
(
2
):
182
8
.
10.
Rindi
G
,
Petrone
G
,
Inzani
F
.
The 2010 WHO classification of digestive neuroendocrine neoplasms: a critical appraisal four years after its introduction
.
Endocr Pathol
.
2014
;
25
(
2
):
186
92
.
11.
Rindi
G
,
Moch
H
,
McCluggage
WG
.
Neuroendocrine neoplasms, non-endocrine organs
. In:
WHO Classification of Tumours Editorial Boardeditor. WHO classification of tumours endocrine and neuroendocrine tumours
. 5th ed.
Lyon, France
:
International Agency for Research on Cancer (IARC)
;
2022
.
12.
Rindi
G
,
Mete
O
,
Uccella
S
,
Basturk
O
,
La Rosa
S
,
Brosens
LAA
, et al
.
Overview of the 2022 WHO classification of neuroendocrine neoplasms
.
Endocr Pathol
.
2022
;
33
(
1
):
115
54
.
13.
Busico
A
,
Maisonneuve
P
,
Prinzi
N
,
Pusceddu
S
,
Centonze
G
,
Garzone
G
, et al
.
Gastroenteropancreatic high-grade neuroendocrine neoplasms: histology and molecular analysis, two sides of the same coin
.
Neuroendocrinology
.
2020
;
110
(
7–8
):
616
29
.
14.
Milione
M
,
Maisonneuve
P
,
Spada
F
,
Pellegrinelli
A
,
Spaggiari
P
,
Albarello
L
, et al
.
The clinicopathologic heterogeneity of grade 3 gastroenteropancreatic neuroendocrine neoplasms: morphological differentiation and proliferation identify different prognostic categories
.
Neuroendocrinology
.
2017
;
104
(
1
):
85
93
.
15.
Shen
C
,
Chen
H
,
Chen
H
,
Yin
Y
,
Han
L
,
Chen
J
, et al
.
Surgical treatment and prognosis of gastric neuroendocrine neoplasms: a single-center experience
.
Bmc Gastroenterol
.
2016
;
16
(
1
):
111
.
16.
Dasari
A
,
Shen
C
,
Devabhaktuni
A
,
Nighot
R
,
Sorbye
H
.
Survival according to primary tumor location, stage, and treatment patterns in locoregional gastroenteropancreatic high-grade neuroendocrine carcinomas
.
Oncologist
.
2022
;
27
(
4
):
299
306
.
17.
Xu
G
,
Xiao
Y
,
Hu
H
,
Jin
B
,
Wu
X
,
Wan
X
, et al
.
A nomogram to predict individual survival of patients with liver-limited metastases from gastroenteropancreatic neuroendocrine neoplasms: a US population-based cohort analysis and Chinese multicenter cohort validation study
.
Neuroendocrinology
.
2022
;
112
(
3
):
263
75
.
18.
Poleé
IN
,
Hermans
BCM
,
van der Zwan
JM
,
Bouwense
SAW
,
Dercksen
MW
,
Eskens
FALM
, et al
.
Long-term survival in patients with gastroenteropancreatic neuroendocrine neoplasms: a population-based study
.
Eur J Cancer
.
2022
;
172
:
252
63
.
19.
Tierney
JF
,
Poirier
J
,
Chivukula
S
,
Pappas
SG
,
Hertl
M
,
Schadde
E
, et al
.
Primary tumor site affects survival in patients with gastroenteropancreatic and neuroendocrine liver metastases
.
Int J Endocrinol
.
2019
;
2019
:
9871319
.
20.
Tai
WM
,
Tan
SH
,
Tan
DMY
,
Loke
KSH
,
Ng
DCE
,
Yan
SX
, et al
.
Clinicopathologic characteristics and survival of patients with gastroenteropancreatic neuroendocrine neoplasm in a multi-ethnic asian institution
.
Neuroendocrinology
.
2019
;
108
(
4
):
265
77
.
21.
Kessel
E
,
Naparst
M
,
Alpert
N
,
Diaz
K
,
Ahn
E
,
Wolin
E
, et al
.
Racial differences in gastroenteropancreatic neuroendocrine tumor treatment and survival in the United States
.
Pancreas
.
2021
;
50
(
1
):
29
36
.
22.
Donadio
MD
,
Brito
ÂB
,
Riechelmann
RP
.
A systematic review of therapeutic strategies in gastroenteropancreatic grade 3 neuroendocrine tumors
.
Ther Adv Med Oncol
.
2023
;
15
:
17588359231156218
.
23.
Tierney
JF
,
Chivukula
SV
,
Wang
X
,
Pappas
SG
,
Schadde
E
,
Hertl
M
, et al
.
Resection of primary tumor may prolong survival in metastatic gastroenteropancreatic neuroendocrine tumors
.
Surgery
.
2019
;
165
(
3
):
644
51
.
24.
Dijke
K
,
Kuhlmann
KFD
,
Levy
S
,
Tesselaar
MET
.
Surgical management of the primary tumor in stage IV small intestinal neuroendocrine tumors: to operate or not to operate, that is the question
.
Curr Oncol Rep
.
2023
;
25
(
6
):
679
88
.
25.
Thornblade
LW
,
Warner
SG
,
Melstrom
L
,
Ituarte
PHG
,
Chang
S
,
Li
D
, et al
.
Does surgery provide a survival advantage in non-disseminated poorly differentiated gastroenteropancreatic neuroendocrine neoplasms
.
Surgery
.
2021
;
169
(
6
):
1417
23
.
26.
Pommergaard
H-C
,
Nielsen
K
,
Sorbye
H
,
Federspiel
B
,
Tabaksblat
EM
,
Vestermark
LW
, et al
.
Surgery of the primary tumour in 201 patients with high-grade gastroenteropancreatic neuroendocrine and mixed neuroendocrine-non-neuroendocrine neoplasms
.
J Neuroendocrinol
.
2021
;
33
(
5
):
e12967
.
27.
Hiyoshi
Y
,
Daitoku
N
,
Mukai
T
,
Nagasaki
T
,
Yamaguchi
T
,
Akiyoshi
T
, et al
.
Risk factors for lymph node metastasis of rectal neuroendocrine tumor and its prognostic impact: a single-center retrospective analysis of 195 cases with radical resection
.
Ann Surg Oncol
.
2023
;
30
(
7
):
3944
53
.
28.
Garcia-Carbonero
R
,
Anton-Pascual
B
,
Modrego
A
,
Del Carmen Riesco-Martinez
M
,
Lens-Pardo
A
,
Carretero-Puche
C
, et al
.
Advances in the treatment of gastroenteropancreatic neuroendocrine carcinomas: are we moving forward
.
Endocr Rev
.
2023
;
44
(
4
):
724
36
.
29.
Mao
R
,
Li
K
,
Cai
J-Q
,
Luo
S
,
Turner
M
,
Blazer
D
, et al
.
Adjuvant chemotherapy versus observation following resection for patients with nonmetastatic poorly differentiated colorectal neuroendocrine carcinomas
.
Ann Surg
.
2021
;
274
(
2
):
e126
33
.
30.
Schmitz
R
,
Mao
R
,
Moris
D
,
Strickler
JH
,
Blazer
DG
.
Impact of postoperative chemotherapy on the survival of patients with high-grade gastroenteropancreatic neuroendocrine carcinoma
.
Ann Surg Oncol
.
2021
;
28
(
1
):
114
20
.
31.
Zhang
P
,
Li
J
,
Li
J
,
Zhang
X
,
Zhou
J
,
Wang
X
, et al
.
Etoposide and cisplatin versus irinotecan and cisplatin as the first-line therapy for patients with advanced, poorly differentiated gastroenteropancreatic neuroendocrine carcinoma: a randomized phase 2 study
.
Cancer
.
2020
;
126
(
Suppl 9
):
2086
92
.
32.
Robinson
MD
,
Livesey
D
,
Hubner
RA
,
Valle
JW
,
McNamara
MG
.
Future therapeutic strategies in the treatment of extrapulmonary neuroendocrine carcinoma: a review
.
Ther Adv Med Oncol
.
2023
;
15
:
17588359231156870
.
33.
Fazio
N
,
La Salvia
A
.
Precision medicine in gastroenteropancreatic neuroendocrine neoplasms: where are we in 2023
.
Best Pract Res Cl En
.
2023
;
37
(
5
):
101794
.
34.
Sorbye
H
,
Kong
G
,
Grozinsky-Glasberg
S
.
PRRT in high-grade gastroenteropancreatic neuroendocrine neoplasms (WHO G3)
.
Endocr Relat Cancer
.
2020
;
27
(
3
):
R67
77
.
35.
Alese
OB
,
Jiang
R
,
Shaib
W
,
Wu
C
,
Akce
M
,
Behera
M
, et al
.
High-grade gastrointestinal neuroendocrine carcinoma management and outcomes: a national cancer database study
.
Oncologist
.
2019
;
24
(
7
):
911
20
.
36.
Xie
S
,
Li
L
,
Wang
X
,
Li
L
.
Development and validation of a nomogram for predicting the overall survival of patients with gastroenteropancreatic neuroendocrine neoplasms
.
Medicine
.
2021
;
100
(
2
):
e24223
.
37.
Zhao
F
,
Huang
L
,
Wang
Z
,
Wei
F
,
Xiao
T
,
Liu
Q
.
Epidemiological trends and novel prognostic evaluation approaches of patients with stage II-IV colorectal neuroendocrine neoplasms: a population-based study with external validation
.
Front Endocrinol
.
2023
;
14
:
1061187
.
38.
Murakami
M
,
Fujimori
N
,
Nakata
K
,
Nakamura
M
,
Hashimoto
S
,
Kurahara
H
, et al
.
Machine learning-based model for prediction and feature analysis of recurrence in pancreatic neuroendocrine tumors G1/G2
.
J Gastroenterol
.
2023
;
58
(
6
):
586
97
.
39.
La Salvia
A
,
Marcozzi
B
,
Manai
C
,
Mazzilli
R
,
Landi
L
,
Pallocca
M
, et al
.
Rachel score: a nomogram model for predicting the prognosis of lung neuroendocrine tumors
.
J Endocrinol Invest
.
2024
.
40.
Jaeger
BC
,
Long
DL
,
Long
DM
,
Sims
M
,
Szychowski
JM
,
Min
Y-I
, et al
.
Oblique random survival forests
.
Ann Appl Stat
.
2019
;
13
(
3
):
1847
83
.
41.
Jaeger
BC
,
Welden
S
,
Lenoir
K
,
Speiser
JL
,
Segar
MW
,
Pandey
A
, et al
.
Accelerated and interpretable oblique random survival forests
.
arXiv [Preprint]
.
42.
Zhang
X
,
Ye
Z
,
Xiao
G
,
He
T
.
Prognostic signature construction and immunotherapy response analysis for Uterine Corpus Endometrial Carcinoma based on cuproptosis-related lncRNAs
.
Comput Biol Med
.
2023
;
159
:
106905
.
43.
Binder
H
,
Allignol
A
,
Schumacher
M
,
Beyersmann
J
.
Boosting for high-dimensional time-to-event data with competing risks
.
Bioinformatics
.
2009
;
25
(
7
):
890
6
.
44.
Friedman
JH
.
Stochastic gradient boosting
.
Comput Stat Data
.
2002
;
38
(
4
):
367
78
.
45.
Guleken
Z
,
Jakubczyk
P
,
Paja
W
,
Pancerz
K
,
Wosiak
A
,
Yaylım
İ
, et al
.
An application of Raman spectroscopy in combination with machine learning to determine gastric cancer spectroscopy marker
.
Comput Meth Prog Bio
.
2023
;
234
:
107523
.
46.
Lee
C
,
Light
A
,
Saveliev
ES
,
van der Schaar
M
,
Gnanapragasam
VJ
.
Developing machine learning algorithms for dynamic estimation of progression during active surveillance for prostate cancer
.
Npj Digit Med
.
2022
;
5
(
1
):
110
.
47.
Katzman
JL
,
Shaham
U
,
Cloninger
A
,
Bates
J
,
Jiang
T
,
Kluger
Y
.
DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network
.
Bmc Med Res Methodol
.
2018
;
18
(
1
):
24
.
48.
Zhao
L
,
Feng
D
.
DNNSurv: deep neural networks for survival analysis using pseudo values
.
arXiv [Preprint]
.
49.
Gensheimer
MF
,
Narasimhan
B
.
A scalable discrete-time survival model for neural networks
.
Peerj
.
2019
;
7
:
e6257
.
50.
Kvamme
H
,
Borgan
Ø
.
Continuous and discrete-time survival prediction with neural networks
.
arXiv [Preprint]
.
51.
Lang
M
,
Binder
M
,
Richter
J
,
Schratz
P
,
Pfisterer
F
,
Coors
S
, et al
.
mlr3: a modern object-oriented machine learning framework in R
.
JOSS
.
2019
;
4
(
44
):
1903
.
52.
Kang
L
,
Chen
W
,
Petrick
NA
,
Gallas
BD
.
Comparing two correlated C indices with right-censored survival outcome: a one-shot nonparametric approach
.
Stat Med
.
2015
;
34
(
4
):
685
703
.
53.
Uno
H
,
Cai
T
,
Tian
L
,
Wei
LJ
.
Evaluating prediction rules for t-year survivors with censored regression models
.
J Am Stat Assoc
.
2007
;
102
(
478
):
527
37
.
54.
Spytek
M
,
Krzyziński
M
,
Langbein
SH
,
Baniecki
H
,
Wright
MN
,
Biecek
P
.
Survex: an R package for explaining machine learning survival models
.
arXiv [Preprint]
.
55.
Di Franco
M
,
Zanoni
L
,
Fortunati
E
,
Fanti
S
,
Ambrosini
V
.
Radionuclide theranostics in neuroendocrine neoplasms: an update
.
Curr Oncol Rep
.
2024
;
26
(
5
):
538
50
.
You do not currently have access to this content.