Article PDF first page preview

Article PDF first page preview

Introduction: Heart failure (HF) is a major global public health concern. The application of machine learning (ML) to identify individuals at high risk and enable early intervention is a promising approach for improving HF prognosis. We aim to systematically evaluate the performance and value of ML models for predicting HF prognosis. Methods: PubMed, Web of Science, Scopus, and Embase online databases were searched up to April 30, 2023, to identify studies on the use of ML models to predict HF prognosis. HF prognosis primarily encompasses readmission and mortality. The meta-analysis was conducted by MedCalc software. Subgroup analyses include grouping based on types of ML models, time intervals, sample sizes, the number of predictive variables, validation methods, whether to conduct hyperparameter optimization and calibration, data set partitioning methods. Results: A total of 31 studies were included. The most common ML models were random forest, boosting, support vector machine, neural network. The area under the receiver operating characteristic curve (AUC) for predicting HF readmission was 0.675 (95% CI: 0.651–0.699, p < 0.001), and the AUC for predicting HF mortality was 0.790 (95% CI: 0.765–0.816, p < 0.001). Subgroup analyses revealed that models with the prediction time interval of 1 year, sample sizes ≥10,000, the number of predictive variables ≥100, external validation, hyperparameter tuning, calibration adjustment, and data set partitioning using 10-fold cross-validation exhibited favorable performance within their respective subgroups. Conclusion: The performance of ML models in predicting HF readmission is relatively poor, while its performance in predicting HF mortality is moderate. The quality of the relevant studies is generally low, it is essential to enhance the predictive capabilities of ML models through targeted improvements in practical applications.

1.
Heidenreich
PA
,
Bozkurt
B
,
Aguilar
D
,
Allen
LA
,
Byun
JJ
,
Colvin
MM
, et al
.
2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the american college of cardiology/american heart association joint committee on clinical practice guidelines
.
Circulation
.
2022
;
145
(
18
):
e895
1032
.
2.
GBD 2017 Disease and Injury Incidence and Prevalence Collaborators
.
Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease study 2017
.
Lancet
.
2018
;
392
(
10159
):
1789
858
.
3.
Savarese
G
,
Becher
PM
,
Lund
LH
,
Seferovic
P
,
Rosano
G
,
Coats
A
.
Global burden of heart failure: a comprehensive and updated review of epidemiology
.
Cardiovasc Res
.
2023
;
118
(
17
):
3272
87
.
4.
Go
AS
,
Mozaffarian
D
,
Roger
VL
,
Benjamin
EJ
,
Berry
JD
,
Blaha
MJ
, et al
.
Heart disease and stroke statistics--2014 update: a report from the american heart association
.
Circulation
.
2014
;
129
(
3
):
e28
92
.
5.
Ventura
HO
,
Pina
IL
.
Comprehensive heart failure management
.
Prog Cardiovasc Dis
.
2017
;
60
(
2
):
169
70
.
6.
Olsen
CR
,
Mentz
RJ
,
Anstrom
KJ
,
Page
D
,
Patel
PA
.
Clinical applications of machine learning in the diagnosis, classification, and prediction of heart failure
.
Am Heart J
.
2020
;
229
:
1
17
.
7.
Greener
JG
,
Kandathil
SM
,
Moffat
L
,
Jones
DT
.
A guide to machine learning for biologists
.
Nat Rev Mol Cell Bio
.
2022
;
23
(
1
):
40
55
.
8.
Huang
XY
,
Ao
TJ
,
Zhang
X
,
Li
K
,
Zhao
XQ
,
Champreda
V
, et al
.
Developing high-dimensional machine learning models to improve generalization ability and overcome data insufficiency for mixed sugar fermentation simulation
.
Bioresour Technol
.
2023
;
385
:
129375
.
9.
Van Calster
B
,
Wynants
L
.
Machine learning in medicine
.
N Engl J Med
.
2019
;
380
(
26
):
2588
.
10.
Deo
RC
.
Machine learning in medicine
.
Circulation
.
2015
;
132
(
20
):
1920
30
.
11.
Jiang
T
,
Gradus
JL
,
Rosellini
AJ
.
Supervised machine learning: a brief primer
.
Behav Ther
.
2020
;
51
(
5
):
675
87
.
12.
Ma
H
,
Li
C
,
Shi
X
,
Yuan
Y
,
Wang
G
.
Deep unsupervised active learning on learnable graphs
.
Ieee T Neur Net Lear
.
2024
;
35
(
2
):
2894
900
.
13.
Krishnan
R
,
Rajpurkar
P
,
Topol
EJ
.
Self-supervised learning in medicine and healthcare
.
Nat Biomed Eng
.
2022
;
6
(
12
):
1346
52
.
14.
Kim
M
,
Rho
K
,
Kim
YD
,
Jung
K
.
Action-driven contrastive representation for reinforcement learning
.
PLoS One
.
2022
;
17
(
3
):
e0265456
.
15.
Ju
C
,
Zhou
J
,
Lee
S
,
Tan
MS
,
Liu
T
,
Bazoukis
G
, et al
.
Derivation of an electronic frailty index for predicting short-term mortality in heart failure: a machine learning approach
.
Esc Heart Fail
.
2021
;
8
(
4
):
2837
45
.
16.
Mpanya
D
,
Celik
T
,
Klug
E
,
Ntsinjana
H
.
Machine learning and statistical methods for predicting mortality in heart failure
.
Heart Fail Rev
.
2021
;
26
(
3
):
545
52
.
17.
Mohammad
MA
.
Advancing heart failure research using machine learning
.
Lancet Digit Health
.
2023
;
5
(
6
):
e331
2
.
18.
Moher
D
,
Liberati
A
,
Tetzlaff
J
,
Altman
DG
;
PRISMA Group
.
Preferred reporting items for systematic reviews and meta-analyses: the prisma statement
.
Open Med
.
2009
;
339
:
b2535
.
19.
Moons
KG
,
de Groot
JA
,
Bouwmeester
W
,
Vergouwe
Y
,
Mallett
S
,
Altman
DG
, et al
.
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the charms checklist
.
PLoS Med
.
2014
;
11
(
10
):
e1001744
.
20.
Wolff
RF
,
Moons
K
,
Riley
RD
,
Whiting
PF
,
Westwood
M
,
Collins
GS
, et al
.
Probast: a tool to assess the risk of bias and applicability of prediction model studies
.
Ann Intern Med
.
2019
;
170
(
1
):
51
8
.
21.
Hajian-Tilaki
K
.
Receiver operating characteristic (roc) curve analysis for medical diagnostic test evaluation
.
CASP J INTERN MED
.
2013
;
4
(
2
):
627
35
.
22.
Newcombe
RG
.
Confidence intervals for an effect size measure based on the mann-whitney statistic. Part 2: asymptotic methods and evaluation
.
Stat Med
.
2006
;
25
(
4
):
559
73
.
23.
Hanley
JA
,
McNeil
BJ
.
The meaning and use of the area under a receiver operating characteristic (roc) curve
.
Radiology
.
1982
;
143
(
1
):
29
36
.
24.
Alsharif
DJ
,
Alsharif
FJ
,
Aljuraiban
GS
,
Abulmeaty
M
.
Effect of supplemental parenteral nutrition versus enteral nutrition alone on clinical outcomes in critically ill adult patients: a systematic review and meta-analysis of randomized controlled trials
.
Nutrients
.
2020
;
12
(
10
):
2968
.
25.
Kang
Y
,
McHugh
MD
,
Chittams
J
,
Bowles
KH
.
Utilizing home healthcare electronic health records for telehomecare patients with heart failure: a decision tree approach to detect associations with rehospitalizations
.
Comput Inform Nurs
.
2016
;
34
(
4
):
175
82
.
26.
Mahajan
S
,
Burman
P
,
Hogarth
M
.
Analyzing 30-day readmission rate for heart failure using different predictive models
.
Stud Health Technol Inform
.
2016
;
225
:
143
7
.
27.
Mortazavi
BJ
,
Downing
NS
,
Bucholz
EM
,
Dharmarajan
K
,
Manhapra
A
,
Li
S
, et al
.
Analysis of machine learning techniques for heart failure readmissions
.
Circ Cardiovasc Qual Outcomes
.
2016
;
9
(
6
):
629
40
.
28.
Frizzell
JD
,
Liang
L
,
Schulte
PJ
,
Yancy
CW
,
Heidenreich
PA
,
Hernandez
AF
, et al
.
Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches
.
JAMA CARDIOL
.
2017
;
2
:
204
9
.
29.
Shameer
K
,
Johnson
KW
,
Yahi
A
,
Miotto
R
,
Li
LI
,
Ricks
D
, et al
.
Predictive modeling of hospital readmission rates using electronic medical record-wide machine learning: a case-study using mount sinai heart failure cohort
.
Pac Symp Biocomput
.
2017
;
22
:
276
87
.
30.
Golas
SB
,
Shibahara
T
,
Agboola
S
,
Otaki
H
,
Sato
J
,
Nakae
T
, et al
.
A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data
.
Bmc Med Inform Decis
.
2018
;
18
(
1
):
44
.
31.
Ahmad
T
,
Lund
LH
,
Rao
P
,
Ghosh
R
,
Warier
P
,
Vaccaro
B
, et al
.
Machine learning methods improve prognostication, identify clinically distinct phenotypes, and detect heterogeneity in response to therapy in a large cohort of heart failure patients
.
J Am Heart Assoc
.
2018
;
7
(
8
):
e008081
.
32.
Miao
F
,
Cai
Y
,
Zhang
Y
,
Fan
X
,
Li
Y
.
Predictive modeling of hospital mortality for patients with heart failure by using an improved random survival forest
.
Ieee Access
.
2018
;
6
:
7244
53
.
33.
Mahajan
SM
,
Ghani
R
.
Using ensemble machine learning methods for predicting risk of readmission for heart failure
.
Stud Health Technol Inform
.
2019
;
264
:
243
7
.
34.
Kwon
J
,
Kim
K
,
Jeon
K
,
Lee
SE
,
Lee
H
,
Cho
H
, et al
.
Artificial intelligence algorithm for predicting mortality of patients with acute heart failure
.
PLoS One
.
2019
;
14
(
7
):
e0219302
.
35.
Allam
A
,
Nagy
M
,
Thoma
G
,
Krauthammer
M
.
Neural networks versus logistic regression for 30 days all-cause readmission prediction
.
Sci Rep
.
2019
;
9
(
1
):
9277
.
36.
Angraal
S
,
Mortazavi
BJ
,
Gupta
A
,
Khera
R
,
Ahmad
T
,
Desai
NR
, et al
.
Machine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction
.
JACC Heart Fail
.
2020
;
8
(
1
):
12
21
.
37.
Adler
ED
,
Voors
AA
,
Klein
L
,
Macheret
F
,
Braun
OO
,
Urey
MA
, et al
.
Improving risk prediction in heart failure using machine learning
.
Eur J Heart Fail
.
2020
;
22
(
1
):
139
47
.
38.
Chen
P
,
Dong
W
,
Wang
J
,
Lu
X
,
Kaymak
U
,
Huang
Z
.
Interpretable clinical prediction via attention-based neural network
.
Bmc Med Inform Decis
.
2020
;
20
(
Suppl 3
):
131
.
39.
Desai
RJ
,
Wang
SV
,
Vaduganathan
M
,
Evers
T
,
Schneeweiss
S
.
Comparison of machine learning methods with traditional models for use of administrative claims with electronic medical records to predict heart failure outcomes
.
JAMA Netw Open
.
2020
;
3
(
1
):
e1918962
.
40.
Liu
W
,
Stansbury
C
,
Singh
K
,
Ryan
AM
,
Sukul
D
,
Mahmoudi
E
, et al
.
Predicting 30-day hospital readmissions using artificial neural networks with medical code embedding
.
PLoS One
.
2020
;
15
(
4
):
e0221606
.
41.
Jing
L
,
Ulloa Cerna
AE
,
Good
CW
,
Sauers
NM
,
Schneider
G
,
Hartzel
DN
, et al
.
A machine learning approach to management of heart failure populations
.
JACC Heart Fail
.
2020
;
8
(
7
):
578
87
.
42.
Pandey
M
,
Xu
Z
,
Sholle
E
,
Maliakal
G
,
Singh
G
,
Fatima
Z
, et al
.
Extraction of radiographic findings from unstructured thoracoabdominal computed tomography reports using convolutional neural network based natural language processing
.
PLoS One
.
2020
;
15
(
7
):
e0236827
.
43.
Tokodi
M
,
Schwertner
WR
,
Kovacs
A
,
Toser
Z
,
Staub
L
,
Sarkany
A
, et al
.
Machine learning-based mortality prediction of patients undergoing cardiac resynchronization therapy: the SEMMELWEIS-CRT score
.
Eur Heart J
.
2020
;
41
(
18
):
1747
56
.
44.
Wang
Z
,
Wang
B
,
Zhou
Y
,
Li
D
,
Yin
Y
.
Weight-based multiple empirical kernel learning with neighbor discriminant constraint for heart failure mortality prediction
.
J Biomed Inform
.
2020
;
101
:
103340
.
45.
Lv
H
,
Yang
X
,
Wang
B
,
Wang
S
,
Du
X
,
Tan
Q
, et al
.
Machine learning-driven models to predict prognostic outcomes in patients hospitalized with heart failure using electronic health records: retrospective study
.
J Med Internet Res
.
2021
;
23
(
4
):
e24996
.
46.
Riester
MR
,
McAuliffe
L
,
Collins
C
,
Zullo
AR
.
Development and validation of the tool for pharmacists to predict 30-day hospital readmission in patients with heart failure (ToPP-Hf)
.
AM J Health-syst PH
.
2021
;
78
(
18
):
1691
700
.
47.
Tokodi
M
,
Behon
A
,
Merkel
ED
,
Kovács
A
,
Tősér
Z
,
Sárkány
A
, et al
.
Sex-specific patterns of mortality predictors among patients undergoing cardiac resynchronization therapy: a machine learning approach
.
Front Cardiovasc Med
.
2021
;
8
:
611055
.
48.
König
S
,
Pellissier
V
,
Hohenstein
S
,
Bernal
A
,
Ueberham
L
,
Meier Hellmann
A
, et al
.
Machine learning algorithms for claims data-based prediction of in-hospital mortality in patients with heart failure
.
ESC Heart Fail
.
2021
;
8
(
4
):
3026
36
.
49.
Zhao
H
,
Li
P
,
Zhong
G
,
Xie
K
,
Zhou
H
,
Ning
Y
, et al
.
Machine learning models in heart failure with mildly reduced ejection fraction patients
.
Front Cardiovasc Med
.
2022
;
9
:
1042139
.
50.
Park
J
,
Hwang
IC
,
Yoon
YE
,
Park
JB
,
Park
JH
,
Cho
GY
.
Predicting long-term mortality in patients with acute heart failure by using machine learning
.
J Card Fail
.
2022
;
28
(
7
):
1078
87
.
51.
Cai
A
,
Chen
R
,
Pang
C
,
Liu
H
,
Zhou
Y
,
Chen
J
, et al
.
Machine learning model for predicting 1-year and 3-year all-cause mortality in ischemic heart failure patients
.
Postgrad Med
.
2022
;
134
(
8
):
810
9
.
52.
Tian
P
,
Liang
L
,
Zhao
X
,
Huang
B
,
Feng
J
,
Huang
L
, et al
.
Machine learning for mortality prediction in patients with heart failure with mildly reduced ejection fraction
.
J Am Heart Assoc
.
2023
;
12
(
12
):
e029124
.
53.
Chen
S
,
Hu
W
,
Yang
Y
,
Cai
J
,
Luo
Y
,
Gong
L
, et al
.
Predicting six-month re-admission risk in heart failure patients using multiple machine learning methods: a study based on the Chinese heart failure population database
.
J Clin Med
.
2023
;
12
(
3
):
870
.
54.
Ru
B
,
Tan
X
,
Liu
Y
,
Kannapur
K
,
Ramanan
D
,
Kessler
G
, et al
.
Comparison of machine learning algorithms for predicting hospital readmissions and worsening heart failure events in patients with heart failure with reduced ejection fraction: modeling study
.
JMIR Form Res
.
2023
;
7
:
e41775
.
55.
Li
Y
,
Sperrin
M
,
Ashcroft
DM
,
van Staa
TP
.
Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar
.
BMJ
.
2020
;
371
:
m3919
.
56.
Huang
S
,
Cai
N
,
Pacheco
PP
,
Narrandes
S
,
Wang
Y
,
Xu
W
.
Applications of Support Vector Machine (SVM) learning in cancer genomics
.
Cancer Genom Proteom
.
2018
;
15
(
1
):
41
51
.
57.
Fan
S
,
Yu
Y
,
Wu
Y
,
Kai
Y
,
Wang
H
,
Chen
Y
, et al
.
Altered brain entropy and functional connectivity patterns in generalized anxiety disorder patients
.
J Affect Disord
.
2023
;
332
:
168
75
.
58.
Lin
Z
,
Qin
X
,
Li
J
,
Zohaib Aslam
M
,
Sun
T
,
Li
Z
, et al
.
Machine learning approach for predicting single cell lag time of salmonella enteritidis after heat and chlorine treatment
.
Food Res Int
.
2022
;
156
:
111132
.
59.
Xu
M
,
Zhou
H
,
Hu
P
,
Pan
Y
,
Wang
S
,
Liu
L
, et al
.
Identification and validation of immune and oxidative stress-related diagnostic markers for diabetic nephropathy by WGCNA and machine learning
.
Front Immunol
.
2023
;
14
:
1084531
.
60.
Shen
J
,
Casie Chetty
S
,
Shokouhi
S
,
Maharjan
J
,
Chuba
Y
,
Calvert
J
, et al
.
Massive external validation of a machine learning algorithm to predict pulmonary embolism in hospitalized patients
.
Thromb Res
.
2022
;
216
:
14
21
.
61.
Cooney
C
,
Korik
A
,
Folli
R
,
Coyle
D
.
Evaluation of hyperparameter optimization in machine and deep learning methods for decoding imagined speech EEG
.
Sensors
.
2020
;
20
(
16
):
4629
.
62.
Cearns
M
,
Hahn
T
,
Clark
S
,
Baune
BT
.
Machine learning probability calibration for high-risk clinical decision-making
.
Aust Nz J Psychiat
.
2020
;
54
(
2
):
123
6
.
63.
Kushwaha
S
,
Srivastava
R
,
Jain
R
,
Sagar
V
,
Aggarwal
AK
,
Bhadada
SK
, et al
.
Harnessing machine learning models for non-invasive pre-diabetes screening in children and adolescents
.
Comput Meth Prog Bio
.
2022
;
226
:
107180
.
64.
Fleuren
LM
,
Klausch
T
,
Zwager
CL
,
Schoonmade
LJ
,
Guo
T
,
Roggeveen
LF
, et al
.
Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy
.
Intens Care Med
.
2020
;
46
(
3
):
383
400
.
65.
Moons
KG
,
Altman
DG
,
Reitsma
JB
,
Ioannidis
JP
,
Macaskill
P
,
Steyerberg
EW
, et al
.
Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration
.
Ann Intern Med
.
2015
;
162
(
1
):
W1
73
.
You do not currently have access to this content.