The Value of the Surprise Question to Predict One-Year Mortality in Idiopathic Pulmonary Fibrosis: A Prospective Cohort Study

Idiopathic pulmonary fibrosis (IPF) is one of the most progressive interstitial lung diseases (ILDs), with a median survival of only 3–5 years when not treated [1]. Within IPF, disease course can be heterogeneous; some patients have a slow disease progression with even periods of relative stability, and others experience a rapid decline or a disease trajectory with acute deteriorations [2]. In daily practice, clinicians often struggle to accurately predict disease progression for individual patients. Most patients have a high symptom burden and an increasingly impaired quality of life [3]. Even though IPF has a worse prognosis than many malignant diseases, timely initiation of palliative care and advance care planning are currently lacking [4-6]. Previous studies revealed that discussions regarding end of life are less frequently reported in patients with ILD than in patients with cancer, that palliative care is only initiated in a small minority of patients with IPF, and that referral to palliative care services generally occurs very late in the disease course [5-8]. Consequently, the majority of patients with IPF die in the hospital or even in the intensive care unit, despite the fact that most patients prefer to die at home [8-11]. A few studies showed that early integrated palliative care may improve quality of life and survival in ILD, including IPF, although the number of ILD patients in these studies was small [12, 13]. In IPF, a study with an advance care planning program reduced hospital visits in the last year of life and increased the number of patients who died at home [14]. Main barriers to timely discuss palliative care and end-of-life care include the unpredictable disease course of ILDs and organizational gaps, such as lack of time [4, 15]. Hence, one of the proposed topics for future research in a recent expert statement was to focus on identifying tools to predict end of life [4].

During the last decades, a number of prediction models for mortality has been developed and validated in IPF [16, 17]. The most widely used prediction model and staging system is the gender age physiology (GAP) model, developed in 2012 [18]. Currently, the GAP model is primarily used to classify patients in different disease stages for research purposes, and not as a clinical prognostic tool to initiate end-of-life care in individual patients. In other chronic diseases, the “surprise question” has been demonstrated to be a useful tool to predict mortality and increase awareness among healthcare providers that patients are nearing the end of life [19-21]. This tool comprises only 1 question: “Would you be surprised if this patient died within the next year?” If the answer on this question is “no,” this should be a trigger to initiate advance care planning [22]. The surprise question is increasingly used in daily care and incorporated in the National Gold Standards Framework for palliative care in the UK [23]. Nevertheless, until now, the value of the surprise question has never been evaluated in patients with IPF.

The primary aim of this study was to assess the sensitivity, specificity, and predictive value of the surprise question for 1-year mortality in IPF. Secondary aims were to evaluate which factors were associated with 1-year mortality and compare the predictive values of the surprise question and the validated GAP model.

This was a prospective cohort study at the Erasmus Medical Center, a European reference center for ILD in Rotterdam, The Netherlands. Because of the noninterventional design, this study was exempt from ethics approval. Participants provided written informed consent before the study entry. All patients with a diagnosis of IPF according to the ATS/ERS/JRS/ALAT guideline, who visited the outpatient clinic between May 2018 and January 2019, were eligible for inclusion [24].

For each included patient, the surprise question was answered by their treating physician (pulmonologist or pulmonologist in training) or ILD specialist nurse. The treating physician had access to all clinical data, including lung function, medical history, and medication use. The surprise question was completed on paper, immediately after the consultation. The 2 possible answers on this question are “surprised” or “not surprised,” with “not surprised” meaning that the healthcare provider would not be surprised if the patient died within the next year. After the outpatient clinic visit, patients completed the Dutch version of the Medical Research Council (MRC) dyspnea scale [25]. The MRC dyspnea scale consists of 1 item to grade the severity of dyspnea on a scale from 0 (not breathless at all) to 5 (too breathless to leave the house). In addition, patient characteristics, lung function parameters, and data about the use of supplemental oxygen and comorbidities were collected from the medical records. Collected lung function parameters were forced vital capacity (FVC) and diffusion capacity of the lung for carbon monoxide (DLCO), performed during the baseline visit. The GAP score was calculated for all included patients. All patients were followed up for a year after inclusion to collect information on survival.

Simple 2 × 2 tables were used to calculate the sensitivity, specificity, positive predictive value, and negative predictive value of the surprise question with regard to 1-year mortality. Baseline variables of the “surprised” and “not surprised” group were compared with the independent Student’s t-test and χ² test. Unadjusted survival analysis (Kaplan-Meier analysis and the logrank test) was used to assess between-group differences in survival. Multivariable logistic regression analysis was performed to evaluate which factors were associated with 1-year mortality. We used a least absolute shrinkage and selection operator (LASSO) analysis for variable selection as the number of variables that can be included in the multivariable model is limited by the number of events, to prevent overfitting (10 events per variable) [26]. After standardization of the variables, the sum of the regression coefficients was forced below a certain threshold by setting the “lambda” (amount of shrinkage). By doing so, coefficients of the less contributing variables will shrink toward zero and will automatically be eliminated from the LASSO model. To obtain unbiased estimates for the regression coefficients, the selected variables were used in a multivariable logistic regression model. Discriminative performance of the surprise question and the GAP model were evaluated and compared with the area under the receiver operator curve (also called C-statistic). A p value <0.05 was considered statistically significant. Statistical analyses were done using SPSS (IBM Statistics for Windows, version 25) and R statistics version 3.6.2 (www.r-project.org). Data are presented as mean (SD) or number (percentage).

In total, 140 consecutive outpatients with IPF were included in this study. The mean age ± SD was 74 ± 6.5 years, and 87% were male. The mean ± SD FVC in the entire cohort was 2.87 ± 0.83 L or 74.0 ± 18.3% of predicted, and the mean DLCO % of predicted was 37.4 ± 13.4 (Table 1). At baseline, the majority of patients (78.6%) used anti-fibrotic medication (i.e., nintedanib or pirfenidone), 15.7% of patients were about to start on medication, and 5.7% did not use anti-fibrotic medication. One-year all-cause mortality was 20% (N = 28). The surprise question was answered by a pulmonologist (N = 122), a pulmonologist in training (N = 15), or ILD specialist nurse (N = 3). In total, the surprise question was answered by 14 different healthcare providers. In 72% of cases (N = 101), healthcare providers would be surprised if the patients were to die within the next year. In 28% of patients (N = 39), they would not be surprised. Lung function parameters, GAP stage, MRC score, and supplemental oxygen use at baseline were significantly different between the “surprised” and “not surprised” groups. There were no between-group differences in age, gender, BMI, and comorbidities between both groups (Table 1).

By using the surprise question, healthcare providers identified patients with a survival of <1 year with a sensitivity of 68% (19/28), a specificity of 82% (92/112), a positive predictive value of 49%, and a negative predictive value of 91% (Table 2). Using Kaplan-Meier analysis, a “surprised” answer was associated with a significantly better survival than a “not surprised” answer (p < 0.001, Fig. 1).

LASSO regression analysis was used to select 3 variables to include in the multivariable logistic regression model for the prediction of 1-year mortality. Age, gender, GAP stage, FVC, use of supplemental oxygen, and comorbidities were the least contributing factors and were eliminated from the LASSO model. The most strongly related factors were the surprise question, MRC score, and DLCO % predicted. These variables were subsequently included in the logistic regression analysis (Table 3). The surprise question (OR 3.69, p = 0.019) and MRC score (OR 1.66, p = 0.033) significantly predicted 1-year mortality in the multivariable model, whereas DLCO did not (OR 0.95, p = 0.086).

The C-statistic of the surprise question to predict mortality was 0.75 (95% CI 0.66–0.85), with an overall accuracy of 79%. The C-index of the GAP model in this cohort was 0.68 (95% CI 0.58–0.77), with an overall accuracy of 69%. The difference between both C-statistic values was not statistically significant (difference 0.07, p = 0.077). A model composed of the surprise question, MRC score, and DLCO and had a C-statistic of 0.82 (95% CI 0.73–0.91). This was not significantly higher than the C-statistic of the surprise question (difference 0.07, p = 0.154).

In this study, we aimed to assess the value of the surprise question to predict 1-year mortality in IPF. The answer to the surprise question significantly predicted all-cause mortality after 1 year in a multivariable model, and receiver operating characteristic curve analysis showed that discriminative performance was acceptable. The specificity and negative predictive value of the surprise question were high, implying that healthcare providers were able to accurately identify those patients who were still alive after 1 year. The sensitivity and positive predictive value were somewhat lower. This finding was not unexpected as the surprise question is not designed to give an exact prognostication, but rather to reflect on the possibility that a patient dies within a year. By stimulating healthcare providers to reflect, there may be an overclassification of patients in the “not surprised” category, in contrast to the common tendency among healthcare providers to overestimate survival time [27]. This approach ensures timely initiation of advance care planning as all patients in the “not surprised” group will likely benefit from advance care planning, even if they live longer than 1 year [28].

Interestingly, the surprise question had a numerically higher predictive accuracy for 1-year mortality than the GAP score (79 vs. 69%). Moreover, the difference in discriminative performance (C-statistic) between the surprise question and GAP score tended toward statistical significance. The discriminative performance of the GAP score in the current cohort was in line with the original validation study, which reported a C-statistic of 0.69 [18]. A recent study, which integrated comorbidities into the GAP model, showed that this new model (called TORVAN) slightly improved the prediction of survival. Nevertheless, the C-statistic of the TORVAN model (0.70–0.71) was lower than that of the surprise question in the present study (0.75) [29]. Importantly, a model composed of the surprise question and the other variables identified by LASSO regression analysis (the MRC score and DLCO) did not significantly improve the prediction of 1-year mortality in our cohort; however, these findings should be interpreted with caution as this study might have been not sufficiently powered to detect significant differences between these models. The most likely reason for the good performance of the surprise question in predicting mortality is that physicians take into account the main factors included in existing models, combined with clinical intuition, and previous experiences. This is also supported by the finding that patients in the “not surprised” group had a significantly lower FVC and DLCO and were more likely to use supplemental oxygen compared to the “surprised” group.

Besides the fact that clinicians seem as good, or better, in predicting mortality than more complex, multidimensional models, the simplicity of the surprise question is another big advantage for implementation in clinical practice. We believe that structured incorporation of the surprise question in daily practice for patients with IPF will be feasible, can stimulate timely end-of-life conversations, and aid decision-making. Moreover, routine use of the surprise question will increase awareness about advance care planning among healthcare providers, which is especially relevant, given the high rate of IPF patients in whom advance care planning and end-of-life care are currently not initiated until the last days of life [7, 8, 10].

The surprise question and the MRC score were the only factors significantly associated with 1-year mortality in multivariable regression analysis. Similar to the surprise question, the MRC scale could easily be integrated in routine care as it consists of only 1 simple question answered by patients themselves. Previously, others have also advocated for the use of the modified MRC scale as a simple screening tool to identify patients with palliative care needs [30]. Several decision-aid tools have been proposed to assess palliative care needs and facilitate shared decision-making in ILD as patients’ needs, values, and wishes are the most important aspects of advance care planning [31, 32]. A retrospective study indicated that the use of a simple “supportive care decision aid” tool increased documented end-of-life conversations as well as palliative care referral rates, implying that the use of simple tools is feasible and effective in clinical practice [31]. Moreover, the palliative care needs assessment tool for ILD (NAT: PD-ILD) has been developed as a communication and decision tool for clinical practice. Although this tool seems valid and reliable, its clinical effectiveness needs to be determined in future studies [32]. A combination of the surprise question and structured needs assessment can potentially be of further added value in daily practice but has never been prospectively investigated in IPF [33].

One of the limitations of this study is that it was conducted in a single ILD expert center, and results have not been validated in a separate cohort. Consequently, our findings need to be extended and validated in larger multicenter cohorts, preferably including community sites with less experience in IPF and also including patients with other forms of progressive pulmonary fibrosis. Thus, the surprise question should currently only be used and applied in specialized IPF centers. Finally, we did not compare the accuracy of the answer on the surprise question between different healthcare provider disciplines as it was answered by experienced pulmonary physicians in the vast majority of cases. In future studies, it would be interesting to assess whether there are any differences between physicians and nurses, and whether work experience also influences the answer on the surprise question.

In IPF care, the surprise question can be a useful and efficient tool to trigger timely conversation about advance care planning and, where needed, referral to specialist palliative care services. The routine use of the simple surprise question can increase the awareness for palliative care needs among healthcare providers and hopefully improve both the quality of life and quality of dying for patients with IPF.

This research was conducted in accordance with the World Medical Association Declaration of Helsinki. Because of the noninterventional design, this study was exempt from ethics approval. All patients provided written informed consent.

N.T., C.O., S.B., and C.R. have nothing to disclose. C.M. reports grants and fees from Boehringer-Ingelheim, outside of the submitted work. All grants and fees were paid to her institution. J.M. reports fees from Boehringer-Ingelheim, Hoffman la Roche, and Chiesi pharmaceuticals, outside of the submitted work. M.W. reports grants and fees from Boehringer-Ingelheim and Hoffman la Roche, and fees from Galapagos, Respivant, Savara, and Novartis, outside of the submitted work. All fees and grants were paid to her institution.

The authors did not receive any funding.

All authors contributed to the conception and design of the study. N.T., C.M., J.M., and M.W. were involved in data acquisition. C.M., N.T., S.B., and M.W. performed data analyses. C.M., N.T., and M.W. drafted the manuscript, and all authors revised it critically for important intellectual content and approved the final version of the manuscript.

Catharina C. Moor and Nelleke C. Tak van Jaarsveld share first authorship.