All published articles of this journal are available on ScienceDirect.
Predictive Performance of two Measures of Prognostic Mortality of Cancer Patients in Intensive Care Unit in Jordan: A Comparative Single-Centre Study
iD
Mohammad K. Aldalaykeh2
iD
Zyad T. Saleh1
iD
Khalil M. Yousef1
iD
Mohammad M. Alnaeem1
iD
Authors Info & Affiliations
Abstract
Background:
Information is presently insufficient about using Acute Physiology and Chronic Health Evaluation (APACHE) mortality predicting models for cancer patients in intensive care unit (ICU).
Objective:
To evaluates the performance of APACHE II and IV in predicting mortality for cancer patients in ICU.
Interventions/Methods:
This was a retrospective study including adult patients admitted to an ICU in a medical center in Jordan. Actual mortality rate was determined and compared with mortality rates predicted by APACHE II and IV models. Receiver operating characteristic (ROC) analysis was used to assess the sensitivity, specificity and predictive performance of both scores. Binary logistic regression analysis was used to determine the effect that APACHE II, APACHE IV and other sample characteristics have on predicting mortality.
Results:
251 patients (survived=80; none-survived=171) were included in the study with an actual mortality rate of 68.1%. APACHE II and APACHE IV scores demonstrated similar predicted mortality rates (43.3% vs. 43.0%), sensitivity (52.6% vs. 52.0%), and specificity (76.3%, 76.2%), respectively. The area under (AUC), the ROC curve for APACHE II score was 0.714 (95% confidence interval [CI] 0.645–0.783), and AUC for APACHE IV score was 0.665 (95% CI 0.595–0.734).
Conclusions:
As APACHE ӀӀ and ӀV mortality models demonstrate insufficient predicting performance, there is no need to consider APACHE IV in our ICU instead of using APACHE ӀӀ as it has more variables and need longer data extraction time.
Implications for Practice:
We suggest that other approaches in addition to the available models should be attempted to improve the accuracy of cancer prognosis in ICU. Further, it is also required to adjust the available models.
1. INTRODUCTION
Cancer continues to be one of the leading causes of death worldwide [1, 2]. In Jordan, cancer is the second leading cause of death after chronic cardiovascular and circulatory disease with an estimated 4,000 new patients diagnosed annually [3, 4]. Most new cancer patients present at advanced stages III/IV, with majority of those dying within one year of diagnosis [3, 5]. Consequently, an increasing number of cancer patients is at risk of admission to intensive care unit (ICU) due to cancer-related complication or the infectious and toxic adverse effects of its treatments [6-8]. Despite the aggressive management provided in ICU, the mortality rate associated with cancer remains high [7, 9, 10]. Minimizing the admission of terminally ill cancer patients to ICU is considered a common practice in industrialized countries [11]. However, admitting those patients to ICUs is more common in Jordan [12-14]. Since ICUs in the developing countries have limited resources, an objective indicator predicting the mortality among cancer patients is needed to guide clinical decisions, avoid unnecessary procedures, and ensure optimal use of ICU resources [15-17].
Illness severity measures are widely used to predict mortality among the patients with wide range of health conditions in ICUs [18]. Acute Physiology and Chronic Health Evaluation (APACHE) is commonly utilized validated measure in research and clinical practices [19]. APACHE scores estimate disease severity based on the presence of several clinical conditions and physiological and laboratory parameters obtained at admission and/or during the first 24 hours after ICU admission. APACHE has been modified several times for an attempt to improve the prediction accuracy of patients’ outcomes. APACHE II is the simplest validated version in predicting mortality and still utilized in ICUs worldwide as well as in Jordan. [18, 20] APACHE IV is the latest updated version that has been designed to include other significant health status parameters to predict health outcomes. Studies validating the predictive performance of APACHE ӀӀ scoring system among ICU patients with cancer suggests that its predicting ability remains suboptimal [20-23]. A contrary result was also documented [24, 25]. APACHE ӀV model on the other hand was validated only in one study [24]. The study only included critically ill patients with non-haematological and non-metastatic cancer and demonstrated a good predicting ability
However, it remains unclear whether APACHE IV is superior to APACHE II in predicting mortality in the ICU among patients with different types and stages of cancer. Comparing the predictive performance of APACHE II and APACHE IV in predicting mortality will potentially help in risk stratification and utilization of proper resources for patients with cancer. The aim of this study was to compare the predictive performance of APACHE II and APACHE IV in predicting ICU mortality using data at the time of ICU admission among Jordanian patients with cancer.
2. METHODS
2.1. Study Design
This was a retrospective single-center cohort study conducted in an ICU of a hospital located in a comprehensive medical center in Jordan (King Hussein Medical Center). The center is a medical compound of five hospitals affiliated with the major Jordanian royal medical services hospitals. The center provides medical service to 38% of the Jordanian population and is considered one of the most important medical centers in Middle East and the whole world. The study hospital has thirty six beds ;adult ICU provide a Western-style critical medicine for critically ill patients with different medical illnesses including cancer.
This study is a part of a project aimed at examining demographics, mortality, cause of death and healthcare utilization for those who died in the studied ICU over a 3-year period (2014, 2015, and 2016). Data from previous years were difficult to retrieve due to poor record keeping system. APACHE II and APACHE IV scores were calculated using an online calculator. The clinical conditions and the physiological and laboratory parameters required to compute the scores of both models were retrieved from medical record. The scores for APACHE II and APACHE IV models were calculated based on the first 24 hrs of ICU admission. ICU mortality was assessed through electronic medical records. Data were collected by a team of researchers between January 2017 and January 2018. The researchers retrieved relevant data for a small subset of records and calculated APACHE IV scores to make sure the data are collected in a uniform way by all reviewers.
2.2. Ethical Considerations
The study was reviewed by the School of Nursing Research Ethics Committee, University of Jordan (Reference: 26-4-2016 SON) and by the Ethics Committees of the study medical center (Reference: TF 3/1/Ethics/13686). The ethics boards determined that the requirement to obtain informed consent from patients was waived due to the study’s retrospective nature
2.3. Statistical Analysis
Data were analysed using Statistical Package for Social Science software 22 (IBM Corporation, Armonk, NY, USA). Descriptive statistics were used to describe the study sample and to summarize APACHE II and IV scores. Predicted mortality rates were determined using APACHE II and APACHE IV models and compared with actual mortality rate. Receiver operating characteristic (ROC) analysis was used to assess the sensitivity, specificity and predictive performance for mortality by both models. Lastly, two multiple binary logistic regression models were used to compare the predictive ability of APACHE II and APACHE IV for mortality. In both models, the outcome (dependent) variable was mortality whereas, APACHE II and APACHE IV were the independent variables. Both models were adjusted for the following covariates gender, type of cancer, and sepsis. An alpha level of less than 0.05 was used to determine statistical significance. The 95% confidence interval was also used to test for significance in this study. The 95% confidence interval offers evidence regarding the range of the actual data as well as magnitude of the relationship between the independent and the dependent variable. Therefore, using the 95% confidence interval allows for assessing the clinical significance in addition to the statistical significance. It also offers information about the nature of the relationship between the variables, i.e. negative or positive.
3. RESULTS
Two hundred and fifty-one patients (survived=80; none-survived=171) were included in the study with an average age of 54.9±15.4 years (survived=53.4±16.8; none-survived= 55.7±14.7). The percentages of male and female in the sample were 56.3% and 43.7%, respectively. Compared to survivors, the proportion of patients with sepsis was higher among non-survived patients (p=.003). Patients who survived their ICU stay had significantly lower APACHE II and APACHE IV score (19.3±7.5 and 76.5±22.3, respectively) compared to those who did not survive (25.4±7.9 and 90.8±22.4, respectively), p<.001. The predicted mortality of APACHE II and APACHE IV were significantly lower among survivors (30.5±22.1% and 37.3±21.2%, respectively) compared to non-survivors (51.0±23.3% and 59.1±22.9%, respectively), p<.001. The characteristics of the study sample and comparison between survival and non-survival patients are provided in Table 1.
Figs. (1 and 2) show the actual mortality rate for each APACHE II and IV score category, respectively. For instance, there was only 1 patient in the category of APCHE II score of 0-4 and no survivors in this category which resulted in 100% mortality rate, while the category of 5-9 had no non-survivors which resulted in 0% mortality rate. The figure of APACHE II model shows that the higher the APCHE II score, the greater the mortality rate. For APACHE IV model, there was a general increase in mortality rate with increasing model scores.
In this study, the actual mortality rate in our sample was 68.1%. APACHE II and APACHE IV models demonstrated similar predicted mortality rates (43.3% vs. 43%), sensitivity (52.6% vs. 52%), and specificity (76.3%, 76.2%), respectively. Table 2 shows APACHE II and APACHE IV predicted outcome versus observed outcome cross tabulation. The area under the curve (AUC) in APACHE II score (71.4%) was slightly higher than the AUC for APACHE IV score (66.5%) (Table 3). Multivariate binary logistic regression revealed that APACHE II score (OR 1.11, 95% CI 1.073-1.17, p = .03) and APACHE IV score (OR 1.03, 95%CI 1.01-1.05, p = .01) were independently associated with higher mortality in cancer patients who needed ICU support (Table 4).
4. DISCUSSION
This retrospective cohort study of 251 cancer patients evaluated the performance of APACHE ӀӀ and APACHE ӀV scores in predicting mortality in cancer patients. This is one of the few studies on assessing the performance of these models among critically ill cancer patients; the first in Middle East. The results revealed that higher scores of both APACHE II and IV at ICU admission are associated with higher mortality rate. However, the predictive and performance of both scores was insufficient for predicting ICU mortality. Both prognostic models had similar predictive sensitivity and specificity and provided similar ability prediction for mortality in this sample. Furthermore, the AUC of the ROC curve for both models demonstrate a comparable moderate predicting ability. Even though there a statistical significance in the regression models, OR were very small suggesting weak clinical significance for both APACHE II and APACHE IV in predicting mortality in this sample.
Our findings regarding APACHE ӀӀ score corroborates other findings in the literature [21-24]. Evidence suggests that while APACHE ӀӀ score is a significant predictor for ICU mortality, it can be used to prognosticate about cancer patients in ICU and remains suboptimal predictor of mortality in patients with cancer. To a lesser extent, other studies reported a good predictive performance for APACHE ӀӀ model [25, 26].
| Variable | Survived (N=80) | None-survived (N=171) | P |
|---|---|---|---|
| Average Age ± SD (Range) | 53.4±16.9 (21.82) | 55.7±14.7 (20-91) | .096a |
| Gender N (%) | - | - | - |
| Male | 49 (61.3) | 92(53.8) | .278b |
| Female | 31 (38.8) | 79(46.2) | - |
| Type of cancer N (%) | - | - | - |
| Hematological | 26 (32.5 | 67 (39%) | .329b |
| Solid | 54 (67.5) | 104 (61%) | - |
| Sepsis N (%) | - | - | - |
| Yes | 36 (37.5) | 111 (65%) | .003b |
| No | 44 (62.5) | 60 (35%) | - |
| APACHE II Score (Mean ± SD) | 19.33±7.51 | 25.43±7.89 | <.001a |
| APACHE IV Score (Mean ± SD) | 76.50±22.30 | 90.79±22.35 | <.001a |
| APACHE II Predicted Mortality (Mean ± SD) | 30.5±22.1% | 51.0±23.3% | <.001a |
| APACHE IV Predicted Mortality (Mean ± SD) | 37.3±21.2% | 59.1±22.9% | <.001a |
| Outcome | Total | |||
|---|---|---|---|---|
| Non-survival (N) | Survival (N) | |||
| APACHE II Prediction | Non-survival | 90 | 19 | 109 |
| Survival | 81 | 61 | 142 | |
| APACHE IV Prediction | Non-survival | 89 | 19 | 108 |
| Survival | 82 | 61 | 143 | |
| Observed | 171 | 80 | 251 | |
APACHE II predicted mortality rate 43.4%
APACHE IV predicted mortality rate 43%
APACHE II: Sensitivity=52.6%, Specificity=76.3%, Cut off value=24.5
APACHE IV: Sensitivity=52%, Specificity=76.2%, Cut off value=90.5
| - | Area | Std. error | Sig. | 95% CI | |
|---|---|---|---|---|---|
| Lower bound | Upper bound | ||||
| APACHEII | 0.714 | 0.035 | 0.000 | 0.645 | 0.783 |
| APACHEIV | 0.665 | 0.035 | 0.000 | 0.595 | 0.734 |
| Variable | OR | P | 95%CI for OR | |
|---|---|---|---|---|
| Lower | Upper | |||
| APACHE II | 1.119 | <.001 | 1.073 | 1.167 |
| Gender | 1.412 | .258 | .777 | 2.566 |
| Sepsis | 4.064 | .001 | 1.742 | 9.484 |
| Type of cancer | 1.846 | 1.846 | .776 | 4.391 |
| APACHE IV | 1.031 | <.001 | 1.017 | 1.046 |
| Gender | 1.466 | .198 | .819 | 2.626 |
| Sepsis | 3.763 | .002 | 1.636 | 8.659 |
| Type of cancer | 2.312 | .059 | .969 | 5.518 |
There is a paucity of information in the literature about APACHE ӀV score in critically ill cancer patients. In a recent study in China [25], the predictive performance of APACHE ӀӀ and APACHE ӀV scores was evaluated in 981 cancer patients admitted to ICU and both scores demonstrated good discrimination and calibration ability. However, this study did not include patients with haematological cancer and the majority of patients had non-metastatic tumors. In our study, more than one third of patients had haematological cancer and the majority had stage III and IV tumor. These differences may explain the discrepancy in findings and also suggest the need for modified version of APACHE scores specific for patients with cancer. To our knowledge, this is the only study that validated APACHE ӀV model in critically ill cancer patients. Validating this score among Jordanian patients with specific cancer characteristics might yield a different result.
In this study, sepsis was found to have a significantly unfavourable association with cancer mortality in ICU with an OR of 4.1 and 3.8 in APACHE II and APACHE IV models, respectively. Other prognostic variables in addition to sepsis were reported in the literature including poor patient performance status, stage or nature of malignancy, provision of intensive treatments (ventilator support, vasoactive medication, and renal-replacement therapy), and the time of last chemotherapy [21, 23]. Adjusting for these variables may improve the predictive performance of APACHE scores. For example, Martos-Benítez et al. adjusted APACHE II prediction model for clinical stage of cancer, use of mechanical ventilator, and reason for admission (malignancy and non-malignancy) [21] The adapted model was validated with 522 patients admitted to ICU with solid cancer and demonstrated a better discrimination and calibration compared to the original one.
There are several limitations for our study. For example this was retrospective study which restricted the ability to collect further data about the sample. Also, only a single medical center was utilized. Lastly, we did not collect data about variables that could have been important in our prediction models such as the stage of cancer and the length of ICU stay.
CONCLUSION
Predicting mortality in ICU cancer patients is difficult due to variability in stages, performance status, and other cancer specific criteria. Even though the predictive ability of APACHE ӀӀ and APACHE ӀV models was established, their performances were insufficient. There is a need to further explore and consider other typical prognostic variables related to cancer patients. A prospective study that may adjust for cancer specific variables is recommended to improve the predictability of APACHE scores.
ETHICS APPROVAL AND CONSENT TO PARTICIPATE
The study was reviewed by the School of Nursing Research Ethics Committee, University of Jordan (Reference: 26-4-2016 SON) and by the Ethics Committees of the study medical center (Reference: TF 3/1/Ethics/13686).
HUMAN AND ANIMAL RIGHTS
Not applicable.
CONSENT FOR PUBLICATION
Not applicable.
AVAILABILITY OF DATA AND MATERIAL
The data that support the findings of this study are available from the corresponding author. Restrictions apply to the availability of these data, which were used under license for this study. Data are available from the corresponding author with the permission of the ethical committee of King Hussein Medical Center.
FUNDING
The work was supported by the Deanship of Scientific Research, University of Jordan, Jordan (26-4-2016 SON).
CONFLICT OF INTEREST
The authors declare no conflict of interest, financial or otherwise.
ACKNOWLEDGMENTS
The authors would like to thank all participating staff from King Hussein Medical Center for assistance with data collection. We express our gratitude to the University of Jordan for the logistic support provided during the study.
