Research Article - Journal of Intensive and Critical Care Nursing (2019) Volume 2, Issue 1

Evaluation of performance in intensive care unit

Alfateh Sayed Mohammed Noor¹, Abdulrahman M Al-Harthy¹, Omar E Ramadan^1,2, Ahmed F Madi^1,3, Mohammed A Al-Odat¹, Basim M Huwait¹, Shahzad A Mumtaz¹, Waleed T Aletreby^1*

Laboratory of mechanisms of stem cells regulation, Stock Company “R-Pharm”, Moscow, Russian Federation

*Corresponding Author:

Waleed T Aletreby
Critical Care Department King Saud Medical City Riyadh Saudi Arabia

Accepted Date: December 16, 2019

Citation: Mohammed Noor AS, Al-Harthy AM, Ramadan, et al. Evaluation of performance in intensive care unit. J Intensive Crit Care Nurs. 2019;2(1):6-11.

Abstract

Abstract: The intensive care unit (ICU) of any hospital consumes a large portion of the hospital’s budget, which mandates continuous evaluation of the performance of the unit to substantiate its expenditure. Measuring the quality of the performance in the ICU is difficult and complex, however, two identified performance indicators of ICU in terms of effectiveness are length of stay (LOS) of patients, and severity adjusted standardized mortality ratio utilizing a severity related prediction model. Aims: To evaluate the performance in an adult ICU in terms of effectiveness, using predefined targets for the length of stay and standardized mortality ratio, as well as comparison to predicted values. Methods: All discharged patients from our ICU during 2018 were included, the average LOS for all and acute patients, and the mortality rate were calculated, and compared to values predicted by APACHE 4 scoring system. Results: During 2018 we discharged 2769 patients, and 2484 patients met the inclusion criteria.The median LOS for all patients [ 5 ( 2 - 12)] was significantly higher than predicted value of 4 (2 – 11) days (p=0.013), the same was observed for the LOS of acute patients (who spend less than 21 days in ICU), the actual and predicted medians were 4 (2 – 10) and 3 (2 – 6) days respectively (p=0.02), however both LOS calculations were within our pre-set targets of 15 days for all and 5 days for acute patients. The actual mortality rate of 12.5% (95% CI 11.2 – 13.9) was significantly lower than that predicted by the APACHE 4 scoring system (14.6%). Using the actual and predicted mortality rates, the standardized mortality ratio was 0.86. Comparison of the year 2018 to 2017 show a significant reduction of LOS for all patients (p= 0.03), and an insignificant trend toward reduction of mortality rate (p = 0.07) Conclusions: The LOS values for all and acute patients are within targets, and comparable to figures reported in some studies, being above values predicted may be attributed to the fact that APACHE 4 scoring system underestimates LOS. The mortality rate was significantly lower than predicted, and lower than that reported in similar studies. With a standardized mortality ratio of less than 1, there is evidence of an acceptable quality of care in the ICU. However, interventions in the form of performance improvement projects are required to improve the indicators, and consequently the quality of care. There is also improvement in the performance and outcome of our ICU in 2018 as compared to 2017.

Introduction

The intensive care unit (ICU) is an integral part of any acute care health facility, as it provides highly specialized and intense close monitoring for critically ill patients with life threatening conditions [1], the very nature of services and interventions provided in an ICU such as mechanical ventilation, diagnostic procedures, invasive monitoring techniques, and the utilization of medications and blood products lead to increased expenditure and daily costs per patient, this in addition to the development of complications [2–4], furthermore, expenses in ICU are also dependent on the severity of illness assumed to be highest in an ICU patient [5]. Consequently, ICUs can utilize up to 20% of a hospital’s budget [6,7], with an estimated mean cost of 31,574 $/day for ventilated patients in USA [8], which translates into $81.7 billion of critical care costs in USA in year 2005 [9].

As a result of this high economic burden of ICUs, evidence must be provided to prove both effectiveness and efficiency of an ICU [10] through assessment of the quality of performance in ICU [11]. While measuring the quality of ICU performance is complex and difficult [12], several performance indicators are being utilized by healthcare institutions worldwide such as length of stay (LOS) and mortality rates (MR), both of which are effectiveness measures [13], particularly when used in correlation to the severity of illness as a supplement to structure, procedure, outcome classical measurement tools [14]. Two of the most commonly used performance indicators in correlation to severity scores are severity adjusted ICU LOS and standardized mortality ratio (SMR) [14-16].

Aim of the Study

To evaluate the quality of care and performance in the ICU in terms of effectiveness and efficiency, during the year 2018 at King Saud Medical City (KSMC), Riyadh, Saudi Arabia. KSMC is a tertiary referral hospital with 1200 bed capacity, the ICU has 127 beds, divided into surgical, medical, trauma, burn, and maternity sections, with 14 isolation beds, it is a closed ICU covered by intensivists 24 hours 7 days, with nurses to patient ratio of 1:1, in the ICU at KSMC we predefined some targets for our patients in our strategic plan for 2018. Such targets include an average LOS for all patients of 15 days or less, an average LOS for acute patients (patients who spend less than 21 days in the ICU) of 5 days or less, a mortality rate of 20% or less, and a standardized mortality ratio (observed mortality rate/predicted mortality rate) of less than one.

Study Design

This is a retrospective observational study, in the ICU at KSMC. All patients discharged from the ICU during the year 2018 were included in the study, with the exclusion of:

• Patients of less than 18 years of age.

• Burn patients.

• Patients with Do Not Resuscitate (DNR) order excluded only from the calculation of mortality rate.

For every included patient the following data were collected:

Demographic data: Age and gender.

Source of admission to ICU.

Broad category of admission diagnosis (medical – surgical – trauma – maternity-post operative).

Length of stay (LOS) in ICU.

Binary ICU outcome (dead or alive).

APACHE 4 score, predicted mortality rate, and predicted LOS.

This study is a report of the mandatory requirement of continuous monitoring, evaluation, and reporting of the ICU performance, required by the total quality management department (TQM) in our hospital.

Statistical Method

LOS for each patient is calculated as day of discharge – day of admission, the mortality rate is defined as the number of patients discharged from the ICU as ‘dead” divided by the total number of patients discharged from the ICU in the same period, excluding patients with Do Not Resuscitate (DNR) orders from both the numerator and denominator.

The predicted LOS and MR are the arithmetic mean of the corresponding values for each patient as predicted by APACHE 4 severity scoring system. Delayed discharge will be considered for patients who remain in ICU for at least a complete day (24 hours) after the order to transfer out and before physical discharge from the ICU.

If continuous variables satisfied the assumptions of parametric tests, they were summarized as mean (SD) and compared with student t test, otherwise they were summarized as median (Q1- Q3) and compared with Mann Whitney U test, Kolmogrov Smirnov test was used to assess normality of continuous variables. Attribute data were summarized as number (%), and compared with chi square test. To compare the actual MR and predicted MR, the average predicted MR will be considered as a proportion to be compared to the actual MR which is a proportion.

All statistical tests were two tailed, and considered significant if p value<0.05, and analysed using a commercially available statistical package (SPSS® version 19; IBM Corporation, Armonk, NY, USA).

Outcomes

The primary outcomes were to report the average actual LOS for all discharged patients, and the mortality rate as compared to average predicted values, as well as the SMR of the year 2018. Secondary outcomes included: LOS for acute patients as compared to predicted in the year 2018, furthermore, we compared the MR and average LOS in the year 2018 to 2017. Also as a secondary outcome the percentage of patients with delayed discharge and the average duration of delayed discharge will be compared between 2017 and 2018, with correlation to the expenditure per ICU bed per day, calculated based on our local institute’s health economics estimation of 5000 Saudi Riyals (SR) (equivalent to about 1,300 $) average cost of ICU bed/day including ventilated and non-ventilated patients.

Results

None of the continuous variables in our study satisfied the assumptions of parametric tests, consequently, all continuous variables were reported as median (Q1-Q3), and compared with Mann Whitney U test.

During the year 2018 there were 2769 discharges from the ICU, 2484 patients met the inclusion criteria (247 patients less than 18 years of age and 38 burn patients were excluded), 62.3% were males. The average age of all discharged patients was 42 (28 – 59) years, Out of all discharged patients 284 died, and 1988 patients were discharged alive, whereas 212 patients were not considered in the calculation of MR as they were labelled as DNR. The majority (42.5%) of the discharges were medical patients, followed by surgical (19.4%) then maternity (17.3%), followed by 12.5% post-operative patients, and least were trauma patients (8.3%). Most of the patients (49.2%) were initially admitted from the emergency department, the rest were either inpatients or patients received directly in the ICU from other hospitals, the discharged patients had an average APACHE 4 score of 72 (34–109), which corresponds to an average PMR of 14.6 (5–35.6), and a predicted LOS of 4 days (2–11), demographic data of patients discharged in 2018 are summarized in Table 1, and compared to 2017 patients.

Table 1: Study patients’ demographics.

The annual average LOS for all patients was 5 (2–12) days, while the annual average LOS for acute patients was 4(2–10) days. The corresponding predicted values were 4(2–11) days for all patients and 3(2 – 6) days for acute patients. Both predicted values of LOS for all patients and for acute patients were significantly lower than the actual values, respective p values were 0.013 and 0.02 (Table 2), similarly, there was a statistically significant difference between the actual mortality rate of the year 2018 and the predicted mortality rate.

Table 2: Actual and predicted annual LOS and Mortality Rate-2018.

The actual MR was 12.5% (95% CI: 11.2–13.9) whereas the predicted MR (average of PMR of all patients) was 14.6%, p value=0.034, apart from the statistical comparison between the actual and predicted mortality rate, a more accepted and utilized measure of evaluation is the standardized mortality ratio (SMR), calculated by dividing the actual MR by the predicted MR, in our study in 2018 the SMR was 0.86 which is a favourable result since it is less than the null value of one.

During the year 2017 there were 2442 discharges who met the inclusion criteria (out of a total of 2622), among which 349 died, resulting in a mortality rate of 14.3% (95% CI: 12.9– 15.8), during the same year the average LOS of all patients was 5 (2–15) days.

Comparison of both years MR resulted in an insignificant trend toward decreased mortality in 2018 (p=0.07), meanwhile there was a statistically significant reduction in the average LOS for all patients between 2017 and 2018 (p=0.03).

However, there was no significant difference in LOS for acute patients in 2018 and 2017 (p=0.06) although the LOS was lower in 2018. In 2018 683 patients stayed at least 24 hours in the ICU after the transfer order (27.5%, 95% CI: 25.8–29.3), those patients remained in ICU an average of 2 (1–4) days before physical discharge, the corresponding values for 2017 were 712 patients with delayed discharge (29.2%, 95% CI: 27.4–31.05), accounting for an average delay in discharge of 2 (1 – 5) days. Comparison of the two variables yielded an insignificant difference in proportion of patients (p=0.2) while a significant reduction of the average duration of delayed discharge (p<0.001). (Table 3 summarizes the comparison between 2017 and 2018.

Table 3: Comparison of: MR, Avg. LOS, Avg. LOS acute patients between 2017 and 2018.

The average cost of ICU patients during 2018 was 62.1 million SR (24.8 – 149 million SR), whereas during 2017 the cost was 61 million SR (24 – 183 million SR) (p=0.7), delayed discharged patients cost 6.8 million SR (3.4–13.7 million SR) in 2018, similar patients in 2017 cost 7.1 million SR (3.6–17.8 million SR) (p=0.6) (Table 4).

Table 4: Comparison of Cost of all and delayed discharge patients.

Discussion

The results of our study show that although the average LOS for all patients is within our target of 15 days or less, it is significantly higher than the average predicted LOS. Despite that, the average LOS of 5 days is lower than that reported by some authors and close to others. Novin et al. [17]reports an average of 19.3 days, while Onnen et al. [5] reports a median LOS of 12.1, however, the most commonly reported LOS is much lower than that, it ranges from 4 to 6 days in many studies [18,19]. Our hospital does not have a step down unit, which leads to a prolonged LOS because of chronic patients who require only nursing care beyond the acute phase of illness, which is the reason why we also calculate the average LOS for acute patients (staying less than 21 days) in order to exclude such patients. The average LOS of acute patients (4 days) is also within our target of 5 days, and quite comparable to many studies [12,14,19,20], where the average or median LOS is as low as 3.04 and as high as 4 days, despite being significantly higher than the predicted LOS, which is a finding that was reported by many authors, who find that APACHE 4 underestimates LOS [17-20]. Our average mortality rate was also within our target of 20% or less, and significantly lower than the average mortality rate of all patients predicted by APACHE 4, the mortality rate of 12.5% is lower than that reported by many authors in different regions of the world [21-24] reporting a MR as high as 60.71% for septic shock and 46% for all patients, whereas a MR as low as 5.3% was reported by Bekele et al. [12], mortality rates in the range of 12.1–28.4% were reported by others [5,14,23]. Our SMR of 0.86 shows better outcome than predicted, since it was less than (although close to) the null value of 1, a similar finding is reported by Tomasz et al. [25] who reports a SMR of 0.98 in a single centre study like ours, while in a much larger study on 12000 patients, the SMR of a mixed ICU was 61% [26].

The secondary outcomes in our study show a significant reduction in the average LOS of all patients from 2017 to 2018, and a trend of reduction of mortality rate and LOS for acute patients, although both didn ’ t reach the level of statistical significance. These improvements may not be attributed to a single particular intervention per se, since there were several performance improvement projects (PIP) started at the same time in our ICU in the year 2018, in addition to the increasing number of clinical practice guidelines and protocols being adopted by our ICU, consequently, whether this improvement in performance from 2017 to 2018 may be attributed to any of these PIPs if not to all of them together remains unclear, and warrants studying the outcome of each project separately. Other secondary outcomes pertained to delayed discharge after transfer order, this outcome is directly related to the performance of case managers in ICU, and the efficiency of early discharge planning, in 2018 there was a reduction of the percentage of patients with delayed discharge compared to 2017, although it didn ’ t reach the level of statistical significance, the average duration of delayed discharge, however, was significantly reduced from 2017 to 2018, the breakdown of the most important reasons of delay showed that unavailability of beds in the general ward was the main reason of delay, and efforts of early discharge planning made by the ICU resulted in reduction of that delay time. Our results show an increase in the ICU expenditure in 2018 as compared to 2017, although not statistically significant, and this may be explained by the increase in the number of admitted patients, the cost of delayed discharge patients in 2018 is less than that of 2017, which can be explained by the actual reduction of the LOS of those patients, although being fewer may also be a contributing factor. Keeping in mind that expenditure results are not very accurate, since they are based on estimations, which do not account for the case severity. We believe that efforts are highly demanded to alleviate performance of all healthcare professionals particularly those in an ICU. Efforts such as regular measurement and interventions to tackle burnout syndrome, for example by job rotation and enrichment, in addition to facilitation of the daily routine work through implementation of electronic healthcare records (EHR) and reduction of routine paperwork.

Recommendations

Since our major finding is a prolonged LOS, we would like to recommend some interventions aiming at its reduction:

Full activation and implementation of Do Not Resuscitate (DNR) policy, which can be achieved through better communication with relatives and families [27, 28].

Implementation of the services of case managers within the ICU, as their efforts were shown to reduce LOS through minimizing delays in surgical and invasive procedures [29].

Starting a step down unit to receive chronic patients who require mainly nursing care from the ICU, which may lead to decrease in average LOS, and evacuate highly demanded ICU beds for critical cases. As well as better collaboration with higher authorities in the Ministry of Health to facilitate the transfer of those patients to nursing homes or rehabilitation centres.

Initiation of a quality improvement project of early mobilization of chronic and ventilated patients in the ICU, which not only decreases LOS, but improves quality of life after discharge as well [30].

Conclusions

• There is evidence of acceptable performance of the ICU with regards to mortality rate.

• We still need to improve our average LOS for all patients.

• There is improvement in 2018 performance compared to 2017.

• Insignificant changes in expenditure between the two comparison periods.

Limitation

This study has a number of limitations, to start with it is a retrospective observational study, it included patients discharged in only one year, and so a small sample size, leading to under power of the study, the different outcomes of the study were not correlated to diagnoses, nor the LOS was adjusted to severity as our hospital is yet to adopt ICD-10.

References

1. Mälstam J, Lind L. Therapeutic intervention scoring system (TISS)-a method for measuring workload and calculating costs in the ICU. Acta Anaesthesiol Scand. 1992; 36: 758-763. 
2. Teres D, Rapoport J, Lemeshow S, et al. Effects of severity of illness on resource use by survivors and nonsurvivors of severe sepsis at intensive care unit admission. Crit Care Med. 2002; 30:2413-2419. 
3. Dasta JF, McLaughlin TP, Mody SH, et al. Daily cost of an intensive care unit day: the contribution of mechanical ventilation. Crit Care Med. 2005; 33:1266-1271.
4. Cox CE, Carson SS, Lindquist JH, et al. Differences in one-year health outcomes and resource utilization by definition of prolonged mechanical ventilation: a prospective cohort study. Crit Care. 2007; 11:R9. 
5. Moerer O, Plock E, Mgbor U, et al. A German national prevalence study on the cost of intensive care: an evaluation from 51 intensive care units. Crit Care. 2007;11(3):R69.  
6. Pittoni GM, Scatto A. Economics and outcome in the intensive care unit. Curr Opin Anaesthesiol. 2009; 22: 232-236. 
7. Zilberberg MD, Luippold RS, Sulsky S, et al. Prolonged acute mechanical ventilation, hospital resource utilization, and mortality in the United States. Crit Care Med. 2008; 36: 724-730. 
8. Dasta JF, McLaughlin TP, Mody SH, et al. Daily cost of an intensive care unit day: the contribution of mechanical ventilation. Crit Care Med. 2005; 33(6):1266-71.  
9. Fakhry SM, Martin B, Al Harakeh H, et al. Proportional costs in trauma and acute care surgery patients: dominant role of intensive care unit costs. J Am Coll Surg. 2013; 216: 607-614.
10.  Timmers TK, Verhofstad MH,  Moons KG, et al. Intensive care performance: How should we monitor performance in the future?. World J Crit Care Med. 2014; 3(4): 74?79. 
11. Moreno RP, Hochrieser H, Metnitz B, et al. Characterizing the risk profiles of intensive care units. Intensive Care Med. 2010; 36: 1207-1212. 
12. Afessa B, Keegan MT, Hubmayr RD, et al. Evaluating the Performance of an Institution Using an Intensive Care Unit Benchmark. Mayo Clin Proc. 2005;80(2):174-180. 
13. Carla A, Chrusch, Claudio M, et al. The Quality Improvement in Critical Care Project, ?Quality Improvement in Critical Care: Selection and Development of Quality Indicators. Canadian Respiratory Journal. 2016;p: 11.
14. De?a K, ?ustiæ A, Goluboviæ V, et al. Evaluation of Single Intensive Care Unit Performance by Simplified Acute Physiology Score II System. Croat Med J. 2005;46(6):964-969 
15. Quality indicators in critically ill patients. Madrid (Spain): Spanish Society of Intensive and Critical Care and Units Coronary (SEMICYUC); 2011. 185 p.  
16. Quality Indicators for Critical Care in Scotland Version 2.0 January 2012. Webpage:  www.sicsag.scot.nhs.uk/SICSQIG-report-2012-120209.pdf. 
17. Nicbakhsh N, Amri Maleh P, Monadi M, et al. Prediction Of Patient?s Mortality Rate Of Intensive Care Unit Based On APACHE IV. J Urmia Nurs Midwifery Fac. 2016;14(2): 183190. 
18. Milic M, Goranovi T, Katan Holjevac J. Correlation of APACHE II and SOFA Scores with Length of Stay in Various Surgical Intensive Care Units. Coll. Antropol. 33 2009; 3: 831?835.
19. Zimmerman JE, Kramer AA, McNair DS, et al. Shaffer. Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med. 2006; 34:2517?2529. 
20. Hu Y, Zhang X, Liu Y, et al. APACHE IV Is Superior to MELD Scoring System in Predicting Prognosis in Patients after Orthotopic Liver Transplantation. Clinical and Developmental Immunology. 2013; p:5.  
21. Saleh A, Ahmed M, Sultan I, et al. Comparison of the mortality prediction of different ICU scoring systems (APACHE II and III, SAPS II, and SOFA) in a single-center ICU subpopulation with acute respiratory distress syndrome. Egyptian Journal of Chest Diseases and Tuberculosis. 2015;64, 843?848.  
22. Shoukat H, Muhammad Y, Gondal KM et al. Mortality Prediction in Patients Admitted in Surgical Intensive Care Unit by Using APACHE IV. Journal of the College of Physicians and Surgeons Pakistan. 2016; 26 (11): 877-880. 
23. Chan T, Bleszynski MS, KBuczkowski A. Evaluation of APACHE-IV Predictive Scoring in Surgical Abdominal Sepsis: A Retrospective Cohort Study. Journal of Clinical and Diagnostic Research. 2016;10(3): PC16-PC18.  
24. Dabhi AS, Khedekar SS, Mehalingam V. A Prospective Study of Comparison of APACHE-IV & SAPS-II Scoring Systems and   Calculation of Standardised Mortality Rate in Severe Sepsis and Septic Shock Patients. J Clin Diagn Res. 2014; 8(10): MC09?MC13. 
25. Siegel T, Adamski J, Nowakowski P, et al. Prospective assessment of the standardized mortality ratio (SMR) as a measure of quality of care in an intensive care unit ? a single-centre study. Anaesthesiology Intensive Therapy. 2015; 47(4) 328?332. 
26. B Afessa, M Keegan, J Naessens, et al. Comparison of intensive care unit mortality performances: standardized mortality ratio vs absolute risk reduction. Critical Care. 2007;11(Suppl 2):P470. 
27. Lilly CM, De Meo DL, Sonna LA, et al. An intensive communication intervention for the critically ill. Am J Med. 2000;15?109(6):46975.
28. Schneiderman LJ, Gilmer T, Teetzel HD, et al. Effect of ethics consultations on nonbeneficial lifesustaining treatments in the intensive care setting: a randomized controlled trial. JAMA. 2003; 290(9):116672. 
29. The Effect of Case Management on US Hospitals. Nurs Econ. 2004?22(2).
30. Hunter A, Johnson L, LCoustasse A. Reduction of intensive care unit length of stay: The case of early mobilization. The Health Care Manager. 2014;33(2), 128-135.