TITLE: EFFECT OF VARYING THE CASE MIX ON THE STANDARDIZED MORTALITY RATIO AND THE W STATISTIC: A SIMULATION STUDY

AUTHORS: Laurent G. Glance, MD¹, Turner Osler, MD², Tamotsu Shinozaki, MD²
AFFILIATION: ¹University of Rochester Medical Center, Rochester, NY; ²University of Vermont Medical College, Burlington, VT.

INTRODUCTION: The ability to evaluate ICU performance is central to quality assurance. Crude mortality rates do not permit meaningful interhospital comparisons because they do not adjust for differences in patient populations. If we are to use risk- adjusted measures of outcomes such as the SMR and the W statistic to evaluate ICU performance, we must assume that those measures are independent of case mix. This study tests the hypothesis that the SMR and W statistic are independent of variations in case mix.
METHODS: APACHE II data was collected prospectively from the SICU at a single institution between 1990 and 1997 (cardiac, burn and pediatric patients were excluded). The patients in this data set were first ranked according to their APACHE II predicted mortality and then divided into ten groups (decile of risk) of equal size: 0-3.1%, 3.2-4.7%, …, 39-100%. A computer simulation was used to create 10,000 different cases mixes with mortality rates between 5% and 16%. The number of patients in each decile (x₁ from the first decile, x₂ from the 2^nd decile, …, and x₁₀ from the 10^th decile) constituted a distinct case mix. A virtual ICU (VICU) with a given case mix was created by randomly resampling (with replacement) x₁ from the first decile, x₂ from the 2^nd decile, …, x₁₀ from the 10^th decile. One hundred different VICUs with identical case mixes were created for each of the 10,000 case mixes. The SMR and W statistic were first calculated for each of the 100 VICUs within a case mix, and then averaged together to obtain the value for a given case mix.
RESULTS: With increases in simulated mortality rate, the SMR decreased in a linear fashion (R-Sq=.66; p=0.000). Similarly, with increases in simulated mortality rate, the W statistic increased in a linear manner (R-Sq=.69; p=0.000). These results suggested that VICUs with higher mortality rates were superior to those with lower mortality rates.
CONCLUSION: VICUs were created from a data set of actual patients treated at one institution to test the hypothesis that the SMR and W statistic do not vary with case mix. Since all the patients were cared for at a single institution by the same staff, the SMR and W statistic should not have varied with changes in case mix. We found instead that the SMR and W statistic were very sensitive to case mix. This simulation suggests that the SMR and the W statistic based on APACHE II cannot be used to compare outcomes in ICUs.