Comparative Effectiveness of Risk-adjusted Cumulative Sum and Periodic Evaluation for Monitoring Hospital Perioperative Mortality

Background: National surgical quality improvement (QI) programs use periodic, risk-adjusted evaluation to identify hospitals with higher than expected perioperative mortality. Rapid, accurate identification of poorly performing hospitals is critical for avoiding potentially preventable mortality and represents an opportunity to enhance QI efforts. Methods: Hospital-level analysis using Veterans Affairs (VA) Surgical Quality Improvement Program data (2011-2016) to compare identification of hospitals with excess, risk-adjusted 30-day mortality using observed-to-expected (O-E) ratios (ie, current gold standard) and cumulative sum (CUSUM) with V-mask. Various V-mask slopes and radii were evaluated - slope of 2.5 and radius of 1.0 was used as the base case. Results: Hospitals identified by CUSUM and quarterly O-E were identified midway into a quarter [median 47 days; interquartile range (IQR): 24-61 days before quarter end] translating to a median of 129 (IQR: 60-187) surgical cases and 368 (IQR: 145-681) postoperative inpatient days occurring after a CUSUM signal, but before the quarter end. At hospitals identified by CUSUM but not O-E, a median of 2 deaths within a median of 5 days triggered a signal. In some cases, these clusters extended beyond CUSUM identification date with as many as 8 deaths undetected using O-E. Sensitivity and negative predictive values for CUSUM relative to O-E were 71.9% (95% confidence interval: 66.2%-77.1%) and 95.5% (94.4%-96.4%), respectively. Conclusions: CUSUM evaluation identifies hospitals with clusters of mortality in excess of expected more rapidly than periodic analysis. CUSUM represents an analytic tool national QI programs could utilize to provide participating hospitals with data that could facilitate more proactive implementation of local interventions to help reduce potentially avoidable perioperative mortality.