The limited utility of ranking hospitals based on their colon surgery infection rates

Daniel A. Caroff, Rui Wang, Zilu Zhang, Robert Wolf, Ed Septimus, Anthony D. Harris, Sarah S. Jackson, Russell E. Poland, Jason Hickok, Susan S. Huang, Richard Platt

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Background: The Centers for Medicare and Medicaid Services (CMS) use colon surgical site infection (SSI) rates to rank hospitals and apply financial penalties. The CMS' risk-adjustment model omits potentially impactful variables that might disadvantage hospitals with complex surgical populations. Methods: We analyzed adult patients who underwent colon surgery within facilities associated with HCA Healthcare from 2014 to 2016. SSIs were identified from National Health Safety Network (NHSN) reporting. We trained and validated 3 SSI prediction models, using (1) current CMS model variables, including hospital-specific random effects (HCA-adapted CMS model); (2) demographics and claims-based comorbidities (expanded-claims model); and (3) demographics, claims-based comorbidities, and NHSN variables (claims-plus-electronic health record [EHR] model). Discrimination, calibration, and resulting rankings were compared among all models and the current CMS model with published coefficient values. Results: We identified 39 468 colon surgeries in 149 hospitals, resulting in 1216 (3.1%) SSIs. Compared to the HCA-adapted CMS model, the expanded-claims model had similar performance (c-statistic, 0.65 vs 0.67, respectively), while the claims-plus-EHR model was more accurate (c-statistic, 0.70; 95% confidence interval,. 67-.73; P=.004). The sampling variation, due to the low surgical volume and small number of infections, contributed 74% of the total variation in observed SSI rates between hospitals. When CMS model rankings were compared to those from the expanded-claims and claims-plus-EHR models, 18 (15%) and 26 (22%) hospitals changed quartiles, respectively, and 10 (8.3%) and 12 (10%) hospitals changed into or out of the lowest-performing quartile, respectively. Conclusions: An expanded set of variables improved colon SSI risk predictions and quartile assignments, but low procedure volumes and SSI events remain a barrier to effectively comparing hospitals.

Original languageEnglish (US)
Pages (from-to)90-98
Number of pages9
JournalClinical Infectious Diseases
Issue number1
StatePublished - Jan 1 2021


  • colon surgery
  • risk adjustment
  • surgical site infection

ASJC Scopus subject areas

  • Microbiology (medical)
  • Infectious Diseases


Dive into the research topics of 'The limited utility of ranking hospitals based on their colon surgery infection rates'. Together they form a unique fingerprint.

Cite this