A comparative evaluation of risk-adjustment models for benchmarking amputation-free survival after lower extremity bypass

• Published in: Journal of vascular surgery Vol. 63; no. 4; pp. 990 - 997
• Main Authors: Simons, Jessica P., MD, MPH, Goodney, Philip P., MD, MS, Flahive, Julie, MS, Hoel, Andrew W., MD, Hallett, John W., MD, Kraiss, Larry W., MD, Schanzer, Andres, MD
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.04.2016
• Subjects: Aged; Aged, 80 and over; Amputation; Benchmarking - standards; Decision Support Techniques; Disease-Free Survival; Female; Humans; Lower Extremity - blood supply; Male; Middle Aged; Patient Selection; Peripheral Arterial Disease - diagnosis; Peripheral Arterial Disease - mortality; Peripheral Arterial Disease - physiopathology; Peripheral Arterial Disease - surgery; Postoperative Complications - diagnosis; Postoperative Complications - mortality; Postoperative Complications - surgery; Predictive Value of Tests; Reoperation; Reproducibility of Results; Retrospective Studies; Risk Assessment; Risk Factors; Surgery; Time Factors; Treatment Outcome; Vascular Grafting - adverse effects; Vascular Grafting - mortality; Vascular Grafting - standards
• ISSN: 0741-5214; 1097-6809
• DOI: 10.1016/j.jvs.2015.09.051

Background Providing patients and payers with publicly reported risk-adjusted quality metrics for the purpose of benchmarking physicians and institutions has become a national priority. Several prediction models have been developed to estimate outcomes after lower extremity revascularization for critical limb ischemia, but the optimal model to use in contemporary practice has not been defined. We sought to identify the highest-performing risk-adjustment model for amputation-free survival (AFS) at 1 year after lower extremity bypass (LEB). Methods We used the national Society for Vascular Surgery Vascular Quality Initiative (VQI) database (2003-2012) to assess the performance of three previously validated risk-adjustment models for AFS. The Bypass versus Angioplasty in Severe Ischaemia of the Leg (BASIL), Finland National Vascular (FINNVASC) registry, and the modified Project of Ex-vivo vein graft Engineering via Transfection III (PREVENT III [mPIII]) risk scores were applied to the VQI cohort. A novel model for 1-year AFS was also derived using the VQI data set and externally validated using the PIII data set. The relative discrimination (Harrell c -index) and calibration (Hosmer-May goodness-of-fit test) of each model were compared. Results Among 7754 patients in the VQI who underwent LEB for critical limb ischemia, the AFS was 74% at 1 year. Each of the previously published models for AFS demonstrated similar discriminative performance: c -indices for BASIL, FINNVASC, mPIII were 0.66, 0.60, and 0.64, respectively. The novel VQI-derived model had improved discriminative ability with a c -index of 0.71 and appropriate generalizability on external validation with a c -index of 0.68. The model was well calibrated in both the VQI and PIII data sets (goodness of fit P  = not significant). Conclusions Currently available prediction models for AFS after LEB perform modestly when applied to national contemporary VQI data. Moreover, the performance of each model was inferior to that of the novel VQI-derived model. Because the importance of risk-adjusted outcome reporting continues to increase, national registries such as VQI should begin using this novel model for benchmarking quality of care.