Feedback

Quality Measurement in Cardiac Surgery

David M. Shahian, MD, Jeffrey P. Jacobs, MD, Marshall L. Jacobs, MD, Sean M. O’Brien, PhD
Quality Measurement in Cardiac Surgery is a topic covered in the Adult and Pediatric Cardiac.

To view the entire topic, please or .

STS Cardiothoracic Surgery E-Book from The Society of Thoracic Surgeons provides expert guidance for Cardiac and Thoracic Surgery. Sections include Pearson’s General Thoracic, Esphageal, Adult Cardiac, and Pediatric and Congenital Cardiac Surgery. Explore these free sample topics:

-- The first section of this topic is shown below --

History of Quality Measurement in Cardiac Surgery

Release of Unadjusted Coronary Artery Bypass Grafting Outcomes by the Federal Government

Cardiothoracic surgeons have been systematically measuring quality and using those data to improve performance since the late 1980s, longer than any other healthcare specialty. The proximate stimulus for these efforts was the seminal publication by the Health Care Financing Administration (HCFA, the predecessor of the Centers for Medicare and Medicaid Services, or CMS) of hospital mortality reports (at the time, referred to by some as the “death lists”) beginning in 1986.[1] In addition to overall results, several individual conditions and procedures were profiled, including coronary artery bypass grafting (CABG) surgery.

The Society of Thoracic Surgeons Response

Unfortunately, the outcomes used in these reports were minimally risk adjusted,[2],[3],[4],[5],[6] leading many hospitals and cardiac surgery programs to complain to The Society of Thoracic Surgeons (STS) that the acuity of their patients was not being accounted for adequately. STS responded by forming an Ad Hoc Committee on Risk Factors, led by Dr. Nick Kouchoukos. In their subsequent report,[7] the authors presciently defined a number of critical principles of performance measurement (often referred to today as provider profiling) that have changed little over the succeeding decades:

“In order to arrive at an estimate of ‘quality of care,’ critical analysis of the population under study is required. Such analysis should include, but should not be limited to, determination of hospital mortality rates. Allof the risk factors that are predictive of operative mortality must be identified and subjected to appropriate statistical analysis before comparisons of mortality rates between institutions can be made. Although the HCFA analysis included such variables as age, sex, and race (indicators of case mix), it did not, in the opinion of some critics, adequately assess case severity, a serious deficiency when examining mortality.....

Thus, comparison of raw mortality rates among hospitals, without adequate information regarding the distribution of these and other important variables associated with increased mortality, and correction for differences in the frequency distribution of these variables if they exist, is inappropriate and misleading.... Analyses incorporating these and possibly other variables permit stratification and identification of subgroups of patients that are at different levels of risk for coronary artery surgery. Such analyses will also permit computation of risk models and expected operative mortality rates. Comparison of these expected or predicted operative mortality rates with observed mortality rates from different institutions will provide a more meaningful assessment of quality of care than the data provided by HCFA. Postoperative complications and other indices of hospital morbidity should also be incorporated into any assessment of quality of care. Such data are not available in the HCFA study.”

In this 30-year-old statement from STS, the foundations of responsible, scientifically valid quality measurement were clearly stated, and they remain central to STS practice today. Risk adjustment is essential for outcomes measures, and this must include all important risk factors. Optimally, robust, standardized clinical data are needed, not claims or billing data. A broadly representative benchmark population is necessary so that the results of each provider can be compared with what would have been expected for their patient mix (see risk adjustment section below). Finally, the authors correctly observed that mortality alone, even if risk adjusted, is not an adequate end point. Years later, this was the impetus for the development of STS multi-domain composite measures that incorporate both postoperative mortality and complications and, for some procedures, process measure compliance.

Development of the Society of Thoracic Surgeons Database

The recommendations of this ad hoc committee led to the recognition that a standardized clinical data source, collected prospectively by trained data managers, was necessary for robust risk adjustment and national benchmarking. STS leaders, led by Dr. Richard Clark, called for the development of a national cardiac surgery clinical registry,[8] and in response, the STS National Database was implemented in 1989.

Early Society of Thoracic Surgeons Risk Models

During the same time frame that the STS National Database was being developed, Dr. Fred Edwards introduced some of the first models for cardiac surgery risk adjustment, using Bayesian approaches.[9],[10],[11],[12] Multiple iterations of these models have been introduced over the succeeding years, most of which used logistic or hierarchical regression modeling.[13],[14],[15],[16],[17],[18],[19],[20],[21],[22],[23]

Confidential Feedback of Society of Thoracic Surgeons Results to Database Participants

The motivation for development of the STS National Database and its associated performance measures was the public release of flawed, inadequately risk-adjusted CABG mortality results by the federal government. Although created in response to misleading public report cards, the STS data and performance reports were initially used only for confidential feedback of nationally benchmarked results to participants, thereby facilitating performance improvement. The salutary effects of this approach were demonstrated by steady declines in rates of adverse outcomes and observed to expected (O/E) ratios, and increased use of evidence-based practices such as internal mammary artery grafts for CABG.[24],[25] Feedback of detailed performance information to participants remains a core function of the STS National Database today, some 3 decades after its initiation. These include quarterly reports of the participants’ risk factors, intraoperative and postoperative care practices and outcomes, benchmarked against STS national results as well as similar hospitals by size and teaching status (Figure 1); semiannual reports of STS composite measure performance with overall and domain-specific star ratings (Figure 2) and drill-down capability for process measures to guide improvement activities (Figure 3); performance on National Quality Forum (NQF)-endorsed measures; executive summaries, including overall STS national data; and extensive educational materials.

Figure 1
Descriptive text is not available for this image
Sample CABG Feedback Report to Participant .
Figure 2
Descriptive text is not available for this image
Sample STS Composite Performance Report for Coronary Artery Bypass Grafting.
Figure 3
Descriptive text is not available for this image
Sample STS Composite Performance Report for Coronary Artery Bypass Grafting: Process Domain “Drill-Down”.

Regional Data-Driven Collaboratives and Performance Improvement

At about the same time the STS Database was initiated, the Northern New England Cardiovascular Disease Study group (NNECVDSG), a consortium of 5 regional cardiothoracic programs in northern New England, developed their own data registry and risk models for CABG surgery.[26],[27] They used these data as the basis of a highly structured, best practice collaborative program that substantially improved results in the region.[28],[29] Some studies from that era have demonstrated that NNECVDSG experienced comparable improvements from confidential feedback and performance improvement as other states did with public reporting.[30],[31],[32] Highly successful registry-based cardiac surgery quality initiatives have subsequently been developed in Michigan[33] and Virginia[34],[35],[36] and have dramatically improved both clinical outcomes and costs.

-- To view the remaining sections of this topic, please or --

History of Quality Measurement in Cardiac Surgery

Release of Unadjusted Coronary Artery Bypass Grafting Outcomes by the Federal Government

Cardiothoracic surgeons have been systematically measuring quality and using those data to improve performance since the late 1980s, longer than any other healthcare specialty. The proximate stimulus for these efforts was the seminal publication by the Health Care Financing Administration (HCFA, the predecessor of the Centers for Medicare and Medicaid Services, or CMS) of hospital mortality reports (at the time, referred to by some as the “death lists”) beginning in 1986.[1] In addition to overall results, several individual conditions and procedures were profiled, including coronary artery bypass grafting (CABG) surgery.

The Society of Thoracic Surgeons Response

Unfortunately, the outcomes used in these reports were minimally risk adjusted,[2],[3],[4],[5],[6] leading many hospitals and cardiac surgery programs to complain to The Society of Thoracic Surgeons (STS) that the acuity of their patients was not being accounted for adequately. STS responded by forming an Ad Hoc Committee on Risk Factors, led by Dr. Nick Kouchoukos. In their subsequent report,[7] the authors presciently defined a number of critical principles of performance measurement (often referred to today as provider profiling) that have changed little over the succeeding decades:

“In order to arrive at an estimate of ‘quality of care,’ critical analysis of the population under study is required. Such analysis should include, but should not be limited to, determination of hospital mortality rates. Allof the risk factors that are predictive of operative mortality must be identified and subjected to appropriate statistical analysis before comparisons of mortality rates between institutions can be made. Although the HCFA analysis included such variables as age, sex, and race (indicators of case mix), it did not, in the opinion of some critics, adequately assess case severity, a serious deficiency when examining mortality.....

Thus, comparison of raw mortality rates among hospitals, without adequate information regarding the distribution of these and other important variables associated with increased mortality, and correction for differences in the frequency distribution of these variables if they exist, is inappropriate and misleading.... Analyses incorporating these and possibly other variables permit stratification and identification of subgroups of patients that are at different levels of risk for coronary artery surgery. Such analyses will also permit computation of risk models and expected operative mortality rates. Comparison of these expected or predicted operative mortality rates with observed mortality rates from different institutions will provide a more meaningful assessment of quality of care than the data provided by HCFA. Postoperative complications and other indices of hospital morbidity should also be incorporated into any assessment of quality of care. Such data are not available in the HCFA study.”

In this 30-year-old statement from STS, the foundations of responsible, scientifically valid quality measurement were clearly stated, and they remain central to STS practice today. Risk adjustment is essential for outcomes measures, and this must include all important risk factors. Optimally, robust, standardized clinical data are needed, not claims or billing data. A broadly representative benchmark population is necessary so that the results of each provider can be compared with what would have been expected for their patient mix (see risk adjustment section below). Finally, the authors correctly observed that mortality alone, even if risk adjusted, is not an adequate end point. Years later, this was the impetus for the development of STS multi-domain composite measures that incorporate both postoperative mortality and complications and, for some procedures, process measure compliance.

Development of the Society of Thoracic Surgeons Database

The recommendations of this ad hoc committee led to the recognition that a standardized clinical data source, collected prospectively by trained data managers, was necessary for robust risk adjustment and national benchmarking. STS leaders, led by Dr. Richard Clark, called for the development of a national cardiac surgery clinical registry,[8] and in response, the STS National Database was implemented in 1989.

Early Society of Thoracic Surgeons Risk Models

During the same time frame that the STS National Database was being developed, Dr. Fred Edwards introduced some of the first models for cardiac surgery risk adjustment, using Bayesian approaches.[9],[10],[11],[12] Multiple iterations of these models have been introduced over the succeeding years, most of which used logistic or hierarchical regression modeling.[13],[14],[15],[16],[17],[18],[19],[20],[21],[22],[23]

Confidential Feedback of Society of Thoracic Surgeons Results to Database Participants

The motivation for development of the STS National Database and its associated performance measures was the public release of flawed, inadequately risk-adjusted CABG mortality results by the federal government. Although created in response to misleading public report cards, the STS data and performance reports were initially used only for confidential feedback of nationally benchmarked results to participants, thereby facilitating performance improvement. The salutary effects of this approach were demonstrated by steady declines in rates of adverse outcomes and observed to expected (O/E) ratios, and increased use of evidence-based practices such as internal mammary artery grafts for CABG.[24],[25] Feedback of detailed performance information to participants remains a core function of the STS National Database today, some 3 decades after its initiation. These include quarterly reports of the participants’ risk factors, intraoperative and postoperative care practices and outcomes, benchmarked against STS national results as well as similar hospitals by size and teaching status (Figure 1); semiannual reports of STS composite measure performance with overall and domain-specific star ratings (Figure 2) and drill-down capability for process measures to guide improvement activities (Figure 3); performance on National Quality Forum (NQF)-endorsed measures; executive summaries, including overall STS national data; and extensive educational materials.

Figure 1
Descriptive text is not available for this image
Sample CABG Feedback Report to Participant .
Figure 2
Descriptive text is not available for this image
Sample STS Composite Performance Report for Coronary Artery Bypass Grafting.
Figure 3
Descriptive text is not available for this image
Sample STS Composite Performance Report for Coronary Artery Bypass Grafting: Process Domain “Drill-Down”.

Regional Data-Driven Collaboratives and Performance Improvement

At about the same time the STS Database was initiated, the Northern New England Cardiovascular Disease Study group (NNECVDSG), a consortium of 5 regional cardiothoracic programs in northern New England, developed their own data registry and risk models for CABG surgery.[26],[27] They used these data as the basis of a highly structured, best practice collaborative program that substantially improved results in the region.[28],[29] Some studies from that era have demonstrated that NNECVDSG experienced comparable improvements from confidential feedback and performance improvement as other states did with public reporting.[30],[31],[32] Highly successful registry-based cardiac surgery quality initiatives have subsequently been developed in Michigan[33] and Virginia[34],[35],[36] and have dramatically improved both clinical outcomes and costs.

There's more to see -- the rest of this entry is available only to subscribers.

Last updated: April 6, 2020