The quest for better health care, driven by measuring safety and quality, is well intentioned and has notable achievements. But like the Biblical story about building a better city, the measurement effort has become a cacophonous muddle that is distracting clinicians, raising the cost-of-care delivery, and not helping consumers make better health care choices.
The problem isn’t requiring measurement; health care needs meaningful measures. It needs measures to accelerate improvement in what matters most — health outcomes for patients during and after the care experience. Health care also desperately needs to measure the actual cost of achieving those outcomes. But most measures reported today are not focused on these critical determinants of value for patients and families.
Recent studies analyzing the different approaches highlight problems in the design of measurement efforts. In a recent Health Affairs study, a group of researchers compared the safety and quality conclusions of four national organizations — Consumer Reports, U.S. News and World Report, Leapfrog, and Health Grades. Not surprisingly, there’s very little agreement among the four on which hospitals are the best. These organizations’ reports explicitly acknowledge multiple purposes, they rely on different indicators of quality, and are often overly aggregated. These are predictable issues given the way that the measurement programs have developed.
But there clearly is something wrong in the architecture of measures when contradictory results emerge in studies of the same measure set. Critics and supporters of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) are at odds after two recent studies came to apparently conflicting conclusions.
After analyzing more than a decade’s worth of data from more than 1.2 million seniors who underwent one of 11 surgical procedures, researchers behind one report concluded that there was no difference in surgical safety between patients treated at the 263 hospitals that participated in ACS-NSQIP and the more than 500 hospitals that did not. Hot on the heels of that study came one exploring the incidence of post-surgical adverse events based on seven years of ACS-NSQIP data. That study concluded that participating “in ACS-NSQIP is associated with reductions in adverse events after surgery.”
Today’s required reporting has too many metrics. But despite their vast number, the metrics don’t measure the things that define health for patients and success for clinicians.
Too Many Required Measures
Health plans, medical societies, certification organizations, as well as county, state, and federal government agencies are all requiring health care providers to report safety and quality information. In the quest for better metrics, more and more measures are suggested, adopted, and their reporting mandated. The quantity of measures, unreasonable in most clinicians’ minds, adds to their workload and engenders frustration with measurement reporting efforts.
The comprehensive health measures database of the Agency for Healthcare Research and Quality, the National Quality Measures Clearinghouse, lists a total of 4,456 health measures. The Department of Health and Human Services’ database indicates its agencies currently use 3,801 measures. The Centers for Medicare and Medicaid Services (CMS) also maintains a Measures Inventory that lists 987 measures, including some, categorized as “pipeline measures,” that are still in development.
The lack of measurement standards results in multiple, slightly different measures focused on underlying data that is essentially identical. The National Quality Forum, a Washington-based organization that endorses health measures, recently cited three different studies that document how unmanageable the measurement reporting burden has become. There are 863 unique measures that must be tracked by organizations that deliver care under 33 different federal programs and there are more than 500 unique measures among about 50 state and regional sets of measures. On the commercial side, a collection of almost 30 commercial health plans require providers to report 550 different measures, few of which overlap with those required by government agencies.
Processes And Not Outcomes
Most of the measures presently in use track process compliance and not the outcomes of care. For example, the Hospital Compare website maintained by CMS includes 123 different measures, of which 102 track process compliance. To some extent, the measurement burden on providers is self-inflicted. Because clinicians don’t regularly measure and compare outcomes, regulators and payers focus attention on measuring inputs such as processes, certifications, and facility standards. Assessing activities performed rather than resulting achievements necessitates long, detailed lists of process checks that add markedly to the volume of measures. Requiring clinicians to measure and report the inputs, but not the outputs, adds to bureaucracy and administrative costs without requiring improvement in value for patients.
Good process is critically important. Moreover, process measurement is a good starting place for organizations to develop a culture of improvement. Yet, the plethora of process measures raises a question about whether the reporting of all of these measures is necessary. While a hospital, health system, or provider should track many things for the purpose of learning, comparison, and improvement, required reporting of such a vast array of health care inputs creates burdens far outweighing the benefits.
Clearly every clinician and hospital should be following evidence-based protocols and processes that improve safety. However, the critical, patient-centered questions address results. “What happened for the patient?” Measurement should be, first and foremost, about whether the patient got better (or had a better end-of-life experience) as a result of the care delivered. Health care is hamstrung in making improvements unless clinicians and leaders have information about the functional outcomes of care—what patients are able to do as a result of their care—and the reduction of suffering during care.
Perpetuating The Single Grade Delusion
The measures currently in use focus primarily on two dimensions of care delivery — safety and quality. They are obviously related. Unsafe care that injures a patient is clearly low quality. But these two dimensions of care delivery must be measured differently.
Safety can be measured across a hospital, much as workplace safety can be measured across an employer. A spate of hospital-induced infections or surgical mistakes likely indicates an unsafe environment for patients. Identifying and addressing those problems can be done across an entire hospital or clinic.
Quality is more nuanced. People seek care for a specific ailment or condition, and quality is defined within that context. Consequently, every type of measure need not be reported for every patient. While functional outcomes such as the ability to swallow and talk are critical to the quality of life for patients with head and neck cancer, these measures are not relevant for patients with prostate cancer, for whom potency and continence are the critical functional issues.
Quality for a well-baby check means something different than it does for heart surgery. Because quality in health care is so context dependent, it cannot be reported as a single letter grade or numerical score for a hospital or health system.
Consider a hospital that, based on accepted measures of health outcomes, is exceptionally good at knee replacement surgery and significantly below average at stroke care. What letter grade should this hospital receive? Tagging it a “poor” hospital would mean discouraging patients who need knee replacements from going there, although its results for this procedure are excellent. Recognizing it as a “good” hospital would likely cause some stroke patients to go there when care for that condition is poor. Evaluating quality requires a more nuanced approach, one recognizing that care at a single institution, department, or service line (stroke care versus epilepsy care, or knee care versus back care), can be both good and bad, depending on the medical circumstances for which the patient is being treated.
Health care needs to migrate—quickly—to measuring the functional outcomes of care for each patient. Health care’s contribution to quality of life and dignity of death depend largely on how care changes the functional outcomes that determine a patient’s capabilities, as well as the experiential outcomes of comfort and calm. Functional measures characterize health as something other than an absence of sickness. They do so in positive terms as the ability to participate in activities, interact with loved ones, live and die with less suffering and without healthcare-induced mayhem. When the patient is truly at the center, empathy demands attention to these outcomes.
The Patient Reported Outcomes Measurement Information System (PROMIS) and the International Consortium for Health Outcomes Measurement (ICHOM) have, respectively, national and international measure sets largely focused on functional patient-reported measures of outcomes. Registries that compare and benchmark these metrics will fuel improvement in care delivery and patient health outcomes.
Until outcomes of care are regularly measured, patients will continue to be confused by conflicting information and by the perpetuation of “eminence-” based medicine in which ambiguous reputation measures substitute for meaningful outcome metrics. The best thing health care can do—for patients and for the professionalism of caregivers—is to accelerate widespread comparisons based on measuring functional outcomes.