The Federal Government has made an unprecedented effort to publicly finance research on the effectiveness of medical treatments.  The Patient Protection and Affordable Care Act of 2010 established both the Patient-Centered Outcomes Research Institute and the Center for Medicare and Medicaid Innovation (CMMI) to promote this research and identify areas where new evidence on treatments is most needed.  While most would argue that health care is filled with examples of uncertainty about which treatments work best compared to others, surprisingly little evidence exists on the relative value, or “comparative effectiveness,” of a vast array of treatments.  The comparative effectiveness research (CER) movement is a response to this uncertainty in medical care.

Despite the promise of CER to generate better evidence about what works — and does not work — in health care, its success or failure will depend entirely on how the results are used.  In this perspective, we identify four important challenges to harnessing the promise of CER.

First, CER must strive to remain patient-centric.  CER studies are similar to randomized clinical trials (RCT) in that they identify treatments that are most effective on average for a given disease population.  Clinical reality is typically far more complex, however, than even the best studies can accommodate.   Patients respond to treatments differently and a treatment that is more effective for a study population on average may be less effective than other treatments for a particular individual.  Furthermore, physicians may be more familiar with a particular therapy, and hence more successful with that therapy than what CER data would suggest.  So, while clinical decisions should be guided by the best available CER, CER evidence will not always supersede decision-making by a knowledgeable physician treating heterogeneous patients.

Figure 1 (click to enlarge) provides stylized examples of CER in the case of two treatments (of equal cost).  Each point represents the outcomes of a patient under two treatment regimens.  In an RCT, Treatment 1 will perform better than Treatment 2, on average, because there are more patients below than above the equal efficacy ray.  However, the minority of patients above the ray will respond better to Treatment 2.  These patients will have worse outcomes if they are ‘steered’ to Treatment 1.  In Case A, it is difficult to distinguish the patients.  In Case B, however, a well-done CER study could help differentiate the clinical populations and drive better treatment decisions.

                                                          FIGURE 1


Figure 1 highlights the importance of recognizing patient heterogeneity when deciding how to implement the results of CER.  For example, one approach would be a utilization management strategy (such as prior authorization) which steers all patients towards Treatment 1.  This approach will maximize patient outcomes if physicians are unable to discern whether a particular patient will respond better to Treatment 1 or 2.  However, in instances where patients have already arrived at medical regimens that are optimal for them – perhaps through trial and error prior to new CER evidence – or physicians are able to identify which patients will benefit most from which treatments, utilization management that is indiscriminately applied will worsen overall patient outcomes.  The amount by which patient outcomes would be worsened will be greater in Case B in which patients clearly benefit differently from the two treatments.

Schizophrenia provides a useful illustration.  Many treatments exist, each with its own unique side effect profiles, some of which are severe and permanent.  In 1999, the National Institutes of Health funded the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) – a study comparing first-generation anti-psychotics with newer second-generation medications.  CATIE found that newer anti-psychotics were no more effective than older medications, leaving some to call for reduced use of second-generation antipsychotics.

Although olanzapine, a second-generation medication, was found to be only non-inferior to first-generation anti-psychotics in the trial, there was a subset of patients for whom the second-generation anti-psychotic may have been more effective, leading some to argue for individualization of treatments in schizophrenia.  It has been estimated that requiring all patients with schizophrenia to be treated with first-generation anti-psychotics (rather than allowing patients who by trial-and-error were on second generation anti-psychotics prior to CATIE) would increase health care spending through increased hospitalizations and other health care utilization.

The successful implementation of CER must therefore allow for patients to seamlessly transition to other treatments when they fail the initial treatment supported by CER or when study circumstances do not apply.  Similarly, third-party payers must consider whether applying new CER evidence to patients already undergoing therapy may disrupt already optimal medical care.

Second, the successful implementation of CER will require better decision support for physicians and some degree of utilization management.  As emphasized by recent guidelines from the Patient-Centered Outcomes Research Institute (PCORI), most would agree that publishing new CER data by itself cannot change the practice of medicine unless it is coupled with measures that facilitate the adoption of new information.  Decision support tools integrated within the care delivery process – e.g. automated prompts reminding physicians to consider an aspirin, statin medication, and beta-blocker during acute myocardial infarction – are needed to ensure that CER data is not limited to the pages of scientific journals, but is carried into practice.  In fact, the ability to incorporate the latest CER evidence into daily processes of care is often cited as a reason why health care systems such as Intermountain Health and Kaiser are so efficient.

Although indiscriminate use of CER in utilization management may lead to worse overall patient outcomes, appropriate use of utilization management based on CER still has a role.  Particularly in instances where heterogeneity in patient responses to treatment is low or where physicians are unable to optimally select treatments for a given patient (as may be the case for new therapies for which a physician has little prescribing experience), CER-based utilization management can appropriately steer patients towards the most effective therapies on average.

Third, CER must evolve beyond medications and procedures and evaluate the comparative effectiveness of health care systems.  Most CER studies focus on the comparative effectiveness of medications or procedural interventions.  While these comparisons can generate substantial improvements in patient outcomes and cost-savings if properly implemented, much more can be gained from a better understanding of the comparative effectiveness of different health care delivery approaches. It is well known that health care delivery systems vary tremendously in the use of office visits, disease management, specialty consultations, hospital care, and imaging, yet the optimal mix of these services is unknown.  As much as 30 percent of health care spending in the U.S. could be reduced if low-efficiency health care systems were able to adopt the practice patterns of the highest efficiency systems.

At a hospital level, medical centers have a greater than two-fold range in the risk-adjusted cost of caring for patients with acute myocardial infarction, primarily due to how often patients are seen and referred to specialists and the extent of diagnostic testing and imaging that occurs.  The cost effectiveness of inpatient care for AMI ranges across hospitals from approximately $5,000 per life year saved to greater than $100,000, with very little of this variation related to rates of percutaneous coronary interventions within a hospital.  The efficiency of hospitals has little to do with the technologies available, but more to do with better organizing inpatient care and coordinating post-acute care.  Identifying components of hospital efficiency that can be translated nationwide should be a major focus of CER, a point recognized by PCORI in its recently issued list of national priorities.

Fourth, priority-setting for federally-funded CER should consider not only clinical priorities but the likelihood that evidence will be generated by private rather than federal funding.  CER need not be financed solely through federal research dollars, and indeed most of the effort in this area will be done by the private sector.  For branded medications which have incurred significant R&D costs, incentives already exist to conduct CER, either by a treatment’s manufacturer or competing manufacturers.  In fact, many effectiveness studies in oncology and heart disease are already financed privately.

In contrast, manufacturers of generic medications have little incentive to invest in CER since the benefits of new evidence accrue to the industry rather than the individual manufacturer sponsoring the study.  Recognizing these incentives is perhaps more important for surgical procedures, which tend to be costly, account for a substantial portion of health care costs, and yet have limited evidence on their effectiveness compared to other less costly and less invasive approaches.   Because private ownership stakes exist less often for surgical procedures, federal funding of CER in this area will always be necessary.  Priority-setting for CER should not only target prevalent diseases where more information is valuable, but should also recognize areas where private funding is unlikely to occur.

In sum, CER, done correctly, can enlighten the practice of medicine, improve the organization of the health care delivery system, and ultimately lead to better patient health.  However, this will require more – and better – evidence, shining a light in corners of the health care system that have traditionally remained in the dark.  If CER simply becomes a tool for steering all patients to the most effective – or cheapest – therapy on average, it may miss important patient heterogeneity and other opportunities to improve patient health.