Editor’s note: In addition to Jonathan Darrow, this post is also coauthored by Aaron Kesselheim. 

The federal Food, Drug, and Cosmetic Act gives the Food and Drug Administration (FDA) the authority to evaluate all prescription drugs and high-risk medical devices before they can be marketed to physicians and patients to ensure that they are safe and effective.

However, there is growing pressure to lessen the traditional standards for defining “safe and effective” for particularly promising therapies and accelerate patient access to these products.

A recent national health policy conference in Washington, D.C., explored the nature of the evidence needed for the regulatory approval of new therapeutics and the implications for patient care. The conference was organized by the Program On Regulation, Therapeutics, And Law (PORTAL) at Brigham and Women’s Hospital/Harvard Medical School, the National Center for Health Research, and the American Association for the Advancement of Science (AAAS).

It provided a forum for reviewing the data that exist about the FDA’s current medical product approval criteria, and the implications of providing patients with expedited access to investigational prescription drugs and medical devices.

Conflicting Pressures

Among the themes explored during the day-long event were the various and sometimes conflicting pressures that the FDA faces as it engages in policymaking to promote the public interest. In the face of rising development costs, large pharmaceutical manufacturers and their lobbying organizations frequently appeal to the FDA and Congress to develop more efficient drug approval programs.

Patients, some desperately ill, urge the FDA to approve new drugs more quickly and can seek pre-approval access to investigational drugs on a compassionate use basis. Yet when FDA-approved products are linked to deadly adverse events, all eyes turn to the FDA for answers and its leaders may be brought before Congress to address why the public did not know about these outcomes sooner.

There are currently a number of regulatory pathways through which medical products that appear in very early testing to have an important impact on public health can be brought to market. For example, the accelerated approval pathway, formalized in 1992, allows certain drugs to be approved on the basis of meaningful surrogate endpoints such as cholesterol level or blood pressure rather than clinical endpoints such as stroke or death.

Originally described as a “highly specialized mechanism for speeding the approval of drugs or biologics that promise significant benefit over existing therapy for serious or life-threatening illnesses,” accelerated approval has accounted for approximately 10% of all new molecular entities in the past decade.

Using surrogate endpoints can shorten trials, but drugs approved under accelerated approval tend to be tested on fewer patients in their pivotal trials, averaging only 142 patients in the intervention group. This can lead to dilemmas for regulators and clinical conundrums for physicians and patients.

Shorter or smaller trials combined with the use of surrogate endpoints can also lead to evidence that is difficult to interpret: In a small trial, bedaquiline (Sirturo), a drug intended to treat multi-drug resistant tuberculosis, showed efficacy in converting sputum from M. tuberculosis positive to negative (a surrogate endpoint), but 2.5 times as many people died from tuberculosis in the bedaquiline group than in the control group. By contrast, sipuleucel-T (Provenge), a prostate cancer drug, failed to establish efficacy on the basis of time to disease progression (a surrogate endpoint) but demonstrated a 4.1 month survival benefit. Both drugs were approved.

Imperfect Knowledge

So how confident should physicians and patients be in using these drugs? Patients with multi-drug resistant tuberculosis or castration resistant prostate cancer may be willing to accept increased risks in return for the possibility of effective treatment. However, despite the legal requirement that patients consent to treatment, patients and even their physicians may not sufficiently comprehend the uncertainties inherent in the use of surrogate measures.

Imperfect knowledge arises not only from the complexities inherent in clinical trial data interpretation, but from inadequate communication of this information to physicians and patients. Draft FDA guidance promulgated in March 2014 addressed both problems — uncertainty and inadequate communication — by suggesting (but not necessarily requiring) that drug companies use the following statement when labeling drugs approved via the accelerated approval pathway:

“An improvement in {identify the specific clinical benefit that remains to be established} has not been established. Continued approval for this indication may be contingent upon…confirmatory trials.”

Although this language is intended to increase consumer awareness of the uncertainty inherent in relying on less-than-well-established surrogate endpoints, PhRMA, the industry trade group, expressed concern that the language “could be misinterpreted to mean that products approved under the Accelerated Approval pathway have not met the statutory requirements of safety and efficacy required of all FDA-approved drugs.”

Evidentiary gaps in the approval of new medical products may be even more pronounced with respect to medical devices. One study found that of 78 approval decisions related to the highest-risk cardiovascular devices, only 24 (31 percent) were supported by at least 1 randomized study, 10 (13 percent) by at least 1 blinded study, and 4 (5 percent) by at least 2 blinded, randomized trials.

In some cases, promising results from randomized but uncontrolled or non-blinded trials of medical devices have been called into question after larger and more rigorous controlled trials were conducted. For example, a catheter-based renal denervation treatment of resistant hypertension showed positive early results in unblinded comparisons between the device and medical therapy. These results led to cost reimbursement and widespread adoption.

However, a subsequent larger sham placebo-controlled trial failed to confirm the expected benefit. Other cases are even more troubling: A 45-person nonrandomized study for the treatment of intracranial atherosclerotic stenosis appeared to “provide compelling evidence supporting the safety of…the Wingspan stent system for stroke prevention,” but 6 years after FDA approval, a 451-patient randomized trial was stopped after higher rates of stroke or death were observed in the Wingspan group.

Extended follow-up reported in 2014 confirmed that the stroke or death risks associated with the Wingspan device persisted over time. Following an FDA-convened expert panel meeting in 2012 in which panel members agreed that there was no evidence of benefit for most patients, the FDA decided to narrow the indication that appeared on the Wingspan’s label but did not require withdrawal.

Premature approval of drugs or devices that may later be shown to be harmful or ineffective is a so-called Type I error. These errors are cause for concern, but it is also important to consider the harm that could result by unnecessarily withholding approval of drugs or devices that are safe and effective (a Type II error), particularly for patients with no treatment options.

Frequency of error, therefore, is a critical piece of data. One study examined 47 new indications associated with 35 oncology drugs approved between 1992 and 2010 on the basis of surrogate endpoints via the accelerated approval pathway: it found that 26 (55 percent) of the 47 were subsequently validated, while 3 (6 percent) were either withdrawn from the market or had their distribution or indications restricted (the remaining 18 (38 percent) were pending).

Ethical Questions

It is also important to consider the ethical questions associated with evidence sufficiency. Should the government restrict the ability of fully-informed patients to choose whether to accept treatment based upon available, if incomplete, evidence? Most people would likely say that expedited programs, such as accelerated approval, are appropriate for patients with terminal cancer or other seriously diseases who have no other options.

However, certain patient groups — even among those groups representing patients in the most dire situations — understand the complicated risks inherent in seeking access to unproven therapies.

While advocacy by AIDS activists in the 1980s helped spearhead the creation of the accelerated approval pathway and other expedited FDA review programs, some of those same activists later witnessed the trouble that can arise from widespread availability of poorly tested drugs of unknown effectiveness. In 1994, one Treatment Action Group member was quoted as saying, “We don’t just need drugs, we need information.”

Even worse, accelerated approval can impose substantial additional burdens on those already vulnerable from illness. In the 1990s, 41,000 breast cancer patients notoriously endured high-dose chemotherapy with autologous bone marrow transplantation based on the promise shown in early disease models; unfortunately, the therapy was later shown to confer no survival advantage.

Considering the FDA Brand

The medical community and the public rely on the FDA to keep ineffective and unsafe drugs from the market while efficiently approving important new medicines. Industry also benefits from the confidence and credibility bestowed on a product by the “FDA-approved” label. However, just as a car manufacturer’s brand can be undermined by having to recall a faulty ignition switch, so too can the FDA’s reputation be called into question by frequent withdrawals of approved drugs and devices across multiple companies.

The stakes are particularly high for the FDA not only because its products are intended to address matters of life and health, but also because Type I approval errors in this marketplace are unusually resistant to correction. The inability to conclusively establish or disprove efficacy at the individual patient level — combined with placebo effect and optimism bias — has driven significant company and patient resistance to FDA attempts at withdrawing products.

The difficulty of removing drugs from the market is not new, but, in the midst of intensifying public pressure for ever faster access, the number and variety of expedited approval programs has increased in recent years, and with them the potential for premature approval.

While timely regulatory review is a useful goal and expedited pathways may be appropriate in certain circumstances, the FDA’s default position should not change. Continued insistence on robust evidence development prior to approval is essential to maintain the high quality standards the public has come to expect from FDA-approved products.