Editor’s Note: Should drugmakers and medical device manufacturers be allowed to provide physicians with medical and scientific journal articles concerning uses of their products that have not been approved by the Food and Drug Administration? Recently, the FDA issued draft guidance that would permit this practice with certain regulatory restraints. In the post below, Jerome Kassirer argues that the guidance would allow the distribution of potentially biased information and would discourage rigorous FDA review of new uses of drugs and devices. Scott Gottlieb, however, argues that the guidance would facilitate access to potentially life-saving information while ensuring that any research distributed is credible and transparent.
Two of the principal criteria of the Food and Drug Administration’s (FDA’s) new guidance on distribution of reprints describing off-label drug use are that the journal of origin must be peer reviewed and have a disclosure policy. The notion that peer review and disclosure will protect the public is, in my judgment, magical thinking. Anthropologist Philips Stevens Jr. says that magical thinking invests symbols with special powers and forces. Peer review and disclosure are two of these powerful symbols, and, in my opinion, both have been afforded far more credibility than they deserve.
It goes without saying that the veracity of medical information is a key element of optimal therapeutic decision making. Until recently, insufficient attention was paid to the accuracy of such data. Physicians generally knew that journal supplements, almost invariably sponsored by a drug company, were frequently unworthy of their trust, but the quality of articles in peer-reviewed journals generally had a better reputation.
But even among peer-reviewed journals there is a widely understood hierarchy. Some, particularly those journals with robust full-time editorial staffs and venerable reputations, have the toughest peer-review standards. Others, mostly those staffed by editors whose day job is running a department or division, may not have the expertise to deal with the crush of new manuscripts. Their journals often receive papers rejected from the top tier of journals: it is fair to say that there is often a reason why these papers were rejected; yet they often end up published in a peer-reviewed journal and could be distributed by drug representatives. Some of the editors of these lower-tier journals simply rely on the recommendations of reviewers: if two reviewers check off “accept,” they accept the paper; if two check off “reject,” they reject the paper; and if the two reviewers disagree, they send the paper to a third reviewer. This is not to say that information in the best medical journals is invariably scientifically sound or credible; it is not.
Unfortunately, the objectivity of the recommendations in articles published in peer-reviewed journals can vary considerably. Peer review is a human endeavor. It may be the best system we have to deal with the information generated by a massive clinical research enterprise, but there is no guarantee that any particular published paper is not fatally flawed or biased in favor of a company’s drug.
Across the hierarchy of journals, the quality of clinical research also varies by study design. Observational studies that have no controls are always suspect: it’s the reason why they have been largely replaced by better designs. Observational studies and case-control studies do have value, but they are most helpful in identifying hypotheses that are ripe for further testing; as such, they might not be directly applicable to patient care. Nonetheless, they too could be eligible for distribution. Double-blind, controlled trials have the most credibility, although even they could be unreliable or biased. Concern about flawed research has risen considerably with the concomitant increase in the fraction of clinical research supported by the drug industry and often run by physicians who are being paid substantial fees by the companies whose lifelines depend on positive outcomes.
Many approaches have been employed to rig the outcomes of company-sponsored studies: inappropriate study designs, lead-time biases, failure to report all the outcomes, restricting access to investigators of primary data, excluding patients after randomization, unblinding, suppressing unfavorable or negative results, and, finally, using inexperienced “investigators” to run the trials. It is magical thinking to believe that just because a paper is published in a peer-reviewed journal, and even if its editorial board consists of experts (another requirement of the FDA guidance), the authenticity and objectivity of its drug recommendations are assured.
On reading the disparaging and trenchant remarks of two major journal editors, Richard Smith and Richard Horton, one would conclude that the entire clinical research enterprise has been corrupted by industry and that we should believe nearly none of it! The closer one gets to the primary data (from opportunities in which information has been exposed in lawsuits against major drug manufacturers), the more one is inclined to believe that their judgments are correct.
Disclosure: A Necessary But Not Sufficient
Remedy For Conflicts Of Interest
Another example of magical thinking is the cleansing property of disclosure. According to many, disclosure is the “great disinfectant” — the solution to conflict of interest, and the proper way to manage conflicts of interest. That is why medical journals insist on disclosing authors’ financial conflicts, and why accrediting bodies insist that lecturers to medical audiences must disclose their financial conflicts in some visible fashion even before launching their talks. Let the audience know that these conflicts exist and then go on to say anything you want! Medical journals across the spectrum have been openly chastised by the press when they failed to elicit or publish authors’ financial conflicts. This brouhaha over lack of disclosures continues to this day, even though the lack of disclosure is not the problem. The conflict is the problem, not the lack of disclosure.
Simply stated, disclosure is a necessary but not a sufficient solution. Why? Once a financial conflict is disclosed in a paper, it becomes the task of the reader to determine what to make of it. What was the author’s motivation? Did the author believe he was being completely unbiased, or did the powerful financial connection between the author and some drug company unconsciously move him to bias a study design or interpretation of evidence in favor of the company paying his honorarium? Could it be possible that the writer has biased her interpretation consciously to incur favor with the company? This uncertainty regarding the motives of conflicted authors underlines the problem of interpreting disclosures of conflict of interest: It simply leaves us guessing about what to believe. In fact, many people believe that disclosing a financial conflict — or two or three or twenty — gives the conflicted person a ticket to say just about anything, biased or not, with impunity.
Setting aside the inadequacy of the protective power of peer review and disclosure, acceptance of the new FDA guidance raises many other concerns. Would journal publishers be more eager to publish biased trials just to sell more reprints, as their sagging subscriptions erode their profits? Are political motives, designed to favor pharmaceutical company marketing, behind the new guidance? Has the FDA once again kowtowed to industry? Is the FDA’s user-fee system to blame for its tilt toward drug companies? What is the legitimate role of government in protecting the public from inordinately aggressive marketing by the drug industry? How can we continue to allow the drug companies to be the principal source of physician continuing education? At what point will our willingness to pay for the escalating cost of drugs trump the profits of the drug industry?
Where Should Doctors Get Information
On Off-Label Uses? Not From Manufacturers.
And finally, where should doctors get their information about off-label drugs? For my money, once there is a reasonable suspicion that an already approved drug might have a new use, it must be examined in the same rigorous way as the approved use. At the very least, detailed post-marketing surveillance, as practiced in other countries, should be mandatory. All new indications for a drug should be examined by the FDA for efficacy and toxicity before a company can market the drug for a new use. Regulatory scrutiny of off-label drugs should not be relaxed.
By requiring peer review, disclosure, and editorial-board expertise, the new FDA guidance offers a veneer of credibility and respectability, but if you scratch this glossy surface, you find more marketing, more inappropriate drug use, more expense, and more adverse consequences.
If I learned that my doctor was getting his information from industry-paid lecturers, in seminars sponsored by drug companies, or from reprints selectively given to him from drug reps, I’d get another doctor.