Blog Home


The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari

March 12th, 2012

Our recent Health Affairs article linking increased test ordering to electronic access to results has elicited heated responses, including a blog post by Farzad Mostashari, National Coordinator for Health IT.  Some of the assertions in his blog post are mistaken.  Some take us to task for claims we never made, or for studying only some of the myriad issues relevant to medical computing.  And many reflect wishful thinking regarding health IT; an acceptance of deeply flawed evidence of its benefit, and skepticism about solid data that leads to unwelcome conclusions.

Dr. Mostashari’s critique of our paper, will, we hope, open a fruitful dialogue.  We trust that in the interest of fairness he will direct readers to our response on his agency’s site.

Our study analyzed government survey data on a nationally representative sample of 28,741 patient visits to 1187 office-based physicians.  We found that electronic access to computerized imaging results (either the report or the actual image) was associated with a 40% -70% increase in imaging tests, including sharp increases in expensive tests like MRIs and CT scans; the findings for blood tests were similar.  Although the survey did not collect data on payments for the tests, it’s hard to imagine how a 40% to 70% increase in testing could fail to increase imaging costs.

Dr. Mostashari’s statement that “reducing test orders is not the way that health IT is meant to reduce costs” is surprising, and contradicts statements by his predecessor as National Coordinator that electronic access to a previous CT scan helped him to avoid ordering a duplicate and “saved a whole bunch of money.” A Rand study, widely cited by health IT advocates including President Obama, estimated that health IT would save $6.6 billion annually on outpatient imaging and lab testing.  Another frequently quoted estimate of HIT-based savings projected annual cost reductions of $8.3 billion on imaging and $8.1 billion on lab testing.

We focused on electronic access to results because the common understanding of how health IT might decrease test ordering is that it would facilitate retrieval of previous results, avoiding duplicate tests.  Indeed, it’s clear from the extensive press coverage that our study was seen as contravening this “conventional wisdom”.

Nonetheless, Dr. Mostashari criticizes us for analyzing the impact of physicians’ electronic access to imaging and test results, but not other aspects of electronic health record (EHR) use.    We did, however, analyze the relationship of EHRs to test ordering in a subsidiary analysis.  While physicians use of a full EHR was associated with a 19% increase in image ordering, as we noted in the paper this finding was not statistically significant.  While we cautiously (and properly) interpreted this as a “null” finding, these data are inconsistent with Mostashari’s optimistic view that use of a full EHR reduces costs.

He asserts that our 2008 data are passe, and that health IT meeting today’s “meaningful use” criteria definitely saves money. The data we analyzed were the latest available data when we initiated the study.  While the proportion of outpatient physicians utilizing health IT has grown since 2008, we are unaware of any “game changing” health IT developments in the past four years that are would produce substantially different results if the study were repeated today.  The EHR vendors that dominated the market in 2008 remain, by and large, today’s market leaders, and their products have undergone mostly modest tweaks.  Mostashari’s contention that 2012  EHRs  – incorporating decision support and electronic information exchange – save money in ways not possible in 2008 should be tested through additional research but remains merely a hypothesis. We hope that some day his predicted savings can be achieved.

Dr. Mostashari offers his own explanation for our findings, suggesting that doctors who are inclined to order more tests are also inclined to purchase health IT for viewing test results electronically rather than on paper.  He offers no evidence for this assertion and ignores the fact that we explored (and rejected) this explanation by analyzing subgroups of doctors who are unlikely to be the decision maker for IT purchases – e.g. employed physicians, those working in an HMO setting etc.  In other words, electronic access to results predicted more test ordering whether or not the ordering physician was responsible for health IT purchases.

He incorrectly states that our analysis did not take into account patients’ severity of illness, physicians’ level of training, and the nature of physicians’ financial arrangements.  In fact, we reported subsidiary multivariate analyses that included several serious diagnoses; all of our models included physician specialty (which we specified in several different ways); and all models included adjustment for an extensive list of indicators of financial arrangements (e.g. whether the physician owned the practice or was an employee; the type of office; whether the practice was owned by a hospital; whether the physician was a solo practitioner; whether the physician’s compensation was based, in part or whole on “profiling”; and whether the practice was predominantly prepaid).  We also performed a series of subsidiary analyses that explored whether physicians with a proclivity to “self refer” patients for imaging tests accounted for our finding; they didn’t.

Dr. Mostashari criticizes us for failing to assess whether health IT improved the quality or appropriateness of care.  Of course, these were not the topic of our research.  Those are different studies for a different time.  However, we would note that other large-scale studies have found no, or trivial quality improvements associated with HIT outside of a few flagship institutions4-6.

Dr. Mostashari’s strongest claim is that observational studies like ours (and most other health policy studies, including some by Dr. Mostashari himself) cannot prove causation. This is surely true.  As long time teachers of evidence based medicine we took care to couch our conclusions in cautious terms, stating only that “Computerization, whatever its other benefits, remains unproven as a cost control strategy.”

But Dr. Motashari is less cautious, asserting that the case for HIT is closed.  The paper he cites to buttress this claim (authored by members of his own agency) culled studies reporting any impact of HIT on virtually any aspect of care, and accepted authors’ claims of benefit without regard to study quality or statistical niceties.  Thus, a focus group’s impressions of benefit are accorded the same weight as nationwide studies of Medicare data showing virtually no impact of computerization on quality measures.  Reports of a reduction from 70% to 38 % in “missed billing opportunities” or a $7,000 reduction in office supply costs are among the 92% of studies judged “positive”.  While the literature review he cites is interesting, nothing in it contradicts our findings.

Dr. Mostashari is also correct in reiterating that randomized trials are the best way to assess health IT.  In fact, no randomized trial has ever been published that examines patients’ outcomes or costs associated with off-the-shelf health IT systems that dominate the U.S. market.  No drug or new medical device could pass FDA review based on such thin evidence as we have on health IT.  Yet his agency is disbursing $19 billion in federal funds to stimulate the adoption of this inadequately evaluated technology. Dr. Mostashari is perhaps the only person in our nation who commands the resources needed to mount a well done randomized controlled trial to fairly assess the impact of health IT, and the comparative efficacy of the various EHR options.

Finally, Dr. Mostashari’s unbridled faith in technology is mirrored by his belief that ACOs are the next panacea for health costs and quality.  That health policy flavor-of-the-month also remains wholly unproven.

Email This Post Email This Post Print This Post Print This Post

 to the #1 source of health policy research.

4 Trackbacks for “The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari”

  1. What do we really know about e-health? Not much. - Science-ish -
    November 23rd, 2012 at 1:46 am
  2. from Healthcare Information System from CSI | Healthcare Information System from CSI
    March 28th, 2012 at 9:01 am
  3. Techno-Babble | Policy Prescriptions
    March 26th, 2012 at 8:05 am
  4. ICMCC News Page » The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari
    March 13th, 2012 at 3:13 am

5 Responses to “The Effect Of Physicians’ Electronic Access To Tests: A Response To Farzad Mostashari”

  1. ctashjian Says:

    Ah, where to begin.  I am a rural physician with 25 years of patient care.  I find the study simplistic and like some of the others “chippy”.  How does one study something that has multiple factors and reduce it to one?  

    We know if a physician owns an imaging center she will order more imaging.  We also know if one gets paid more for ordering more tests, they will order more.  So in some ways I agree with the authors that Dr. Mostashari can not attribute everything to the EHR.  

    However that is where my agreement with them ends. Kaiser has done a good job at eliminating those factors listed above.   In Minneapolis and St. Paul the physician community (yes including the radiologists) got together and drafted criteria for ordering MRIs.  Then a protocol was written and imbedded in the  EHR of all the major care systems. By using this protocol the cost of healthcare has been reduced in the twin cities and the care has been maintained.   This is not hypothetical but real.  

    This is why giving physicians the tools isn’t enough.  They have to be incentivized to use them.  As long as you pay physicians more to order more they will.  The whole idea behind ACOs is to bend the cost curve.  Those of us that are old enough can remember the HMO days, they DID bend the cost curve.  However at too high a price…choice.  Having access to information will be critical to improving care.   The EHR is the best way to obtain and use that information.  

    In our small clinic we implemented an EHR in 2010.  Like most offices the first year was challenging but with a dedicated staff we made it work.  Now we are using our EHR to build a registry and PROSPECTIVELY manage our patients — something we could never do in the paper world.   We now are taking on “Total Cost of Care” contracts for 2012 and we believe we can lower costs, while maintaining quality  care and participate in the savings, something we could NEVER do before.  

    In summary, I believe, like Dr. Mostashari, the EHR can (and has) reduce the cost of care, even in radiology, but only if the physicians are willing to use it in a triple aim fashion. 

  2. Says:

    Providers are at the mercy of information. The speed and visibility of information allows providers ability to make more concurrent decisions reducing delays in care. Accuracy and completeness of information is only as good as the source. Honestly, the fact that we can now measure reasons for ordering will hopefully drive an ability to truly manage overutilization. Without the informatics platforms this would not be possible. Historically it was me the provider doing the right thing for the patient and my justification was not codified.

  3. John D Waddell Says:

    HIT enhances knowledge. Knowledge is good. Nuff said.

  4. Peter Basch Says:

    McCormick et al. hoped that their response to Dr. Mostashari would lead to a “fruitful dialogue.” So how come after 3 days there is only brief response? I think the one comment posted provides the answer. Sure, there is lots of passion expressed by many who work within health IT fields. And I would agree that Dr. Mostashari’s blog post passionately articulated the potential benefits of health IT, and why one would not necessarily expect that the conditions and technology studied by McCormick et al. would show anything relevant to the Meaningful Use program. It was not a personal attack on the study authors. However, I have trouble reading their response to him without cringing. The one posted comment noted the gratuitous slap at ACOs; to that I would add that their response reads as an unnecessarily vicious and personal attack on Dr. Mostashari and his predecessor Dr. Blumenthal.

    Before I find myself added to this list, a few words… As my brief bio on the Health Affairs Blog shows, I am a practicing physician who is also an early adopter of an EHR; and for the past decade I have also worked on EHR implementation and policy issues. What my brief bio does not show is that my first posting to Health Affairs was a sharp criticism of a piece predicting enormous cost savings of health information technology and exchange.

    So in the interests of engaging in a fruitful dialogue without provoking further personal attacks, here are some general comments about conditions necessary to see benefit from studies of health IT systems.

    Health IT (which includes EHRs) per se is at best an enabling technology with no inherent ability to do good things. Mature health IT can improve the speed of complex processes; and that can lead to more spending or less; higher quality and safer practice – or the opposite. What establishes the directionality (and/or friction) for and within the processes that health IT enables are (in no order, all 3 are equally important) software (maturity, implementation, training, etc.), workflow, and incentives (i.e., payment policy). The obvious implication of this statement; aside from adding an interesting observation, unless a study carefully addresses these three dimensions (software, workflow, and incentives); its value is severely limited.

    For example, many of the reports that show value from health IT systems come from Kaiser Permanente. Kaiser Permanente has a particular business model that makes it possible to reward non-visit based care, proactive care, and population management. They also have finely tuned and ever evolving workflows that support their care models. And they use an excellent and highly customized EHR system. These published studies speak to workflow and how the EHR was used to achieve organizational or practice objectives. One would never expect their EHR vendor to foolishly claim that anyone using their product (regardless of implementation, workflow, and business drivers) would achieve the same results.

    Getting back to the McCormick paper… Yes, they may have read some poorly framed papers where uncritical over exuberance led to a statement such as “using a health IT system will reduce costs.” And Dr. Blumenthal likely recounted a personal narrative about how using an EHR system helped him to avoid ordering a CT scan. That said, neither outdated thinking nor a single illustrative story informed policy. Health IT policy has gradually matured over the past 10 years, and by the time the HITECH Act was passed in 2009, policy makers on both sides of the aisle clearly understood that the potential “magic” of health IT can only be unleashed with more mature product, used in meaningful ways, and under aligned incentives. Hence the “Meaningful Use” program and not the “Cash for Computers Program.”

    And if I may state this now, as I am near certain that studies will be published in the very near future, which will look at providers using certified EHR technology who successfully attested for Meaningful Use Stage 1. These studies will compare their quality scores to doctors who have not yet attested, are not using certified EHRs, and/or still using paper records. And more than likely the results will of course show that Stage 1 Meaningful Use did not improve quality. Details are important. Stage 1 of Meaningful Use is designed only to start doctors and hospital on their journey toward optimization of a health IT infrastructure. There is nothing in Stage 1 that would be expected to necessarily improve quality, safety, or value. So researchers – please don’t waste your time looking here. Your conclusions would be analogous to looking at a dark room with a newly installed lamp, and instead of commenting that the switch was in the off position and thus the room was expected to be dark, concluding that lamps do not produce light. It will not be until we are well into Stage 3, where the daily use of EHR systems is pointed towards improving performance on multiple dimensions that we could begin to see useful results.

    And finally, as thoughtful as the Meaningful Use program is, I would also expect that when it concludes, unless the US modifies payment policy such that volume, duplication and fragmentation are no longer primary drivers – I would expect that our (by then) excellent health IT systems would be “turned on their heads” to once again support the unsustainable volume-driven healthcare system that we are trying to move away from.

  5. jgogek Says:

    While in general I’m glad to see researchers vigorously defend their research, I’m left wondering about the gratuitous slam against ACOs in the last paragraph of the above blog post. Calling ACOs “flavor-of-the-month” and “wholly unproven” is simply pejorative and has nothing to do with the substance of the research in dispute. It tarnishes the authors’ objectivity. In fact, it makes them seem kind of chippy.

Leave a Reply

Comment moderation is in use. Please do not submit your comment twice -- it will appear shortly.

Authors: Click here to submit a post.