Catalyst for Payment Reform (CPR) will spend significant time in 2015 looking at the features payment reform programs must have to be workable for purchasers and sustainable for providers. We look forward to sharing in this space what we learn, as we have done each month for over a year. But this month, we’ll take a slight detour to devote some “ink” to an issue near and dear to our heart—price transparency, a critical building block for payment reform.

Across the board, price transparency tools from vendors and plans are improving, but are the price estimates tools give consumers accurate? Unfortunately, the answer is often no. In general, as we discovered in work with our colleagues from HCI3, many price transparency tools suffer from one or more common methodological flaws. Below, we examine these flaws and their remedies.

First, many tools have incomplete definitions of medical episodes, which means they give consumers inaccurate price estimates for their full episodes of care. The classic example of this is a tool that shows consumers a price for “hip replacement surgery,” but the price doesn’t include outpatient costs like rehabilitation. The consumer who chooses the “lowest-priced provider” may end up spending thousands of additional dollars on out-patient follow-up care.

The tools can also throw consumers off when they don’t show sufficient quality information or details about potentially avoidable complications. Short term, choosing the lowest-priced provider may save a few co-pays. However, by the end of the episode, the patient might find they’ve spent a lot more than advertised, and also incurred a harmful complication.

Second, many tools need to do more to help consumers distinguish between needed and unneeded care, and how that fits into the price estimates consumers see. Consider the example of a consumer trying to pick a low-priced, high-quality provider for diabetes care. If the tool doesn’t parse out what is included in the price estimates, the consumer may end up picking a low-priced provider who doesn’t perform needed tests. Tools need to explain what care is included in price estimates and show apples-to-apples price comparisons among providers.

Third, some tools suffer from the problem of small “sample size.” A basic rule in math is that you cannot get a reliable average from a small sample of data. This is tough for the employer who wants its employees to have a price transparency tool, but may have a relatively small number of employees in a given geographic area. A small employee population means a small set of claims data. This means the price estimates derived from that data set may not be very reliable, and the actual experience of the plan member is likely to be different than the estimate. In trying to compensate for small sample sizes, some vendors may use multiple years of historical claims data to derive average episode or service prices. However, these prices may not reflect currently negotiated rates.

Fortunately, there are several steps employers, plans, and vendors can take to help address these common shortcomings. Foremost, tools should have well defined episodes of care. Tools should also educate consumers about quality and help them understand the frequency and costs (financial and health-related) of potentially avoidable complications, as well as what constitutes needed and unneeded care.  In situations where data sets are small (there is a limited amount of claims data) tools should be “transparent” about the price estimates they provide, and explain clearly that the estimations are rough approximations.

Of course, the accuracy of the price estimates that price transparency tools provide is only one piece of an elaborate puzzle. First, consumers must to be motivated to seek price information; this requires a robust engagement strategy and, often, innovative benefit designs. Then consumers must understand that price and quality vary. Both are important, and no, there is no correlation between high prices and high quality. Once consumers pass through these gates, they can look at price estimates, and concerns about accuracy become relevant.

Right now, the conversation around price transparency tools tends to be all about usability and engagement, which are indeed important. But plans and vendors can win these battles but lose the war if they get consumers to use a tool regularly that doesn’t provide reliable price estimates. The first time a consumer gets an explanation of benefits (EOB) with amounts radically different from the estimates the tool generated, her confidence in the tool will be shattered.