The Centers for Medicare and Medicaid Services (CMS) recently announced a Notice of Proposed Rulemaking (NPRM) for Medicare Shared Savings Program (MSSP) Accountable Care Organizations (ACOs). The rulemaking contains several proposals that if enacted, would fundamentally change the underlying incentives for providers to participate in the program. These proposed reforms address issues such as data sharing, renewals of participation agreements, beneficiary attribution, incentives to move to two-sided risk, and lastly, reforms to the benchmark calculations against which ACOs compete to earn savings.

The NPRM comes on the heels of a September 16, 2014 release of performance results for MSSP ACOs that began their performance years by 2013. Under the current program rules, ACOs that successfully reported quality performance data and whose savings exceeded their “minimum savings rate” were eligible to share in savings with Medicare. The MSSP program allows ACOs to choose either one-sided risk (Track 1, only upside potential to earn savings) or two-sided risk (Track 2, both upside and downside potential to earn savings/incur losses) with the final sharing amount based on achieving quality targets (up to 50 percent for Track 1 and 60 percent for Track 2). A vast majority of ACOs enrolled in Track 1, the one-sided risk option. Of the 220 ACOs in the program that participated in the first performance year, 53 earned shared savings, 52 saved money but not enough to meet the required “minimum savings rates,” and the other 115 did not accrue savings (spending on patients assigned to the ACO was greater than projected).

In February 2014, the CMS asked stakeholders for input as to how to improve its ACO programs, feedback which they used to generate the NPRM. Many ACOs and other stakeholders argued that failures to achieve savings over and above minimum savings rates were a partial result of residing in low spending areas. In this post, we examine the merits of this contention and consider the policy implications of our results and their bearing on some of the modifications of the MSSP program that CMS has proposed.

We also discuss other strategies for improving the program CMS did not mention in the NPRM. Specifically, we suggest that CMS consider graduated savings distributions that align financial incentives more closely with policy goals while acknowledging the diversity of contexts; for example by giving ACOs a greater share of initial savings and a progressively smaller share as the amount of savings achieved increases, and by giving ACOs in low spending areas a greater share of savings than their counterparts in high spending areas. We also propose that CMS consider rewarding ACOs that produce even small savings several years in a row, and that the agency look at ways to encourage ACOs to move non-Medicare payers into outcome-based contracts.


An ACO’s expected level of spending is calculated based on the prior 3 years of risk adjusted, fee-for-service expenditures among beneficiaries attributed to the ACO. This benchmark is established at the start of the 3-year agreement period and inflated annually by adding the absolute amount of growth in national per capita fee-for-service (FFS) Medicare expenditures. Although the annual inflationary adjustment allows for greater spending growth, percentage wise, in low spending regions, some respondents to the February 2014 CMS query argued that the methodology behind the initial benchmark projection makes it easier to meet or exceed spending targets in high spending regions due to its reliance on historical performance.

In essence, ACOs from low spending regions argue that the formula used to calculate spending benchmarks inappropriately rewards prior poor performance. On the other hand, the provision of low-value care in fee-for-service medicine is thought to be pervasive, and ACOs have incentives to ferret out such waste regardless of where they operate, thanks to the shared savings arrangements. Brent James, chief quality officer at Intermountain Healthcare, based in Utah – notably a low-cost region – has estimated that about half of all medical dollars spent are wasted in that they do not contribute to patient health. While others put this range somewhat lower – from 20 percent to 30 percent  – the potential to achieve savings without harming outcomes for patients appears substantial. Whether all ACOs are well suited to convert “wasted” dollars into shared savings, and the extent to which other factors like geographic payment differences matter, are open questions.

How Much Does Being In High Spending Or Low Spending Regions Matter For ACOs?

The timing of both the NPRM and the release of shared savings results by ACO in the MSSP program offers a unique opportunity to assess whether the claims about the importance of geography in achieving success are true. To address this question, we placed each MSSP ACO in the appropriate Dartmouth Hospital Referral Region (HRR), a commonly used geographic measure of where patients from a particular area receive a plurality of their tertiary medical care, and divided the HRRs into quintiles by 2010 Medicare spending. Further details on methods and limitations are available in the methodological appendix at the end of this post.

In our analysis, ACOs located in higher spending HRRs were significantly more likely to achieve shared savings with a significant trend observed as one moves across quintiles. ACOs located in the lowest spending HRR quintile were 35.5 percent less likely (6.2 vs. 41.7 percent; p<0.01) to achieve shared savings than ACOs located in an HRR in the highest spending quintile (see Table 1).


This finding supports the argument that ACOs in low spending regions are at some disadvantage in achieving shared savings relative to ACOs in high spending regions. While just 1 of 16 ACOs in the lowest spending HRR quintile achieved savings, this frequency grew bigger as spending increased, and 20 of 48 in the highest HRR quintile achieved savings.

Why might this be the case? One potential answer is what we might term the “low-hanging fruit argument:” ACOs in low spending regions have been performing well for long enough that an alternative payment model like the one in MSSP may not offer adequate incentives for their organizations to spend less than they already are. In fact, it could be that having already harvested the low hanging fruit, doing so might negatively impact quality. The reverse might be true for those ACOs in high spending regions, those without long histories of managing defined populations or utilization. This seems plausible, if practice patterns as described in Atul Gawande’s landmark New Yorker piece have really changed as much as it appears they have. However, to assess these claims, more data will be needed to perform direct comparisons on the processes of care between ACOs, and to compare performance longitudinally instead of this single snapshot in time.

Additionally, we note that the risk adjustment algorithm used by CMS to calibrate expected expenditures among an ACO’s attributed beneficiaries may add to the bias in favor of high cost regions. The methodology, known as the Hierarchical Condition Category (HCC) score, relies on demographic information and diagnoses derived from fee-for-service claims to predict expected spending. Yet, due to practice variations and coding bias, the frequency of diagnosis and the intensity of care may travel together with substantial geographic variation over and above any differences in health status from one area to the other. Therefore, we argue that a further evaluation of the relative contribution of the HCC method of risk adjustment to an ACO’s benchmark and its regional variation is warranted. This analysis may guide us to seek a different risk adjustment methodology.

Implications for Policy

As reflected in the comments submitted to CMS earlier this year, many ACOs and stakeholders argue that the incentives to provide accountable care are outweighed by the associated costs. If the benchmarking process makes it harder for ACOs in low spending regions to achieve savings — they may be less likely to form ACOs and less likely to stay in the program.

A survey of executive members of the National Association of Accountable Care Organizations revealed that a sizable majority (67 percent) of MSSP members were unlikely to sign up for a second 3-year contract in which they must accept 2-sided risk. CMS likely heeded this in the NPRM, which will allow ACOs to continue in Track 1 at a lower sharing rate for an additional contract. Some ACOs have also highlighted the rebasing of the benchmarks in the second performance contract as an additional disincentive.

Another important factor is the magnitude of other “ACO-like” performance contracts (e.g. with commercial payers) an organization may engage in, which may further drive efforts to redesign care; as more total patient revenues are aligned away from fee-for-service, the greater the incentives to invest in the necessary infrastructure to successfully manage population health and reduce total spending.

The NPRM from CMS seeks input on several alternatives to the current benchmark formula, for which our analysis may have relevance.

1. Equal weighting of 3 benchmark years 
Under the current formula, an ACO’s historical FFS performance in years 1-3 is weighted 60 percent in most recent year, 30 percent in 2nd most recent year and 10 percent in least recent year. This weighted average is intended to most accurately reflect the recent health spending of beneficiaries. The NPRM seeks input on a change that would weight each benchmark year equally (33.3 percent), which would likely give ACOs a more generous ending benchmark from which they may share in savings in future contracts.

2. Accounting for shared savings in future benchmark formula 
This option would increase the benchmark in subsequent contracts by the amount equal to the average per beneficiary savings earned. This reform would increase the benchmark in future contract periods offering ACOs a greater starting point, and alleviating concerns that they are “competing against themselves” in future contracts. Whether the adjustment should be for the full amount in the highest spending regions should be open to debate.

3. Using regional FFS spending to update benchmark
Current policy uses national FFS spending to trend and update the benchmark, which some have argued does not accurately reflect the impact of local markets on costs. CMS discusses an alternative that would instead use a methodology similar to one used in the Physician Group Practice (PGP) demonstration. The PGP calculated and updated benchmarks based on the cost experience from a local comparison group residing in the same service area as those beneficiaries participating in the demonstration, but which did not meet the assignment criteria. CMS discusses certain technical adjustments that may be necessary in order to identify a reliable comparison group, especially in areas with significant ACO penetration or perhaps wherein a single provider organization is dominant. The consequences of this change are difficult to predict because of the poor correlation between growth rates and levels of spending, while the current method (updating by national fee-for-service growth amount) does benefit ACOs in low spending regions.

4. Holding constant ACO historical FFS spending relative to regional FFS spending for all future agreements
In this scenario, CMS proposes to replace the current policy with respect to rebasing the benchmark with an alternative methodology. It would do so by holding constant an ACO’s per beneficiary FFS performance relative to its local market per beneficiary FFS in future agreements. This would guarantee to ACOs that gains realized within one performance contract would not be “paid back” in a future contract in the form of a lower benchmark.

5. Transition ACOs to solely rely on regional FFS spending over several agreements
CMS also discusses a potential reform of the way it sets and updates the benchmark based only on regional spending, as opposed to the current method that is based on ACO-specific spending. This reform proposal, endorsed by the Medicare Payment Advisory Commission, among others, would set the benchmark according to average, county-level FFS spending, which ACOs would compete against to earn savings. CMS posits that it could successfully embrace this approach over several agreement periods to ease such a transition. The advantage of such a move would be to give ACOs a prospective target, a change from the current model.

Based on our analysis, it is important to improve the financial model for ACOs residing in low spending regions. It is unclear how most of the potential reform ideas might impact these ACOs. For instance, Proposal 1 posits equalizing the weights of benchmark years would provide a larger benchmark from which to manage and earn savings, assuming the earlier years were more costly. Yet what if an ACO has been a historically efficient and low-cost organization for more than 3 years? Similarly, Proposal 2 is problematic for low cost ACOs in that, as our analysis shows, few are likely to earn shared savings, so incorporating this into their benchmark likely does little for them. Proposal 3 may benefit low-cost ACOs to some extent in that they would be competing against a regionally driven reference benchmark, but might come at the cost of not driving improvement among high cost ACOs. In each case, the proposals deserve careful analysis of their impact for all participants.

How else might we improve the model? We briefly discuss three additional strategies that did not appear in the NPRM, below.

Graduated Savings Distributions. An additional strategy that CMS might consider to promote greater adoption and maintain stability is to adjust the sharing rate to ACOs based upon the magnitude of per beneficiary savings achieved and regional spending levels. The goal would be to encourage participation, reward all ACOs for achieving savings, but ensure that the Medicare Trust Fund also benefits as the amount of savings achieved increases. Under this scenario, the saving rate would be higher for the first 5 percent of savings (perhaps 80 percent), and decline as savings increased (perhaps declining to 20 percent if per-beneficiary savings exceeded 15 percent). It could be further modified based upon the initial per beneficiary spending rate. In this scenario, an ACO in the lowest spending benchmark category would be eligible to receive a greater proportion of savings than would an ACO in the highest spending benchmark category. Graduating savings in this way would entail some controversy in defining the cut-off points for eligibility in each category, and its rate of sharing/loss, but would address the concerns elicited by our analysis.

Sustained Excellence Bonuses. Another approach that may unlock value for ACOs is for CMS to provide sizable bonus payments for those ACOs that save money for Medicare for a period of several years in a row even if they do not reach the minimum savings threshold in any single year. This sort of positive reinforcement may compel ACOs that may not immediately realize savings to continue within the program.

Incentives for Achieving Greater Payer Alignment Around Value. While ACOs may have substantial “spillover effects,” where the positive care improvements permeate to even non-ACO patients, systems where substantial patient revenues are still derived from traditional fee-for-service continue to have incentives to increase volume. Some ACO participants believe that multi-payer alignment has been critical to their ability to achieve greater clinical integration, necessary analytic investments, and so on. This is in line with what CMS has already put forth in the Pioneer program, requiring participants to move a majority of the ACO’s total revenue towards “outcomes-based contracts.” While mandating this requirement would likely be counterproductive, creating positive incentives to do so could help accelerate adoption and improvement.


Depending on one’s perspective, the early results of the MSSP are either promising or disappointing. On the one hand, nearly a quarter of qualifying ACOs achieved shared savings in the first year of performance; on the other, three-quarters either did not lower spending or did so but failed to exceed the minimum savings rate. Our analysis suggests that, using Hospital Referral Region spending as a proxy, the way medicine is practiced (or at least has been practiced) in a region is important to the ACO’s ability to generate shared savings under current benchmarking methdology. We conclude that some adjustments to the Medicare Shared Savings Program are warranted, but that those adjustments should not disrupt positive movements in other areas of the country. Improvements in the benchmarking formula, the risk adjustment methodology, and perhaps additional incentives to sustain high performance over a longer period are worth exploring.

Methodological Appendix: The Influence Of Geography On Likelihood Of Shared Savings For MSSP ACOs

We addressed the question by using the most current list of participating MSSP ACOs available on the CMS website, which includes United States Postal Service zip codes associated with each ACO’s operating address. Using the ACO’s zip code, we linked the September 2014 list of the MSSP ACOs to its corresponding Dartmouth Hospital Referral Region (HRR). The HRR is a commonly used geographic measure of where patients from a particular area receive a plurality of their tertiary medical care. Twenty-six (of 220) ACOs appeared on the 9/16/14 performance results list but not on the separate list of all MSSPs available from CMS, and were thus excluded, as they could not be accurately linked to an HRR. We categorized HRRs into quintiles by total 2010 Medicare reimbursements per beneficiary (adjusted for price, age, sex and race) as estimated by the Dartmouth Atlas of Health Care. To match the methodology used by CMS, only ACOs that generated surplus savings over and above the minimum savings rate while also successfully reporting quality measures were considered to have achieved shared savings. Finally, we used Pearson’s chi-squared and the Cochran-Armitage test for trend to examine the relationship of an ACO’s HRR spending quintile to the outcome of having achieved shared savings as reported by CMS.


These analyses are inherently limited, largely by the nature of these data. First, while the hospital referral region is a frequently used measure of geographic distribution of care delivery, an ACO’s population may not reside entirely within a single HRR, or may represent only a fraction of the population in an HRR, whose aggregated spending level obscures substantial, preexisting internal variation. Second, due to the exclusion of those ACOs that appear on the performance results but which could not be accurately assigned to an HRR, the size of our sample was reduced by 26 to 194, as opposed to 220 (88 percent of total). However, if the distribution of ACOs by HRR spending quintile held true for the rest of the sample, the results would remain effectively unchanged. Third, because we lack information on the care processes used by each ACO, it is not possible to assert that regional spending differences alone determine shared savings – other factors that we cannot observe and adjust for may, in fact, be more important. Finally, we note that the observed results in the first performance year of the MSSP may differ from future results; ACOs in lower cost regions may simply require more time to identify and achieve savings.

Editor’s note: Please note that the empirical model and analysis discussed in this post have not been subjected to external peer review.

The authors would like to thank Wade Harrison for his helpful comments and assistance on statistical analysis.