On August 25, the Centers for Medicare & Medicaid Services (CMS) announced Performance Year Two (PY2) results for the Medicare Shared Savings Program (MSSP) Accountable Care Organizations (ACOs). (CMS also announced results for the Pioneer ACO demonstration, but this discussion is limited to the MSSP ACOs.)

In sum (Note 1), among the 333 participating PY2 ACOs, 86, or 26 percent, earned shared savings payments, meaning their claims costs for 2014 were below their financial benchmarks by an amount exceeding their minimum savings rates (MSR) (Note 2), and they reported complete quality information. Eighty-four Track 1 ACOs (upside risk only) and two Track 2 ACOs (upside and downside risk) earned shared savings. Total savings equaled $777 million for these 86 ACOs and, of this amount, the ACOs received $341 million in shared savings payments. Shared savings was less than 50 percent to 60 percent of total savings ACOs were eligible to receive. Had these ACOs achieved quality scores at or above 90 percent, they would have earned an additional $41 million.

Below, we briefly summarize the financial (see Exhibit 1) and quality performances of the PY2 MSSP ACOs, and then offer several lessons that policymakers should draw from these results.

Financial Performance Summarized

The 86 ACOs that earned shared savings were likely to have participated in MSSP for a longer time: Among the 111 ACOs that joined the program in 2012, 35 percent earned savings, compared to 18 percent of the 119 2014 ACOs. More experienced ACOs also generated greater savings per beneficiary: ACOs with an April 2012 start date generated $329 per beneficiary in shared savings, compared to $238 per beneficiary for ACOs with a 2014 start date.

Experience, however, was overshadowed by the fact that successful ACOs had on average 9 percent higher benchmarks compared to non-successful ACOs, or $10,500 per beneficiary versus $9,700 per beneficiary. (PY2 ACOs on average also had a 9 percent higher benchmark per beneficiary compared to the national fee-for-service [FFS] average.)

Savings were concentrated among the 86 successful ACOs: Five ACOs earned over $60 million in shared savings, and 30 earned well over $200 million. ACOs that were physician-based did better than hospital-based ACOs, and those with a federally qualified health center (FQHC) or a rural health clinic (RHC) performed better still. The number of beneficiaries served in successful ACOs was not a determining factor.

Exhibit 1


1 Calculated based upon only the number of assigned beneficiaries in the ACOs that earned shared savings.
2 Calculated only for those ACOs that earned shared savings.
Source: Author’s analysis of ACO MSSP 2014 Public Use File (PUF).

Quality Performance Summarized

PY2 was the MSSP’s first pay-for-quality performance year. This means earned shared savings were ultimately determined by how well each ACO delivered care as scored against 19 of the total 33 MSSP quality measure set. (The 14 other measures in the set remained pay for reporting in PY2.) There was no correlation between high quality performance and savings: Among the 60 ACOs that earned quality scores at or above 90 percent, only 22 earned shared savings.

The average quality score was 83 percent across all 333 ACOs, which is 7 percent below the 90 percent required to earn the full 50 percent of Track 1 shared savings. The average quality score among the 86 that earned shared savings was, however, only slightly higher at 86 percent. The 4 percent difference resulted in earned shared savings being reduced by $41 million for those 86 ACOs.

Quality performance within the preventive health domain was particularly modest. Scores for depression screening, pneumococcal vaccination, colorectal cancer screening, influenza immunization, mammography screening, and weight screening ranged from 39 percent to 67 percent.

What Do Two Years Of Performance Results Suggest?

To the extent two years of performance year data are meaningful, at least five observations can be made.

A Modest Number Of Successful ACOs Yields Modest Savings

First, that only 25 percent of ACOs have been successful explains the very modest amount of savings to date. Two-year MSSP savings equaled approximately 0.1 percent of total Medicare spending over the two performance years, or $1.5 billion on a base of more than $1 trillion.

ACO Success Has Been Driven By Higher Benchmarks

Second, success has been primarily determined by comparatively higher benchmarks. Lower utilization mattered only marginally. This point is likely best made by noting that the 92 ACOs eligible for shared savings in PY2 spent only $29 less per beneficiary per year ($10,537 versus $10,566) compared to the 67 ACOs that fell below their negative MSR.

However, in comparison to ACOs with expenditures above their respective benchmarks, ACOs that earned savings did achieve greater reductions in hospitalizations; emergency department (ED) visits leading to hospitalizations; skilled nursing, home health, and hospice utilization; and imaging. One notable trend among all 333 ACOs (also reflected in the FFS data) was the substantial increase in primary care visits attended by nurse practitioners (NPs), physician assistants (PAs), or clinical nurse specialists (CNSs).

Going forward, counting on a higher benchmark is neither a winning nor sustainable strategy over time for participating ACOs, since ACOs will see their benchmarks decrease over time — as long as benchmarks are determined historically. More generally, providers will be less likely to participate in the MSSP under the current rules knowing they have a one in four chance of being successful (and less still if their established benchmarks were low in comparison to, or more in line with, the national FFS average).

Reforming The Calculation Of Benchmarks Is Paramount

Third, CMS is well aware that the underlying regulatory issue largely determining ACO performance success is how financial benchmarks are calculated. CMS did decide in the agency’s June Final Rule to re-weight the three benchmark years from 10 percent-30 percent-60 percent respectively to equal weights of 33 percent for each year. The agency also agreed in that rule to include the ACO’s portion of shared savings in resetting its benchmark, to mitigate the effect of an ACO being penalized by its own good performance or having to chase diminishing returns.

Later this year, CMS has promised to issue a proposed rule that will outline methods by which ACO financial benchmarks can be updated using a regional growth factor. The theory behind such a method suggests that a benchmark based on regional costs, or a blend of regional and national costs, may create a more level playing field than the current method (whereby an ACO’s benchmark is based on the ACO’s three previous benchmark year costs and updated by a national growth factor). Time will tell.

Reforms In Calculating MSRs Are Also Needed

Fourth, another primary regulatory reason shared savings have been limited is the MSR. The ACA’s MSSP authorizing provision, Section 3022, requires the Secretary to account for an “appropriate percent …for normal variation in expenditures.” Track 1 ACOs that make up nearly the entire MSSP have an MSR ranging from 2 percent to 3.9 percent. The three Track 2 ACOs have a flat 2 percent MSR.

Among the unsuccessful PY2 ACOs, 174, or 52 percent of all ACOs, had claims costs or spending fall within their MSR and therefore did not qualify for shared savings. The 89 ACOs, or 27 percent of all ACOs, that fell within their positive MSR, meaning they held claims costs below their benchmark but not by a percent higher than their MSR, were not eligible to receive any of the $168 million in total savings they achieved. (Sixty-seven PY2 ACOs, or 20 percent, fell below their negative MSR. Their claims costs against their benchmark exceeded normal variation. Had these 67 been Track 2 ACOs, they would have had to repay the amount of claims exceeding their negative MSR to CMS.)

To the agency’s credit, CMS has shown it can be flexible in how it applies the MSR. In the June MSSP Final Rule, CMS decided to allow at-risk Track 2 and newly created Track 3 ACOs (which take on greater upside and downside risk) to choose a 0 percent MSR or one from 0.5 percent to 2.0 percent in 0.5 percent increments. CMS could also modify the MSR for Track 1 ACOs such that it’s not all or nothing. Under current rules, the ACO either exceeds its MSR and is rewarded with shared savings or does not exceed its MSR and is rewarded with nothing. Dartmouth has suggested that if an ACO spends below its benchmark then CMS could pay the first dollar of earned shared savings by paying decreasing shared savings percentages in tranches.

Lost Shared Savings Due To Quality Could Have Been Greater

Finally, the $40 million lost shared savings would likely have been more if all 33 quality measures were pay-for-performance. The way quality scores are calculated reinforces the ACO provider community’s view that quality performance benchmarking is simply punitive. It’s penalty only. Quality performance measurement acts as a multiplier; even if an ACO scores the maximum number of quality points, its quality multiplier is 1.0. This distinguishes the MSSP quality benchmarking from the Medicare Advantage (MA) program, where CMS pays bonuses to plans with four or more quality stars, or to one-third of all MA plans.

While CMS did decide in the agency’s June Final Rule to award additional quality points for significant year-over-year quality improvement measurement, scoring remains penalty only. As with MA, CMS could award a bonus to top-tier quality performing ACOs. For example, CMS could increase earned shared savings by some additional percent above the current 50 percent for Track 1 and 60 percent for Track 2 participants.

Looking Forward

Considering the limited success of the MSSP to date, one external factor is worth noting. In the ecology of Medicare payment reform, CMS can never do one thing. In July CMS announced a mandatory five-year demonstration, the Comprehensive Care for Joint Replacement (CCJR), to test 90-day bundled payment care episodes for hip and knee replacement surgeries in 75 markets beginning this January.

The CCJR could have several unintended adverse effects on the MSSP. Because the CCJR is redundant with the MSSP (ACOs are already responsible for total beneficiary Part A and B costs), the demonstration threatens to shift savings away from the MSSP and/or leave ACOs with lost opportunity costs. The demonstration as proposed inadequately protects against already considerable over-utilization or unwarranted variation, thereby indiscriminately adding claims costs to ACO benchmarks.

As proposed, the CCJR demonstration makes no effort to knit together the CCJR and the MSSP or to create synergy with the MSSP. This means CCJR hospitals are not required to collaborate and/or gain share with ACOs in instances where the ACO and hospital are unaligned and the hospital is treating an ACO-assigned beneficiary.

The Medicare Shared Savings Program has become the ACA’s flagship Medicare reform program. It is an essential component to CMS’s efforts to transform Medicare service delivery from volume to value or to alternative payment models. With over 400 ACO participating provider groups, support for and participation in the program has been substantial.

If CMS wants to maintain and grow provider participation, results to date are both instructional and cautionary (Note 3). They clearly suggest that success so far has been based on leveraging past performance through higher benchmarks with little correlation to quality. CMS should quickly revise its benchmarking methodology to reduce—if not eliminate—participation bias towards those with higher benchmarks, improve financial incentives, reward quality, waive antiquated payment rules, and encourage beneficiary participation.

Note 1

All data provided is obtained or derived from three sources: The CMS public use file “Medicare Shared Savings Program Accountable Care Organizations Performance Year 2014 Result“; the CMS public use file “Medicare Shared Savings Program Accountable Care Organizations Performance Year 1 Results,”; and the slide deck “Medicare Shared Savings Program Webinar: Performance Year 2014 Quality Performance and Financial Reconciliation Results for ACOs with 2012, 2013 and 2014 Start Dates.”

Note 2

The MSR is designed to account for normal spending variations; ACOs are not eligible to share in savings, or (for Track II ACOs) liable to share in losses, until they exceed their positive and negative MSRs, respectively.

Technically, 85 not 86 ACOs earned shared savings in PY2, since one advanced payment ACO received a shared savings payment but owed more in repayment for its advanced payment. Six ACOs did not qualify for shared savings because they did not satisfactorily report their quality measures; they generated $29 million in total savings.

Note 3

PY1 results were very similar to the PY2 results. Nearly the same percent of PY1 ACOs earned shared savings: 24 percent, or 52 out of 220 PY1 ACOs, earned shared savings, including two of five PY1 Track 2 ACOs. Total PY1 program savings, $705 million, were almost as high as PY2 savings despite there being one-third fewer PY2 participants. Average savings per PY1 ACO were $6 million, substantially higher than the $4 million average savings per PY2 ACO, due in part to the fact that quality performance benchmarking in PY1 was simply pay-for-reporting.

Savings were also concentrated among select ACOs in PY1. Twelve PY1 ACOs earned approximately half of all savings. PY1 also had a similar percent, 54 percent or 119 ACOs, that fell within their MSR. The same 20 percent, or 43 ACOs, fell below their negative MSR. Again, successful ACOs had comparatively higher benchmarks and were able to reduce hospitalizations and ED visits, skilled nursing and home health utilization and, again, leveraged NPs, PAs and CNSs in comparison to unsuccessful ACOs.

Authors’ Note

These comments are the authors alone, they do not necessarily represent the views of their employers.