The Centers for Medicare and Medicaid Services (CMS) recently released financial and quality performance data for its accountable care organization (ACO) programs for the fourth performance year (PY4) of the Pioneer ACO Program and third performance year (PY3) of the Medicare Shared Savings Program (MSSP). CMS data reveal that the Pioneer and MSSP programs produced savings of $466 million in 2015 for Medicare, a small increase of about $55 million over savings generated in 2014. Additionally, average quality scores continued to increase in both programs.

In this post, we provide brief comments on the Pioneer ACO results with a more detailed look at the MSSP results, along with some reflections on the program going forward. Our analysis builds upon similar analyses of Pioneer and MSSP performance results from previous years.

Overview of Pioneer ACO Results

At the end of its fourth performance year (2015), the Pioneer ACO Program had 12 participating organizations, a decrease from its original enrollment of 32 organizations. While some organizations have abandoned accountable care initiatives for financial or strategic reasons, seven organizations transitioned into the Next Generation ACO Model, which launched in January 2016 with greater financial incentives and more regulatory flexibility than the Pioneer Model.

Eight of the 12 ACOs participating in the Pioneer Program in 2015 were able to hold spending below their financial benchmark, six of which reduced spending enough to earn shared savings. Four ACOs spent above their benchmark, but only one was responsible for repayment of losses. Average program quality scores increased from 2014 to 2015 (87.2 percent to 92.26 percent), while nine of the 12 ACOs had quality scores above 90 percent.

Nine ACOs chose to remain in the Pioneer ACO Model in 2016, the last year for the program. Six of those participants earned shared savings in 2015 and only one failed to reduce spending in 2015. With the Pioneer Program ending, these organizations could be motivated to transition into the Next Generation ACO model in 2017, but that will depend in large part on individual Pioneer assessments about the adequacy of rewards for quality and cost improvement and the sufficiency of the regulatory flexibility provided to implement needed payment and practice changes.

Overview of MSSP Results

Glass Half Full: Program Growth And Some Positive Trends

The number of participating ACO organizations in MSSP has grown each year. If current trend continues, the program could have over 500 participating organizations in 2017. With nearly 9 million Medicare beneficiaries attributed to the Medicare ACO programs (MSSP, Pioneer, and Next Generation Model), ACOs remain the largest type of alternative payment model (APM) in the Medicare Program. ACOs are likely to continue to be among the predominant APMs, particularly as providers are incented to move into APMs as the Quality Payment Program is implemented as required by the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA).

MSSP ACOs in 2015 generated total program savings (including all savings and losses relative to financial benchmarks) of $429 million, with 119 sharing in savings and none of the three Track 2 ACOs (subject to two-sided financial risk) required to repay losses to CMS. Eighty-three ACOs reduced costs but not enough to share savings, 189 spent above the benchmark but were not responsible for repayment of losses, and one ACO would have been eligible for shared savings but failed to satisfactorily report quality. One encouraging trend is that the percentage of ACOs earning shared savings has increased each year of the program from 2013 to 2015: 23.6 percent (52/220), 25.8 percent (86/333), and 30.4 percent (119/392), respectively. However, the percentage of ACOs reducing costs relative to their benchmark decreased in 2015 (51.8 percent or 203/392), down from 53.6 percent (118/220) in 2013 and 54.4 percent (181/333) in 2014.

The shared savings payment for eligible ACOs ranged from $855,000 to $41.9 million, with an average shared savings payment of $5.4 million (up from an average of $4 million 2014). Ten ACOs earned shared savings of greater than $10 million. Of the three ACOs that participated in Track 2 (opportunity for a sharing rate of up to 60 percent), one earned shared savings of $12.7 million; one reduced costs by 1.48 percent, not enough to earn shared savings; and one increased costs by 0.50 percent, not enough to trigger repayment of losses. Finally, one Track 1 ACO could have earned shared savings of greater than $800,000, but failed to meet the minimum quality standard to qualify (70 percent).

CMS noted that experience in the program is correlated with a higher likelihood of earning shared savings. Of those ACOs joining the program in 2012, 42 percent earned shared savings, compared to 37 percent of those that started in 2013 and 22 percent that started in 2014. This is an encouraging trend, as it seems to indicate that as ACOs get more experience, the likelihood of shared savings increases; although it is not known whether the correlation is impacted by other factors (e.g., organizations joining in earlier years may have had more population health experience than those joining in later years).

To probe deeper into the three-year trend of those ACOs joining the program in its first performance period we took a sample of the top 20 performing ACOs (highest savings rate) and bottom 20 performing ACOs (highest loss rate) [note: we excluded an outlier ACO with a loss rate in excess of 30 percent] and tracked their financial performance over the three-year performance period. In Exhibit 1 (top performers) and Exhibit 2 (bottom performers), each ACO is represented by a line with points showing their savings rate in Performance Year 1 (PY1) ending in 2013, Performance Year 2 (PY2) ending in 2014, and Performance Year 3 (PY3) ending in 2015. Those ACOs with a dashed line dropped out of the program following the second performance year.

Although this is a small sample of the overall program, it does suggest that low performing ACOs are capable of significant improvement over time. Four bottom performing ACOs in PY1 were able to achieve savings by PY3, and three earned shared savings; on the other hand, two top performers in PY1 sustained losses in PY3. Of the ACOs remaining in MSSP for all three performance periods, the low performers, on average, had a greater percent change increase in savings (63 percent) than the top performers (5 percent).

Exhibit 1. Change in Savings Among PY1 Top Performers, PY1 to PY3


Exhibit 2. Change in Savings Among PY1 Bottom Performers, PY1 to PY3


Note: Exhibit excludes one outlier ACO with losses in excess of 30 percent

Among all ACOs joining in PY1, 34 ACOs that had costs above their financial benchmark in PY1 were able to reduce costs relative to their benchmark in PY3, including approximately one-third of which were able to earn shared savings.

In addition to a growing proportion of participants earning shared savings, quality continues to improve across the program. CMS announced that ACOs reporting quality in both 2014 and 2015 improved on 84 percent of quality measures, with an average quality improvement of over 15 percent for four specific measures: falls risk screening; depression screening and follow-up; blood pressure screening and follow-up; and administration of pneumonia vaccinations. Additionally, 91 percent of ACOs in at least their second year of participation increased their quality improvement score in one or more of the four quality measure domains. ACOs earning shared savings had, on average, a higher quality score (92.7 percent) than the participants as a whole (91.4 percent).

Glass Half Empty: Overall Program Savings Lower Than Hoped And Many ACOs Still Struggling

Despite indications of positive trends in shared savings and quality improvement, some analysts continue to question the true impact of the Medicare ACO programs in meaningfully reducing costs both for individual ACOs and for the federal government. Although the percentage of ACOs earning shared savings has increased year-to-year, fewer than one in three participants have been able to reduce costs enough to be rewarded for their efforts and only about half of ACOs have been able to reduce costs at all. Without greater assurance of potential shared savings, some ACOs could choose to abandon their efforts altogether, particularly if they see no return on investment from the significant start-up and operations costs necessary to enter and maintain participation in the MSSP.

Additionally, there is a weak overall correlation between quality scores and the savings or loss percentage of MSSP participants. As Exhibit 3 illustrates, although this correlation is positive, rather than negative as in 2014, many organizations have not been able to simultaneously reduce costs and improve quality. It is perhaps not surprising to see quality improvement before significant improvement on cost control, but it remains unclear if, and at what point, the two will become more closely aligned. On the other hand, if current measures are not indicative of true quality, we might not expect high quality to correlate with better financial performance.

Exhibit 3. Overall Quality Performance Score Vs. Savings Rate for MSSP ACOs


Note: Exhibit excludes five ACOs with quality scores below 40 percent and 2015 starters (under pay for reporting)

More importantly, ACOs will need to perform well on their quality measures in order to maximize their shared savings rate (up to 50 percent in Track 1 and up to 60 percent in Track 2) and avoid leaving millions of dollars on the table. As noted earlier, ACOs earning shared savings achieved, on average, higher quality scores than those not earning shared savings. Track 1 ACOs achieved a shared sharing rate of 46 percent out of a maximum possible rate of 50 percent.

Much attention has been paid to the impact of financial benchmarks on individual ACO performance; indeed, Ashish Jha found that on average those ACOs able to reduce costs had both higher per capita spending and a higher per capita benchmark. Exhibit 4 shows the relationship between per capita benchmarks and savings rate. Similar to analysis done last year, there is a significant (p=<.001) correlation between the per capita benchmark and financial performance of a given ACO. While there are many factors driving ACO financial success, it is clear the benchmark is at least one factor of performance, potentially advantaging ACOs that have disproportionately less healthy beneficiaries or higher-than-average regional health care costs.

Exhibit 4. Per Capita Expected Spending vs. Percent Savings


Note: Exhibit excludes one outlier ACO with losses in excess of 30 percent

CMS is reminded with each new performance year that there will be attrition in MSSP. Between PY1 and PY2, six ACOs left the program, while 32 ACOs left the program between PY2 and PY3. Another six ACOs transitioned into the Next Generation Model following PY3. However, growth in the program has far outpaced attrition to date and may change once more as incentives increase for participation in APMs under MACRA.

Nonetheless, CMS must continue to consider concerns that drive participation decisions: the attribution model; financial benchmarking process; lack of return on investment (ROI); insufficient, unclear, or time-delayed data on performance; and lack of regulatory flexibility to allow innovations such as the use of telehealth. Although CMS has addressed some of these concerns with incremental changes to the program (e.g., recent changes to the benchmarking process and additional benefits including prospective attribution, certain regulatory waivers, higher sharing rates, etc.), these inducements are available only to those participating in two-sided risk models.

Finally, the federal government has not seen significant financial benefits from the MSSP so far. Jha observed that the amount paid out in shared savings to MSSP ACOs in 2015 ($645 million) actually exceeded savings to CMS ($429 million), resulting in a net impact to CMS of -$216 million. To Jha’s point, the Congressional Budget Office (CBO) projected positive program savings of $4.9 billion through 2019 in scoring MSSP savings for the ACA in 2010. However, CBO expected the vast majority of those savings to be generated in the latter years (2016-2019) of the MSSP. More recently, CBO emphasized this point for other non-statutory ACA payment reform initiatives.

The low rate of savings we have seen in the first three performance periods of MSSP is due at least in part to the fact that so few ACOs are participating in two-sided risk models, which encourage higher savings and also could result in a repayment of some losses back to CMS. As the number of ACOs in Track 2 and Track 3 increases, as was the case in 2016 with a total of 22 ACOs, the net financial impact to CMS may become more favorable. Whether MSSP will be able to meet the long-term CBO savings projections is uncertain, but expected changes in the risk profile of more MSSP ACOs suggest that a positive savings trend is still possible in the coming performance years.

Looking Ahead

As our analysis suggests, one can easily adopt a positive or a negative point of view on the performance results of the Medicare ACO program. On the positive side, quality continues to increase and the number of ACOs receiving shared savings has shown sustained growth; on the negative side, there is a significant gap between those doing well and those doing poorly, and in the aggregate the program has not made a significant contribution to controlling the rate of growth in Medicare costs. Additional research is needed to better assess what factors (size, value-based care experience, level of integration, geography, underlying financial attributes, patient risk profile, etc.) are helping to drive success for some organizations in MSSP over others.

However, it is worth noting that physician-led and smaller ACOs have continued to perform better on average than their larger counterparts. Exhibit 5 illustrates that ACOs with less than 6,500 attributed patients achieved average savings in 2015 nearly twice as high as those with 6,500 to 9,000 attributed patients — an increase in the trend from 2014. (To improve comparisons we preserved the same clusters used for 2014 evaluation, which resulted in less even quintiles for 2015.) Furthermore, ACOs with 13,000 or more patients, on average, experienced financial losses in 2015. These results continue to support the argument that physician-led ACOs may be well equipped to succeed at accountable care despite often having less start-up capital and less experience at bearing financial risk.

Exhibit 5: Average Savings Rate by Number of Beneficiaries


Regardless of one’s point of view, it appears evident that the Medicare ACO program will continue to grow, particularly as providers seek to join APMs to prepare for financial benefits under the MACRA Quality Payment Program — currently set to impact physician payment in 2019, but based on performance in 2017. For this reason, there may a larger than normal increase in Medicare ACO participation when the next class of organizations enters the MSSP beginning in January 2017. New entrants may want to use MSSP as a vehicle to gain experience with managing population health on a track to learn more about assuming financial risk. Many current ACOs may want to accelerate their move to Track 2 and Track 3 — although this would require more flexibility from CMS on program rules if done in the middle of a performance period.

In addition to increasing flexibility on moving between program tracks, CMS could play more of a role in easing this transition to two-sided risk models. CMS should consider accelerating innovations in the program, such as further refinements in setting and updating financial benchmarks, along with associated risk adjustment changes; experimenting with additional regulatory waivers; and other steps to help these organizations prepare for accepting downside financial risk. CMS should continue and perhaps expand its ACO Investment Model (AIM), which provides upfront and monthly payments to smaller and more rural organizations wishing to join MSSP. As some have suggested, CMS should also consider whether and how it might factor in some portion of an organization’s start-up investment into calculating the risk level, performance, and associated shared savings of a given ACO; if an organization is able to demonstrate return on investment with shared savings in its early years of program participation, it may have greater confidence in its ability to assume and manage two-sided risk.

In addition to enabling a more rapid move to two-sided risk, CMS should consider broader program changes to support all ACOs. CMS should continue to explore ways to provide more frequent and actionable data to organizations that can help them better understand how they are likely to perform over time and what areas need further improvement. As CMS learns more about the impacts of prospective attribution and voluntary beneficiary alignment in the Next Generation ACO model, it may also consider providing some of these benefits to advancing ACOs as soon as possible.

Progress has been incremental and variable, but nonetheless we are seeing improvement in the performance of ACOs in the MSSP. ACOs that are committed to the shift to accountable care recognize that there is still a lot of hard work ahead and financial success may not come immediately. For many organizations, accountable care will be a journey requiring significant time, resources, and creative innovation. As the most recent MSSP performance results illustrate, we are still in the early stages of that journey.

Authors’ note

This content represents the views of the authors only, and does not necessarily represent the views or professional advice of KPMG LLP.