Estimates of ACO savings in the presence of provider and beneficiary selection

Background: Medicare's accountable care organizations (ACOs)—designed to improve quality and lower spending—were associated with growing savings in previous studies. However, savings estimates may be biased by beneficiary sorting among providers based on healthcare needs and by providers opting into the program based on anticipated gains. Methods: Using Medicare administrative claims (2009–2014), we compared annual spending changes after provider organizations joined ACOs to changes in non-ACOs (controls). To address provider selection, using novel data to identify non-ACO organizations, we restricted controls to comparably large provider organizations. To address beneficiary selection, we (a) estimated within-organization (including non-ACO comparison organizations) spending changes, (b) estimated within-beneficiary spending changes, (c) incorporated beneficiaries without qualifying healthcare expenses, and (d) used a fixed beneficiary ACO assignment using the pre-ACO period. Results: Each year, 19% of Medicare beneficiaries switched provider organizations. Spending was higher for switchers than stayers ($3163, p < .001) and grew more the next year ($2004; p < .001). Starting from a baseline regression modeled on previous ACO evaluations, estimated savings varied widely as we sequentially introduced methods to address selection. Combining methods, however, generated more stable estimated ACO savings of $46 (p = .022), averaged across cohorts. Conclusions: When implementing a comprehensive suite of methods to adjust for provider and beneficiary selection, we estimated ACO savings that grew over time. Our estimates are in line with, but smaller than, previous estimates in the literature. Implementing piecemeal adjustments produced misleading results. Implications: Our results confirm the importance of selection for savings estimates and for provider organizations managing costs and quality. Attribution rules that consider multiple years may help mitigate the impact of beneficiary churn for providers and payers. Implementing payment reform by randomizing early participants, or implementing fully across selected markets, may better serve efforts to evaluate and improve payment models. Level of evidence: Level 3.