Are ACOs living up to their promise?

Accountable Care Organizations (ACOs) may not be as effective as many believe when it comes to reducing healthcare costs and improving outcomes, according to a new study.

The study, published in Annals of Internal Medicine, examines data from ACOs participating in the Medicare Shared Savings Program (MSSP). It finds that savings and quality improvements previously attributed to the design of ACOs may actually be the result of clinicians with higher-than-average costs leaving the organizations.

However, the National Association of ACOs (NAACOs) has challenged those findings, citing a study from several Harvard economists that questions the data and methodology used to reach the Annals study’s conclusions.

The authors of the Annals paper examined claims data from 2008 through 2014 from about 835,000 Medicare beneficiaries, 30,300 clinicians, 6,900 provider groups and 337 ACOs participating in the MSSP. They looked at per-beneficiary spending and quality performance in glycated hemoglobin testing, LDL cholesterol testing, diabetic retinal examinations and mammography.

Similar to many previous studies, they used an adjusted longitudinal model-one that included secular trends, market factors and beneficiary characteristics-to compare costs and outcomes between patients treated in practices that were MSSP ACO participants and nonparticipants. In addition, the study looked at the data using an instrumental variable model that included the impact of clinicians’ nonrandom entry into and exiting from the MSSP.

The authors note that despite the numerous variables that affect ACO participation, such as demonstrated ability to provide high-quality, efficient care, “research to date has not identified the degree to which selection bias may affect evaluations of the MSSP.”

The results showed that while MSSP participation was associated with spending reductions and clinical improvements under the longitudinal model, few or no improvements were evident under the instrumental variable model. In addition, the researchers found a strong likelihood of doctors with high average spending leaving the program compared with those whose spending was closer to the median level.

These results, the authors say, point to the conclusion that any savings Medicare has derived from MSSP ACOs may be “driven by nonrandom exit of high-cost clinicians and their patient panels” from the program. Moreover, “these findings suggest caution in extending ACOs to other settings and patients without stronger evidence that the program saves money or improves quality of care.”

But in a study posted on the website The Incidental Economist, several Harvard researchers take issue with the Annals study. NAACOS, in a written statement, says the study “identified multiple sources of bias in the [Annals] methodology, leading to erroneous conclusions,” and that contrary to assertions from the Annals paper’s authors, “prior published studies that demonstrated ACO savings controlled for the types of risk selection” discussed in the Annals study.

In its statement, NAACOS adds that multiple studies, including a recent one from the Medicare Payment Advisory Committee, have shown that ACOs save money for Medicare, and that the Annals paper “is not an indictment of the real savings ACOs have generated, but further evidence of how the model is unfolding.”

The Annals study, “Performance in the Medicare Shared Savings Program After Accounting for Nonrandom Exit,” appears in the June 18 issue.