NEJM report of ACO financial model fails to include risk of delayed start on transformation

On March 23, 2011, Trent Haywood, MD, JD, and Keith Kosel, PhD, MBA published the results in the New England Journal of Medicine (NEJM) web site of a financial model of a hypothetical Accountable Care Organization (ACO).  This model shows that ACOs are likely to lose money on the Medicare Shared Savings Program called for in the Patient Protection and Affordable Care Act during the first three years of implementing the ACO model, based on the up-front investment expected to be required.  The authors conclude that “the high up-front investments make the model a poor fit for most physician group practices.”  They call for modifications to the Medicare Shared Savings Program to make it more generous to participating ACOs.

The model is based on assumptions derived from data from the Physician Group Practice (PGP) Demonstration, carried out by CMS from 2005 to 2010.  In the PGP, the average up-front investment by participants was $1.7M, or $737 per primary care physician (PCP).  The authors calculate that an “unlikely” 20% margin would be required to break even during the 3-year time frame of the Medicare Shared Savings Program scheduled to start in 2012.

Haywood and Kosel are to be commended for taking the time to develop a financial model and publish results.  I think that such models are extremely helpful to real-world decision-makers because they force people to be explicit about the assumptions they are making, and they provide some quantitative estimates of the outcomes relevant to the comparison of available alternatives so people can make better choices.  Unfortunately, in my opinion, the authors misconceptualized the model, creating a risk that people will use the negative results of the model to justify inaction, to their own detriment.

Every decision is a choice among available alternatives.  To create a useful model to support decision-making, an analyst must follow the following four basic steps:

  1. Identify the available alternatives being compared
  2. Identify the outcomes that are relevant to the decision-maker and that are thought to be potentially materially different across the available alternatives
  3. Make quantitative estimates of  the magnitude and uncertainty of all such outcomes for all the available alternatives, and
  4. Apply values (including ethical principles and preferences) to determine which set out outcomes is most desirable or optimal

Although this basic process seems simple and straight-forward, experienced analysts know that each of these steps is devilishly difficult.  In the case of Haywood and Kosel’s financial analysis, in my opinion, they ran into trouble with the first two steps; they failed to identify the available alternatives and misconceptualized the choice or decision that the model is designed to support, and therefore failed to recognize non-Medicare outcomes that differ across the available alternatives.  Of course, an error in any particular step cascades to the remaining steps.

Haywood and Kosel did not explicitly explain the decision their model was intended to support.  But, one could infer from the conclusion that among the intended decisions they were supporting was the decision by physician organizations whether or not to make a $737 per PCP up-front investment and then sign-up for the optional Medicare Shared Savings Program in order to reap a return in the form of increased Medicare revenue.  But, the up-front investment required to create a successful ACO takes the form of fundamental transformation of care processes and the organizational structures, human resources, information systems, and cultural changes required to support them.  Such fundamental transformations affect the entire population served by the nascent ACO, not just Medicare patients.  And, they don’t just affect the providers’ relationship with payers, they also affect the providers’ competitive standing with respect to other providers and their relationship with other stakeholders such as employers, state and federal legislators, accreditation organizations, etc.

The correct conceptualization of the decision facing provider organizations right now is a choice between (1) getting started now with ACO-type transformation or (2) waiting until later to decide if such a transformation is necessary.  Physicians and hospitals that are contemplating the formation of ACOs would be wise to invest in the creation of a model to make explicit estimates of all the relevant financial and non-financial outcomes for the available alternatives.  Such a model will, by necessity, include many assumptions not supported by solid data.  That’s not the fault of a model, nor a reason to justify making decisions based only an intuition (what David Eddy calls “global subjective judgement”).  Rather, prudent health care leaders will invest the time to create and use a model to really understand the sensitivity of the results to various assumptions and the dynamics of the outcomes (how outcomes are likely to play out over time).

My prediction is that, when properly conceptualized as a “start transformation now” vs. “put transformation off until later” decision, such a model is likely to show what personal retirement planning models always show — it pays to get started on things that take a long time to achieve.  If you fall too far behind competitors, you may be unable to catch up later.  On the other hand, if provider organizations opt to get started on transformation, obviously there are many smaller decisions that need to be made, such as which care processes to start on, which particular payer-specific deals to cut, which IT investments to prioritize, etc.

One last point:  Although “pay back period” can sometimes be a useful summary measure of a financial analysis, my advice to to avoid over-simplifying the reporting of model results by reducing it down to a single summary measure.  Model authors would serve decision-makers better by presenting a table with their estimates of all the relevant outcomes for all the alternatives being considered, and possibly showing when those results occur over time.  Then, decision-makers can understand the drivers of their decisions and subsequently summarize the results in various ways that communicate their thinking most effectively using various summary measures such as net present value, return-on-investment, internal rate of return, pay-back period, cost per quality-adjusted life-year, cost-benefit ratio, etc.



3 thoughts on “NEJM report of ACO financial model fails to include risk of delayed start on transformation”

  1. Excellent argument! Totally agree and as posted elsewhere on LinkedIn, the ‘devil is always in the details’, i.e., footnotes and assumptions in the proforma(s). You correctly draw attention to the absence of ‘lost opportunity costs’ to inaction, or deferred innovation.

    Can we get you on ACO Watch: A Mid Week Review to discuss (


  2. Pingback: Part 1 of Critique of Milliman’s Whitepaper on Non-Medicare ACOs: Misinterpreting the Milliman “Well Managed” Actuarial Model?

  3. Pingback: Part 1 of Critique of AHA analysis showing higher ACO-related costs: Still underestimating care management cost

Leave a Comment

Your email address will not be published.

Free Subscription to Blog

Recent Posts