Part 1 of Critique of Milliman’s Whitepaper on Non-Medicare ACOs: Misinterpreting the Milliman “Well Managed” Actuarial Model?

Victoria Boyarsky, Howard Kahn, David Mirkin, and Rob Parke from Milliman recently released a Healthcare Reform Briefing Paper entitled “ACOs beyond Medicare.

In the paper, the authors argue that the Medicare Shared Savings Program (and the associated regulations regarding the formation of ACOs), as defined in the draft regulations released on March 31, 2011, will make financial success elusive in the Medicare population for emerging ACOs.  They argue that providers should consider focusing instead on commercial (non-Medicare) populations.  To “measure the benefits and costs of forming an ACO”  in commercial populations,  the Milliman authors used their actuarial models. Using these models, the authors estimate that the per member per month (PMPM) costs of a typical commercial health plan delivered by “loosely managed, uncoordinated fee-for-service providers” would be $347.42.   According to these models, a “well-managed, well-organized multi-specialty group” would be expected to be able to provide care to the same population for $250.64.  This means that they estimate that forming an ACO will lead to a 28% reduction in cost.

Before I dive into criticism, let me thank Milliman for being willing to prepare a quantitative model and disseminate results freely.  As I noted in my recent critique of another economic model of ACO transformation, I think that such models are extremely helpful to real-world decision-makers because they force people to be explicit about the assumptions they are making, and they provide some quantitative estimates of the outcomes relevant to the comparison of available alternatives so people can make better choices.

But, Milliman’s “Well Managed” Model Cannot Be Used to Estimate Expected Outcomes.

Let me explain.  In the “Non-Medicare ACO” whitepaper, and in many other Milliman documents I’ve seen in recent years, the outputs from Milliman’s “Loosely Managed” vs. “Well Managed” models are described in ways that suggest they were designed to estimate outcomes.  In this whitepaper, the authors describe the “well managed” numbers as using “the utilization rates observed from well-managed multi-specialty groups when treating the same population.”  The authors go on to assert that the “difference between loosely-managed and well-managed claim cost PMPMs illustrates the amount that would be available to share among stakeholders.”  So, they are asserting that their model is producing an estimate of expected outcomes.

But, in my opinion, this is not a valid interpretation.  The Milliman model of “well managed” care is not based on the average performance of real provider organizations that are classified as “well managed.”  Rather, Milliman uses an “extensive amount of data,” augmented with their own experience and judgment, to find the “benchmark” or “target” performance, rather than the average performance.  Moreover, they determine this “benchmark”  separately for each of 60 categories of health care services.  The performance they are reporting as being expected for “well managed” entities is really the performance that would be expected for a theoretical entity that simultaneously achieved benchmark performance in all 60 categories.

Of course, any real organization’s actual performance during any particular time period is partly a result of their excellence and partly a matter of luck (random variation).   The joint probability of any real ACO being lucky enough to be the benchmark performer in each of 60 categories during the same time period is very remote.  The Milliman “well managed” model, therefore, is designed to show a sort of exaggerated hypothetical stretch goal, rather than to estimate the average expected outcome for a real ACO (or health plan) that adopts the set of management interventions that Milliman favors.  When I’ve seen “well managed” numbers presented by Milliman in the past, it was in the context of health plan data.  I was always amazed at how far any particular health plan was from being “well managed” – a performance gap that serves a useful purpose as a call to action for Milliman’s clients who are managers wanting to avoid having their organization labeled as “not well managed.”  But, once I learned what the numbers really meant, I wished that they had been presented in a way that made their true nature clearer.  And, I wished that Milliman published data about the overall performance of actual organizations with different characteristics and using different types of clinical programs.

In this non-Medicare ACO context, the implication of this misinterpretation of the Milliman actuarial model is that the 28% cost reduction associated with well managed ACOs is an over-estimate.




2 thoughts on “Part 1 of Critique of Milliman’s Whitepaper on Non-Medicare ACOs: Misinterpreting the Milliman “Well Managed” Actuarial Model?”

  1. Pingback: Part 2 of Critique of Milliman’s Whitepaper on Non-Medicare ACOs: Overly optimistic about inpatient use management, alarmingly pessimistic about primary care process transformation

  2. Pingback: Part 1 of Critique of AHA analysis showing higher ACO-related costs: Still underestimating care management cost

Leave a Comment

Your email address will not be published. Required fields are marked *

Free Subscription to Blog

Recent Posts