HHS Releases Final ACO Rule

The Department of Health & Human Services (HHS) today released the final rule for accountable care organizations (ACO).

The new rule includes a number of changes designed to make the Medicare Shared Savings Program more palatable for health care providers who had a largely negative response to the draft rule released last March. The changes include the following:

  • Allow providers to choose to participate without any downside financial risk during the initial contract period, rather than requiring all participants to take downside risk during the third year of the contract period
  • Provide up front financial support to physician-owned ACOs to support investments in building ACO capabilities, to be repaid through gain sharing rewards in subsequent years
  • Reduce the up front investment needed by eliminating the requirement for meaningful use of electronic health records
  • Reduce the number of quality measures from 65 to 33
  • Allow providers to choose from a number of available start dates throughout 2012
  • Allow community health centers and rural health clinics to serve as ACOs
  • Prospective identification of the Medicare beneficiaries for whom the ACO will be held accountable, rather than deriving such care relationships after the accountability period
  • Eliminates the mandatory anti-trust review for newly-formed ACOs
  • Puts the burden on the federal government, rather than nascent ACOs, to gather data regarding local market share
The  text of the rule is available here, and the associated final waiver rules are available here.

In my opinion, the elimination of the requirement to accept downside risk is likely to substantially increase the willingness of providers to participate in the program, while simultaneously reducing the likelihood that participation will lead to meaningful transformation of the care process within those participants.  But, given the strong opposition to the draft rule, CMS had little choice but to dilute the requirements to at least get some players to take the field.


Read More

Observed over expected (O/E) analysis is commonly misapplied to performance comparisons. Please don’t.

A few years ago, I had a shocking and enlightening discussion about analytic methods with a group of epidemiologists and biostatisticians from Blue Cross Blue Shield of Michigan.

PGIP Quarterly Meeting in Lansing

We were sitting around a table at a conference center in Lansing, where we were debriefing from a meeting of the Physician Group Incentive Program. We were talking about the methods for performance comparison. Everyone knew that we needed to “risk adjust” to take into account differences in patient characteristics when comparing the performance of different physicians, clinic practice units, and physician organizations. If we failed to properly risk adjust, the poorly performing providers would surely argue “my patients are sicker.”

Traditional Risk Adjustment using Standardization

When epidemiologists want to compare the performance of two health care providers on a level playing field, the traditional method is to do risk adjustment using an approach called standardization.    The concept is to determine which patient or environmental variables influence the outcome of interest.  These are called confounding variables, because differences in the mix of patients based on these variables can confound the performance comparison unless they are taken into consideration. Examples of such confounding variables include age, gender, the presence of co-morbid conditions, etc.  If any of the confounding variables are continuous numbers, like age, the epidemiologist must first convert them to discrete categories, or groupings.  For example, if age was the confounding variable, the epidemiologist might define categories for “adults” and “children.”  Or, the epidemiologist might break age into ten-year ranges.  If there is more than one confounding variable, the categories are defined based on the combinations of values, such as “adult females,” “adult males,” etc.  These categories are sometimes called “risk strata” or “risk cells.”  Then, for each of these categories, for each of the providers being compared, the epidemiologist calculates the outcome measure of interest, such as the mortality rate or the total cost of care.  The result is a matrix of measure values for each of the risk cells for each provider.  This matrix can be conceptualized as a “model” of the actual performance of each provider.

To create a level playing field for comparisons, the epidemiologist then creates a standardized population.  The standardized population is simply the number or percentage of patients in each of the risk cells.  Then, the model of each provider’s performance is applied to that standardized population to determine what the outcomes would have been if that provider had cared for the standardized mix of patients.  For each of the risk cells, the standardized number of patients is multiplied by the actual performance that provider achieved for such patients.  Then, the results for all the risk cells are aggregated to obtain that provider’s risk-adjusted performance. Another way of thinking about this is that the risk adjusted outcome is the weighted average outcome for all the risk cells, where the weights are the proportion of patients in that risk cell in the standardized, level-playing-field population.  If the provider’s actual mix of patients was “sicker” or “higher risk” than the standardized population, then the risk adjusted outcome will be more favorable than the unadjusted outcome.

“Observed Over Expected” Analysis

In the literature, even in respected journals, I have seen many examples of performance comparisons that used a different analytic approach called “observed over expected” or “O/E,” rather than using the traditional standardization approach.  A recent example is a paper regarding the mortality-avoidance performance of childrens’ hospitals.  Just as with standardization, the O/E method begins by identifying confounding variables — the patient characteristics that are predictors of the outcome of interest.   With O/E analysis, confounding variables that are continuous, like age, do not have to be converted to discrete categories or groupings.  All the confounding variables are used as independent variables in a regression model.  Then, the resulting regression model is applied to each individual patient observation, inserting the values of the predictor variables for that patient into the regression formula to obtain the “expected” value of the outcome of interest.  At that point, you have the actual observed value and the expected value for each patient (or case).  Then, you sum up the observed values for all the patients for a given provider.  And, you sum up the expected values for all the patients for a given provider.  Finally, you divide the sum of observed values by the sum of expected values to get the O/E ratio.  If the ratio is greater than one, that is interpreted to mean that the provider has a higher-than-expected value for that outcome.  If the outcome variable is undesirable, such as mortality rate, complication rate or cost, an O/E ratio of greater than one is interpreted to mean that the provider performed poorly compared to the other providers.  People have been routinely using O/E analysis as if it was a way of doing risk-adjusted performance comparisons — as a way of “leveling the playing field” to do performance comparisons that take into account differences in patient characteristics.

But, sitting around the table in Lansing, I was shocked to realize that O/E analysis is not actually applicable for this purpose.  Why? Because O/E analysis does not actually create a level playing field.

On the contrary, O/E analysis is conceptually the opposite of the traditional standardization approach.  In traditional standardization, a model of each provider’s actual performance is applied to a standardized population.  In O/E analysis, the regression model is essentially a model of typical performance.  That regression model is applied to the actual population that received care from a particular provider.  The problem is that different providers can see a different mix of patients.  Consider the following simple calculations.

In this simple example, we are comparing just two providers.  We are considering just one confounding variable, age.  And, we are breaking that variable into just two categories, adults and children.  As shown in the example, provider A sees mostly adults, while provider B sees mostly children.  Provider B performs poorly in those children, but actually performs better that A in the adult population.  Because provider B sees more children, the poor performance in children dominates the O/E calculation, so provider B looks bad in terms of an O/E ratio of 1.09.  But, since there are more adults than children in the overall population, which we are using as the “standardized” population, provider B’s superior performance in adults dominates the risk adjustment.  So, provider B has a risk-adjusted mortality rate that is better than provider A.  In other words, if you use O/E analysis for level-playing-field performance comparisons, you may get the wrong answer.

Risk adjustment using standardization is not computationally difficult.  But, it does require a bit of programming work to convert continuous variables such as age into categorical variables, such as breaking age into ten-year ranges.  In my opinion, O/E analysis should not be used for risk-adjusted performance comparisons.  O/E analysis does have the advantage of being computationally more convenient, particularly when you have many confounding variables and when some of them are continuous variables.  But, it is not that much more work to do it right.

Read More

CMS Innovations Center announces new Medicare Bundled Payments Initiative. I predict it will be popular.

On top of the Medicare Shared Savings Program (the original ACO initiative) and the Pioneer ACO Program, CMS is now adding four more optional approaches for providers to get reimbursed for Medicare services.  The new models all involve bundled payments, where a single negotiated payment covers all professional and facility services for a particular type of episode, such as a hip replacement.  This approach has been shown in a previous Medicare demonstration project and numerous bundled payment initiatives of commercial payers to create an incentive for physicians and hospitals to do a better job coordinating care and reducing the use of potentially unnecessary services during the episode of care, resulting in lower overall cost.

The four new models differ in terms of what the bundle includes and whether the payment is made prospectively or retrospectively.

Under the program, data about the historical cost for the services associated with the bundle will be provided by CMS. Then the provider will negotiate a discount percentage off of the historical cost, assuring that CMS receives some savings.  Then, if providers can reduce their actual cost more than the discount percentage, they get to keep the additional savings.  It’s a fair bet that, over time, CMS will ratchet down the bundled payment one way or another.

So, the idea is for providers to voluntarily sign-up for a program that will reduce their revenue.  Why would they do that?  For two reasons:

  1. They could make more bottom line profit if they decreased their fixed and variable cost faster than CMS decreases the bundled payments, and
  2. They know the market is heading back in the direction of providers bearing more risk, and this is a way to get their feet wet and establish their capabilities for care coordination, care process transformation, utilization management and the supporting information technology and analytics.

I predict that the bundled payment approach will be more popular among health care organizations than the more global ACO approach because is feels more gradual, more controllable, and more understandable.  As a result, it is less risky and scary.

More information about these bundled payment options are available on the CMS website.

Read More

Hospitalists have been focused on reducing hospital length of stay, but not so much on smooth transitions to ambulatory care

A new study published in the  Annals of Internal Medicine compared the economic outcomes of hospital episodes managed by hospitalists to those managed by the patients’ primary care physicians in a Medicare population. Previous studies focused only on the cost of the hospital stay itself, and showed that hospitalists were able to reduce length of stay and hospital cost. These economic savings accrue primarily to hospitals who are reimbursed with a fixed DRG-based payment for most hospital stays. These hospital savings have motivated hospitals to hire more hospitalist physicians. According to the Society of Hospital Medicine, 80 percent of hospitals with more than 200 beds now have hospitalists.  There are now 30,000 hospitalist physicians, and the specialty continues to grow more rapidly than any other.

But, the new study measures the economic outcomes of the entire hospital episode, including care received after the patient is discharged from the initial hospital stay.   The study shows that hospital stays managed by hospitalists had an average length of stay that was 0.64 days shorter, saving an average of $282. But, those patients were more likely to return to the emergency department and more likely to be readmitted to the hospital, leading to post-discharge costs that averaged $332 higher than for hospital episodes managed by the patients’ own primary care physicians. Thus, the use of hospitalists ends up costing Medicare $50 more per episode, increasing overall costs by $1.1 billion annually.

The study authors, Yong-Fang Kuo, PhD and James S. Goodwin, MD from the University of Texas Medical Branch in Galveston, hypothesized that “hospitalists, who typically are employed or subsidized by hospitals, may be more susceptible to behaviors that promote cost shifting.” The implication is that, if hospitalists were employed by primary care-based Accountable Care Organizations (ACOs) that were being held responsible for the total cost of care for a defined population of patients, they might be more strongly encouraged to focus on improving care transitions to reduce downstream complications and associated emergency department visits and hospital re-admissions.

Even without ACOs, there has been a great deal of effort over the last few years to improve transitions of care for patients discharged from acute care hospitals.  Most of these efforts attempt to improve both the “pitch” by the hospital-based team and the “catch” by ambulatory care providers.  But, some efforts, such as the BOOST program funded by the Hartford Foundation, have a primary emphasis on the pitch.  Other efforts, such as the STAAR program of the Commonwealth Fund and the Institute for Healthcare Improvement (IHI), tend to emphasize the catch. Hopefully such programs will lead to wide-spread improvements in quality of care and reductions in total cost of hospital episodes.  ACOs could catalyze and accelerate those improvements by linking hospital care more tightly to primary care and supporting this linkage through investments in health information exchange (HIE) capabilities designed to foster thoughtful, smooth transitions of care.

Read More

Dr. Ward to speak at 5th Annual Predictive Modeling Summit, November 8-9, 2011, Washington, DC

The 5th Annual Predictive Modeling Summit is described by conference organizers as the “leading forum on predictive analytics applied to key health care functions, settings and populations.”

Dr. Ward will be giving a presentation entitled “Using Intervention Models and Predictive Models to Optimize Patient Selection for Care Management in ACOs” at 1pm on November 9, 2011.

Conference details:

Hope to see you there!


Read More

Klar 3: The necessity of re-qualifying the population to avoid regression-toward-the-mean bias in historical comparison groups

Ron Klar, MD, MPH

Ron Klar, MD, MPH is a health care consultant with a long history of involvement in federal health care policy and health plan innovation. He published a recent series of three posts regarding the draft rules for the Medicare Shared Savings Program (MSSP) in the Health Affairs Blog, an influential forum for debating health policy issues. This is my third in a series of 4 posts describing areas of agreement and disagreement with Dr. Klar. In my first post, I described areas of agreement.  In my second, I covered my disagreements about Dr. Klar’s proposed changes regarding care relationship derivation.  In this post, I will describe my disagreement regarding Klar’s proposed changes to the approach to selecting a comparison group for savings assessment.

In the draft MSSP rules, CMS proposed two “options” for methods of selecting the comparison group for determining savings. The rules, following the lead of the health reform legislation language, mislabels the comparison group as a “benchmark.” CMS is not really trying to determine if an ACO is better than or comparable to the best-performing provider organization, as is implied by using the term “benchmark.”  What they really intend is to compare the actual cost to the cost that would have been expected to occur if the same beneficiaries had been cared for by non-ACO providers. CMS indicates in the draft rule that they prefer option 1, which involves using the same assignment algorithm in the prior time period as is used for the accountability/performance period. This approach is described as “requalification” in the care management evaluation standards published by the Disease Management Association of America (DMAA). Option 2, for which CMS is seeking feedback, involves using historical information for the cohort of beneficiaries that was actually assigned to the ACO.

In Dr. Klar’s first post, he explained that he prefers option 2, arguing that option 1 has no “face validity” because the individual beneficiaries will be different. I strongly disagree.

As I noted in my blog post last week, when claims-based patient selection logic is applied, the selection is determined based not only on unchanging characteristics of the person (like gender), but also on data regarding health care events that happened at particular points in time. The person-months in the years before meeting the assignment criteria do not have the same risk as the person-months after meeting the assignment criteria. There is randomness in the timing of events, as people experience peaks and valleys of individual risk. When you select people based on recent health care events, you are not selecting randomly. You are preferentially picking people who tend to be in a risk peak as evidenced by recent health care utilization. Without any intervention, continuing random variation will cause the risk of the selected population to decrease over time, toward the mean risk of the overall population. This is known as a regression-toward-the-mean bias.  This type of bias is strongest when the patient is being purposefully selected based on being a high risk outlier, such as when a predictive model is used to generate a risk score used to select patients to be targeted for outreach for a care management program.  But, this type of bias exists in a weaker form for any patient selection based on recent health care utilization.  Patients naturally have higher risk in the time periods just before and after health care utilization, since they seek health care in response to illness episodes that drive cost.  To avoid regression-toward-the-mean bias, I prefer option 1, which offers a symmetrical selection process for the ACO intervention population and the historical comparison population.

Dr. Klar correctly points out that if no risk adjustment is done, ACOs could be incentivized to preferentially seek care relationships with lower risk patients. I feel this should be solved by doing risk adjustment (as has been proposed in the rule), rather than by using option 2.

Klar goes on to propose a variety of additional modifications to the rules that illustrate the complications of using the option 2 pre-post design, such as having to apply a weighted average scheme to deal with people with different numbers of years of available history and people who died during the performance period.



Read More

Identifying and Understanding Analysis Tricks: Regression Toward the Mean

Imagine that you are a new homeowner, shopping for insurance for your new house.  You live in an area prone to earthquakes, and you are not a big risk-taker.  You decide that you should have earthquake insurance.  You are on the web researching earthquake insurance policies. You come across the web site of Acme Insurance, an international leader in earthquake damage coverage.  The web site says they are the best earthquake insurance company because they not only pay for earthquake damage, they have an innovative program to actually prevent earthquakes experienced by their beneficiaries. The program involves assigning an earthquake prevention coordinator (EPC) to each homeowner.  The EPC does one session of telephonic earthquake prevention coaching, sends some earthquake prevention educational materials by e-mail, and makes a follow-up call to assure that the homeowner is exhibiting good earthquake prevention behaviors.   This is a proprietary program, so more details are only available to Acme Insurance beneficiaries.  The program is proven to reduce earthquakes by 99%.  You click on the link to view the research study with the documented proof.

The study was conducted by Acme Analysis, a wholly-owned earthquake informatics subsidiary of Acme Insurance.  The study begins by noting an amazing discovery.  When Acme analyzed its earthquake claims for 2010, it noted that 90% of its earthquake damage cost occurred in only 10% of its beneficiaries.  It noted that these high cost beneficiaries were living in particular cities.  For example, it noted high earthquake claims cost in Port au Prince, Haiti for damage incurred during the January 12, 2010 earthquake there.  It developed an innovative high risk classification approach based on the zodiac sign of the homeowners’ birth date and the total earthquake claims cost for damage incurred in the prior month.  On February 1, 2010, they applied this risk classification to identify high risk homeowners, most of which were Libras or Geminis living in Port au Prince.  They targeted 100 of those high risk homeowners for their earthquake prevention program.    The EPCs sprung into action, making as many earthquake prevention telephone coaching calls and sending as many earthquake prevention e-mails as they could, considering the devastated telecommunications infrastructure in Port au Prince.

The program evaluation team then compared the rate of earthquakes exceeding 6.0 on the Richter scale and average earthquake damage claims for those 100 people for the pre-intervention period vs. the post intervention period. Among the 100 beneficiaries targeted by the program, the average number of major earthquakes plummeted from 1 in the pre-intervention period (January, 2010) to 0 in the post-intervention period (March, 2010), and the number of minor earthquakes (including aftershocks) dropped from 20 down to just 10. But the program was not just good for the beneficiaries wanting to avoid earthquakes.  It was a win-win for Acme Insurance.  Earthquake damage claims had dropped from an average of $20,000 per beneficiary during the January, 2010 pre-intervention period to an average of just $200 for damage incurred during the post-intervention period in March, 2010, when two of the targeted beneficiaries experienced damage from an aftershock.  The program effectiveness was therefore 1 – (200/20,000) = 0.99.  That means the innovative program was 99% effective in preventing earthquake damage claims cost.  After considering the cost of the earthquake prevention coordinators and their long-distance telephone bills, the program return on investment (ROI) was calculated to be 52-to-1.  The program was a smashing success, proving that Acme Insurance is the right choice for earthquake coverage.

Can you spot the problem? Can you extrapolate this insight to the evaluation of health care innovations such as disease management, care coordination, utilization management, patient-centered-medical home, pay-for-performance, accountable care organizations, etc.?

The problem is called “regression toward the mean.” It is a type of bias that can affect the results of an analysis, leading to incorrect conclusions.  The problem occurs when a sub-population is selected from a larger population based on having extreme values of some measure of interest.  The fact that the particular members had an extreme value at that point in time is partly a result of their underlying unchanging characteristics, and partly a matter of chance (random variation).   Port au Prince, like certain other cities along tectonic plate boundaries, is earthquake prone.  This is an unchanging characteristic.  But, it was a matter of chance that a major earthquake hit Port au Prince in the particular month of January, 2010.  If you track Port au Prince in subsequent months, their theoretical risk of an earthquake will be somewhat higher because it is still an earthquake prone area.  But, chances are that, in any typical month, Port of Prince will not have a major earthquake.

An analogous effect can be observed when you identify “high risk” patients based on having recently experienced extreme high rates of health care utilization and associated high cost.  The high cost of such patients is partly driven by the underlying characteristics of the patients (e. g. age, gender, chronic disease status), and partly based on random chance.  If you track such patients over time, their cost-driving characteristics will lead them to have somewhat higher costs than the overall population.  But, the chance component will not remain concentrated in the selected patients.  It will be spread over the entire population.  As a result, the cost for the identified “high risk” patients will decrease substantially.  It will “regress” toward the mean.  With high risk classification methods typically used in the health care industry, my experience is that this regression is in the 20-60% range over a 3-12 month period, without any intervention at all.  Of course, the overall population cost will continue to follow its normal inflationary trend line.

This regression-toward-the-mean phenomenon has been at play in many, many evaluations of clinical programs of health plans and wellness and care management vendors.  Sometimes unwittingly.  Sometimes on purpose.  Starting back in the 1990s, disease management vendors were fond of negotiating “guarantees” and “risk sharing” arrangements with managed care organizations where they would pick high risk people and guarantee that their program would reduce cost by a certain amount.   Based on regression toward the mean, the vendor could rely on the laws of probability to achieve the promised result, regardless of the true effectiveness of their program.  The vendor would get their negotiated share of the savings.  Good work if you can get it.  It lasted for a few years until the scheme was widely discredited.  But not widely enough, it appears.  Wellness and care management vendors still routinely compare cost before and after their intervention for a cohort of patients selected based on having extreme high cost in the pre-intervention period.  Health plans and employers eat up the dramatic savings numbers, happy to see that the data “proved” that they made wise investments.

Study designs suffering from obvious regression-toward-the-mean bias will usually be excluded from publication in peer-reviewed scientific journals.  But, they do show up in less formally-reviewed clinical program evaluations by hospitals and physician organizations.  For example, in a recent analysis of a patient-centered medical home (PCMH) pilot, the authors concluded that the program had caused a “48 percent reduction in its high-risk patient population and a 35 percent reduction in per-member-per-month costs” as shown in the following graphic.

In this PCMH program, a total of 46 “high risk poly” members were selected based on having high recent health care utilization involving 15 or more health care providers.  The intervention consisted of assigning a personal health nurse that developed a personal health plan,  having a personal health record (based on health plan data), and providing reimbursement for monthly 1-hour visits with a primary care physician.  The analysis involved tracking of the risk category (based on health plan claims data) and the per-member-per-month (PMPM) cost for the cohort of 46 patients, comparing the pre-intervention period (2009) to the intervention period (2010).   I’m sure the program designers and evaluators for this PCMH pilot are well meaning and not trying to mislead anybody.  I share their enthusiasm for the PCMH model of primary care delivery.  But, I think the evaluation methodology is not up to the task of proving whether the program did or did not save money. Furthermore, even with a better study design to solve the problem of regression-to-the-mean bias, the random variation in health care costs is far too large to be able to detect even a strong effect of a PCMH program in a population of only 46 patients.  Or even 4,600 patients for that matter.  I’d guess that proper power calculations would reveal that at least 46,000 patients would be required to have a chance of proving PCMH effectiveness in reducing cost.

So, how do you solve the problem of Regression Toward the Mean?

As with any type of bias, the problem is with the comparability of the comparison group.  The gold standard study design is, of course, a randomized controlled trial (RCT), where the comparability of the comparison group is assured by randomly assigning subjects between the intervention group and the comparison group (also called the “control group”).

If randomization is not possible, one can try to find a concurrent comparison group that is not eligible for the program and which is thought to be otherwise similar to the eligible population.  The same selection criteria that is applied to the eligible population to pick targets is also used in the ineligle population to pick “simulated targets” for a comparison group.  Note that in such a concurrent control design, the comparison should be between targets and the simulated targets, without considering which of the targets volunteered to actually participate in the intervention.  This aspect of the design is called an “intention to treat” approach, intended to avoid another type of bias called “volunteer bias.” (More on that in a future post.)

Often, the evaluators do not have access to concurrent data from an ineligible population to provide a concurrent comparison group.  In such a case, an alternative is “requalification” of the population in the pre-intervention period and the post-intervention period. Requalification involves using the exact same selection criteria used to pick targets at baseline and shifting it forward in time to pick a comparison group. The result will be a comparison group that is a different list of patients than the ones picked for the intervention. Some of the targets of the intervention may be requalified to be in the comparison group. Others will not. Some members of the comparison group will be members that did not qualify to be in the intervention group.  It is counter-intuitive to some people that such an approach creates a better comparison group than just tracking the intervention group over time.  But, with requalification, you are assured that the same influence that luck had in selecting people based on recent utilization will be present in the comparison group.  The idea is to avoid bias in the selection process by trying to make the selection process symmetrical between intervention and comparison groups.

If I apply these remedies for regression toward the mean bias, does that mean I will be assured of reliable evaluation results?

Unfortunately, no. The bad news is that clinical programs are devilishly hard to properly evaluate.  There are many other sources of bias intrinsic to many evaluation designs.  And, the random variation in the measures of interest are often very large compared to the relatively weak effects of most clinical programs.   This creates a “signal to noise” problem that is particularly bad when trying to evaluate small pilot programs or the performance of individual providers over short time periods.

If you really want to create a learning loop within your organization, there is no alternative to building a team that includes people with the expertise required to identify and solve thorny evaluation and measurement problems.

Read More

Klar 2: The great attribution debates: Include specialists or not? Plurality or majority? Retrospective or prospective? Derived or declared?

Ron Klar, MD, MPH

Ron Klar, MD, MPH is a health care consultant with a long history of involvement in federal health care policy and health plan innovation.  He published a recent series of three posts (post 1, post 2, and post 3) regarding the draft rules of the Medicare Shared Savings Program (MSSP) in the Health Affairs Blog, an influential forum for debating health policy issues. In a recent post of my own, I described where I agree with Dr. Klar.  In this post, I’ll describe some areas of disagreement related to the methods of defining the population for which the ACO is to be held accountable.  In two future posts, I’ll cover some additional areas of disagreement.

First, let’s define some terms.

I use the term “care relationship” to describe the data linking patients to providers.  Care relationship information can be “derived” based on other data such as encounter claims records.  Care relationships can be “declared” explicitly when the participants in the relationship – patients and providers – indicate that they intend for the relationship to exist or when they explicitly validate care relationship data that has previously been derived.  Or, care relationship data can be created and maintained through a mixture of derivation and declaration.  Others typically use the terms “attribution,” “assignment,” or “alignment” to describe care relationships, revealing their tendency to think only in terms of derived care relationships.   Derived care relationships can be determined “prospectively,” in advance of an accountability period.  Or, they can be determined “retrospectively,” at the end of an accountability period.

The draft rules for the MSSP proposes to define the population using derived care relationships.  The rules call for accomplishing this derivation by selecting the primary care physician that provided a plurality of the evaluation and management (E&M) encounter claims for a beneficiary, using an assignment process that is partly prospective and partly retrospective.

Dr. Klar proposed to change many aspects of the rules regarding care relationship derivation:

  1. Include specialists, rather than just primary care physicians
  2. The selection should be based on providing a majority (more than half) of the E&M services for a beneficiary, rather than just a plurality (more than anyone else)
  3. The derivation should be purely retrospective

Include specialists or not?

Klar’s proposal to include specialists is based partly on the fact that it will increase the proportion of beneficiaries assigned to an ACO.  Some beneficiaries have visits to specialists, but not visits to PCPs.  Such beneficiaries will only be assigned to an ACO if the assignment includes specialists.  Klar also asserts that including specialists in the assignment will stimulate organizations to “tie” specialists into the ACO.

For both of these same reasons, we originally included specialists in the “attribution” algorithms in the Physician Group Incentive Program (PGIP) at Blue Cross Blue Shield of Michigan (BCBSM).  But, we determined it was necessary to switch to what we called a “pure PCP” algorithm due to unanticipated consequences of including specialists in the attribution. When attribution includes specialists, beneficiaries with expensive conditions requiring specialist care are relatively more likely to be assigned to a physician organization (PO) or ACOs that include specialists, while PO/ACOs that don’t include specialists will tend to have a lower risk population.  Within the PO/ACO, a primary care physician that manages more of the heart failure in her panel of patients will have those patients assigned to her.  Another primary care physician who chooses instead to refer his heart failure cases to higher cost cardiologists will end up with those patients being assigned to the cardiologist.  As a result, the PCP that refers out heart failure management will have a more favorable utilization and cost profile.  These biases make it difficult to interpret performance comparisons when specialists are included in attribution.  I strongly prefer the “pure PCP” attribution approach.

Use plurality or majority?

In the PGIP program, as in the draft rule for the MSSP, we derived care relationships based on a plurality of E&M services, not a majority.  Whether the topic is managed care, patient-centered medical home, organized systems of care, or accountable care organizations, the idea is for providers to take responsibility for the care of a defined population.  Patients that flutter among many PCPs and don’t see any one PCP the majority of the time are still part of the population.  In fact, convincing such patients to have a more stable, exclusive relationship with one physician, or at least one primary care practice unit, should be a key objective of an ACO.  A majority standard would leave more members of the population without a derived care relationship with a PCP.  Therefore, a plurality standard is better than a majority standard.

Prospective or retrospective?

The draft rule for the MSSP proposes an assignment process that is partly prospective and partly retrospective. Many critics of the draft rule have called for a purely prospective derivation, arguing that ACO providers should only be held responsible for the cost and quality of care for patients that they knew about in advance.  But, Dr. Klar went against the crowd, calling for a purely retrospective derivation. He argued that the delay in claims data used for the derivation is too long, resulting in too much inaccuracy in care relationship data due to people switching their actual care relationships during the year.  Based on 25-33% annual turnover in care relationships, 44-55% of beneficiaries assigned before the start of a performance year would not still be assigned after the end of that performance year.  On that point, I agree with Dr. Klar.

But then Dr. Klar went on to provide another argument against any prospective assignment.  He asserted that prospective assignment would create an “undesirable distinction” among Medicare beneficiaries, causing prospectively assigned beneficiaries to be “treated differently” by providers.  He considered such distinctions to be inconsistent with expectations for the traditional Medicare fee-for-service program.  On this point, Dr. Klar has a lot of company.  Many advocates for Medicare beneficiaries are strongly defensive of the unlimited choice of providers currently intrinsic to the traditional Medicare program.  In that spirit, the health reform legislation prohibits restrictions limiting beneficiaries’ ability to  seek care from any participating Medicare provider. This prohibition could be interpreted as implicitly forbidding providers from having care relationship declaration processes where patients document their intention to have a primary care physician relationship, since that would possibly give the impression of “lock-in.”

The underlying debate about the role of the PCP

When I step back from the technical details and look at the bigger picture, it seems to me that Dr. Klar, like many others engaged in discussions about ACOs, seems to have a different conceptualization of the role of PCPs in ACOs than I do.  In proposing the inclusion of specialists in care relationship derivation, and by expressing concern about even giving the impression of fettering beneficiaries’ choice of providers, Klar reveals a conceptualization of an ACO that emphasizes the value of the organization, but does not necessarily emphasize the central role of PCPs.

I feel that a powerful, influential care relationship between a patient and her primary care physician is the main active ingredient in achieving ACO cost savings. In this context, the process of having patients declare or validate their care relationships is an important tool to creating the type of care relationship consistent with the vision of the patient-centered medical home (PCMH). In a PCMH care relationship, the patient understands the roles and responsibilities of the members of the team, and conceptualizes the patient and family as engaged members of that team.  In a strong PCMH-style primary care relationship, the primary care team can influence the patient’s behavior, encouraging adherence to the care plan, and promote effective self-management, involvement in informed medical decision-making, and healthy lifestyle behaviors.  Moreover, in a strong PCMH primary care relationship, the PCP can influence referrals for specialty and facility care, steering the patient toward specialists and facilities that are efficient and prudent. Such a role, when enforced through HMO-style mandatory referral authorization, can seem undesirable from the patient’s perspective, earning the pejorative title “gatekeeper.”  But, in a PCMH and ACO context, the primary care physician is challenged to effectively fulfill the gatekeeper function with one hand tied behind his back.  In an ACO, the patient is not required to seek a mandatory referral authorization from the PCP.  Therefore, to have influence over referral patterns, the PCP is challenged to earn the trust of patients and their families by demonstrating clinical competence and offering excellent service.  They are challenged to exert referral influence in softer ways designed to be satisfying or at least acceptable to the patient.  This influence causes more specialty and facility care to be delivered by more efficient providers.  And, it incentivizes all specialists and facilities to be more efficient.  In my estimation, this form of influence is the strongest active ingredient driving savings in ACOs – stronger than care coordination, stronger than patient-self management support, stronger than avoiding gaps in care through clinical decision support, and stronger than the avoidance of duplication of services through health information exchange.

Of course, there needs to be clear communication to beneficiaries of the voluntary nature of care relationships.  It must be clear that any declared care relationship information maintained by ACOs will not be used to determine shared savings or for any other CMS program administration purposes. But, the worry that ACO providers might implicitly influence patients to have an exclusive primary care relationship with them is not a risk.  In fact, the success of the ACO concept depends on it.

In summary, I’m willing to join Dr. Klar in his contrarian idea of using a purely retrospective care relationship derivation to determine MSSP reward payments.  But, I feel that the care relationship should be “pure PCP,” and the derivation algorithm should cast a wide net with a plurality criteria.  And, MSSP rules should make it clear that ACOs are permitted to create their own processes to track current care relationships, including processes that involve physician and patient declarations of care relationships.


Read More

Klar 1: Key points of agreement about changes needed for Medicare Shared Savings Program

Ron Klar, MD, MPH

Ron Klar, MD, MPH is a health care consultant with a long history of involvement in federal health care policy and health plan innovation.  He recently did an analysis of the draft rules for the Medicare Shared Savings Program (MSSP), the Medicare program that establishes accountable care organizations (ACOs).  Dr. Klar published his critique as a series of three posts in the Health Affairs Blog, an influential forum for discussion of health policy issues.  The first post focused on the sharing model and issues regarding assignment, identification and beneficiary choice.   The second post focused on methods for computing the savings and other financial issues. The third post focused on the approach for assessing quality, rates for shared savings and losses, limits to payments and losses, and sharing of CMS data.

Dr. Klar’s series differed from many critiques of the MSSP by offering detailed proposed changes and by explaining his reasoning.  The posts are quite long, going into technical details that can be tedious (but, who am I to throw such stones!)  They are a valuable contribution to the debate about the MSSP.  They also cover issues that are generally applicable to all value-based contracting arrangements between payers and providers.

Just to make it a more interesting read, I will break up my review of Dr. Klar’s three posts into four posts of my own.  In this post, I’ll describe where I agree with Dr. Klar.  In the remaining posts, I’ll describe some areas of strong disagreement.

First, I agree that the MSSP needs to include a pure shared savings model, with no sharing of losses. Dr. Klar proposes that CMS treat the model that involves sharing of losses as an experiment and make it optional.  He acknowledges that CMS was concerned about the risk of paying a reward that might be undeserved, and that CMS wanted providers to have “skin in the game.”  But, he argues that the program already reduces this risk by:

  1. Requiring ACOs to describe their roadmap for development of population management capabilities
  2. Requiring annual attestation of progress according to this roadmap
  3. Establishing a size-related minimum savings rate to reduce the role of lucky random variation

He proposes to add additional “skin in the game” commitment by adding public transparency in the form on an “ACO Compare” website.

Second, I heartily agree that the savings to be shared should include the full difference between actual vs. expected cost, without subtracting the 2 percent minimum savings rate. Klar points out that the proposed rule does not conform to clear language in the legislation.  This seemingly technical modification by CMS rule makers would remove a huge portion of likely shared savings if not overturned.

Third, I agree that the MSSP should determine eligibility and calculate the amount of payment based on a smaller set of claims-based outcomes measures, and have all other proposed measures serve only for reporting and monitoring purposes. This is a reasonable suggestion except for the elimination of valid claims-based process measures.  Dr. Klar argues that only outcomes measures should be used.  I prefer a mixture of process and outcomes measures, since process measures are more causally proximal and less likely to be obscured by unrelated downstream factors, while outcomes measures more directly reflect  the ultimate value.

Forth, I agree that the MSSP should assess quality performance by comparing an overall summary measure of performance to a local comparison group, not by comparing each separate measure to an arbitrary target. The idea is to assure that ACOs are not harming overall quality.  Klar proposes to impart a bias on these performance measures by doubling the weight for particular measures where the ACO performed more poorly than the local comparison population, and by requiring the ACO to exceed the local comparison group by a growing margin, starting at zero and growing to 5-10% over three years.  That sounds like a reasonable compromise for those that expect ACOs to eventually perform better, not just the same as non-ACO Medicare providers.

Fifth, I agree that the shared savings should be restructured to include a pre-determined “base” rate, plus an additional rate that depends on the quality performance, up to a more generous maximum sharing rate. Klar correctly points out that this will allow prospective ACOs to plan on achieving the base shared savings level (presuming they will achieve minimum quality and other eligibility provisions), while viewing the remainder as a possible bonus.  Klar proposes a base sharing rate of 40 percent for his proposed pure shared savings model and 50% for the shared savings and losses model.  Dr. Klar proposes to increase the maximum sharing rate to 52.5% for pure shared savings, and 75% for the shared savings and losses model, not including the 0.5% to 2.5% “special additions” for having a high rate of FQHC or rural health care.  That sounds at least partly responsive to those who want a more certain and generous sharing of savings.

Sixth, I agree that it is important to eliminate the 25% withholding of shared shavings (intended to cover possible future losses) and limit the shared losses paid during any particular year to 10%. Dr. Klar correctly advocates for establishing symmetry between the base sharing rates and the losses sharing rates.   This proposal seems to go against that principle.  But, it nevertheless seems justified to me based on the recognition that people have a strong fear of losses, and they discount gains if they are earned farther in the future.

Klar argues convincingly that, to be successful, the MSSP must win high participation rates from providers.  The loud chorus of negative feedback from prospective ACO participants suggests that, as currently structured, the program will certainly not achieve significant participation. Even the leading provider organizations thought to be best suited for the program have signaled their reluctance to participate, including Cleveland Clinic, Intermountain, Geisinger and Mayo.

If these six important changes were made, I would feel that CMS had been responsive to skeptical providers, and I would be disappointed if those providers did not seriously consider participating in the program.


Read More

How to get a clearer picture of economic performance: Standard Cost and Payer Neutral Revenue

This morning, I read a set of slides published by the American Hospital Association (AHA) and the Health Research and Education Trust (HRET) called “Striving for Top Box: Hospitals Increasing Quality and Efficiency.”

The report showcased the clinical integration and performance improvement efforts of three hospital-based health systems: Novant (North Carolina), Piedmont (Georgia) and Banner (Arizona). The report identified some common cultural characteristics and strategies responsible for their success, including precise execution, accountability in performance improvement, engaged physicians, focused action plans, consistent communication, team-oriented approaches, transparent data sharing, data dependent atmosphere and standardization in processes.   The slides are clearly written and are a worthwhile read.  I agree with essentially all the points made.  A concise summary based on actual case studies is a good thing.

One point, however, went beyond stating the obvious and got me thinking.   Novant, a health system with 12 hospitals serving the Charlotte and Winston-Salem metropolitan areas, described one of its two key strategies as “moving toward a payer-neutral revenue (PNR) system.” Novant considers all payers as if they were Medicare to prepare for a day when lower payments could be a reality.  They do this by running claims to be submitted to other health plans through a payment algorithm based on Medicare reimbursement rules and rates.  They use the resulting data to prepare pro forma financial statements showing which services lines would be profitable if all health plans paid like Medicare.

How does Payer Neutral Revenue relate to standard cost and standard revenue?

On the health plan side, clinical utilization efficiency measures have traditionally been based on “units” on paid claims.  But, units for different procedures codes within a single category of utilization can have vastly different economic implications.  Three MRIs plus one chest x-ray equal four units of radiology utilization.  If I eliminate the inexpensive chest x-ray, is it meaningful to say I improved clinical efficiency by 25%?

In response to these concerns, financial analysts in health plans have traditionally focused on per-member-per-month (PMPM) paid claim dollars.  But, if the health plan negotiated different fees for different providers, then the PMPM paid claims measures reflect a mixture of the providers’ clinical prudence and the health plan’s effectiveness in negotiating lower fees.  If a provider organization in the health plan’s provider network has a high paid claims PMPM cost, it could be because of unwarranted clinical utilization, or it could be that the provider organization successfully negotiated a higher price per service.

To create a measure that more clearly reflects only clinical efficiency, I have used what managerial accountants would call a “standard cost” approach.  This involves multiplying units of utilization of different procedures by a standardized dollar amount for each procedure.  The result is a utilization measure which is weighted by the economic value of each unit of utilization, after removing the effects of fee variation.   By using standard cost PMPM metrics, with proper risk-adjustment, the health plan can compare the clinical efficiency of different providers in its network on a more level playing field from the perspective of the individual clinician, who usually does not feel responsible for negotiating higher or lower fees.

In a sense, Novant’s PNR approach represents the inverse of health plans’ standard cost PMPM metrics. Just as a health plan can use standard cost to remove the variation in reimbursement of different providers, Novant is using PNR to remove the variation in reimbursement from different payers.  As such, I would suggest that a more generic term for PNR would be “standard revenue.”  After all, every organization’s cost is another’s revenue.

Is Novant’s Payer Neutral Revenue really neutral?

But there is one difference between Novant’s PNR approach and the standard cost approach that I typically use.  I typically set the standard cost for each procedure to the average fee paid for that procedure code during a prior time period.  That way, the standard cost PMPM metrics are measured in dollars, and overall standard cost will roughly equal overall actual cost.  There is no upward or downward bias in standard cost vs. actual cost.

But, Novant’s PNR approach does not try to be “neutral” in this sense.  They picked Medicare to be the basis of their standard.  Medicare generally pays substantially less than private health plans.  I presume this is because Novant is trying to create a sense of urgency.  They use Payer Neutral Revenue as the banner over their initiatives to reduce cost and unnecessary healthcare utilization, including their efforts to create a data-driven culture and improve their analytic capabilities.  I assume they are trying to emphasize the bottom-line connection of those efforts.  They currently have relatively rosy financials, based largely on the currently-available opportunity to shift costs from public to private payers.  By showing their financials in the more pessimistic future scenario where all reimbursement was at Medicare level, when cost-shifting is no longer possible, they are apparently trying to convey a greater sense of urgency.  They want people’s heads to be in the future, so they are more motivated to get started with preparations and make difficult choices today.

Read More
Page 2 of 3«123»