CMS Announces 32 Pioneer ACOs, including 3 in Michigan

The Centers for Medicare and Medicaid Services (CMS) announced the final list of 32 health care provider organizations that are to participate in the “Pioneer ACO” program during 2012.

The Pioneer ACO program was intended to allow provider organizations that had experience and sophisticated population management and care coordination capabilities to get started under a gain-sharing arrangement for Medicare more quickly than the Medicare Shared Savings Program (MSSP).  Compared to the MSSP participants, these Pioneer ACOs will take on greater risk and will be eligible to receive higher gain-sharing payments. They will then have an opportunity to move more rapidly from a gain-sharing to a population-based full-risk capitation payment model in year three, as long as they successfully earned shared savings awards during the first two years.  They will then be allowed to continue through an optional fourth and fifth year. The Pioneer ACOs all commit to negotiating “outcomes-based” reimbursement arrangements with other payers by the end of the second year.

Of the 32 Pioneer ACOs, more than one third are physician organizations, with the remainder being integrated delivery systems or other structures that include both hospital facilities and physicians.

The majority of the Pioneer ACOs are concentrated in 5 states, with California predictably leading the pack with 6 Pioneers, followed by Maine, Michigan, Minnesota and Texas.

The three Michigan-based ACOs include:

  • Genesys PHO: a collaboration between Genesys Health System and 160 primary care physicians with 400 participating specialist physicians who deliver health care services in Genesee, Lapeer, Shiawassee, Tuscola and northern Oakland counties.
  • Michigan Pioneer ACO: To be managed by the Detroit Medical Center PHO, a partnership of The Detroit Medical Center and its 1100 physicians, who include employed and faculty physicians, but consisting mostly of private practice primary care physicians. The Detroit Medical Center is a large academically integrated system in metropolitan Detroit, owned by Vanguard Health Systems and serving as a teaching and research site for the Wayne State University School of Medicine.
  • The University of Michigan Health System, which includes the U-M Faculty Group Practice, part of the U-M Medical School, includes all of the nearly 1,600 U-M faculty physicians who care for patients at the three U-M hospitals and 40 U-M health centers. Although U of M’s ACO was categorized by CMS as an “integrated delivery system,” the Pioneer ACO will also include participation by IHA Health Services Corporation, an Ann Arbor-based group practice that is part of Trinity Healthcare with 175 physicians in 32 practices.

The full list of Pioneer ACOs follows:

CMS released a fact sheet with more details about the Pioneer ACO program.

Read More

The Humunculus is a metaphor for Clinical Process Improvement Frameworks

During the last 20 years, we have experienced wave after wave of new frameworks for improving health care.  Each had its own terminology, ardently promoted and enforced by its zealous advocates.  Each had a lifecycle that began with a long incubation period, followed by a period of explosive growth in popularity and influence, rapidly leading to unrealistic expectations, followed by a period of decline during which the framework was declared to have been ineffective.  We’ve been through health maintenance, outcomes management, clinical effectiveness, managed care, disease management, chronic care, care management, practice guidelines, care maps, evidence-based medicine, quality functional deployment, continuous quality improvement, re-engineering, total quality management, and six sigma.  We’re still in the thick of lean, patient-centered care,  value-based benefits, pay-for-performance and accountable care.

Four things I’ve noticed about this lifecycle of health care improvement frameworks:

  1. They are formulated by conceptual thinkers, but then get taken-over by more tactically-oriented people.  The tactical folks often focus too much on the tools, terminology and associated rituals.  The framework always gets “simplified” to be more suitable for mass consumption.  For example, continuous quality improvement somehow morphed into being primarily about assigning a timekeeper during team meetings and communicating progress on a felt-backed “story board,” rather than finding people with systems-thinking talent and applying that talent to understand sources of variation in complex processes.
  2. During the early part of the growth phase, the advocates are always desperate for examples of success, and shower a great deal of attention on early projects that are described using the terminology of the framework and that appear to have succeeded.  The desperation usually leads advocates to lower their standards of evidence during this phase.  This leads to over-promising and unrealistic expectations.  It stimulates lots of superficial imitation by people interested in hopping on the bandwagon.  And, it plants the seeds for the eventual decline, when people determine that their inflated expectations were not met.
  3. The decline phase, when the framework is declared to be ineffective, seems to always happen before the framework was ever really implemented in the way envisioned by the original formulators during the incubation phase.
  4. All the frameworks are really just restatements of the same underlying concepts, but with different terminology and tools, and different emphasis.  In other words, they all have the same anatomy, but different parts of the anatomy are emphasized.

This last point reminds me of the “humunculus,” also called the “little man.” When I was in medical school in the late 1980s, we used heavy text books that generally did a bad job of teaching the information. One notable exception was clinical neuroanatomy. We used a small, paperback text book playfully entitled “Clinical Neuroanatomy Made Ridiculously Simple” by Stephen Goldberg, MD. It contained a collection of clever drawings designed to explain the structures and functions of the brain and spinal cord. Perhaps the most famous of the drawings was the humunculus.

Cross section of somatosensory cortex, showing mapping to sensory input sources

This drawing was adapted from earlier work by an innovative neurosurgeon named Wilder Penfield, who invented new surgical procedures for patients with epilepsy during the late 1930s.  During those procedures, he used electrodes to stimulate different points on the surface of the brain.  He drew diagrams similar to the drawing above showing that the surface of the brain contained a little man hanging upside down. The diagram shows that a disproportionate portion of the brain surface is dedicated to the sense of touch and muscle movements in certain parts of the body.  Lots of brain surface is dedicated to highly sensitive and nimble areas like the lips, tongue, hands and feet.  Very little brain surface is dedicated to the arms, legs and back.  Many anatomic illustrators have drawn the humunculus as a cartoon character showing how this disproportional emphasis on different parts of the body looks on the little man.

The Humunculus

The humunculus is a great teaching tool, making it easy to remember these aspects of clinical neuroanatomy.  But, I think the humunculus is also a useful metaphor for the distorted emphasis that various health care improvement frameworks have placed on various parts of the underlying anatomy of health care improvement.

Framework

Emphasis

Health maintenance Preventive services
Outcomes Management Measurement of function, patient experience and health status
Clinical Effectiveness Measurement of outcomes in real world settings, rather than laboratory controlled conditions
Managed Care Prospective review of appropriateness of referrals, procedures and expensive drugs, and retrospective review of cost of care
Disease Management Role of nurses in training patients to be more effective in self-management
Chronic Care Teamwork in primary care clinic and importance of organizational and community environment
Care Management Role of nurses in coordinating services delivered by different providers and in different settings
Practice Guidelines Consensus about which ambulatory services are appropriate in which situations
Care Maps Consensus about the sequence of inpatient services for different diagnoses
Evidence-based Medicine Weight of scientific evidence about efficacy of a service (without regard to cost)
Quality Functional Deployment Focus on the demands made by patients
Continuous quality improvement Small experiments to determine if incremental process changes are improvements
Re-engineering Designing new processes from scratch, rather than making incremental changes
Total Quality Management Importance of organizational culture and management processes
Six Sigma Focus on reducing frequency of defects
Lean Focus on eliminating non-value-adding process steps and reducing cycle time
Patient-centered care Focus on the needs of patients and the involvement of patients in their own care
Value-based Benefits Financial incentives to motivate patients to comply with recommended treatments that reduce overall cost
Pay-for-performance Financial incentives to motivate individual physicians to improve quality and reduce cost
Accountable care Financial incentives to motivate health care organizations to improve quality and reduce cost

Over the years, I have assimilated the concepts, terminology and tools from these various improvement frameworks into an approach that attempts to achieve balance, with each aspect of the framework shown without over-emphasis.

This framework puts the patient in the center, surrounded by the health care processes, which are surrounded by improvement processes.  It attempts to balance between focusing on care planning (the clinical decision-making regarding what services are needed) vs. focusing on care-delivery (the teamwork to execute the care plan and provide health care services to the patient).  It balances between measuring outcomes and measuring quality and cost performance.  It balances between implementing best practices through guidelines and protocols vs. improving practices through performance feedback and incentives. By avoiding a distorted over-emphasis on any one part of the anatomy, hopefully it can have greater lasting power than some of the more humunculus-like frameworks that have come and gone.   This framework is described more fully here.

Read More

Health Care Heroes: Don Berwick, MD – Adapting industrial quality improvement principles to the improvement of health care processes

Last week, Don Berwick, MD, announced his resignation as Administrator of the Centers for Medicare and Medicaid Services (CMS).  Now is a good time to explain why Dr. Berwick is one of my all time health care heroes.

Don Berwick as one of the Notre Dame Four Horsemen on Ward's coffee mug

Apparently, I talk about Dr. Berwick a lot. A few years ago, I received one of my most treasured gifts from colleagues at Blue Cross Blue Shield of Michigan (BCBSM).  It was a coffee mug featuring the famous photograph of the Four Horsemen of Notre Dame, a reference to my undergraduate alma mater.  My colleagues replaced the faces of three of the horsemen with the faces of three of my health care heroes, Drs. Paul Ellwood (the person who coined the terms “health management organization” and “outcomes management”), David Eddy (the clearest thinker on the topics of clinical practice policies and the rational allocation of health care resources), and Don Berwick. The face of the forth horseman they replaced with my own face.  I considered it a great honor to be associated with my heroes, at least on a coffee mug.

My team at BCBSM had heard me repeatedly explain Dr. Berwick’s important contribution to adapting the quality improvement  principles that had been successfully used in manufacturing to the health care field.  Others had been involved in promoting “continuous quality improvement,” “statistical process control,” and “total quality management” in health care. Paul Batalden, Brent James, Eugene Nelson, and Jack Billi come to mind, to name but a few. But, in my opinion, it has always been Berwick that has been the most eloquent and persuasive. He connected the statistical tools emphasized by James with the front line worker involvement emphasized by Batalden. And, he was able to describe how these approaches applied to clinical decision-making as well as care delivery.

At the heart of Dr. Berwick’s contribution was teaching us all to distinguish between the “Theory of Bad Apples” and the “Theory of Continuous Improvement.”

According to the Theory of Bad Apples, errors come from “shoddy work” by people with deficient work performance.  Leaders who uphold this theory focus on inspection to identify such deficient performance, indicated by the undesirable tail in the distribution of provider performance as shown on the left side of the diagram above.  Then, such leaders focus on holding the bad performers “accountable” by applying disciplinary measures intended to motivate improvement in performance and by pursuing other interventions intended to re-mediate the bad performance.  In the health care context, the workers are physicians and the shoddy work is poor quality health care. According to Berwick, the predictable defensive response by the physicians who are targeted for such remedial attention includes three elements: (1) kill the messenger, (2) distort the data and (3) blame somebody else.

Berwick advocates instead for the Theory of Continuous Improvement.  The basic principles of this theory are

  • Systems Thinking: Think of work as a process or a system with inputs and outputs
  • Continual Improvement: Assume that the effort to improve processes is never-ending
  • Customer Focus: Focus on the outcomes that matter to customers
  • Involve the Workforce: Respect the knowledge that front-line workers have, and assume workers are intrinsically motivated to do good work and serve the customers
  • Learn from Data and Variation to understand the causes of good and bad outcomes
  • Learn from Action: Conduct small-scale experiments using the “Plan-Do-Study-Act” (PDSA) approach to learn which process changes are improvements
  • Key Role of Leaders: Create a culture that drives out fear, seeks truth, respects and inspires people, and continually strives for improvement

T-Shirt of "Berwickians" -- the staff of epidemiologists and biostatisticians at BCBSM

Berwick argued the point made by Dr. Deming:  if we  can reduce fear, people will not try to distort the data.  When learning is guided by accurate information and sound rules of inference, when suppliers of service remain in dialog with those that depend upon them, and “when the hearts and talents of workers are enlisted in the pursuit of better ways, the potential for improvement in quality in nearly boundless.”

I first was influenced by Dr. Berwick back in the 1980’s when he championed these ideas during his tenure at the Harvard Community Health Plan, and subsequently during the 1990’s when he led the National Demonstration Project on Quality Improvement in Health Care and the Institute for Healthcare Improvement.  His face was already on my coffee mug at the time he was nominated to lead CMS.  I was thrilled that someone from our community of people dedicated to clinical process improvement had been recognized and would be serving in a position of such influence.

The Irony of the Political Opposition to Berwick’s Role as CMS Administrator

Dr. Berwick’s candidacy as CMS Administrator faced stiff opposition from Republican leaders who were angry about anything connected to the health care reform law or, for that matter, the Obama administration itself.  The President made the decision to evade this opposition by making a recess appointment of Dr. Berwick.  But, such recess appointments have a limited lifespan.  As the deadline for making a formal, congressionally sanctioned appointment approached at the end of the 2011 legislative session, 42 Republican senators signed a letter reiterating their disapproval of Dr. Berwick as CMS Administrator.   The arguments against Dr. Berwick’s  candidacy, both at the time of his original nomination and again over the last few months, centered around comments that Dr. Berwick has made praising the British health care system.  They concluded from his comments that he was in favor of redistributing wealth to the poor and of rationing, the dreaded “R” word, the thing done by “death panels!”  He was, therefore both a bleeding heart and heartless at the same time.  Dr. Berwick denied these charges, but the opposition was unconvinced and unwilling to back down from a position of persistent opposition to anything connected to “Obamacare.”

The irony is that, of the heroes on my coffee mug, Dr. Berwick is not the one deserving of praise for having insight and bravery concerning the basic tenets of health economics. Instead, it was Dr. David Eddy’s mug that was on my coffee mug because he was brave enough to publish numerous papers in the Journal of the American Medical Association explaining why rationing was the right thing to do (e.g. this one and another one).  Eddy argued that creating evidence-based “practice policies” that rationally allocated health care resources using “explicit methods” was favorable to using implicit methods supported only by “global subjective judgement.”  What a radical thought!

Despite my great admiration for Dr. Berwick, he was the hero that disappointed me as a rationing denier.  In fact, in a 2009 paper published in Health Affairs entitled “What ‘Patient-Centered’ Should Mean: Confessions of an Extremist,” he eloquently argued that we should give any patient whatever they wanted, regardless of the cost and regardless of the evidence of effectiveness.  He discounted the role of the physician as a steward of resources.  I felt the argument was heartfelt and humanistic.  But, I felt it was a cop out.  How strange, then, that the Republican opposition hoisted him on the rationing petard.

Looking Forward to Berwick’s Next Journey

Although it is disappointing to me that Dr. Berwick will no longer be leading CMS, I am eager to see what he chooses to do next.  I’m sure he will continue to make a great contribution to our field.  Without all the administrative and political duties to clog up his day, perhaps we are about to witness a surge in his ongoing contributions to improving health care.

More information: See Health Affairs article and associated Health Affairs Blog Post praising Dr. Berwick.

Read More

AHRQ guidance: Forget about proving Medical Home effectiveness with small pilot studies attempting to measure all-patient cost savings

The vast majority of already-published and currently-underway studies of the effectiveness of the Patient Centered Medical Home (PCMH) model of care are pilot studies with fewer than 50 PCMH practices.  Most of these studies report or intend to report reductions in hospitalization rates and savings across the entire population of patients served by the PCMH practices.  New  guidance from the Federal Government calls the value of such reports into question.

The AHRQ recently released an excellent pair of white papers offering guidance on the proper evaluation of PCMH initiatives.  The first is a four page overview intended for decisionmakers, entitled “Improving Evaluations of the Medical Home.”   The second is a 56 page document that goes into more detail, entitled “Building the Evidence Based for the Medical Home: What Sample and Sample Size Do Studies Need?”  The whitepapers were prepared by Deborah Peikes, Stacy Dale and Eric Lundquist of Mathematica Policy Research and Janice Genevro and David Meyers from the AHRQ.

The white papers emphasize a number of key points:

  • Base evaluation plans on plausible estimates of the effects of PCMH.  Based on a review of the evidence so far, the white paper suggested that a successful program could plausibly hope to reduce cost or hospitalizations, on average, by 15 percent for chronically ill patients and 5 percent for all patients.
  • Use a proper concurrent comparison group, rather than doing a “pre-post” analysis. Pre-post analyses, although common, are inconclusive because they can easily by confounded by other factors changing during the same time period, such as economic conditions, health care policy changes, advances in technology, etc.
  • Focus on evaluating a large number of practices, rather than a large number of patients per practice.  The authors point out that “a study with 100 practices and 20 patients per practice has much greater power than a study of 20 practices with 100 patients each.”  They warn that small pilot studies with 20 practices or less are unlikely to produce rigorous results without combining the results with many other small studies conducted using the same metrics.  Such pilot studies, which unfortunately are very common, are really only useful for generating hypotheses, not for drawing conclusions.  The authors note that neither positive nor negative results of such small studies should be relied upon.  Small PCMH studies can show no significant impact because they did not have the power to detect such an impact.
  • Focus on evaluating health and economic outcomes in subsets of patients such as those with chronic disease.  Satisfaction can be evaluated across the entire population, but if you use data for the entire population to measure hospitalizations, emergency department visits, inpatient days, or health care costs, the lower risk portions of the population contribute noise that obscures the measurement of the effect that is occurring primarily among those most likely to experience such events in the first place.
  • Use statistical methods that account for “clustering” at the practice level, rather than treating individual patients as the unit of analysis.  Since the intervention is intended to change processes at the practice level, the individual patients within a practice are not independent of one another.  Clustering must be taken into account not only at the end of the study, when evaluating the data.  It must also be taken into account at the beginning, when determining the number of practices and patients to sample.  For example, if a study includes a total of 20,000 patients, but the patients are clustered within 20 practices, then the effective sample size is only 1,820, assuming patient outcomes are moderately clustered within practices.  When statistical methods treat such patients as independent, they are implicitly treating the sample size in such a situation as 20,000 rather than 1,820.   As a result, evaluators making such an assumption are dramatically over-estimating their power to detect the effect of the PCMH transformation.  If they adjust for clustering at the end, their findings are likely to show a lack of a significant effect, even if the PCMH program really worked.  On the other hand, if they don’t adjust for clustering in the end, there is a great risk of reporting false positive findings.  For example, in a PCMH study with 20 practices and 1,500 patients per practice, where the analysis was done without adjusting for clustering and found a positive result, there is a 60% chance that the positive result is false, based on typical assumptions.
These recommendations are based not only on the experience of the authors, but on modeling that they did to explore the implications of different study scenarios with different numbers of patients, intervention practices, control practices and measures of interest.  These models calculate the minimum detectable effect (MDE) based on assumptions regarding typical characteristics of the patient populations, practices, and plausible effects of the PCMH program, based on a review of prior studies and the authors’ experience.  The models illustrate that, when measuring the impact of PCMH on costs or hospitalization rates for all the patients receiving PCMH care, the MDE drops as the number of practices in the PCMH intervention group increases.  But, even with 500 PCMH practices, the studies cannot detect the 5% cost or hospitalization reduction that the authors consider to be the plausible impact of PCMH on the entire population.

The authors re-ran the models, assuming that the measure of cost and hospitalization would consider only the sub-population of patients with chronic diseases.

The model showed that, based on reasonable assumptions, at least 35 PCMH practices, plus an equivalent number of concurrent comparison practices, would be required to detect the 15% effect that the literature suggests is the plausible effect of PCMH on cost and hospitalizations among patients with chronic diseases.  Even when focusing on the chronic disease sub-population, a pilot evaluation with only 10 PCMH practices and 10 comparison practices could not detect an effect smaller than 30%, an effect size they considered implausible.

I found this modeling exercise to be very informative and very worrisome, given the large number of pilot studies underway that are unlikely to provide conclusive results and the risk that people will try to draw incorrect conclusions when those results become available.  Often, health care leaders find these calculations inconvenient and frustrating, as if the bearers of this mathematical news are being overly rigorous and “academic.”

Note that these concepts and conclusions are applicable not only to evaluations of PCMH, but also of other programs intended to improve processes or capabilities at the level of a physician’s practice, a clinic or a physician organization such as health information technology investments, training staff in Lean methods, or implementing gain-sharing or other performance-based incentives.

Read More

Telling a 46 year health care cost growth story in one graph

In a recent post to the Health Affairs Blog, Charles Roehrig, an economist who serves as VP and director of Altarum’s Center for Sustainable Health Spending, presented some very interesting graphics of long term health care cost growth in the U.S.  He shows the often-presented statistic of health care cost as a percent of Gross Domestic Product (GDP) over the 46 year period since Medicare was established in 1965.  The climbing graph is bumpy due to the effect of recessions and recoveries on the GDP measure in the denominator.  To see the underlying health care cost growth more clearly, Roehrig calculates what the GDP would have been during each time period if the economy was at a full employment state, called the “potential GDP.”  He then calculates health care cost as a percentage of potential GDP.  This creates a nice, steady upward ramp from 6% to almost 17%.

Then, using the potential GDP as a base, Roehrig created a great graphic showing how fast hospital, physician, prescription drug and other cost components grew in excess of the potential GDP.  In his blog post, Roehrig tells the story in graphs and words.  I created the following version of Roehrig’s graph to try to incorporate more of the story into the graph itself.

Roehrig concluded that the “policy responses to excess growth for hospital services, physician services, and prescription drugs seem to have been fairly successful.”  But, he referenced Tom Getzen, who warns against assuming that the recent lower growth rate is the “new normal.”  Rather, it may be temporarily low due to the lagged effects of the recent recession.  So, it may be too early to break out the champagne.

I really like showing such a long time horizon and breaking down health care costs into these five categories.  And, I am convinced that using potential GDP represents an improvement over the conventional GDP measure as a denominator for cost growth statistics.  But, I’ve never understood the popularity of expressing costs as a percentage of GDP in the first place.  In my opinion, it is more relevant to just show growth in real (inflation-adjusted) per capita costs, or the insurance industry variant of that, “per member per month” (PMPM) costs. Using GDP or potential GDP in the denominator seems to imply that, as our country gets more populous and richer, we should increase health care costs accordingly.  I agree with the idea that population growth should logically lead to increasing health care expenditures.  Expressing costs on a per capita basis handles that issue.  But, if we are prioritizing health care services as essential services, we would not necessarily need to spend that much more on health care if we got richer.

Read More

Primary care physicians acknowledge over-utilization and blame it on the lawyers.

Catching up on some reading, I came across this article in Medical News Today, describing the results of survey research conducted by Brenda E. Sirovich, MD, MS, and colleagues from the VA Outcomes Group (White River Junction, Vermont), and the Dartmouth Institute for Health Policy and Clinical Practice.   They surveyed primary care physicians and published their results in the Archives of internal Medicine.  They documented that primary care physicians acknowledge over-utilizationof services received by their patients.

Their #1 theory of causation?  “It’s because of malpractice lawyers!” That is not surprising to me, and is consistent with many conversations with both front line PCPs and leaders of primary care physician organizations.

However, I personally believe that this is really the #1 rationalization of the over-utilization.  I feel that there are two main causes:

  1. Low fee-for-service reimbursement, creating the need for many short visits each day to generate enough revenue to make a good living (i.e. the “hamster wheel”).  When visits need to be short, prescriptions and referrals are important to make the patient feel satisfied that their problem is really being addressed.
  2. Lack of effective clinical leadership or even peer interaction over the actual clinical decision-making (i.e. “care-planning”) done on a day-to-day basis by the vast majority of primary care physicians

Beyond the medical school and residency stage, physicians’ care planning occurs all alone, with no-one looking over their shoulder — at least no one with sufficient quantity and quality of information to make any real assessment of clinical decision-making.  Health plans have tried to do so with utilization management programs, but the poor quality of information and the relationship distance between the physician and the health plan are too great to generate much more than antipathy.

If you eliminated malpractice worries and paid primary care physicians a monthly per-capita fixed fee, would wasteful over-utilization go down without also providing deeper clinical leadership and peer review enabled by better care planning data?  Perhaps.  But I would worry that, in that scenario, physicians would still stick with their old habits of hitting the order & referral button out of habit to please the patients who have been habituated to think of “lots of orders and referrals” as good primary care.

The “mindfulness” thing in the invited commentary by Calvin Chou, MD, PhD, from the University of California, San Francisco, is a bit much — trying too hard to coin a term.  I’ve heard that presented before, and I categorized it with “stages of change,” “empowerment,” “self-actualization,” “motivational interviewing,” and “patient activation.”   I’m not saying that such popular psychological/sociological concepts have no merit.  I’m just a Mid-Westerner who starts with more conventional theories of behavior.

Read More

So, is there any good use of O/E analysis? Yes. It’s called Benchmark Opportunity Analysis.

In last week’s post, I argued that observed over expected analysis (O/E) was commonly misused as a method for doing “level playing field” performance comparisons.  I recommend against using it for that purpose.

But, is there some other good use for O/E analysis?

I can’t think of a good use for the O/E ratio itself — the metric derived by dividing observed performance by expected performance.  But, it turns out that the underlying idea of developing a model of performance that has been shown to be achievable is very useful to identify and prioritize opportunities for improvement.  The idea is to apply such a model to a health care provider’s actual population, and then compare those “achievable” results with the actual performance of the provider to see how much room there is for improvement.  I like to call this “opportunity analysis.”

There are two main variations on the “opportunity analysis” theme.  The first approach is to consider the overall average performance achieved by all providers as the goal.  The basic idea is to estimate how much the outcome will improve for each provider if they focused on remediating their performance for each risk cell where they have historically performed worse than average.  The analysis calculates the magnitude of improvement they would achieve if they were able to move their performance up to the level of mediocrity for such risk cells, while maintaining their current level of performance in any risk cells where they have historically performed at or above average.  A good name for this might be “mediocrity opportunity analysis,” to emphasize the uninspiring premise.

The second variation on this approach challenges providers to achieve excellence, rather than just mediocrity. I like to call this “benchmark opportunity analysis.”   The idea is to create a model of the actual performance of the one or more providers that achieves the best overall performance, called the “benchmark providers.” Then, this benchmark performance model is applied to the actual population of each provider, to estimate the results that could be achieved, taking into account differences in the characteristics of the patient populations.  These achievable benchmark results are compared to the actual performance observed.  The difference is interpreted as the opportunity to improve outcomes by emulating the processes that produced the benchmark performance.

As shown in this illustrative graphic, a benchmark opportunity analysis compares different improvement opportunities for the same provider.  In the example, Acme Care Partners could achieve the greatest savings by focusing their efforts on improving the appropriateness of high tech radiology services. In contrast, Acme Care Partners is already achieving benchmark performance in appropriateness of low tech radiology services, and therefore has zero opportunity for savings from improving up to the benchmark level.  That does not mean that they can’t improve.  No analysis can predict the opportunity for true innovation.  Benchmark opportunity analysis is just a tool for pointing out the largest opportunities for emulating peers that already perform well, taking into account the differences in the mix of patients between a provider organization and it’s high performing peers.

This method is generally consistent with the “achievable benchmarks of care” (ABC) framework proposed more than 10 years ago by the Center for Outcomes and Effectiveness Research and Education at the University of Alabama at Birmingham.  However, that group advises against using the method for financial performance measures, presumably out of fear that it could encourage inappropriate under-utilization.  I consider that a valid concern.   To reduce that risk, I advocate for a stronger test of “achievability” for cost and utilization performance measures.  In the conventional ABC framework for quality measures, “achievability” is defined as the level of performance of the highest-performing set of providers that, together, deliver care to at least 10% of the overall population.  Such a definition is preferable to simply setting the benchmark at the level of performance achieved by the single highest-performing provider because a single provider might have gotten lucky to achieve extremely favorable performance.  When I apply the achievable benchmark concept to utilization or cost measures, I set the benchmark more conservatively than for quality measures.  For such measures, I use 20% rather than 10% so as to avoid setting a standard that encourages extremely low utilization or cost that could represent inappropriate under-utilization.

Note that one provider may have benchmark care processes that would achieve the best outcomes in a more typical population, but that same provider may have an unusual mix of patients that includes a large portion of patients for whom they don’t perform well, creating a large opportunity for improvement.  The key point is that opportunity analysis is the right method to compare and prioritize alternative improvement initiatives for the same provider.  But the results of opportunity analyses should not be used to compare the performance of providers.

The following graphic summarizes the comparison of traditional risk adjustment, O/E analysis, and benchmark opportunity analysis.

Simple Example Calculations

For those readers interested in a little more detail, the following table uses the same raw data from the calculations from last week’s post to illustrate the approach.

As shown in this example, Provider A has worse performance (higher mortality rate) than Provider B in adults.  So, Provider B is the benchmark performer in the adult risk cell.  If Provider A improved from 6.41% mortality down to the 5.00% mortality level of Provider B, it could save the lives of 11 adults per year.  Provider B has worse performance in children.  If Provider B improved its performance in children up to the level achieved by Provider A, while still achieving its benchmark level of performance in adults, it could save 1 life per year.

Read More

HHS Releases Final ACO Rule

The Department of Health & Human Services (HHS) today released the final rule for accountable care organizations (ACO).

The new rule includes a number of changes designed to make the Medicare Shared Savings Program more palatable for health care providers who had a largely negative response to the draft rule released last March. The changes include the following:

  • Allow providers to choose to participate without any downside financial risk during the initial contract period, rather than requiring all participants to take downside risk during the third year of the contract period
  • Provide up front financial support to physician-owned ACOs to support investments in building ACO capabilities, to be repaid through gain sharing rewards in subsequent years
  • Reduce the up front investment needed by eliminating the requirement for meaningful use of electronic health records
  • Reduce the number of quality measures from 65 to 33
  • Allow providers to choose from a number of available start dates throughout 2012
  • Allow community health centers and rural health clinics to serve as ACOs
  • Prospective identification of the Medicare beneficiaries for whom the ACO will be held accountable, rather than deriving such care relationships after the accountability period
  • Eliminates the mandatory anti-trust review for newly-formed ACOs
  • Puts the burden on the federal government, rather than nascent ACOs, to gather data regarding local market share
The  text of the rule is available here, and the associated final waiver rules are available here.

In my opinion, the elimination of the requirement to accept downside risk is likely to substantially increase the willingness of providers to participate in the program, while simultaneously reducing the likelihood that participation will lead to meaningful transformation of the care process within those participants.  But, given the strong opposition to the draft rule, CMS had little choice but to dilute the requirements to at least get some players to take the field.

 

Read More

Observed over expected (O/E) analysis is commonly misapplied to performance comparisons. Please don’t.

A few years ago, I had a shocking and enlightening discussion about analytic methods with a group of epidemiologists and biostatisticians from Blue Cross Blue Shield of Michigan.

PGIP Quarterly Meeting in Lansing

We were sitting around a table at a conference center in Lansing, where we were debriefing from a meeting of the Physician Group Incentive Program. We were talking about the methods for performance comparison. Everyone knew that we needed to “risk adjust” to take into account differences in patient characteristics when comparing the performance of different physicians, clinic practice units, and physician organizations. If we failed to properly risk adjust, the poorly performing providers would surely argue “my patients are sicker.”

Traditional Risk Adjustment using Standardization

When epidemiologists want to compare the performance of two health care providers on a level playing field, the traditional method is to do risk adjustment using an approach called standardization.    The concept is to determine which patient or environmental variables influence the outcome of interest.  These are called confounding variables, because differences in the mix of patients based on these variables can confound the performance comparison unless they are taken into consideration. Examples of such confounding variables include age, gender, the presence of co-morbid conditions, etc.  If any of the confounding variables are continuous numbers, like age, the epidemiologist must first convert them to discrete categories, or groupings.  For example, if age was the confounding variable, the epidemiologist might define categories for “adults” and “children.”  Or, the epidemiologist might break age into ten-year ranges.  If there is more than one confounding variable, the categories are defined based on the combinations of values, such as “adult females,” “adult males,” etc.  These categories are sometimes called “risk strata” or “risk cells.”  Then, for each of these categories, for each of the providers being compared, the epidemiologist calculates the outcome measure of interest, such as the mortality rate or the total cost of care.  The result is a matrix of measure values for each of the risk cells for each provider.  This matrix can be conceptualized as a “model” of the actual performance of each provider.

To create a level playing field for comparisons, the epidemiologist then creates a standardized population.  The standardized population is simply the number or percentage of patients in each of the risk cells.  Then, the model of each provider’s performance is applied to that standardized population to determine what the outcomes would have been if that provider had cared for the standardized mix of patients.  For each of the risk cells, the standardized number of patients is multiplied by the actual performance that provider achieved for such patients.  Then, the results for all the risk cells are aggregated to obtain that provider’s risk-adjusted performance. Another way of thinking about this is that the risk adjusted outcome is the weighted average outcome for all the risk cells, where the weights are the proportion of patients in that risk cell in the standardized, level-playing-field population.  If the provider’s actual mix of patients was “sicker” or “higher risk” than the standardized population, then the risk adjusted outcome will be more favorable than the unadjusted outcome.

“Observed Over Expected” Analysis

In the literature, even in respected journals, I have seen many examples of performance comparisons that used a different analytic approach called “observed over expected” or “O/E,” rather than using the traditional standardization approach.  A recent example is a paper regarding the mortality-avoidance performance of childrens’ hospitals.  Just as with standardization, the O/E method begins by identifying confounding variables — the patient characteristics that are predictors of the outcome of interest.   With O/E analysis, confounding variables that are continuous, like age, do not have to be converted to discrete categories or groupings.  All the confounding variables are used as independent variables in a regression model.  Then, the resulting regression model is applied to each individual patient observation, inserting the values of the predictor variables for that patient into the regression formula to obtain the “expected” value of the outcome of interest.  At that point, you have the actual observed value and the expected value for each patient (or case).  Then, you sum up the observed values for all the patients for a given provider.  And, you sum up the expected values for all the patients for a given provider.  Finally, you divide the sum of observed values by the sum of expected values to get the O/E ratio.  If the ratio is greater than one, that is interpreted to mean that the provider has a higher-than-expected value for that outcome.  If the outcome variable is undesirable, such as mortality rate, complication rate or cost, an O/E ratio of greater than one is interpreted to mean that the provider performed poorly compared to the other providers.  People have been routinely using O/E analysis as if it was a way of doing risk-adjusted performance comparisons — as a way of “leveling the playing field” to do performance comparisons that take into account differences in patient characteristics.

But, sitting around the table in Lansing, I was shocked to realize that O/E analysis is not actually applicable for this purpose.  Why? Because O/E analysis does not actually create a level playing field.

On the contrary, O/E analysis is conceptually the opposite of the traditional standardization approach.  In traditional standardization, a model of each provider’s actual performance is applied to a standardized population.  In O/E analysis, the regression model is essentially a model of typical performance.  That regression model is applied to the actual population that received care from a particular provider.  The problem is that different providers can see a different mix of patients.  Consider the following simple calculations.

In this simple example, we are comparing just two providers.  We are considering just one confounding variable, age.  And, we are breaking that variable into just two categories, adults and children.  As shown in the example, provider A sees mostly adults, while provider B sees mostly children.  Provider B performs poorly in those children, but actually performs better that A in the adult population.  Because provider B sees more children, the poor performance in children dominates the O/E calculation, so provider B looks bad in terms of an O/E ratio of 1.09.  But, since there are more adults than children in the overall population, which we are using as the “standardized” population, provider B’s superior performance in adults dominates the risk adjustment.  So, provider B has a risk-adjusted mortality rate that is better than provider A.  In other words, if you use O/E analysis for level-playing-field performance comparisons, you may get the wrong answer.

Risk adjustment using standardization is not computationally difficult.  But, it does require a bit of programming work to convert continuous variables such as age into categorical variables, such as breaking age into ten-year ranges.  In my opinion, O/E analysis should not be used for risk-adjusted performance comparisons.  O/E analysis does have the advantage of being computationally more convenient, particularly when you have many confounding variables and when some of them are continuous variables.  But, it is not that much more work to do it right.

Read More

2 min video of Harold Miller saying hospitals need to be offered a “glide path” to handle revenue reduction anticipated from new payment models

In this 2 minute video, Harold Miller, Executive Director, Center for Healthcare Quality & Payment Reform,talks about the changes that hospitals will face during the transition to new payment models. His remarks were part of the Massachusetts Medical Society’s program, “A Path to Accountable Care Organizations: How Do We Get There From Here?”, held on Sept. 13, 2011.

He argues that savings won’t necessarily come from reducing the revenues of specialist physicians, but that they are likely to come from reducing revenue to hospitals, device manufacturers and others.  He says hospitals need to anticipate getting smaller, and payers need to create a good “glidepath” to make that happen without being too disruptive.

I agree wholeheartedly with this principle. More generally, I am a big fan of Dr. Miller’s work. He thinks clearly about health care structures and processes, and is an effective communicator. His early framing of Accountable Care remains very useful.

https://www.youtube.com/embed/aeF9mLzCrN4

Read More
Page 5 of 12« First...«34567»10...Last »