First take on new CMS Comprehensive Primary Care Plus model

April 23, 2016 by Dr. Ward in Accountable Care Organizations (ACOs), ACO Reimbursement and Gain Sharing, Patient-Centered Medical Home (PCMH) | Leave a comment

CMS CPC IconThis morning, I read about the recently announced next generation version of the CMS Comprehensive Primary Care model, which will require multi-payer participation and will involve up to 5K practices in 20 regions.

Sounds interesting.  I need to study it in more detail.  But based on my initial assessment:
  • I agree with the idea of pursuing payment and delivery system changes on a multi-payer basis to make it more compelling to providers.
  • I also agree with the idea of prepaying some E&M and then paying reduced FFS for E&M to cover only marginal cost of E&M office visits to make providers incentive-neutral on encounter modes.
  • But I disagree with move away from shared savings and implicit abandonment of the idea of non-governmental primary care-based organized systems of care pursuing care process innovation in favor of CMS taking over responsibility for defining a nationally-standardized multi-payer “care delivery model” and injecting it into individual primary care practices using a CMS-developed  “learning system.”
  • I also disagree with the Track 2 idea of partnering with “CMS-convened” IT vendors and contractual commitment to specific IT capabilities.  That approach basically takes MU, which was a huge distraction from real improvement, to even more obnoxious levels of micro-management.
Overall, I share the Fed’s frustration with the limited impact of previous efforts to transform primary care payment and delivery models, but I think the solution should be to define incentives which are more timely, coherent and consequential, enabled by more meaningful transparency requirements, clearer care relationships and some empowerment of primary care delivery organizations to define their own referral networks.
Read More

The Humunculus is a metaphor for Clinical Process Improvement Frameworks

December 10, 2011 by Dr. Ward in Accountable Care Organizations (ACOs), Clinical Practice Guidelines and Protocols, Clinical Process Improvement, Continuous Quality Improvement (CQI), Lean Thinking, Patient-Centered Medical Home (PCMH) | Leave a comment

During the last 20 years, we have experienced wave after wave of new frameworks for improving health care.  Each had its own terminology, ardently promoted and enforced by its zealous advocates.  Each had a lifecycle that began with a long incubation period, followed by a period of explosive growth in popularity and influence, rapidly leading to unrealistic expectations, followed by a period of decline during which the framework was declared to have been ineffective.  We’ve been through health maintenance, outcomes management, clinical effectiveness, managed care, disease management, chronic care, care management, practice guidelines, care maps, evidence-based medicine, quality functional deployment, continuous quality improvement, re-engineering, total quality management, and six sigma.  We’re still in the thick of lean, patient-centered care,  value-based benefits, pay-for-performance and accountable care.

Four things I’ve noticed about this lifecycle of health care improvement frameworks:

  1. They are formulated by conceptual thinkers, but then get taken-over by more tactically-oriented people.  The tactical folks often focus too much on the tools, terminology and associated rituals.  The framework always gets “simplified” to be more suitable for mass consumption.  For example, continuous quality improvement somehow morphed into being primarily about assigning a timekeeper during team meetings and communicating progress on a felt-backed “story board,” rather than finding people with systems-thinking talent and applying that talent to understand sources of variation in complex processes.
  2. During the early part of the growth phase, the advocates are always desperate for examples of success, and shower a great deal of attention on early projects that are described using the terminology of the framework and that appear to have succeeded.  The desperation usually leads advocates to lower their standards of evidence during this phase.  This leads to over-promising and unrealistic expectations.  It stimulates lots of superficial imitation by people interested in hopping on the bandwagon.  And, it plants the seeds for the eventual decline, when people determine that their inflated expectations were not met.
  3. The decline phase, when the framework is declared to be ineffective, seems to always happen before the framework was ever really implemented in the way envisioned by the original formulators during the incubation phase.
  4. All the frameworks are really just restatements of the same underlying concepts, but with different terminology and tools, and different emphasis.  In other words, they all have the same anatomy, but different parts of the anatomy are emphasized.

This last point reminds me of the “humunculus,” also called the “little man.” When I was in medical school in the late 1980s, we used heavy text books that generally did a bad job of teaching the information. One notable exception was clinical neuroanatomy. We used a small, paperback text book playfully entitled “Clinical Neuroanatomy Made Ridiculously Simple” by Stephen Goldberg, MD. It contained a collection of clever drawings designed to explain the structures and functions of the brain and spinal cord. Perhaps the most famous of the drawings was the humunculus.

Cross section of somatosensory cortex, showing mapping to sensory input sources

This drawing was adapted from earlier work by an innovative neurosurgeon named Wilder Penfield, who invented new surgical procedures for patients with epilepsy during the late 1930s.  During those procedures, he used electrodes to stimulate different points on the surface of the brain.  He drew diagrams similar to the drawing above showing that the surface of the brain contained a little man hanging upside down. The diagram shows that a disproportionate portion of the brain surface is dedicated to the sense of touch and muscle movements in certain parts of the body.  Lots of brain surface is dedicated to highly sensitive and nimble areas like the lips, tongue, hands and feet.  Very little brain surface is dedicated to the arms, legs and back.  Many anatomic illustrators have drawn the humunculus as a cartoon character showing how this disproportional emphasis on different parts of the body looks on the little man.

The Humunculus

The humunculus is a great teaching tool, making it easy to remember these aspects of clinical neuroanatomy.  But, I think the humunculus is also a useful metaphor for the distorted emphasis that various health care improvement frameworks have placed on various parts of the underlying anatomy of health care improvement.

Framework

Emphasis
Health maintenance Preventive services
Outcomes Management Measurement of function, patient experience and health status
Clinical Effectiveness Measurement of outcomes in real world settings, rather than laboratory controlled conditions
Managed Care Prospective review of appropriateness of referrals, procedures and expensive drugs, and retrospective review of cost of care
Disease Management Role of nurses in training patients to be more effective in self-management
Chronic Care Teamwork in primary care clinic and importance of organizational and community environment
Care Management Role of nurses in coordinating services delivered by different providers and in different settings
Practice Guidelines Consensus about which ambulatory services are appropriate in which situations
Care Maps Consensus about the sequence of inpatient services for different diagnoses
Evidence-based Medicine Weight of scientific evidence about efficacy of a service (without regard to cost)
Quality Functional Deployment Focus on the demands made by patients
Continuous quality improvement Small experiments to determine if incremental process changes are improvements
Re-engineering Designing new processes from scratch, rather than making incremental changes
Total Quality Management Importance of organizational culture and management processes
Six Sigma Focus on reducing frequency of defects
Lean Focus on eliminating non-value-adding process steps and reducing cycle time
Patient-centered care Focus on the needs of patients and the involvement of patients in their own care
Value-based Benefits Financial incentives to motivate patients to comply with recommended treatments that reduce overall cost
Pay-for-performance Financial incentives to motivate individual physicians to improve quality and reduce cost
Accountable care Financial incentives to motivate health care organizations to improve quality and reduce cost

Over the years, I have assimilated the concepts, terminology and tools from these various improvement frameworks into an approach that attempts to achieve balance, with each aspect of the framework shown without over-emphasis.

This framework puts the patient in the center, surrounded by the health care processes, which are surrounded by improvement processes.  It attempts to balance between focusing on care planning (the clinical decision-making regarding what services are needed) vs. focusing on care-delivery (the teamwork to execute the care plan and provide health care services to the patient).  It balances between measuring outcomes and measuring quality and cost performance.  It balances between implementing best practices through guidelines and protocols vs. improving practices through performance feedback and incentives. By avoiding a distorted over-emphasis on any one part of the anatomy, hopefully it can have greater lasting power than some of the more humunculus-like frameworks that have come and gone.   This framework is described more fully here.

Read More

Identifying and Understanding Analysis Tricks: Regression Toward the Mean

July 9, 2011 by Dr. Ward in Evaluation Methods, Evaluation Tricks to watch out for, Patient-Centered Medical Home (PCMH) | Leave a comment

Imagine that you are a new homeowner, shopping for insurance for your new house.  You live in an area prone to earthquakes, and you are not a big risk-taker.  You decide that you should have earthquake insurance.  You are on the web researching earthquake insurance policies. You come across the web site of Acme Insurance, an international leader in earthquake damage coverage.  The web site says they are the best earthquake insurance company because they not only pay for earthquake damage, they have an innovative program to actually prevent earthquakes experienced by their beneficiaries. The program involves assigning an earthquake prevention coordinator (EPC) to each homeowner.  The EPC does one session of telephonic earthquake prevention coaching, sends some earthquake prevention educational materials by e-mail, and makes a follow-up call to assure that the homeowner is exhibiting good earthquake prevention behaviors.   This is a proprietary program, so more details are only available to Acme Insurance beneficiaries.  The program is proven to reduce earthquakes by 99%.  You click on the link to view the research study with the documented proof.

The study was conducted by Acme Analysis, a wholly-owned earthquake informatics subsidiary of Acme Insurance.  The study begins by noting an amazing discovery.  When Acme analyzed its earthquake claims for 2010, it noted that 90% of its earthquake damage cost occurred in only 10% of its beneficiaries.  It noted that these high cost beneficiaries were living in particular cities.  For example, it noted high earthquake claims cost in Port au Prince, Haiti for damage incurred during the January 12, 2010 earthquake there.  It developed an innovative high risk classification approach based on the zodiac sign of the homeowners’ birth date and the total earthquake claims cost for damage incurred in the prior month.  On February 1, 2010, they applied this risk classification to identify high risk homeowners, most of which were Libras or Geminis living in Port au Prince.  They targeted 100 of those high risk homeowners for their earthquake prevention program.    The EPCs sprung into action, making as many earthquake prevention telephone coaching calls and sending as many earthquake prevention e-mails as they could, considering the devastated telecommunications infrastructure in Port au Prince.

The program evaluation team then compared the rate of earthquakes exceeding 6.0 on the Richter scale and average earthquake damage claims for those 100 people for the pre-intervention period vs. the post intervention period. Among the 100 beneficiaries targeted by the program, the average number of major earthquakes plummeted from 1 in the pre-intervention period (January, 2010) to 0 in the post-intervention period (March, 2010), and the number of minor earthquakes (including aftershocks) dropped from 20 down to just 10. But the program was not just good for the beneficiaries wanting to avoid earthquakes.  It was a win-win for Acme Insurance.  Earthquake damage claims had dropped from an average of $20,000 per beneficiary during the January, 2010 pre-intervention period to an average of just $200 for damage incurred during the post-intervention period in March, 2010, when two of the targeted beneficiaries experienced damage from an aftershock.  The program effectiveness was therefore 1 – (200/20,000) = 0.99.  That means the innovative program was 99% effective in preventing earthquake damage claims cost.  After considering the cost of the earthquake prevention coordinators and their long-distance telephone bills, the program return on investment (ROI) was calculated to be 52-to-1.  The program was a smashing success, proving that Acme Insurance is the right choice for earthquake coverage.

Can you spot the problem? Can you extrapolate this insight to the evaluation of health care innovations such as disease management, care coordination, utilization management, patient-centered-medical home, pay-for-performance, accountable care organizations, etc.?

The problem is called “regression toward the mean.” It is a type of bias that can affect the results of an analysis, leading to incorrect conclusions.  The problem occurs when a sub-population is selected from a larger population based on having extreme values of some measure of interest.  The fact that the particular members had an extreme value at that point in time is partly a result of their underlying unchanging characteristics, and partly a matter of chance (random variation).   Port au Prince, like certain other cities along tectonic plate boundaries, is earthquake prone.  This is an unchanging characteristic.  But, it was a matter of chance that a major earthquake hit Port au Prince in the particular month of January, 2010.  If you track Port au Prince in subsequent months, their theoretical risk of an earthquake will be somewhat higher because it is still an earthquake prone area.  But, chances are that, in any typical month, Port of Prince will not have a major earthquake.

An analogous effect can be observed when you identify “high risk” patients based on having recently experienced extreme high rates of health care utilization and associated high cost.  The high cost of such patients is partly driven by the underlying characteristics of the patients (e. g. age, gender, chronic disease status), and partly based on random chance.  If you track such patients over time, their cost-driving characteristics will lead them to have somewhat higher costs than the overall population.  But, the chance component will not remain concentrated in the selected patients.  It will be spread over the entire population.  As a result, the cost for the identified “high risk” patients will decrease substantially.  It will “regress” toward the mean.  With high risk classification methods typically used in the health care industry, my experience is that this regression is in the 20-60% range over a 3-12 month period, without any intervention at all.  Of course, the overall population cost will continue to follow its normal inflationary trend line.

This regression-toward-the-mean phenomenon has been at play in many, many evaluations of clinical programs of health plans and wellness and care management vendors.  Sometimes unwittingly.  Sometimes on purpose.  Starting back in the 1990s, disease management vendors were fond of negotiating “guarantees” and “risk sharing” arrangements with managed care organizations where they would pick high risk people and guarantee that their program would reduce cost by a certain amount.   Based on regression toward the mean, the vendor could rely on the laws of probability to achieve the promised result, regardless of the true effectiveness of their program.  The vendor would get their negotiated share of the savings.  Good work if you can get it.  It lasted for a few years until the scheme was widely discredited.  But not widely enough, it appears.  Wellness and care management vendors still routinely compare cost before and after their intervention for a cohort of patients selected based on having extreme high cost in the pre-intervention period.  Health plans and employers eat up the dramatic savings numbers, happy to see that the data “proved” that they made wise investments.

Study designs suffering from obvious regression-toward-the-mean bias will usually be excluded from publication in peer-reviewed scientific journals.  But, they do show up in less formally-reviewed clinical program evaluations by hospitals and physician organizations.  For example, in a recent analysis of a patient-centered medical home (PCMH) pilot, the authors concluded that the program had caused a “48 percent reduction in its high-risk patient population and a 35 percent reduction in per-member-per-month costs” as shown in the following graphic.

In this PCMH program, a total of 46 “high risk poly” members were selected based on having high recent health care utilization involving 15 or more health care providers.  The intervention consisted of assigning a personal health nurse that developed a personal health plan,  having a personal health record (based on health plan data), and providing reimbursement for monthly 1-hour visits with a primary care physician.  The analysis involved tracking of the risk category (based on health plan claims data) and the per-member-per-month (PMPM) cost for the cohort of 46 patients, comparing the pre-intervention period (2009) to the intervention period (2010).   I’m sure the program designers and evaluators for this PCMH pilot are well meaning and not trying to mislead anybody.  I share their enthusiasm for the PCMH model of primary care delivery.  But, I think the evaluation methodology is not up to the task of proving whether the program did or did not save money. Furthermore, even with a better study design to solve the problem of regression-to-the-mean bias, the random variation in health care costs is far too large to be able to detect even a strong effect of a PCMH program in a population of only 46 patients.  Or even 4,600 patients for that matter.  I’d guess that proper power calculations would reveal that at least 46,000 patients would be required to have a chance of proving PCMH effectiveness in reducing cost.

So, how do you solve the problem of Regression Toward the Mean?

As with any type of bias, the problem is with the comparability of the comparison group.  The gold standard study design is, of course, a randomized controlled trial (RCT), where the comparability of the comparison group is assured by randomly assigning subjects between the intervention group and the comparison group (also called the “control group”).

If randomization is not possible, one can try to find a concurrent comparison group that is not eligible for the program and which is thought to be otherwise similar to the eligible population.  The same selection criteria that is applied to the eligible population to pick targets is also used in the ineligle population to pick “simulated targets” for a comparison group.  Note that in such a concurrent control design, the comparison should be between targets and the simulated targets, without considering which of the targets volunteered to actually participate in the intervention.  This aspect of the design is called an “intention to treat” approach, intended to avoid another type of bias called “volunteer bias.” (More on that in a future post.)

Often, the evaluators do not have access to concurrent data from an ineligible population to provide a concurrent comparison group.  In such a case, an alternative is “requalification” of the population in the pre-intervention period and the post-intervention period. Requalification involves using the exact same selection criteria used to pick targets at baseline and shifting it forward in time to pick a comparison group. The result will be a comparison group that is a different list of patients than the ones picked for the intervention. Some of the targets of the intervention may be requalified to be in the comparison group. Others will not. Some members of the comparison group will be members that did not qualify to be in the intervention group.  It is counter-intuitive to some people that such an approach creates a better comparison group than just tracking the intervention group over time.  But, with requalification, you are assured that the same influence that luck had in selecting people based on recent utilization will be present in the comparison group.  The idea is to avoid bias in the selection process by trying to make the selection process symmetrical between intervention and comparison groups.

If I apply these remedies for regression toward the mean bias, does that mean I will be assured of reliable evaluation results?

Unfortunately, no. The bad news is that clinical programs are devilishly hard to properly evaluate.  There are many other sources of bias intrinsic to many evaluation designs.  And, the random variation in the measures of interest are often very large compared to the relatively weak effects of most clinical programs.   This creates a “signal to noise” problem that is particularly bad when trying to evaluate small pilot programs or the performance of individual providers over short time periods.

If you really want to create a learning loop within your organization, there is no alternative to building a team that includes people with the expertise required to identify and solve thorny evaluation and measurement problems.

Read More

Reference for Joint Principles of the Patient Centered Medical Home, 2007

February 1, 2011 by Dr. Ward in Bibliography, Patient-Centered Medical Home (PCMH) | Leave a comment

Here is the original reference to the February, 2007, 3-page consensus document that describes the Joint Principles of the Patient Centered Medical Home.  It was created by the four professional societies that are focused on primary care: American Academy of Family Physicians (AAFP), American Academy of Pediatrics (AAP), American College of Physicians (ACP), and American Osteopathic Association (AOA).

Very simple and straight-forward.

http://www.pcpcc.net/content/joint-principles-patient-centered-medical-home

Read More