New HIT ROI lit review is a methodologic tour de force, but misses the point

JAMIA logoRecently, Jesdeep Bassi and Francis Lau of the University of Victoria (British Columbia) published in the Journal of the American Medical Informatics Association (JAMIA) another in a series of review articles that have been written in recent years to summarize the literature regarding the economic outcomes of investments in health information technology (HIT).  Such articles answer the questions

  • “How much do various HIT technologies cost?”
  • “How much do they save?”
  • “Are they worth the investment?”

They reviewed 5,348 citations found through a mix of automated and manual search methods, and selected a set of 42 “high quality” studies to be summarized.  The studies were quite diverse, including a mix of types of systems evaluated, methods of evaluation, and measures included.  The studies included retrospective data analyses and some analyses based on simulation models.  The studies included 7 papers on primary care electronic health record (EHR) systems, 6 on computer-based physician order entry (CPOE) systems, 5 on medication management systems, 5 on immunization information systems, 4 on institutional information systems, 3 on disease management systems, 2 on clinical documentation systems, and 1 on health information exchange (HIE) networks.

Lau HIT ROI results

Key results:

  • Overall, 70% of the studies showed positive economic results, 24% were inconclusive, and 6% were negative.
  • Of 15 papers on primary care EHR, medication management, and disease management systems, 87% showed net savings.
  • CPOE, immunization, and documentation systems showed mixed results.
  • The single paper on HIE suggested net savings, but the authors expressed doubts about the optimistic assumptions made in that analysis about a national roll-out in only ten years.

My take:

Bassi and Lau have made an important contribution to the field by establishing and documenting a very good literature review methodology – including a useful list of economic measures, a nice taxonomy of types of HIT, and many other tools which they graciously shared online for free in a series of appendices that accompany the article.  They also made a contribution by doing some tedious work to sort through lots of papers and sorting and classifying the HIT economics literature.

But, I think they missed the point.

Like many others, Bassi and Lau have implicitly accepted the mental model that health information technology is, itself, a thing that produces outcomes.  They evaluate it the way one would evaluate a drug or a cancer treatment protocol or a disease management protocol.  Such a conceptualization of HIT as an “intervention” is, unfortunately, aligned with the way many healthcare leaders conceptualize their investment decision as “should I buy this software?”  I admit to contributing to this conceptualization over the years, having published the results of retrospective studies and prospective analytic models of the outcomes resulting from investments in various types of health information technologies.

Process PuckBut, I think it would be far better for health care leaders to first focus on improvement to care processes — little things like how they can consistently track orders to completion to assure none fall through the cracks, bigger things like care transitions protocols to coordinate inpatient and ambulatory care team members to reduce the likelihood that the patient will end up being re-hospitalized shortly after a hospital discharge, and really big things like an overall “care model” that covers processes, organizational structures, incentives and other design features of a clinically integrated network.  Once health care leaders have a particular care process innovation clearly in sight, then they can turn their attention to the health information technology capabilities required to enable and support the target state care process.  If the technology is conceptualized as an enabler of a care process, then the evaluation studies are more naturally conceptualized as evaluations of the outcomes of that process.  The technology investment is just a one of a number of types of investments needed to support the new care process.  The evaluation “camera” zooms out to include the bigger picture, not just the computers.

I know this is stating the obvious.  But, if it is so obvious, why does it seem so rare?

This inappropriate conceptualization of HIT as an intervention is not limited to our field’s approach to economic evaluation studies.  It is also baked into our approach to HIT funding and incentives, such as the “Meaningful Use” incentives for investments in EHR technology, and the incentives created by HIT-related “points” in accreditation evaluations and designations for patient-centered medical home (PCMH), accountable care organizations (ACOs), organized systems of care (OSC), etc.  The designers of such point systems seem conscious of this issue.  The term “meaningful use” was intended to emphasize the process being supported, rather than the technology itself.  But, that intention runs only about one millimeter deep.  As soon as the point system designers put any level of detail on the specifications, as demanded by folks being evaluated, the emphasis on technology becomes instantly clear to all involved.  As a result, the intended focus on enabling care process improvement with technology slides back down to a  requirement to buy and install software.  The people being evaluated and incentivized lament that they are being micromanaged and subject to big burdens.  But they nevertheless expend their energies to score the points by installing the software.

So, my plea to Bassi and Lau, and to future publishers of HIT evaluation studies, is to stop evaluating HIT.  Rather, evaluate care processes, and require that care process evaluations include details on the HIT capabilities (and associated one time and ongoing costs) that were required to support the care processes.

Read More

What can we learn from the managed care backlash of the 1990s? Can we avoid an ACO backlash?

Advocates of “accountable care organizations” (ACOs) are careful to avoid the terminology of “managed care,” which is widely viewed as a failed model from the 1980s and 1990s.  But, there are obvious similarities between ACOs and managed care.  Both involve an organization taking responsibility for the quality and cost of care for a defined population.  Both emphasize the importance of primary care as the foundation of a coordinated and efficient health care delivery system.  Both involve economic incentives to physicians to improve quality and slow the upward trend in total cost of care.

But, we all remember the strong backlash against managed care during the late 1990s.   Although almost 10% of the U.S. population are still served by HMOs, the managed care vision has been largely in exile for more than a decade now.   PPOs are now the dominant model, with relatively small financial incentives to patients to seek their care from providers within relatively large provider networks.  Many PPOs have dabbled in “pay for performance,” but the physician incentives involved have been relatively small and the performance bar set relatively low.  The use of more heavy-handed managed care approaches has declined significantly.  For example, plans usually don’t require a referral authorization by a “gatekeeper” primary care physician before granting access to specialists.  And the use of pre-authorization by health plan staff for many expensive procedures has declined significantly.   Health plans did not drop these heavy-handed approached because they became convinced they were ineffective.  They dropped them because they feared they would face a consumer backlash and lose membership.

So, what can we learn from the managed care backlash?  And what can we do differently to avoid an “ACO backlash?”

I went back to some research done during the height of the managed care backlash to refresh my memory of how bad it was, and why it happened.  Most helpful was a paper from 1997 in Health Affairs by Robert Blenden and other researchers at Harvard and the Kaiser Family Foundation. Blenden reported survey results showing that Americans hated managed care companies even more than they hated banks and oil companies.

Blenden’s survey results showed that a significant proportion of Americans experienced hassles and other problems with managed care plans.  These common, minor problems were hypothesized to serve as the seeds of stronger dissatisfaction and distrust.  The survey also showed that the public overestimated the frequency of rare events that are dramatic and threatening.  For example 66% thought that HMOs sometimes or often hold back on a child’s cancer treatment.  73% thought that HMOs send newborn babies home after just one day, in spite of mothers’ concerns about their children’s health.  As shown in the following graph, there was a dose-response relationship with the “heaviness” of the health plan and the degree of mistrust that the health plan would do the right thing if they got sick.

Blenden concluded that the backlash against managed care was primarily driven by mistrust and fear, leading to calls for government regulation and reducing the market demand for managed care.  I created the following “cause-effect” diagram to illustrate this theory.

So, what can we do differently this time around?  We must do a far better job of building trust. That will require actually being trustworthy.  And, it will require being more proactive about communicating trustworthiness.

This topic is so central to the success of ACOs that it deserves a lot more attention by people who have expertise in public opinion, market communication, and brand development.  But, here is my proposed starting point for developing a strategy to build trust in ACOs and other innovative models of health care finance and delivery.

Read More

Dr. Ward’s Slides from AMGA ACO Learning Collaborative, Chicago, July 14, 2011

Presentation entitled “Developing the Economic Model for Success” from day 2 of the American Medical Group Association’s Accountable Care Organization Learning Collaborative, at the Swissotel in Chicago.  Attended by approximately 60 leaders of AMGA member physician organizations and other interested parties.

Full page PDF format.

Hand-out PDF format.

Read More

Explaining alternative types of ACO structures: How do Co-Management arrangements and Hospitalists fit in?

All across the country, physicians and hospitals are talking to each other about creating new organizational structures to enable them to work together more effectively.  Some are trying to create Accountable Care Organizations (ACOs) or Organized Systems of Care (OSCs) to participate in the Medicare Shared Savings Program or similar programs being launched by commercial health plans.  Others are motivated by a desire to accomplish specific goals, such as improving transitions of care or trying to get surgical procedures to start on time.

Any organizational structure represents a trade-off between different “dimensions” that are competing for attention.   For example, in a car company, it may be helpful to have one part of the company focus on the European market, while others focuses on North America, Asia, etc.  Alternatively, it would be good for one part to focus on products that are sporty, while other parts focus on luxury, economy and utility.  Or, one part can focus on marketing and communications, while other parts focus on research & development, manufacturing, and finance.  Whenever one dimension is selected to be the basis for organizational structure, the other dimensions are not selected.  The lack of consistency and focus in the non-selected dimensions often becomes a problem for the company.  Those problems become the justification for the next reorganization of the company.

In health care, there are a number of relevant dimensions crying out to the selected as the basis for organizational structure:

  • Geography:  Regions of the country, or regions within one metropolitan area, or urban vs. rural, or international
  • Professionals (doctors) vs. Facility (hospitals, ASCs, nursing homes, LTACs, etc.) vs. Ancillaries (lab, home health, DME, optical) vs. Insurers
  • Primary Care vs. Specialty Care
  • Academic. vs. Community (“town vs. gown”)
  • Ambulatory Care vs. Inpatient Care
  • Professional Credentials: Doctors vs. Nurses vs. Administrators vs. Lawyers
  • Clinical Department: Surgery vs. Medicine vs. Pediatrics vs. Psychiatry vs. Dermatology vs. etc.
  • Service Lines: Heart & Vascular  vs. Bone & Joint vs. Fetal & Maternal vs. Cancer vs. Pediatrics vs. End Stage Renal vs. Hospice vs. etc.
  • Payer type: Fee For Service vs. Capitated/Managed Care, or Commercial vs. Government
  • etc.

The structure of a local health system cannot be based on all of these dimensions at the same time.  Different local markets have different structures.  Over the last two decades, the typical structures of local health systems have shifted.  Continued shifting is anticipated.

During the 1980s and 1990s, various “health system” and “staff model HMO” structures were established, and many physician practices were acquired by hospitals and other large entities.  But, after the managed care back-lash of the late 1990s, the degree of integration decreased.  In the conventional  organizational alignment in the 2000’s, the payer function was separated from providers, and physician providers were separated from hospitals.  Health plans hired their own care managers or contracted with care management vendors in an attempt to improve health outcomes and reduce cost (or at least to appear to do so).  Primary care physicians and specialists formed various types of physician organizations such as multi-specialty group practices and independent physician associations (IPAs).  Hospitals hired their own hospitalists to coordinate hospital services and reduce length of stay under prospective payment.

In anticipation of health care reform, and in particular the Medicare Shared Savings Program, hospitals are initiating the formation of Accountable Care Organizations (ACOs).  Such organizations integrate physicians and hospitals.  They also intend to include care managers as part of the ACO, rather than relying on health plan-based care managers.  I call this the “integrated delivery system ACO Model.”  It resembles the health systems of the 1980s and 1990s, with the exclusion of the payer function.  They desire to avoid “insurance risk” while retaining “performance risk” — although I am deeply skeptical that these two components of variance can be meaningfully separated.

This integrated delivery system ACO model offers the theoretical advantages of alignment of incentives of primary care physicians, specialists and hospitals — all focused on managing the care of a defined population of patients to improve quality and reduce overall population cost.  However, there are a number of important barriers to this model.  First, in many medium and large communities, there are more than one hospital organization and more than one physician organization, and there is a many-to-many relationship between them.  If a hospital organization convenes an ACO, they must either accept the divided loyalty of the associated physicians or they must disrupt the referral relationships to cleanly separate “their” physicians from those of competing ACOs.  If many primary care physicians continue their association with multiple hospital-based ACOs, then it becomes difficult for any one ACO to make significant investments in primary care practice-based care management staff, clinical information technology, and practice transformation coaching.  Each ACO  fears that their competitors will have a “free ride” on their investments.  If hospital leaders in a community attempt to solve this problem by creating a single large ACO for the community, they threaten the competitiveness of the local health care market and risk attracting the ire of the Federal Trade Commission and other anti-trust regulators.

An alternative model for ACO formation is the “primary care-based ACO” model.  Under this model, primary care physician organizations come together to convene the ACO, such that all of the patients of participating primary care practice units constitute the ACO’s defined population.  The draft regulations for the Medicare Shared Savings Plan define the ACO population this way, although it is obviously limited to only the Medicare patients.  In this primary care-based ACO model, the ACO has an easier time achieving critical mass and avoiding free-rider problems with their investments in practice-based care managers, clinical information technology, and practice transformation coaching.  In such as model, specialists and hospitals become cost centers to the primary care-based ACO.  Such specialists and hospitals will continue to provide services to patients from multiple ACOs.  In this sense, hospitals and specialists are on the periphery of the ACO, rather than at the center of it.

Of course, the diagram above is an over-simplification because it lumps all specialists together.  In fact, as shown in the figure below, the optimal alignment of specialists depends on the focus of their practice.  Traditionally, “PARE” providers (pathology, anethesiology, radiology and emergency) are closely affiliated with hospitals.  In addition, specialists with practices that focus on inpatient care and procedures have a stronger natural alignment with hospital organizations.  Such providers include surgeons, interventional cardiologists, and critical care physicians.

On the other hand, internal medical sub-specialists with practices that focus on evaluation and management services for chronic diseases may find themselves more naturally aligned with primary care-based ACOs.  As the “patient-centered medical home” (PCMH) model takes root, and the primary care base of the U.S. health care system is gradually restored, it is recognized that there will not be enough primary care physicians to play that expanded role.  The supply of mid-level practitioners (nurse practitioners and physician assistants) will also be insufficient to fill the gap.  Therefore, I feel that the logical answer is for primary care-based ACOs to engage chronic disease-oriented internal medical sub-specialists to play the role of clinical leaders of some PCMH practices emphasizing the care of patients with those chronic diseases.  Many sub-specialists have been advocating for the right to be considered PCMH providers.  Unfortunately, in the draft rules for the Medicare Shared Savings Programs, such sub-specialists were not included in the criteria for physicians eligible to have “attributed” patients in that program, seemingly coming down on the other side of this debate.

So, how does “Service Line Co-Management” fit in?

As described in a recent article in the AMA online publication, many hospitals are entering into co-management arrangements with specialists associated with their key service lines.  Under co-management, the specialists and the hospital form and co-own a separate limited liability company (LLC) which is assigned the responsibility to accomplish specified goals for the service line and to generally contribute to improved clinical quality or efficiency of services delivered in that service line.  Such an arrangement must be set up and managed carefully, so as not to run afoul of rules against kickbacks for hospital referrals.  Co-management LLCs are not ACOs, since they are not intended to take responsibility for a defined population of patients.  But, they can serve as a first step to building trust between physicians and hospitals in preparation for the formation of an integrated delivery system ACO model.  For hospitals desiring to convene such ACOs without the benefit of a long track record of effective physician collaboration, I highly recommend consideration of the co-management model as a way to “get some points on the board.”  The more focused co-management arrangements are on specific, achievable and measurable goals, the more successful they will be and the better they will serve this trust-building function.

In the context of a primary care-based ACO model, a hospital-specialist service line co-management entity may play a more interesting long term role.  If primary care-based ACOs view such service lines as cost centers, and if such ACOs have multiple hospital partners that are competing against each other, the co-management entity can potentially become the vehicle for value-based contracts with the ACO, delivering a winning combination of clinical quality and population-level efficiency to assure the success of both the hospital and specialist partners.

Therefore, service line co-management can be a great first step, regardless of whether the eventual ACO model in the community ends up as an integrated-delivery system ACO model or a primary care-based ACO model.

What about Hospitalists?

In a primary care-based ACO model, the ACO may be motivated to “reel in” the hospitalist function in order to get the benefits of smoother transitions of care and to encourage stewardship of inpatient and specialist resources as needed to manage overall population health care cost.  Of course, hospitals will not easily give up the control of the hospitalist function that they typically enjoy.  Hospitals need the hospitalists to be focused on their objectives of reducing length of stay and increasing “keepage” (or reducing “leakage”) of specialty and ancillary services referrals to competing hospitals or specialists.  I anticipate a tug-of-war between hospitals and primary care-based ACOs over alignment with hospitalists.  I would consider the degree of alignment with hospitalists to be a measure of the intensity and sincerity of a primary care-based ACOs’ efforts  to manage population cost.  The next few years will be an interesting time to be a hospitalist.

Read More

NEJM report of ACO financial model fails to include risk of delayed start on transformation

On March 23, 2011, Trent Haywood, MD, JD, and Keith Kosel, PhD, MBA published the results in the New England Journal of Medicine (NEJM) web site of a financial model of a hypothetical Accountable Care Organization (ACO).  This model shows that ACOs are likely to lose money on the Medicare Shared Savings Program called for in the Patient Protection and Affordable Care Act during the first three years of implementing the ACO model, based on the up-front investment expected to be required.  The authors conclude that “the high up-front investments make the model a poor fit for most physician group practices.”  They call for modifications to the Medicare Shared Savings Program to make it more generous to participating ACOs.

The model is based on assumptions derived from data from the Physician Group Practice (PGP) Demonstration, carried out by CMS from 2005 to 2010.  In the PGP, the average up-front investment by participants was $1.7M, or $737 per primary care physician (PCP).  The authors calculate that an “unlikely” 20% margin would be required to break even during the 3-year time frame of the Medicare Shared Savings Program scheduled to start in 2012.

Haywood and Kosel are to be commended for taking the time to develop a financial model and publish results.  I think that such models are extremely helpful to real-world decision-makers because they force people to be explicit about the assumptions they are making, and they provide some quantitative estimates of the outcomes relevant to the comparison of available alternatives so people can make better choices.  Unfortunately, in my opinion, the authors misconceptualized the model, creating a risk that people will use the negative results of the model to justify inaction, to their own detriment.

Every decision is a choice among available alternatives.  To create a useful model to support decision-making, an analyst must follow the following four basic steps:

  1. Identify the available alternatives being compared
  2. Identify the outcomes that are relevant to the decision-maker and that are thought to be potentially materially different across the available alternatives
  3. Make quantitative estimates of  the magnitude and uncertainty of all such outcomes for all the available alternatives, and
  4. Apply values (including ethical principles and preferences) to determine which set out outcomes is most desirable or optimal

Although this basic process seems simple and straight-forward, experienced analysts know that each of these steps is devilishly difficult.  In the case of Haywood and Kosel’s financial analysis, in my opinion, they ran into trouble with the first two steps; they failed to identify the available alternatives and misconceptualized the choice or decision that the model is designed to support, and therefore failed to recognize non-Medicare outcomes that differ across the available alternatives.  Of course, an error in any particular step cascades to the remaining steps.

Haywood and Kosel did not explicitly explain the decision their model was intended to support.  But, one could infer from the conclusion that among the intended decisions they were supporting was the decision by physician organizations whether or not to make a $737 per PCP up-front investment and then sign-up for the optional Medicare Shared Savings Program in order to reap a return in the form of increased Medicare revenue.  But, the up-front investment required to create a successful ACO takes the form of fundamental transformation of care processes and the organizational structures, human resources, information systems, and cultural changes required to support them.  Such fundamental transformations affect the entire population served by the nascent ACO, not just Medicare patients.  And, they don’t just affect the providers’ relationship with payers, they also affect the providers’ competitive standing with respect to other providers and their relationship with other stakeholders such as employers, state and federal legislators, accreditation organizations, etc.

The correct conceptualization of the decision facing provider organizations right now is a choice between (1) getting started now with ACO-type transformation or (2) waiting until later to decide if such a transformation is necessary.  Physicians and hospitals that are contemplating the formation of ACOs would be wise to invest in the creation of a model to make explicit estimates of all the relevant financial and non-financial outcomes for the available alternatives.  Such a model will, by necessity, include many assumptions not supported by solid data.  That’s not the fault of a model, nor a reason to justify making decisions based only an intuition (what David Eddy calls “global subjective judgement”).  Rather, prudent health care leaders will invest the time to create and use a model to really understand the sensitivity of the results to various assumptions and the dynamics of the outcomes (how outcomes are likely to play out over time).

My prediction is that, when properly conceptualized as a “start transformation now” vs. “put transformation off until later” decision, such a model is likely to show what personal retirement planning models always show — it pays to get started on things that take a long time to achieve.  If you fall too far behind competitors, you may be unable to catch up later.  On the other hand, if provider organizations opt to get started on transformation, obviously there are many smaller decisions that need to be made, such as which care processes to start on, which particular payer-specific deals to cut, which IT investments to prioritize, etc.

One last point:  Although “pay back period” can sometimes be a useful summary measure of a financial analysis, my advice to to avoid over-simplifying the reporting of model results by reducing it down to a single summary measure.  Model authors would serve decision-makers better by presenting a table with their estimates of all the relevant outcomes for all the alternatives being considered, and possibly showing when those results occur over time.  Then, decision-makers can understand the drivers of their decisions and subsequently summarize the results in various ways that communicate their thinking most effectively using various summary measures such as net present value, return-on-investment, internal rate of return, pay-back period, cost per quality-adjusted life-year, cost-benefit ratio, etc.

Read More

Satirical YouTube video on Accountable Care Organizations

I’m not sure about the background of this video submitted to YouTube by Centura Health. But, it is hilarious and sad at the same time. I’ve shown it to dozens of people, and it circulated widely among physician organizations in Michigan.

Read More