When doing analysis, it is really important to clarify up front what it is you are actually trying to figure out. This sounds so obvious. But, I am always amazed at how often sophisticated, eager analysts can zip past this important first step.
Health plans, vendors and health care providers are all involved in the design and execution of wellness and care management programs. Each program can be conceptualized as some intervention process applied to some target population. A program is going to add more value if the target population is comprised of the people for whom the intervention process can have the greatest beneficial impact.
I have found it useful to conceptualize the process of selecting the final target population as having two parts. The first part is the process of identification of the population for whom the intervention process is deemed relevant. For example, a diabetes disease management program is only relevant to patients with diabetes. An acute-care-to-ambulatory-care transitions program is only relevant to people in an acute care hospital. A smoking cessation program is only relevant to smokers. The second part is to determine which members of the relevant population are to actually be targeted. To do this, a program designer must figure out what characteristics of the relevant candidates are associated with having a higher opportunity to benefit from the intervention. For example, in disease management programs, program designers often use scores from predictive models designed to predict expected cost or probability of disease-related hospitalization. They are using cost or likelihood of hospitalization as a proxy for the opportunity of the disease management program to be beneficial. Program designers figure that higher utilization or cost means that there is more to be saved. This is illustrated in the graph below, where a predictive model is used to sort patients into percentile buckets. The highest-opportunity 1% have have an expected annual admit rate of almost 4000 admits per 1000 members, while the lowest 1% have less than 200 admits per 1000 members. A predictive model is doing a better job when more of the area under this curve is shifted to the left.
Although it is common to use expected cost or use as a proxy for opportunity, what a program designer would really like to know is how much good the intervention process is likely to do. Other variables besides expected cost or use can contribute to a higher opportunity. For example, in a disease management program, the program might be more worthwhile for a patient that is already motivated to change their self-management behaviors or one that had known gaps in care or treatment non-compliance issues that the intervention process is designed to address.
Applying micro-economics to care management targeting
Once the definition of “opportunity” is determined and operationally defined to create an “opportunity score” for each member of the relevant population, we can figure out which members of the relevant population to actually target for outreach for the program. Conceptually, we would sort all the people in the relevant population by their opportunity score. Then, we would start by doing outreach to the person at the top of the list and work our way down the list. But, the question then becomes how far down the list do we go? As we go down the list, we are investing program resources to outreach and intervention effort directed at patients for which the program is accomplishing less and less. Economists call this “diminishing returns.”
As illustrated by the red line in the graph above, there is some fixed cost to operating the program, regardless of the target rate. For example, there are data processing costs. Then, if the program does outreach to a greater and greater portion of the relevant population, more and more people say “yes” and the costs for the intervention go up in a more or less linear manner. As shown by the green line, the savings increase rapidly at first, when the program is targeting the candidates with the greatest opportunity. But, as the threshold for targeting is shifted to the right, the additional candidates being targeted have lower opportunity, and the green line begins to flatten. The blue line shows the result of subtracting the costs from the savings to get net savings. It shows that net savings increases for a while and then begins to decrease, as the cost of intervening with additional patients begins to outweigh the savings expected to accrue from those patients. In this analysis, net savings is highest when 41% of the relevant population of diabetic patients is targeted for the diabetes disease management program. The black dotted line shows the result of dividing savings by cost to get the return of investment, or ROI. With very low target rates, too few patients are accumulating savings to overcome the fixed cost. So the ROI is less than 1. Then, the ROI hits a peak at a target rate of 18%, and declines thereafter. This decline is expected, since we are starting with the highest opportunity patients and working down to lower opportunity patients. Note that in this analysis, increasing the target penetration rate from 18% to 41% leads to a lower ROI, but the net savings increases by 24%. So, if the goal is to reduce overall cost, that is achieved by maximizing net savings, not by maximizing ROI.
Should we try to maximize accuracy?
In a recent paper published in the journal Population Health Management by Shannon Murphy, Heather Castro and Martha Sylvia from Johns Hopkins HealthCare, the authors describe their sophisticated methodology for targeting for disease management programs using “condition-specific cut-points.” A close examination of the method reveals that it is fundamentally designed to maximize the “accuracy” of the targeting process in terms of correctly identifying in advance the members of the relevant disease-specific population that will end up being among the 5% of members with the highest actual cost. In this context, the word accuracy is a technical term used by epidemiologists. It means the percentage of time that the predictive model, at the selected cut-point, correctly categorized patients. In this application, the categorization is attempting to correctly sort patients into a group that would end up among the 5% with highest cost vs. a group that would not. By selecting the cut point based accuracy, the Hopkins methodology is implicitly equating the value of the two types of inaccuracy: false positives, where the patient would be targeted but would not have been in the high cost group, and false negatives, where the patient would not be targeted but would have been in the high cost group. But, there is no reason to think that, in the selection of targets for care management interventions, false negatives and false positive would have the same value. The value of avoiding a false negative includes the health benefits and health care cost savings that would be expected by offering the intervention. The value of avoiding a false positive includes the program cost of the intervention. There is no reason to think that these values are equivalent. If it is more important to avoid a false positive, then a lower cut-point is optimal. If it is more valuable to avoid a false negative, then a higher cut-point is optimal. Furthermore, the 5% cost threshold used in the Hopkins methodology is completely arbitrary, selected without regard to the marginal costs or benefits of the intervention process at that threshold. Therefore, I don’t advise adopting the methodology proposed by the Hopkins team.
What about cost-effectiveness?
The concept of maximizing ROI or net savings is based on the idea that the reason a health plan invests in these programs is to save money. But, the whole purpose of a health plan is to cover expenses for necessary health care services for the beneficiaries. A health plan does not determine whether to cover hip replacement surgery based on whether it will save money. They cover hip replacements surgery based on whether it is considered a “standard practice,” or whether there is adequate evidence proving that the surgery is efficacious. Ideally, health care services are determined based on whether they are worthwhile — whether the entire collection of health and economic outcomes is deemed to be favorable to available alternatives. In the case of hip replacement surgery, the health outcomes include pain reduction, physical function improvement, and various possible complications such as surgical mortality, stroke during recovery, etc. Economic outcomes include the cost of the surgery, and the cost of dealing with complications, rehabilitation and follow-up, and the savings from avoiding whatever health care would have been required to deal with ongoing pain and disability. When attempting to compare alternatives with diverse outcomes, it is helpful to reduce all health outcomes into a single summary measure, such as the Quality-Adjusted Life Year (QALY). Then, the incremental net cost is divided by the incremental QALYs to calculate the cost-effectiveness ratio, which is analogous to the business concept of return on investment. If the cost-effectiveness ratio is sufficiently high, the health service is deemed worth doing. There is no reason why wellness and care management interventions should not be considered along with other health care services based on cost effectiveness criteria.
The idea that wellness and care management interventions should only be done if they save money is really just a consequence of the approach being primarily initiated by health plans in the last decade. I suspect that as care management services shift from health plans to health care providers over the next few years, there will be increased pressure to use the same decision criteria as is used for other provider-delivered health care services.
