home » Decision Analysis

To resolve conflicts, re-frame polar positions as optimization between undesirable extremes. But, sometimes there is no way to win.

February 18, 2012 by Dr. Ward in ACO Culture, Decision Analysis, Dr. Ward's Methods Notes | 2 Comments

In politics and professional life, achieving success requires the ability to resolve conflicts. I’ve noticed that conflicts often become entrenched because the opposing parties both simplify the conflict in black and white terms. They conceptualize their own interest as moving in one direction. And, they conceptualize their opponent as wanting to move in the opposite, wrong direction. As a result, the argument between the parties generates no light, only heat. Each side only acknowledges the advantages of their direction and the disadvantages of the opposing direction. Neither side seeks to really understand and learn from the arguments offered by the other side.

When I’ve had the opportunity to mediate such conflicts, I almost always used the same strategies.

Step One. I try to move both parties to some common ground, moving back to some basic statement that seemingly nobody could disagree with. This generates a tiny bit of agreement momentum.
Step Two. Apply the momentum generated in step one to getting the parties to agree that, if you took each party’s position to an extreme, the result would be undesirable. The parties are inherently agreeing to re-conceptualize the disagreement from being a choice between polar opposite positions to an optimization problem. The idea is to choose an agreeable value on some spectrum between undesirable extremes. If the parties make this leap, we are half way to sustainable agreement.

Step Three. Get the parties to agree to avoid talking about the answer, and focus on reaching consensus on the factors and assumptions that influence the selection of the optimal answer. Sometimes, this can be done subjectively, simply listing the factors. Other times, it is worthwhile to get quantitative, working together on assumptions and calculations to estimate the magnitude and uncertainty of the outcomes at various points along the spectrum of alternative answers. This quantitative approach has been described as the “explicit method,” and an example of applying it to resolve fierce disagreements about mammography guidelines is described in an earlier post.
Step Four. Finally, ask the parties to apply their values to propose and explain an optimum answer, from their point of view. In this step, the important point is to insist that the parties acknowledge that they are no longer arguing about facts or assumptions, since consensus has already been achieved on those. If not, then go back to step three. The objective is to untangle and separate factual, logical, scientific debates from the discussion of differences in values. If those remain tangled up, the parties inevitably resort to talking at each other, rather than engaging in productive dialog.
Step Five. Try to achieve a compromise answer. In my experience, if you’ve really completed steps 1-3, this ends up being fairly easy.
Step Six. Work to sustain the compromise. Celebrating the agreement, praising the participants for the difficult work of compromise, documenting the process and assumptions, and appealing to people to not disown their participation in the process are all part of this ongoing work. Passive aggressiveness is the standard operating model in many settings, part of the culture of many organizations. And, it is a very difficult habit to break.

Of course, in the routine work of mediating conflicts, I don’t really explicitly go through these six steps. This conflict resolution approach is in the back of my mind. They are really more like habits than steps.

Sometimes this approach works. Sometimes, it does not. It can break at any step.

Notice that break downs in most of the steps are basically people issues. People won’t change their conceptualization. They are unwilling to make their assumptions explicit. They are unwilling to acknowledge differences in values. They are unwilling to compromise.

But, sometimes, the process breaks because of the nature of the issue being debated. Sometimes, conceptualizing the debate as an optimization problem between two undesirable extremes fails because there are really not good choices along the spectrum.

For example, when debating the design of a program or policy, I have often encountered a no-win trade-off between keeping it simple vs. addressing each party’s unique circumstances. If I keep it too simple, people complain that it as a “hammer,” failing to deal with their circumstances. If I add complexity to deal with all the circumstances, people complain that it is a maze or a contraption. If I select some middle level of complexity, the complaints are even worse because the pain of complexity kicks in before the value of complexity is achieved.

I’ve seen this no-way-to-win scenario in my own work, in the design of information systems, wellness and care management protocols, practice guidelines and protocols, analytic models, organizational structures, governance processes, contractual terms, and provider incentive programs. And, I’ve seen this scenario in many public policy debates, such as debates about tax policy, tariffs, banking regulations, immigration, education, and health care reform. In cases when the extremes are more desirable than the middle ground, the only approach I can think of is to bundle multiple issues together so that one party wins some and the other party wins others, to facilitate compromise.

Reports of the death of Cost-Effectiveness Analysis in the U.S. may have been exaggerated: The ongoing case of Mammography

August 13, 2011 by Dr. Ward in ACO Culture, Cost-Effectiveness Analysis (CEA), Decision Analysis, Electronic Medical Records (EMRs), Registries | Leave a comment

Guidelines for the use of mammograms to screen for breast cancer have been the topic of one of the fiercest and longest-running debates in medicine. Back in the early 1990s, I participated in that debate as the leader of a guideline development team at the Henry Ford Health System. We developed one of the earliest cost-effectiveness analytic models for breast cancer screening to be used as an integral part of the guideline development process. I described that process and model in an earlier blog post. Over the intervening 20 years, however, our nation has fallen behind the rest of the world in the use of cost-effectiveness analysis to drive clinical policy-making. As described in another recent blog post, other advanced nations use sophisticated analysis to determine which treatments to use, while Americans’ sense of entitlement and duty have turned us against such analysis — describing it as “rationing by death panels.” Cost-effectiveness analysis and health economics is dead.

But, maybe reports of its death have been exaggerated.

A recent paper published on July 5, 2011 in the Annals of Internal Medicine described the results of an analysis of the cost-effectiveness of mammography in various types of women. The study was conducted by John T. Schousboe, MD, PhD, Karla Kerlikowske, MD, MS, Andrew Loh, BA, and Steven R. Cummings, MD. It was described in a recent article in the Los Angeles Times. The authors used a computer model to estimate the lifetime costs and health outcomes associated with mammography. They used a modeling technique called Markov Microsimulation, basically tracking a hypothetical population of women through time as they transition among various health states such as being well and cancer free, having undetected or detected cancer of various stages and, ultimately, death.

They ran the models for women with different sets of characteristics, including 4 age categories, 4 categories based on the density of the breast tissue (based on the so-called BI-RADS score), whether or not the women had a family history of breast cancer, and whether or not the women had a previous breast biopsy. So, that’s 4 x 4 x 2 x 2 = 64 different types of women. They ran the model for no-screening, annual screening, and screening at 2, 3 or 4 year intervals. For each screening interval, they estimated each of a number of health outcomes, and summarized all the health outcomes in to single summary measure called the Quality-Adjusted Life Year (QALY). They also calculated the lifetime health care costs from the perspective of a health plan. Then, they compared the QALYs and costs for each screening interval, to the QALYs and costs associated with no screening to calculate the cost per QALY. Finally, they compare the cost per QALY to arbitrary thresholds of $50K and $100K to determine whether screening at a particular interval for a particular type of women would be considered by most policy-makers to be clearly costs effective, reasonably cost-effective, or cost ineffective.

The authors took all those cost effectiveness numbers and tried to convert it to a simple guideline:

“Biennial mammography cost less than $100 000 per QALY gained for women aged 40 to 79 years with BI-RADS category 3 or 4 breast density or aged 50 to 69 years with category 2 density; women aged 60 to 79 years with category 1 density and either a family history of breast cancer or a previous breast biopsy; and all women aged 40 to 79 years with both a family history of breast cancer and a previous breast biopsy, regardless of breast density. Biennial mammography cost less than $50 000 per QALY gained for women aged 40 to 49 years with category 3 or 4 breast density and either a previous breast biopsy or a family history of breast cancer. Annual mammography was not cost-effective for any group, regardless of age or breast density.”

Not exactly something that rolls off the tongue. But, with electronic patient registries and medical records systems that have rule-based decision-support, it should be feasible to implement such logic. Doing so would represent a step forward in terms of tailoring mammography recommendations to specific characteristics that drive a woman’s breast cancer risk. And, it would be a great example of how clinical trials and computer-based models work together, and a great example of how to balance the health outcomes experienced by individuals with the economic outcomes borne by the insured population. It’s not evil. It’s progress.

It will be interesting to see if breast cancer patient advocacy groups, mammographers and breast surgeons respond as negatively to the author’s proposal as they did to the last set of guidelines approved by the U.S. Preventive Services Task Force which called for a reduction in recommended breast cancer screening in some categories of women.

NEJM report of ACO financial model fails to include risk of delayed start on transformation

March 30, 2011 by Dr. Ward in ACO Reimbursement and Gain Sharing, ACO Strategy, Cost-Effectiveness Analysis (CEA), Decision Analysis | 1 Comment

On March 23, 2011, Trent Haywood, MD, JD, and Keith Kosel, PhD, MBA published the results in the New England Journal of Medicine (NEJM) web site of a financial model of a hypothetical Accountable Care Organization (ACO). This model shows that ACOs are likely to lose money on the Medicare Shared Savings Program called for in the Patient Protection and Affordable Care Act during the first three years of implementing the ACO model, based on the up-front investment expected to be required. The authors conclude that “the high up-front investments make the model a poor fit for most physician group practices.” They call for modifications to the Medicare Shared Savings Program to make it more generous to participating ACOs.

The model is based on assumptions derived from data from the Physician Group Practice (PGP) Demonstration, carried out by CMS from 2005 to 2010. In the PGP, the average up-front investment by participants was $1.7M, or $737 per primary care physician (PCP). The authors calculate that an “unlikely” 20% margin would be required to break even during the 3-year time frame of the Medicare Shared Savings Program scheduled to start in 2012.

Haywood and Kosel are to be commended for taking the time to develop a financial model and publish results. I think that such models are extremely helpful to real-world decision-makers because they force people to be explicit about the assumptions they are making, and they provide some quantitative estimates of the outcomes relevant to the comparison of available alternatives so people can make better choices. Unfortunately, in my opinion, the authors misconceptualized the model, creating a risk that people will use the negative results of the model to justify inaction, to their own detriment.

Every decision is a choice among available alternatives. To create a useful model to support decision-making, an analyst must follow the following four basic steps:

Identify the available alternatives being compared
Identify the outcomes that are relevant to the decision-maker and that are thought to be potentially materially different across the available alternatives
Make quantitative estimates of the magnitude and uncertainty of all such outcomes for all the available alternatives, and
Apply values (including ethical principles and preferences) to determine which set out outcomes is most desirable or optimal

Although this basic process seems simple and straight-forward, experienced analysts know that each of these steps is devilishly difficult. In the case of Haywood and Kosel’s financial analysis, in my opinion, they ran into trouble with the first two steps; they failed to identify the available alternatives and misconceptualized the choice or decision that the model is designed to support, and therefore failed to recognize non-Medicare outcomes that differ across the available alternatives. Of course, an error in any particular step cascades to the remaining steps.

Haywood and Kosel did not explicitly explain the decision their model was intended to support. But, one could infer from the conclusion that among the intended decisions they were supporting was the decision by physician organizations whether or not to make a $737 per PCP up-front investment and then sign-up for the optional Medicare Shared Savings Program in order to reap a return in the form of increased Medicare revenue. But, the up-front investment required to create a successful ACO takes the form of fundamental transformation of care processes and the organizational structures, human resources, information systems, and cultural changes required to support them. Such fundamental transformations affect the entire population served by the nascent ACO, not just Medicare patients. And, they don’t just affect the providers’ relationship with payers, they also affect the providers’ competitive standing with respect to other providers and their relationship with other stakeholders such as employers, state and federal legislators, accreditation organizations, etc.

The correct conceptualization of the decision facing provider organizations right now is a choice between (1) getting started now with ACO-type transformation or (2) waiting until later to decide if such a transformation is necessary. Physicians and hospitals that are contemplating the formation of ACOs would be wise to invest in the creation of a model to make explicit estimates of all the relevant financial and non-financial outcomes for the available alternatives. Such a model will, by necessity, include many assumptions not supported by solid data. That’s not the fault of a model, nor a reason to justify making decisions based only an intuition (what David Eddy calls “global subjective judgement”). Rather, prudent health care leaders will invest the time to create and use a model to really understand the sensitivity of the results to various assumptions and the dynamics of the outcomes (how outcomes are likely to play out over time).

My prediction is that, when properly conceptualized as a “start transformation now” vs. “put transformation off until later” decision, such a model is likely to show what personal retirement planning models always show — it pays to get started on things that take a long time to achieve. If you fall too far behind competitors, you may be unable to catch up later. On the other hand, if provider organizations opt to get started on transformation, obviously there are many smaller decisions that need to be made, such as which care processes to start on, which particular payer-specific deals to cut, which IT investments to prioritize, etc.

One last point: Although “pay back period” can sometimes be a useful summary measure of a financial analysis, my advice to to avoid over-simplifying the reporting of model results by reducing it down to a single summary measure. Model authors would serve decision-makers better by presenting a table with their estimates of all the relevant outcomes for all the alternatives being considered, and possibly showing when those results occur over time. Then, decision-makers can understand the drivers of their decisions and subsequently summarize the results in various ways that communicate their thinking most effectively using various summary measures such as net present value, return-on-investment, internal rate of return, pay-back period, cost per quality-adjusted life-year, cost-benefit ratio, etc.

The Smoking Intervention Program, a Provider-based Care Management Process

January 28, 2011 by Dr. Ward in Continuous Quality Improvement (CQI), Cost-Effectiveness Analysis (CEA), Decision Analysis, Outcomes Measurement, Provider-based Care Management, Satisfaction Measurement | Leave a comment

Smoking cessation is an important public health concern, and has been the subject of a recent Agency for Health Care Policy and Research (AHCPR) guideline, as well as a HEDIS measure. A point prevalence study conducted with the Henry Ford Health System found a 27.4% prevalence of smoking, and an additional 38.6% former smokers.

The CCE developed a first-generation smoking-dependency clinic which was staffed by trained non-physician counselors and overseen by a physician medical director. The original intervention was a 50-minute initial evaluation and counseling visit, with nicotine replacement therapy prescribed for all patients with a high level of nicotine dependency. This intervention was subsequently updated to reflect the AHCPR recommendation that, unless contraindicated, all smoking cessation patients be prescribed nicotine replacement therapy.

Because relapse is a normal part of smoking cessation, the intervention was explicitly designed to address relapse. This was done through return visits, an optional support group, and follow-up telephone counseling calls throughout the year, as illustrated in the following figure.

The program was designed to be inexpensive and simple to execute within the clinic. This was accomplished by automating the logistics of both the intervention and the collection of outcomes measures. The Flexi-Scan System, an internally developed computer application which helps automate outcome studies and disease-management interventions was used to automate (1) data entry through a scanner, (2) prompting of follow-up calls and mailings, and (3) the generation of medical-record notes and letters to the referring physicians. A database that can be used for outcomes-data analyses is acquired as a part of this process.

As illustrated on the figure below, this first-generation program achieved a twelve-month quit rate of 25%. Such a quit rate is about twice as high as the rate achieved with brief counseling intervention.

To evaluate the cost-effectiveness of this program, a decision analytic model was constructed. This model was constructed using the Markov method. Key assumptions of the model include the following:

One year quit rate for usual care (optimistically assumed to consist of brief physician advice) was 12.5%.
Spontaneous quit rate of 1% per year in “out years.”
Relapse rate for recent quitters of 10%.
Age, Sex distribution based on Smoking Clinic patient demographics
Life expectancy of smokers and former smokers by age and sex based on literature (life tables).
Cost of clinic intervention – $199
Cost of nicotine therapy Smoking Clinic – $101 (Assuming 0.9 Rx/Patient)
Usual Care – $33 (Assuming 0.3 Rx/Patient)
Future health care costs were not considered
Annual discount rate of 5%

The results of this model were presented at the annual meeting of the Society for Medical Decision-Making. The model results are presented in the form a table called a “balance sheet” (a term coined by David Eddy, MD, PhD). As shown below, the model estimated that the first-generation smoking-dependency clinic cost about $1,600 for each life year gained.

To help evaluate whether this cost-effectiveness ratio is favorable, a league table was constructed (see below). The league table shows comparable cost-effectiveness ratios for other health care interventions. Interpretation of the table suggests that the smoking cessation intervention is highly favorable to these other health care interventions.

League Table

Intervention	Cost per Quality-adjusted Life Year Gained
Smoking Cessation Counselling	$6,400
Surgery for Left Main Coronary Artery Disease for a 55-year old man	$7,000
Flexible Sigmoidoscopy (every 3 years)	$25,000
Renal Dialysis (annual cost)	$37,000
Screening for HIV (at a prevalence of 5/1,000)	$39,000
Pap Smear (every year)	$40,000
Surgery for 3-vessel Coronary Artery Disease for a 55 year-old man	$95,000

Although this first generation program was effective and cost-effective, it was targeted only at the estimated 16,500 smokers in the HFMG patient population who were highly motivated to quit.

The estimated 66,000 other smokers in the HFMG patient population would be unlikely to pursue an intervention that involved visiting a smoking dependency clinic. Even for the smokers who were highly motivated to quit, the smoking cessation clinic had the capacity to provide counseling to about 500 people each year, or about 3% of these highly motivated smokers.

Second Generation Smoking Intervention Program

In response to this problem, the CCE developed a “second generation” Smoking Intervention Program.” This program uses a three tiered approach which includes (1) a “front-end” process for primary care and specialty clinics to use to identify smokers and provide brief motivational advice, (2) a centralized telephone-based triage process to conduct assessment and make arrangements for appropriate intervention, and (3) a stepped-care treatment tier.

In the “front-end” process, clinic physician and support staff were trained to screen their patients from smoking status and readiness to quit and provide tailored brief advise. Each participating clinic was provided with a program “kit” including screening forms, patient brochures, and posters to assist them in implementing the program. Patients who are interested in further intervention are referred to a centralized triage counselor for further assessment and intervention. These counselors are trained, non-physician care providers. They proactively call each patient referred, conduct an assessment of the patients smoking and quitting history and triage into a stepped-care intervention program.

An important part of this intervention has been providing information to clinicians, including a quarterly report showing the number of patients they have referred to the Smoking Intervention Program, the status of those patients, the type of intervention they are receiving, and the number of patients who report not having smoked in the preceding six months.

The clinician-specific data is presented in comparison to data for the medical group as a whole. These reports have a strong motivational effect on clinicians, as evidenced by a sharp increase in Smoking Intervention Program referrals after each reporting cycle.

As shown above, the second generation program achieved a six month quit rate of about 25%. This rate is virtually identical to the first generation program. The new program, however, has much larger capacity and lower cost per participant. Patient satisfaction with the Smoking Intervention Program is encouraging, with 85% reporting that they would refer a friend to this program.

Using Eddy’s Explicit Method to Develop Practice Guidelines for Mammography in the 40-49 Age Group

January 27, 2011 by Dr. Ward in Clinical Practice Guidelines and Protocols, Cost-Effectiveness Analysis (CEA), Decision Analysis | Leave a comment

Note that the following write-up is now almost 20 years old! (How did that happen?) Amazingly, the debate about the role and frequency of mammography in the 40-49 age group has raged on the entire time since then. This case study demonstrates the real-world use of the “explicit method” proposed by one of my “health care heroes”, David Eddy. Eddy’s approach involves using decision-analytic models and cost-effectiveness analysis to interpret evidence and incorporate our values to inform practice guideline development. This case shows that the explicit method is not just a “purist” methodology. It is a practical method of achieving agreement among physicians that started the process in angry disagreement.

In the last two decades, our field has largely retreated from difficult discussions about cost-effectiveness and the rational allocation of limited resources — the dreaded “R” word. In the recent debates about health care reform, those opposing reform talked of “death panels” and berated the U.K.’s NICE program which espouses some of the same principles as we used in this case. Such harsh talk has replaced the thoughtful, principled discussions we were having at the Henry Ford Health System and in the field in general back then. True health care reform will require that we go back and cross at this light.

— R. Ward, MD Jan 27, 2010

Background

As shown in the newspaper clipping below, the role of mammography in the 40-49 age group has long been controvertial.

Many organizations recommend mammography during the 40’s, including the American Cancer Society, the American College of Radiology, the American College of Obstetricians and Gynecologists, the American Medical Women’s Association, the National Alliance of Breast Cancer Organizations, and the National Breast Cancer Coalition. Other organizations argue that the benefits of mammography have not been proven, and therefore mammography should not be offered until age 50. These organizations include the American College of Physicians, the American College of Family Practice, the U.S. Preventive Services Task Force, the National Women’s Health Network, the National Center for Medical Consumers, the Darmouth Center for the Evaluative Clinical Sciences, the Canadian National Task Force on the Periodic Examination, and the United Kingdom Public Health Policy Board.

Within Henry Ford Medical Group, this same debate was also raging. In May, 1991, the HFMG Operations Committee approved a Consensus Guideline calling for bi-annual screening in 40-49 age group. Then, in October, 1993, the HFMG Consensus Guideline was updated based on recently published results of the Canadian National Breast Screening Study. The new consensus group formulated a recommendation which said “women age 40-49 not in a high risk group should carefully consider the risks and benefits before scheduling a mammogram.” This draft was approved by the Clinical Practice Committee, but was subsequently rejected by the Operations Committee. Strong letters of opposition were sent to HFMG leadership from the Department of Surgery and the Division of Breast Imaging. This led to a decision by Clinical Practice Committee to re-do analysis using explicit methods.

Objective

A multi-disciplinary team was commissioned by the HFMG Clinical Practice Committee to use an explicit methodology to conduct a clinical policy analysis and develop specific clinical policy recommendations regarding the role of screening mammography for average risk women age 40-49.

Methods

A multi-disciplinary team was assembled to conduct the analysis and formulate policy recommendations. This team included the Chairmen of the departments of Radiology, Obstetrics & Gynecology, and Surgery. It also included the Division Head of Breast Imaging, two general surgeons from the breast clinic, a staff oncologist, a Division Head in Internal Medicine, the Clinical Director of Family Practice, the Section Head of Epidemiology, the Associate Medical Director of the Health Alliance Plan (HMO). The team was led by the Director of the Center for Clinical Effectiveness.

Based on information from the medical literature, internal HFMG data, and expert opinion of team members, a mathematical model was developed and refined in order to gain a greater understanding of the implications and shortcomings of existing scientific evidence and to estimate the health and economic outcomes (with ranges of uncertainty) for three alternative plans: (1) do not recommend mammography until age 50, (2) recommend a program of bi-annual mammography during the 40-49 period, and (3) recommend a program of annual mammography.

Results

Results of Mammography Policy Analysis

As shown in the figure above, compared to not recommending mammograms, a program of 5 bi-annual mammograms for the 2,500 HFMG women entering their 40’s would add about $1.5 million to the net health care cost for the group (90% range of certainty: $918k – $2.1 million). This program could be expected to save between one and six lives, resulting in a gain of about 141 life years (43 – 244). This represents an expenditure of $440 thousand per life saved (undiscounted). With discounting of health and economic outcomes, this represents an incremental cost-effectiveness ratio of $34 thousand per life-year gained (15k – 120k). Earlier detection, in addition to saving lives, would permit the use of breast conserving procedures in about 2 more women, and would permit non-systemic treatment for 4 more women. In addition, such a program of bi-annual mammography could lead to added piece of mind for about 1,700 women receiving all negative screening results. On the down side, 59 more women would suffer the fear, inconvenience and risk associated with falsely positive mammogram leading to a negative biopsy, and an additional 650-950 women would suffer the unneeded worry associated with false positive mammogram results.

The full model results, presented in a table called a “balance sheet”, also shows the calculated health and economic outcomes associated with Plan C, offering annual mammography during the 40-49 age range.

Compared to bi-annual mammography, a program of 10 annual mammograms during the 40’s would cost an additional $900 thousand, saving an additional 0-1 lives for an estimated gain of 26 more life-years (1 – 58). This represents an expenditure of $1.4 million per life saved (undiscounted). With discounting, this represents an incremental cost-effectiveness ratio of $108 thousand per life-year gained (42k – 1.8 million).

Conclusions

On the basis if these estimates, the team recommended bi-annual screening mammograms in average risk women age 40-49. This guideline was intended to serve as a “best-practice”, “minimum practice”, and “maximum practice” guideline, as summarized in the following statement which was unanimously endorsed by the team: “Unless documented, patient-specific circumstances dictate otherwise, it is important to offer screening mammograms every two years during the 40-49 age period. More frequent mammograms are not routinely needed for average risk women during this age period.” This guideline statement was incorporated into the existing HFMG clinical preventive services guideline for breast cancer screening.

Postlude

Shortly after this guideline was approved, a meta-analysis of mammography trials was published in JAMA. The abstract stated “The results of our meta-analysis suggest that screening mammography reduced breast cancer mortality by 26% (95% CI 17-34%) in women aged 50-74 years, but does not significantly reduce breast cancer mortality in women aged 40-49 years.” (emphasis added). The body of the manuscript stated “there were only three clinical trials in which women aged 40-49 years underwent two-view mammography and had 10-12 years of follow-up; in those, the relative risk for reduction in breast cancer mortality . . . was 0.73 (95% CI, 0.54 to 1.0) after 10-12 years of follow-up.” Although the abstract suggested the opposite conclusion as the HFMG clinical policy analysis, apparently based on a confidence interval that touched zero, the point estimate of mammography effectiveness in the 40-49 age group was actually more favorable than the HFMG analysis (27% vs. 25% risk reduction). The fact that the assumptions and calculated outcomes were explicitly documented in the HFMG made this potentially consensus-breaking piece of new information and put it in its appropriate context. This example also illustrates the decision-making criterion implied by the clinical trial and meta-analysis literature: if an intervention has a positive effect which is 95% or more likely to be greater than zero, then it is implicitly recommended, regardless of the magnitude of the outcome in relation or the cost. The explicit method permits policy-makers in health care organizations to use a more sophisticated and philosophically defensible criteria based on an assessment of benefits, costs, and the uncertainties associated with each.