Methods Manual – Part 2: Economic Review Process
Section Overview
Purpose
An economic review, based on published literature, is conducted for interventions that are recommended on the basis of effectiveness by CPSTF. A nine-step economic review process, similar to the ten-step effectiveness review process, assesses the economic impact of an intervention approach recommended by CPSTF.
Economic reviews determine:
- Cost to implement the intervention;
- Economic benefits due to the intervention; and
- Summary of economic estimates that compares cost-to-benefits or cost-to-health outcomes.
Figure 5. Nine Steps in the Community Guide Economic Review ProcessE
Step 1: Prioritize Economic Reviews
All intervention approaches that CPSTF recommends with strong or sufficient evidence of effectiveness become candidates for economic reviews. With limited capacity, it is necessary to prioritize which intervention approaches are reviewed.
To help CPSTF decide which reviews to select for an economic evaluation, CGP staff provide information on the number of published economic evaluations that provide information on cost-effectiveness or cost-benefit, based on an informal desktop search (e.g. Google Scholar or PubMed search).
Step 2: Create Coordination Team
The coordination team (the team) assembled for the effectiveness review generally participates in the economic review. An economist directs the team with guidance from the senior scientist leading the effectiveness review.
The team’s tasks for the economic review include developing and approving the following:
- Economic analytic framework
- Economic research questions
- Search strategy and inclusion/exclusion criteria
- Components of an intervention, including drivers which are important contributors to intervention cost, intervention benefit, and summary economic outcomes of cost-benefit and cost-effectiveness
- Quality assessment method
- Draft CPSTF economic finding statements
Each of these are described in the following steps.
Step 3: Conceptualize Economic Review
Several products are used for both the effectiveness and the economic review.
- Intervention definition, which defines the activities or actions that take place in the intervention.
- Analytic framework, which specifies the conceptual approach to evaluate the primary health outcomes and other non-health related outcomes of the intervention.
- Effectiveness search strategy, which describes the formal search method and sources searched.
The team develops an economic analytic framework, based on the analytic framework in the effectiveness review, but adds the economic components to visually represent the long- and short-term intervention cost and benefits. (See example economic analytic framework).
The team also develops economic research questions to elicit estimates for the following standard economic outcomes:
- Intervention cost, which is the cost of resources necessary to implement the intervention.
- Intervention economic benefits due to improved health, such as
- Averted healthcare cost,
- Improved worksite productivity, and
- Increased number and quality of years lived.
- Other intervention economic benefits due to effects that may not be related to health.
- Comparison of intervention cost to intervention economic benefit.
- Assessment of cost per quality-adjusted life year and disability-adjusted life year for cost-effectiveness.
The team defines inclusion and exclusion criteria to select papers for the economic review. In addition to meeting effectiveness review criteria, each study for the economic review must also provide estimates for at least one economic outcome described in the research questions.
Step 4: Search
CGP staff work with the CDC Library staff on literature searches to identify potential studies to include in the evidence. The team may also include potential studies identified during the effectiveness review, suggested by subject matter experts, and from references cited within studies.
The economic search strategy, which builds on the search strategy used for the effectiveness review, may include the following.
- Database sources with an economic focus, as well as the EconLit, Cochrane Economic Evaluations, and Campbell Collaboration databases
- Search terms relating to economics, such as cost, economic benefit, cost-benefit, and cost-effectiveness, and the effectiveness search terms
- Sources beyond the peer-reviewed literature, such as government and non-governmental organization reports, depending on the type of intervention
Step 5: Screen Literature
The search yield is screened in multiple stages, as described in Part I.
Step 6: Abstract
Once the team narrows the search yield through the screening process, the team abstracts relevant data into a detailed evidence table for each study as described in Part 1 above. (See example economic detailed evidence table.)
Quality Assessment
The team assesses the quality of evidence collected from the included studies. Each economic estimate is assigned a quality of good, fair, or limited based on how well it captures the components known to be drivers of the estimate and the measurement methods used in estimation. A good quality estimate includes all or most of the components known to be drivers of its magnitude and has none or few limitations points assigned for shortfalls in statistical and measurement methods. A fair quality estimate captures some of the known drivers or have some limitations in statistical and measurement methods. A limited quality estimate fails to include all or most components known to be drivers of its magnitude or also has numerous limitation points assigned for shortfalls in statistical and measurement methods.9 The components considered to be drivers and the areas in which the measurement aspects are assessed will be different for each economic review. Limited quality estimates are excluded from the body of evidence. (see example from a completed economic review)
Key Elements of Quality Assessments
- Capture of drivers in estimates of cost and benefit
- Appropriateness of measurement in the areas of perspective, population in consideration, time horizon, valuation method, measurement of intervention effect, estimation of modeled outcomes, and any intervention-specific areas not covered in previous areas
- Appropriate perspective, including the societal perspective (preferred for the Community Guide economic reviews) or a more restricted perspective from healthcare systems, employers, or government
- Analytic period of the intervention and its effects, usually a lifetime to determine effects on morbidity and mortality as measured by quality and disability-adjusted years lived
- Components and drivers of cost and benefit
Step 7: Analyze
Data Standardization
The team standardizes estimates to enhance comparability across studies that differ in sample size, intervention duration, year of implementation, and currency in which monetary values are expressed. Estimates are expressed in per person per year terms and monetary values are converted to U.S. dollars for a common year using purchasing power parities from the World Bank and the consumer price index from the Bureau of Labor Statistics.
Analysis Methods
The team summarizes and organizes the review’s results as economic evidence that address each of the research questions about intervention cost, intervention benefit, cost-benefit, and cost-effectiveness. Gaps in the economic evidence are also identified.
The evidence collected from multiple studies for each of the economic outcomes of interest are summarized and presented as medians and interquartile intervals (IQI). In cases where there are three or fewer estimates, all estimates are presented and not summarized.
The economic review conclusion, whether the intervention is cost-beneficial or cost-effective, is based on all the estimates for cost-benefit or cost-effectiveness, respectively, considered together.
- A cost-benefit estimate is favorable (cost-beneficial) for the intervention when economic benefits exceed the cost of intervention.
- A cost-effectiveness estimate is favorable when the cost per quality-adjusted life year (QALY) saved is less than a conservative threshold of $50,000 OR when the cost per disability-adjusted life year (DALY) averted is less than the conservative benchmark of per capita gross domestic product of the country of implementation.
Step 8: Make CPSTF Economic Finding
CGP staff present results of systematic reviews, on behalf of the coordination team, to CPSTF for their deliberation and decisions about economic findings. CPSTF members review and discuss the evidence, consider input from partners, and issue an economic finding statement.
Primary consideration for reaching a CPSTF economic finding is determined by whether the benefits exceed the cost or the cost-effectiveness estimate is below a threshold.
CPSTF Economic Decision Tables
As part of the presentation to CPSTF, the coordination team presents CPSTF Economic Decision Table for Cost-Benefit Findings and CPSTF Economic Decision Table for Cost-Effectiveness Findings, as appropriate.
The economic decision tables display evidence of an intervention’s cost-benefit or cost-effectiveness findings based on the quality of estimates, consistency of the results, and criterion for determination.
When estimates are inconsistent in indicating whether benefits exceed cost or if the intervention is cost-effective, the interquartile interval for the estimates is consulted. CPSTF reaches a finding of cost-effectiveness if all values in the interquartile interval are less than the threshold or reaches a finding that benefits exceed cost if all net benefit values in the interquartile interval are greater than zero. Otherwise, there is no CPSTF economic finding. A CPSTF economic finding statement appears in the finding section when the results for cost-benefit or cost-effectiveness are favorable. Results from an economic review appear in the rationale section of the TFFRS.
Evidence Gaps
As in the effectiveness review, the economic review may reveal gaps in the research that fail to adequately address one or more of the research questions. Evidence gaps may also be drawn from elements in the economic analytic framework that are not estimated or addressed in the current research.
Step 9: Disseminate Conclusions
The updated TFFRS from the effectiveness review, now including the economic findings, is disseminated through the same channels used for the effectiveness review. Associated dissemination materials, such as email and social media, are also updated and redistributed.
Printer Friendly Version
Printer friendly version of this portion of the methods manual [PDF – 217 KB]