Abstract

OBJECTIVES: A growing body of evidence indicates that the delivery of public health activities varies widely across states and communities, creating missed opportunities for prevention as well as inequities in health protection. Consequently, HHS recently used a consensus-based process to develop a framework for improving quality in public health and to identify a set of nine priority areas for improvement. As next steps in operationalizing this framework, measures of quality in public health are needed to guide public health improvement initiatives and to support research on the comparative effectiveness of alternative public health strategies. This presentation profiles the methodology developed for the Multi-network Practices and Outcomes Variation Examination Study (MPROVE), which uses the infrastructure of six Public Health Practice-Based Research Networks (PBRNs) across the U.S. to develop and validate a “starter set” of public health quality measures and to analyze geographic variation in these measures across diverse public health settings.

STUDY DESIGN: The research design includes five major components: (1) a review of existing data sources available within the six PBRNs to identify common measurement capabilities; (2) an expert panel process modeled after the RAND/UCLA Appropriateness Method to specify candidate quality measures based on the HHS Framework using the best available scientific evidence combined with the collective judgment of experts; (3) a novel value-of-information (VOI) analysis used to prioritize candidate quality measures based on the expected health impact (EHI) of reducing unwarranted variation in the measures; and (4) validity and reliability testing of priority measures by collecting data from a sample of local public health agencies that participate in the six public health PBRNs.

POPULATION: The study population includes approximately 210 local public health settings in six states that participate in one of six public health PBRNs.

PRINCIPAL FINDINGS: Data resources within the six participating PBRNs indicate that the strongest capabilities for quality measurement exist in the two HHS domains of timeliness and adherence to evidence-based practices, and in the service domains of environmental health and communicable disease control. Preliminary estimates from VOI analysis suggest that quality measures in the HHS domains of reach, adherence, and equity offer the largest EHI values, particularly in the service domains of tobacco control, nutrition and physical activity promotion, and maternal and child health. Appropriateness Method results and findings from validity and reliability testing suggest additional strategies for specifying public health quality measures that balance rigor, feasibility, and relevance to practice.

CONCLUSIONS: A mixed-method approach to measurement development that includes expert judgment and VOI analysis allows explicit trade-offs to be made among feasibility, practice relevance, validity and reliability, and expected health impact.

IMPLICATIONS FOR POLICY AND PRACTICE: Carefully developed quality measures can enable research on the comparative effectiveness of public health services and systems and ultimately drive health-improving advances in practice.

Document Type

Presentation

Publication Date

6-27-2012

Notes/Citation Information

A presentation at the AcademyHealth Public Health Systems Research Interest Group Meeting in Orlando, FL.

Download

Share

COinS