Form #1 Form #1 CHIPRA PQMP Candidate Measure Submission Form

In 2009, the Children’s Health Insurance Program Reauthorization Act (CHIPRA) reauthorized the Children’s Health Insurance Program (CHIP) originally established in 1997.¹ Title IV of the law included a number of provisions aimed at improving health care quality and outcomes for children. Section 401(a) of CHIPRA called for the identification of an initial core set of health quality measures for children enrolled in Medicaid or CHIP based on measures available in 2009. The initial core set² was recommended by the Agency for Healthcare Research and Quality (AHRQ) National Advisory Subcommittee on Children’s Health Quality Measures for Medicaid and CHIP (SNAC), posted for public comment by the Secretary of the U.S. Department of Health and Human Services (HHS) on December 29, 2009, and made available for voluntary use by State Medicaid and CHIP programs in February 2011,³ along with technical specifications.⁴

Section 401 (b) of CHIPRA created the Pediatric Quality Measures Program (PQMP) to improve the initial core set of pediatric quality measures and increase the portfolio of evidence-based measures available to public and private purchasers of children’s health care services, providers, and consumers. Improved core measures are to be posted annually beginning January 1, 2013. The PQMP is a partnership between AHRQ and the Centers for Medicare & Medicaid Services (CMS). As part of the PQMP, there are seven Centers of Excellence (COEs)—a consortium of academic institutions, State partners, consumers, and others—that will develop and test measures over the course of the program for categories specified by CHIPRA and topics identified by CMS and AHRQ.⁵ In addition to the measures submitted by the COEs, public nominations for quality measures will be solicited in the spring of each year. All submitted measures will be reviewed by a SNAC⁶ of the AHRQ National Advisory Council on Research and Quality (NAC). The SNAC will make recommendations to the NAC, which advises the director of AHRQ, who in turn will make recommendations to CMS and the Secretary of HHS.

CHIPRA notes that measures in the improved core sets should be evidence based; cover a full range of services, conditions, and ages; be able to identify disparities by race, ethnicity, socioeconomic status, and special health care need; be risk adjusted as appropriate; and designed to ensure that data are collected and reported in a standard format that permits comparison of quality and data at a State, plan, and provider level.

This template, the CHIPRA Pediatric Quality Measures Program (PQMP) Candidate Measure Submission Form (CPCF) was developed by the COEs, the SNAC, and AHRQ as a standardized form to be used for all nominations for pediatric quality measures under the CHIPRA legislation. The first part of the CPCF template provides guidance on the submission process. The template then includes opportunities for all measure submitters to provide a basic description of their measure, and address a number of desirable measure attributes for pediatric quality measures. The desirable measure attributes include importance, evidence or other rationale for focus of the measure, scientific soundness of the measure itself, identification of disparities, feasibility, levels of aggregation, understandability, and health information technology. The form also requests identification of the limitations of the measure being submitted. It then provides an opportunity to summarize why the measure should be recommended by the SNAC, taking into account the measure’s advantages and limitations in relation to the desirable measure attributes. The template requires measure submitter information, public disclosure requirement requiring signed written statement, and an opportunity to upload supplementary material including graphics, figures, tables, and any other information to facilitate review of the measure by the SNAC. Attachments may be in PDF, Microsoft Word, or text format. The final section of the template provides a glossary of terms. Many of the desirable attributes are similar to those called by other leading entities that solicit measures, but several are CHIPRA specific (e.g., more child focused, spotlight on disparities, and attention to specific levels of aggregation). The SNAC will interpret the extent to which the measure is suitable for voluntary use by Medicaid, CHIP, or other public and private programs, purchasers, plans, providers and consumers using the information provided in the template.

NOTE: If a section is not applicable to the measure, please write ‘Not applicable’ in the text field before progressing to the next section. If the information is not available, please write “Not available” in the text field before progressing to the next section.

<< >> indicates the name of a text field in the online version of CPCF.

+ indicates a field to upload attachment in the online version of CPCF.

Section I .
Basic Measure Information

I.A. Measure Name

«Measure_Name»

I.B. Measure Number (auto-generated)

«Measure_Number»

I.C. Measure Description

Please provide a non-technical description of the measure that conveys to a broad audience what it measures.

«Measure Description»

I.D. Measure Owner

«Measure_Owner»

I.E. National Quality Forum (NQF) ID (if applicable)

«NQF_ID»

I.F. Measure Hierarchy

Please use this section to note if the measure is part of a measure hierarchy_{or is part of a measure group or composite measure}. The following definitions are used by AHRQ’s National Quality Measures Clearinghouse and are available at http://www.qualitymeasures.ahrq.gov/about/hierarchy.aspx:

I.F.1. Please identify the name of the collection of measures to which the measure belongs (if applicable). A Collection is the highest possible level of the measure hierarchy. A Collection may contain one or more Sets, Subsets, Composites, and/or Individual Measures.

«Measure_Collection_ Name»

I.F.2. Please identify the name of the measure set to which the measure belongs (if applicable). A Set is the second level of the hierarchy. A Set may include one or more Subsets, Composites, and/or Individual Measures.

«Measure_Set_ Name»

I.F.3. Please identify the name of the subset to which the measure belongs (if applicable). A Subset is the third level of the hierarchy. A Subset may include one or more Composites and/or Individual Measures.

«Measure_Subset_ Name»

I.F.4. Please identify the name of the composite measure to which the measure belongs (if applicable). A Composite is a measure with a score that is an aggregate of scores from other measures. A Composite may include one or more other Composites and/or Individual Measures. Composites may comprise component measures that can or cannot be used on their own.

«Measure_Composite_ Name»

I.G. Numerator Statement

«Numerator»

I.H. Numerator Exclusions (as appropriate)

«Numerator Exclusions»

I.I. Denominator Statement

«Denominator»

I.J. Denominator Exclusions (as appropriate)

«Denominator Exclusions»

I.K Data Sources

Check all the data sources for which the measure is specified and tested.

Data Source	[Online form will have radio buttons here]
Administrative Data (e.g., claims data)
Paper Medical Record
Survey – Health care professional report
Survey – Parent/caregiver report
Survey – Child report
Electronic Medical Record
Other (If other, please list all other data sources in the field below.)

«Other_Data_Sources»

Section II.
Detailed Measure Specifications

Provide sufficient detail to describe how a measure would be calculated from the recommended data sources, uploading a separate document (+ Upload attachment) or a link to a URL. Examples of detailed measure specifications can be found in the CHIPRA Initial Core Set Technical Specifications Manual 2011 published by the Centers for Medicare & Medicaid Services.⁷ Although submission of formal programming code or algorithms that demonstrate how a measure would be calculated from a query of an appropriate electronic data source are not requested at this time, the availability of these resources may be a factor in determining whether a measure can be recommended for use.

«Measure_Specifications»

+ Opportunity to upload attachment.

Section III.
Importance of the Measure

In the following sections, provide brief descriptions of how the measure meets one or more of the following criteria for measure importance (general importance, importance to Medicaid and/or CHIP, complements or enhances an existing measure). Include references related to specific points made in your narrative (not a free-form listing of citations).

III.A. Evidence for General Importance of the Measure

Provide evidence for all applicable aspects of general importance, including but not limited to the following:

Addresses a known or suspected quality gap or disparity in quality (e.g., addresses a socioeconomic disparity, a racial/ethnic disparity, a disparity for Children with Special Health Care Needs (CSHCN), and/or a disparity for limited English proficiency (LEP) populations.
Potential for quality improvement (i.e., there are effective approaches to reducing the quality gap or disparity in quality).
Prevalence of condition among children under age 21 and/or among pregnant women.
Severity of condition and burden of condition on children, family, and society (unrelated to cost).
Fiscal burden of measure focus (e.g., clinical condition) on patients, families, public and private payers, or society more generally, currently and over the life span of the child.
Association of measure topic with children’s future health—for example, a measure addressing childhood obesity may have implications for the subsequent development of cardiovascular diseases.
The extent to which the measure is applicable to changes across developmental stages (e.g., infancy, early childhood, middle childhood, adolescence, young adulthood).

«Evidence_of_General_Importance»

III.B. Evidence for Importance of the Measure to Medicaid and/or CHIP

Comment on any specific features of this measure important to Medicaid and/or CHIP that are in addition to the evidence of importance described above, including the following:

The extent to which the measure is understood to be sensitive to changes in Medicaid or CHIP (e.g., policy changes, quality improvement strategies).
Relevance to the Early and Periodic Screening, Diagnostic and Treatment benefit in Medicaid (EPSDT).⁸
Any other specific relevance to Medicaid/CHIP (please specify).

«Evidence_of_Importance_to_Medicaid_CHIP»

III.C. Relationship to Other Measures (if any)

Describe, if known, how this measure complements or improves on an existing measure in this topic area for the child or adult population or if it is intended to fill a specific gap in an existing measure category or topic. For example, the proposed measure may enhance an existing measure in the initial core set, it may lower the age range for an existing adult-focused measure, or it may fill a gap in measurement (e.g., for asthma care quality, inpatient care measures).

«Relationship_to_Other_Measures»

Section IV.
Measure Categories

CHIPRA legislation⁹requires that measures in the initial and improved core set, taken together, cover all settings, services, and topics of health care relevant to children. Moreover, the legislation requires the core set to address the needs of children across all ages,¹⁰ including services to promote healthy birth. Regardless of the eventual use of the measure, we are interested in knowing all settings, services, measure topics, and populations that this measure addresses. These categories are not exclusive of one another, so please indicate "Yes" to all that apply.

	Does the measure address this category [Yes/No drop-down]
Care Setting—ambulatory
Care Setting—inpatient
Care Setting—other—please specify		[Add the following choices: home, school, other community and public health settings, long-term care, other---drop-down or radio buttons]
Service—preventive health, including services to promote healthy birth
Service—care for acute conditions
Service—care for children with special health care needs/chronic conditions
Service—other (please specify)
Measure Topic—duration of enrollment
Measure Topic—clinical quality
Measure Topic—patient safety
Measure Topic —family experience with care
Measure Topic—care in the most integrated setting
Measure Topic—other (please specify)		«Other_Topic»
Population—pregnant women		«Age_Range»
Population—neonates (0-28 days) (specify age range)		«Age_Range»
Population—infants (29 days to 364 days) (specify age range)		«Age_Range»
Population—pre-school-age children (1 year through 5 years) (specify age range)		«Age_Range»
Population—school-age children (6 years through 10 years) (specify age range)		«Age_Range»
Population—adolescents (11 years through 20 years) (specify age range)		«Age_Range»
Population—other (please specify)		«Age_Range»
Other category (please specify)		«Other_Category»

Section V.
Evidence or Other Justification for the Focus of the Measure

The evidence base for the focus of the measures will be made explicit and transparent as part of the public release of CHIPRA deliberations; thus, it is critical for submitters to specify the scientific evidence or other basis for the focus of the measure in the following sections.

V.A. Research Evidence

Research evidence should include a brief description of the evidence base for valid relationship(s) among the structure, process, and/or outcome of health care that is the focus of the measure. For example, evidence exists for the relationship between immunizing a child or adolescent (process of care) and improved outcomes for the child and the public. If sufficient evidence existed for the use of immunization registries in practice or at the State level and the provision of immunizations to children and adolescents, such evidence would support the focus of a measure on immunization registries (a structural measure).

Describe the nature of the evidence, including study design, and provide relevant citations for statements made. Evidence may include rigorous systematic reviews of research literature and high-quality research studies.

«Research_Evidence»

V.B. Clinical or Other Rationale Supporting the Focus of the Measure (optional)

Provide documentation of the clinical or other rationale for the focus of this measure, including citations as appropriate and available.

«Clinical_Or_Other_Rationale»

Section VI.
Scientific Soundness of the Measure

Explain the methods used to determine the scientific soundness of the measure itself. Include results of all tests of validity and reliability, including description(s) of the study sample(s) and methods used to arrive at the results. Note how characteristics of other data systems, data sources, or eligible populations may affect reliability and validity.

VI.A. Reliability

Reliability of the measure is the extent to which the measure results are reproducible when conditions remain the same. The method for establishing the reliability of a measure will depend on the type of measure, data source, and other factors. Explain your rationale for selecting the methods you have chosen, show how you used the methods chosen, and provide information on the results (e.g., the Kappa statistic). Provide appropriate citations to justify methods.

«Reliability»

VI.B. Validity

Validity of the measure is the extent to which the measure meaningfully represents the concept being evaluated. The method for establishing the validity of a measure will depend on the type of measure, data source, and other factors. Explain your rationale for selecting the methods you have chosen, show how you used the methods chosen, and provide information on the results (e.g., R² for concurrent validity). Provide appropriate citations to justify methods.

«Validity»

Section VII.
Identification of Disparities

CHIPRA requires that quality measures be able to identify disparities by race, ethnicity, socioeconomic status, and special health care needs. Thus, we strongly encourage nominators to have tested measures in diverse populations. Such testing provides evidence for assessing measure’s performance for disparities identification. In the sections below, describe the results of efforts to demonstrate the capacity of this measure to produce results that can be stratified by the characteristics noted and retain the scientific soundness (reliability and validity) within and across the relevant subgroups.

VII.A. Race/Ethnicity

«Race_Ethnicity_Diversity_In_Measure_Testing»

VII.B. Special Health Care Needs

«Special_Health_Care_Need_Diversity_In_Measure_Testing»

VII.C. Socioeconomic Status

«SES_Diversity_In_Measure_Testing»

VII.D. Rurality/Urbanicity

«Rurality/Urbanicity_In_Measure_Testing»

VII.E. Limited English Proficiency (LEP) Populations

«LEP_In_Measure_Testing»

Section VIII.
Feasibility

Feasibility is the extent to which the data required for the measure are readily available, retrievable without undue burden, and can be implemented for performance measurement.¹¹Using the following sections, explain the methods used to determine the feasibility of implementing the measure.

VIII.A. Data Availability

VIII.A.1. What is the availability of data in existing data systems? How readily are the data available?

«Data_Availability»

VIII.A.2. If data are not available in existing data systems or would be better collected from future data systems, what is the potential for modifying current data systems or creating new data systems to enhance the feasibility of the measure and facilitate implementation?

«Implementation_Opportunities»

VIII.B. Lessons from Use of the Measure

VIII.B.1. Describe the extent to which the measure has been used or is in use, including the types of settings in which it has been used, and purposes for which it has been used.

«Extent of Use_of_Measure»

VIII.B.2. If the measure has been used or is in use, what methods, if any, have already been used to collect data for this measure?

«Data_collection_methods_used»

VIII.B.3. What lessons are available from the current or prior use of the measure?

«Lessons_Learned»

Section IX.
Levels of Aggregation

CHIPRA states that data used in quality measures must be collected and reported in a standard format that permits comparison (at minimum) at State, health plan, and provider levels. Use the following table to provide information about this measure’s use for reporting at the levels of aggregation in the table.

For the purpose of this section, please refer to the definitions for provider, practice site, medical group, and network in Section XVI. Glossary of Terms. .

If there is no information about whether the measure could be meaningfully reported at a specific level of aggregation, please write “Not available” in the text field before progressing to the next section.

+Opportunity to Upload attachment.

Level of aggregation (Unit) for reporting on the quality of care for children covered by Medicaid/CHIP^†	Intended use: Is measure intended to support meaningful comparisons at this level? (Yes/No)	Data Sources: Are data sources available to support reporting at this level?	Sample Size: What is the typical sample size available for each unit at this level? What proportion of units at this level of aggregation can achieve an acceptable minimum sample size?	In Use: Have measure results been reported at this level previously?	Reliability & Validity: Is there published evidence about the reliability and validity of the measure when reported at this level of aggregation?	Unintended consequences: What are the potential unintended consequences of reporting at this level of aggregation?
State level*: Can compare States	Yes No
Other geographic level: Can compare other geographic regions (e.g., MSA, HRR)	Yes No
Medicaid or CHIP Payment model: Can compare payment models (e.g., managed care, primary care case management, FFS, and other models)	Yes No
Health plan*: Can compare quality of care among health plans.	Yes No
Provider-level*
Individual practitioner: Can compare individual health care professionals	Yes No
Hospital: Can compare hospitals	Yes No
Practice, group, or facility:** Can compare: (i) practice sites; (ii) medical or other professional groups; or (iii) integrated or other delivery networks	Yes No

^† There could be other levels of reporting that could be of interest to Medicaid agencies such as markets and referral regions.

* Required in CHIPRA legislation.

** There is no implication that measures that are applicable at one level are automatically applicable at all three of the levels listed in this row.

Section X.
Understandability

CHIPRA states that the core set should allow purchasers, families, and health care providers to understand the quality of care for children. Please describe the usefulness of this measure toward achieving this goal. Describe efforts to assess the understandability of this measure (e.g., focus group testing with stakeholders).

«Understandability»

Section XI.
Health Information Technology

Please respond to the following questions in terms of any health information technology (health IT) that has been or could be incorporated into the calculation of the measure.

XI.A. Health IT Enhancement

Please describe how health IT may enhance the use of this measure.

«Health_IT_Enhancement»

XI.B. Health IT Testing

Has the measure been tested as part of an electronic health record (EHR) or other health IT system?

[Yes/No drop-down]

If so, in what health IT system was it tested and what were the results of testing?

«Health_IT_Testing_Results»

XI.C. Health IT Workflow

Please describe how the information needed to calculate the measure may be captured as part of routine clinical or administrative workflow.

«Health_IT_Workflow»

XI.D. Health IT Standards

Are the data elements in this measure supported explicitly by the Office of the National Coordinator for Health IT Standards and Certification criteria (see: http://healthit.hhs.gov/portal/server.pt/community/healthit_hhs_gov__standards_ifr/1195)?

[Yes/No drop-down]

If yes, please describe.

«Health_IT_Standards»

XI.E. Health IT Calculation

Please assess the likelihood that missing or ambiguous information will lead to calculation errors.

«Health_IT_Potential_Calculation_Errors»

XI.F. Health IT Other Functions

If the measure is implemented in an EHR or other health IT system, how might implementation of other health IT functions (e.g., computerized decision support systems in an EHR) enhance performance on the measure?

«Health_IT_ Other_Functions»

Section XII.
Limitations of the Measure

Describe any limitations of the measure related to the attributes included in this CPCF (i.e., availability of measure specifications, importance of the measure, evidence for the focus of the measure, scientific soundness of the measure, identification of disparities, feasibility, levels of aggregation, understandability, health information technology).

«Limitations_of_Measure»

Section XIII.
Summary Statement

Provide a summary rationale for why the measure should be selected for use, taking into account a balance among desirable attributes and limitations of the measure. Highlight specific advantages that this measure has over alternative measures on the same topic that were considered by the measure developer or specific advantages that this measure has over existing measures. If there is any information about this measure that is important for the review process but has not been addressed above, include it here.

«Summary_Statement»

Section XIV.
Identifying Information for the Measure Submitter

Complete information about the person submitting the material, including the following:

«Name»,
«Title»,
«Organization»,
«Mailing Address»,
«Telephone Number»,
«email address», and
Signed written statement guaranteeing that all aspects of the measure will be publicly available, as defined in the Public Disclosure Requirements.

+Opportunity to Upload attachment.

Public Disclosure Requirements

Each submission must include a written statement agreeing that, should U.S. Department of Health and Human Services accept the measure for the 2014 and/or 2015 Improved Core Measure Sets, full measure specifications for the accepted measure will be subject to public disclosure (e.g., on the Agency for Healthcare Research and Quality [AHRQ] and/or Centers for Medicare & Medicaid Services [CMS] websites), except that potential measure users will not be permitted to use the measure for commercial use. In addition, AHRQ expects that measures and full measure specifications will be made reasonably available to all interested parties. “Full measure specifications” is defined as all information that any potential measure implementer will need to use and analyze the measure, including use and analysis within an electronic health record or other health information technology. As used herein, “commercial use” refers to any sale, license or distribution of a measure for commercial gain, or incorporation of a measure into any product or service that is sold, licensed or distributed for commercial gain, even if there is no actual charge for inclusion of the measure. This statement must be signed by an individual authorized to act for any holder of copyright on each submitted measure or instrument. The authority of the signatory to provide such authorization should be described in the letter (Section XIV: Identifying Information for the Measure Submitter).

Section XV.
Supplementary Material

+ Opportunity to upload attachment with supplementary material including graphics, figures, tables, and any other information to facilitate review of this measure by the SNAC. Attachments may be in PDF, Microsoft Word, or text format.

Section XVI.
Glossary of Terms

	TERM	DEFINITION	SOURCES
	DENOMINATOR	The number or population representing the total universe in which an event might happen: the number at risk used to calculate a rate, proportion, or percentage.	Cohn, 2001
	Medical Group	A medical group is a self-defined “parent” provider organization which may exist within a broader network structure and is generally comprised of multiple practice sites, but can represent a single, large multi-specialty practice site. They often have integrated administrative systems and procedures. Some represent hospital affiliated provider organizations.	PQMP Result Aggregation Workgroup, 2012
	Network	A network is an overarching affiliation of medical groups and/or practice sites with an integrated approach to quality improvement that health plans regard as a contracting entity for these provider organizations. Most represent a collection of ambulatory practice sites whose integrated systems and procedures support clinical and administrative functions (e.g. scheduling, treating patients, ordering services, prescribing, keeping medical records and follow-up). Some embody a collection of hospital affiliated providers.	PQMP Result Aggregation Workgroup, 2012
	NUMERATOR	A subset of those in the denominator who have experienced the event of interest (e.g., death, morbidity, screening) used to calculate a rate, proportion, or percentage.	RTI
	OUTCOME	A particular state of health, often defined for purposes of quality measurement as a result of the performance (or nonperformance) of functions or processes of care.	Adapted from CMS
	OUTCOME MEASURE	Measure that indicates the results of the performance (or nonperformance) of functions or processes. A measure that focuses on achieving a particular state of health.	CMS
	PROCESS MEASURE	Measure that focuses on a health care process that leads to a certain outcome. For a process measure to be valid, a scientific basis exists for believing that the process, when executed well, will increase the probability of achieving a desired outcome.	Adapted from CMS
	PRACTICE SITE	A practice site is one or a group of providers who practice together at a single location (i.e. same mailing address down to the Suite # level). The single location is the site where care is provided during specific periods of time. The same systems and procedures support clinical and administrative functions (e.g. scheduling, treating patients, ordering services, prescribing, keeping medical records and follow-up). Medical records for all patients treated at the practice site are available to and shared by all providers, as appropriate.	Adapted from National Committee on Quality Assurance’s practice site methodology
	PROCESS (of care)	Process of care denotes what is actually done to the patient in the giving and receiving of care. As examples: the provider could immunize the patient against a communicable disease; the provider could prescribe a medication for the patient; the provider could screen an asymptomatic patient for developmental disorders.	Adapted from IOM, 2006, Appendix E
	PROVIDER	Provider is any individual, organization, facility or group that delivers direct health care to children; depending on the measurement context, this may be a hospital, medical group, or individual clinician.	PQMP Result Aggregation Workgroup, 2012
	QUALITY (in health care)	Health care quality has been defined in several ways. In 1990, the Institute of Medicine (IOM) defined quality as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge (IOM, 1990). Eisenberg defined quality as the right care for the right person at the right time in the right way. In 2001, the IOM defined quality as having six aims: Safety, Timeliness, Effectiveness, Equity, Efficiency, and Patient-Centeredness. The Affordable Care Act defines quality of care as a measure of performance on IOM’s six aims for health care. CHIPRA defines a clinical quality measure as “a measurement of clinical care that is capable of being examined through the collection and analysis of relevant information, that is developed in order to assess one or more aspects of pediatric health care quality in various institutional and ambulatory health care settings, including the structure of the clinical care system, the process of care, the outcome of care, or patient experiences in care.”	IOM, 2001; IOM, 1990; Eisenberg, 1997; CHIPRA, 2009; Patient Protection and Affordable Care Act, 2010
	QUALITY MEASURE	A quality measure is in effect a rule (or the result of a rule) that assigns numeric values to a specific quality indicator. Quality measures generally consist of a descriptive statement or indicator, a list of data elements necessary to construct and/or report the measure, detailed specifications that direct how the data elements are to be collected (including the source of data), the population on whom the measure is constructed, the timing of data collection and reporting, the analytic models used to construct the measure, and the format in which the results will be presented.	Adapted from IOM, 2006, Appendix E; NQMC Glossary
	RELIABILITY	Measure reliability: The results of the measure are reproducible a high proportion of the time when assessed in the same population (e.g., the measure has high inter-rater reliability, no calculation errors). Internal consistency reliability assesses the consistency of results across items within a test, where “test” refers to a series of questions, ratings, or other items designed to determine knowledge, ability, or health status. Inter-rater reliability is a measure of the variation in measurements when taken by different individuals but with the same method or instruments. Test-retest is a statistical method used to determine a test's reliability. The test is performed twice; in the case of a questionnaire, this would mean giving a group of participants the same questionnaire on two different occasions. If the correlation between separate administrations of the test is high (~.7 or higher), then it has good test-retest reliability. It is important to consider the time interval between testing and retesting and the nature of the measurement. Quality measures optimally would show improvement in scores over time.	CMS; Wikipedia based on The Standards for Educational and Psychological Testing, 1999***; The Free Dictionary by Farlex
	STRUCTURE	Structure refers traditionally to the attributes of settings in which providers deliver health care, including material resources (e.g., electronic health records), human resources (e.g., staff expertise), and organizational structure (adapted from IOM, Performance Measurement, 2006; Appendix E). Some have suggested that structural attributes should include organizational characteristics such as leadership and culture (Kunkel, 2007) and system attributes beyond individual health care delivery settings.	Adapted from IOM, 2006, Appendix E
	STRUCTURAL MEASURE	Measures of structure assess the capacity of health care professionals and organizations to provide safe, timely, effective, equitable, efficient and patient-centered processes of care and positive health outcomes.	Adapted from AHRQ
	STRUCTURE-PROCESS-OUTCOMES MODEL	As identified by Donabedian (1988), the classic paradigm for assessing quality of care based on a three-component approach. Donabedian’s model proposes that each component has a direct influence on the next, as represented by the arrows in this schematic (Donabedian, 1980): Structure → Process →Outcomes.	IOM, 2006, Appendix E
	VALIDITY	Measure accurately represents the concept being evaluated and achieves the purpose for which it is intended (to measure quality). In science and statistics, validity has no single, agreed-upon definition but generally refers to the extent to which a concept, conclusion, or measurement is well founded and corresponds accurately to the real world. The word "valid" is derived from the Latin validus, meaning strong. Concurrent validity refers to the degree to which the measure correlates with other measures of the same construct that are measured at the same time. Using a testing example, a test administered to current employees and then correlated with their scores on current performance reviews would have good concurrent validity if those who scored well on the test also did well on performance reviews. Construct validity is the extent to which a measure measures the concept or construct that it is intended to measure. For example, a measure that measures the quality of diabetes care by whether a provider conducted an HbA1c test on a patient with diabetes has relatively good construct validity because high HbA1c levels are associated with diabetes crises. Content validity. In psychometrics, content validity refers to the extent to which a measure represents all facets of a given construct. For example, a depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension. Using the diabetes care example, a combination of three different measures (HbA1c testing, foot examinations, and eye examinations) would have better content validity than a single measure of HbA1c testing. Criterion validity involves the correlation between a measure and a criterion variable (or variables) taken as representative of the construct. In other words, it compares the test with other measures or outcomes (the criteria) already held to be valid. For example, IQ tests are often validated against measures of academic performance (the criterion). If the test data and criterion data are collected at the same time, this is referred to as concurrent validity evidence. If the test data are collected first in order to predict criterion data collected at a later point in time, then this is referred to as predictive validity evidence. Face validity is the validity of a measure at face value. Generally face validity means that the measure "looks like" it will work, as opposed to "has been shown to work." Predictive validity refers to the degree to which the measure can predict (or correlate with) other measures of the same construct that are measured at some time in the future. In job selection, for example, this would mean that tests are administered to applicants, all applicants are hired, their performance is reviewed at a later time, and then their scores on the two measures are correlated. If there is a strong correlation between test scores and future performance, the test would be said to have good predictive validity. Measures should be assessed against all relevant criteria at all intended levels of aggregation.	CMS, Wikipedia, based on The Standards for Educational and Psychological Testing, 1999 ***
***A revised version is expected after 2012.

Sources:

Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse http://qualitymeasures.ahrq.gov/tutorial/validity.aspx. Accessed February 6, 2012.

AHRQ, National Quality Measures Clearinghouse Glossary http://qualitymeasures.ahrq.gov/about/glossary.aspx. Accessed February 1, 2012.

CHIPRA, 2009. Children’s Health Insurance Program Reauthorization Act of 2009. Pub. L. no. 111-3, 123 Stat. 8 (2009). http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111. Accessed February 6, 2012.

Cohn, V, 2001. News and numbers: A guide to reporting statistical concepts in health and other fields. Ames, I.A.: Wiley-Blackwell.

CMS. Centers for Medicare & Medicaid Services Quality Measurement and Health Assessment Group Glossary http://www.cms.gov/MMS/19_MeasuresManagementSystemBlueprint.asp#TopOfPage. Accessed February 6, 2012.

Donabedian, A. 1980. Methods for deriving criteria for assessing the quality of medical care. Medical Care Review 37(7):653–698.

Donabedian, A. 1988. The quality of care. How can it be assessed? Journal of the American Medical Association 260(12):1743–1748.

Eisenberg-Testimony on Health Care Quality. John Eisenberg, MD, Administrator, AHCPR, before the House Subcommittee on Health and the Environment, October 28, 1997. Agency for Health Care Policy and Research, Rockville, MD. http://archive.ahrq.gov/news/test1028.htm. Accessed February 6, 2012.

IOM, 1990. Institute of Medicine, Medicare: A Strategy for Quality Assurance, Volume 1. National Academies Press; 1990.

IOM, 2001. Institute of Medicine, Crossing the Quality Chasm: A New Health System for the 21^st Century. Washington, DC: National Academies Press; 2001.

IOM, 2006. Appendix E. Institute of Medicine, Committee on Redesigning Health Insurance Performance Measures, Payment, and Performance Improvement Programs. Performance Measurement: Accelerating Improvement (Pathways to Quality Health Care Series). Appendix E. The National Academies Press; 2006.

Kunkel, S., Rosenqvist, U., and Westerling, R. 2007. The structure of quality systems is important to the process and outcome, an empirical study of 386 hospital departments in Sweden. BMC Health Services Research Jul 9; 7:104.

National Committee on Quality Assurance’s Practice Site Methodology. Definition of Practice Site adapted by the PQMP Result Aggregation Workgroup, 2012.

Patient Protection and Affordable Care Act of 2010. P.L. 111-148. Washington, DC. U.S. Government Printing Office, 2010. http://housedocs.house.gov/energycommerce/ppacacon.pdf. Accessed February 6, 2012.

PQMP Result Aggregation Workgroup, 2012. AHRQ-CMS Pediatric Quality Measures Program, PQMP Performance Result Aggregation Framework; 2012.

The Free Dictionary by Farlex: http://www.thefreedictionary.com/test. Accessed February 6, 2012.

RTI. RTI International Staff www.rti.org.

Wikipedia, based on The Standards for Educational and Psychological Testing, 1999. American Educational Research Association, Psychological Association, & National Council on Measurement in Education.. Washington, DC: American Educational Research Association; 1999***. http://en.wikipedia.org/wiki/Standards_for_Educational_and_Psychological_Testing. Accessed February 6, 2012. ***A revised version is expected after 2012.

Thank you for using the CHIPRA Pediatric Quality Measures Program (PQMP) Candidate Measure Submission Form (CPCF) template for your measure submission.

1 Children’s Health Insurance Program Reauthorization Act of 2009. Public Law No. 111-3, 123 Stat. 8 (2009). Available at: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111.

2 CHIPRA Initial Core Set of Children’s Health Care Quality Measures. Available at: http://ahrq.gov/CHIPRA/#Core http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/CHIPRA-Initial-Core-Set-of-Childrens-Health-Care-Quality-Measures.html.

3 CHIPRA Initial Core Set of Children’s Health Care Quality Measures. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/InitialCoreSetResouceManual.pdf.

4 CHIPRA Initial Core Set of Children’s Health Care Quality Measures: Technical Specifications and Resource Manual for Federal Fiscal Year 2011 Reporting. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/InitialCoreSetResouceManual.pdf.

5 Pediatric Quality Measures Program Centers of Excellence Grant Awards. AHRQ Publication No. 12-P006, March 2012. AHRQ, Rockville, MD. http://www.ahrq.gov/chipra/pqmpfact.htm.

6 AHRQ National Advisory Council on Research and Quality. Subcommittee on Quality Measures for Children’s Health Care. Members List. 2012. Available at: http://ahrq.gov/CHIPRA/qmsnaclist12.htm.

7 Initial Core Set of Children’s Health Care Quality Measures: Technical Specifications and Resource Manual for Federal Fiscal Year 2011 Reporting. Available at http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/InitialCoreSetResouceManual.pdf and http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/CHIPRA-Initial-Core-Set-of-Childrens-Health-Care-Quality-Measures.html.

8 The EPSDT is a comprehensive set of benefits available to children and youth under age 21 who are enrolled in Medicaid. For more information, see http://www.healthlaw.org/images/stories/epsdt/3-ESDPT08.pdf.

9 Children’s Health Insurance Program Reauthorization Act of 2009. Public Law No. 111-3, 123 Stat. 8 (2009). Available at: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111.

10¹⁰Under Section 214 of CHIPRA, States may elect to cover the following groups under Medicaid only or under both Medicaid and CHIP: pregnant women and children up to age 19 for CHIP or up to age 21 for Medicaid.

11 The definition is adapted from: Centers for Medicare & Medicaid Services Quality Measurement and Health Assessment Group glossary. Available at: http://www.cms.gov/MMS/19_MeasuresManagementSystemBlueprint.asp#TopOfPage. Accessed February 6, 2012.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Month 200X
Author	Terry Hall
File Modified	0000-00-00
File Created	2021-01-30