AHRQ - Guidelines for Ensuring Quality of Information Disseminated to the Public

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality

Guidelines for Ensuring the Quality of Information
Disseminated to the Public
Note: Last revised in 2006, these Guidelines also appear on the HHS Web site. AHRQ presents them here with a 2015
Update.

Contents
I. Agency Mission.
II. Scope and Applicability of Guidelines.
III. Types of Information Disseminated by the Agency to the Public.
IV. Types of Dissemination Methods.
V. Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public.
VI. Agency Administrative Complaint Procedures.
VII. Influential Scientific, Financial and Statistical Information.

I.  Agency Mission [Select for Update]
The mission of the Agency for Healthcare Research and Quality (AHRQ) is to develop scientific evidence that enables health
care decision makers—patients and clinicians, those who manage and purchase health care services, and policy makers—to
make more informed health care choices. AHRQ accomplishes its mission by conducting, supporting, and disseminating
scientific research designed to improve the outcomes, quality, and safety of health care, reduce its cost, broaden access to
effective services, and improve the efficiency and effectiveness of the ways health care services are organized, delivered, and
financed.
The Agency was created December 1989 as the Agency for Health Care Policy and Research (AHCPR) and reauthorized on
December 6, 1999, as the Agency for Healthcare Research and Quality. AHRQ is an agency of the Public Health Service
(PHS) within the U.S. Department of Health and Human Services (HHS).
At AHRQ, the Director will have overall responsibility for implementing the AHRQ Information Quality Guidelines.
Return to Contents

II. Scope and Applicability of Guidelines for Agency
AHRQ will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and AHRQ
guidelines. It is AHRQ's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it
disseminates to the public. AHRQ strives to provide information that is accurate, reliable, clear, complete, unbiased, and
useful. It is committed to integrating the principle of information quality into every phase of information development, including
creation, collection, maintenance, and dissemination.
The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002.
The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002,
regardless of when the agency first disseminated the information.  The guidelines apply to information in all media (e.g., print,
electronic) and apply primarily to the dissemination of substantive information (e.g., reports, studies, and summaries relating to
the findings and related products of AHRQ research) rather than on information pertaining to basic Agency operations not
intended for public dissemination.
Many AHRQ activities result in publicly disseminated information that is subject to the OMB Guidelines including:
https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

1/6

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality

Statistical information and databases.
Results of research projects.
Program evaluations.
Software tools.
Consumer and professional health information.

The following types of information produced by the Agency are not covered: scientific information produced by extramural
researchers who conduct and disseminate their own work in the same manner as their academic colleagues and whose views
do not represent the views of the Agency; information in which the results are identified as not being Agency views or
positions; archival information; information intended solely for intra-or-interagency use or sharing of government information;
press releases that support the announcement or give public notice of information that AHRQ has disseminated elsewhere;
correspondence with individuals; responses to requests for Agency records under the Freedom of Information Act (FOIA);
public filings; subpoenas; and the findings and determinations that AHRQ makes in the course of adjudications involving
specific parties.
Return to Contents

III. Types of Information Disseminated by the Agency to the Public
Data products—Includes print and electronic materials that describe or present aggregate statistical information such as data
from the Medical Expenditure Panel Survey (MEPS), a survey of households regarding how they use and pay for health care.
MEPS Public Use Data Files allow access to aggregate data. Another example is the Healthcare Cost and Utilization Project
(HCUP), a family of health care databases and related tools with data provided by state data organizations, state hospital
associations, and other sources. Web-based interactive analytic tools (i.e., MEPSnet, HCUPnet) are also available.
Results of research projects—Research reports and other information products such as fact sheets, research syntheses,
and AHRQ's monthly newsletter Research Activities are produced in both print and electronic formats and describe the
findings and methods of AHRQ-funded research projects. [Select for Update.]
Program evaluation reports—Reports summarizing the results of program evaluations are included in the departmental
catalog of program evaluations maintained by the Office of the Assistant Secretary for Planning and Evaluation. Many of the
Agency's program evaluations are driven by the goals established under the Government Performance and Results Act
(GPRA).
Software tools—Tools developed to assist health care policy makers, health system leaders, purchasers, and others in
carrying out their tasks and making health care choices. An example is the Quality Indicators software that ensures a
standard, trusted approach to quality measurement to support improvements to patient care.
Consumer and professional health information [Select for updated paragraph]—AHRQ produces a wide range of print and
electronic consumer and professional information. Examples include consumer publications on: various health
conditions/diseases, on how to select a health plan, personal health guides, and how to improve the quality of health care you
receive as a patient. Examples of publications for professionals include The Physicians Handbook of Preventive Services and
evidence reports on a wide array of clinical topics that can be used by public and private entities to develop and implement
their own practice guidelines, performance measures, review criteria, and other clinical quality improvement tools.
Return to Contents

IV. Types of Dissemination Methods
AHRQ principally uses print and electronic methods of dissemination.
Print methods include:
Reports, fact sheets, and research syntheses communicating the results of surveys and other research activities.
Publication of Research Activities, AHRQ's monthly newsletter AHRQ [Select for Update].
Electronic methods include:
The AHRQ Web site as well as AHRQ program specific sites.
AHRQ's Electronic Newsletter.
https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

2/6

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality

Secondary methods of dissemination include, but are not limited to: workshops, audiovisual techniques, oral
presentations, and teleconferences. These dissemination mechanisms largely draw on information developed and reviewed
first through one of the primary print or electronic modes of information dissemination.
Return to Contents

V. Agency Quality Assurance Policies, Standards and Processes for
Ensuring the Quality of Information Disseminated to the Public
The quality assurance process begins at the inception of the information development process. AHRQ reviews the quality
(including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral
to every step of the development of information, including creation, collection, maintenance, and dissemination. AHRQ will
demonstrate in its Paperwork Reduction Act (PRA) clearance packages that each draft information collection will result in
information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and AHRQ information
quality guidelines.
Work performed by AHRQ is conducted in compliance with generally accepted professional standards appropriate to the
field(s) of study being undertaken, e.g., clinical, statistics, economics, or qualitative health services research. Moreover, work
produced by the Agency undergoes many levels of review before being finalized and disseminated to ensure quality, i.e.,
utility, objectivity (in substance and presentation), and integrity. Reviews are both hierarchical, i.e., performed by higher level
supervisors and managers, and by peer experts both inside and outside of the Agency. Generally, it is the responsibility of
AHRQ's Center Directors to ensure the utility, objectivity, and integrity of work produced within their research Centers.
The discussion that follows outlines the general quality standards and quality assurance procedures that AHRQ has in place.
More detailed information is available on AHRQ's Web site and that information will be expanded to reflect new products (e.g.
new databases) or revised over time to reflect advances in generally accepted professional standards.

Statistical information and database development
Quality standards: Statistical activities are carried out in accordance with generally accepted professional standards and
modern statistical theory and practice, including scientific sampling, statistical inference and analytical techniques and
practices. All statistical programs employ or have access to experts in statistics and research design.
Quality assurance procedures: Each of the Agency's data products undergoes a series of standardized quality assurance
procedures to assure the validity, reliability, and consistency of the data. These procedures are posted on the AHRQ Web site.
For example, the Medical Expenditure Panel Survey (MEPS), which involves data collection, uses quality assurance
procedures such as the validation of interviewers' work, performance of quality control checks on the source variables across
all analytic groups, and benchmarking the data (e.g. comparisons with prior years and comparable data sources such as the
Current Population Survey of the U.S. Census Bureau and the National Health Interview Survey). The Healthcare Cost and
Utilization Project (HCUP), which aggregates and standardizes existing data from states, has quality control procedures in
place to verify that the databases contain information that is valid, internally consistent, and consistent with established norms.
Procedures include automated edit checks, careful review of all summary statistics, and benchmarking data with other sources
(e.g. comparison with the National Hospital Discharge Survey).
Before disseminating information in any format, data are reviewed internally to ensure that they are scientifically sound and
meet or exceed standards for data quality, statistical integrity, and confidentiality and privacy protections. Similarly, description
and documentation of databases, tools, products, and on-line query systems are carefully reviewed, edited, and re-reviewed
before release to the public.

Results of research projects
Quality standards: AHRQ staff adhere to the highest professional standards in their appropriate/respective fields, e.g.,
economics, social science and statistics. They seek to create transparency in their activities to enable others to fully
understand their research processes, methodologies, and assumptions so that the work can be evaluated and replicated. Staff
must ensure that strong measures are in place to guarantee the integrity of research information and to protect it from
unauthorized access or revision.
Quality assurance procedures: All intramural research projects—whether they involve qualitative or quantitative analysis—
https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

3/6

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality
y
p
p j
y
q
q
y
and all projects on which AHRQ staff collaborate with authors outside the Agency are subject to AHRQ's intramural research

planning process. This is intended to ensure conformance with strategic planning, accountability, peer review, scientific
objectivity, confidentiality, and privacy standards. Research reports that are the products of intramural research undergo many
levels of review prior to release, including hierarchical and peer reviews (sometimes including external reviews by other
agencies and outside experts). Manuscripts submitted to scientific journals are subject to additional peer-review by the journal,
prior to acceptance and publication.
In keeping with widely-accepted scientific research practices, research reports published by AHRQ describe the methods, data
sources, analytical techniques, assumptions, and limitations of the research so the study can be substantially assessed and
replicated.

Program Evaluations
Quality standards: Program evaluations are developed and assessed in accordance with the highest professional standards
of evaluation practice.
Quality assurance procedures: Evaluation proposals are reviewed by Agency management for suitability, utility, and
methodology, in accordance with sound evaluation design and standards of evaluation practice. Many evaluation projects
have technical advisory committees that oversee the design and conduct of the evaluation in accordance with standard
evaluation theory and practice; they often provide an expert review of the final report. Results of evaluation activities are
released to the public only after Agency management has completed a review of the quality, accuracy and completeness of
the report.

Software tools
Quality standards: Software tools are developed by teams of subject area experts in compliance with generally acceptable
professional standards appropriate to their respective fields. Many levels of review and testing are built into the development
process to ensure that the final product is useful, objective, and of high quality.
Quality assurance procedures: Reviews of tools in development are stringent. To ensure that they are needed and useful,
the Agency generally relies upon peer experts and user groups both inside and outside of the Department. The Agency
conducts extensive testing prior to release. Feedback is obtained from users of the tools or software to identify enhancements
that will make the tools or software more fully meet user needs. 

Consumer and professional health information
Quality standards: Consumer and professional information developed by AHRQ is conducted in compliance with generally
accepted professional standards appropriate to the field of study being undertaken, e.g., clinical, economics, statistics, and
social science research, as well as sound health care communication principles.
Quality assurance procedures: AHRQ works closely with stakeholder groups to determine the need for consumer and
professional publications. Publications are based on information emanating from research that has been reviewed for scientific
and medical accuracy and completeness by experts within and outside of the Agency before it is disseminated. Consumer
publications are tested with targeted audiences to ensure relevance, clarity, and comprehension. In addition, AHRQ's
publications containing health care recommendations indicate the level of substantiating evidence.
Return to Contents

VI.  Agency Administrative Complaint Procedures
AHRQ has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated
information that does not comply with OMB, HHS and AHRQ guidelines. Persons seeking correction of AHRQ-produced
information should follow the procedures below.
A. Responsibility of the Complainant 

To seek a correction of information disseminated by the Agency, complainants should follow the procedures described
below. 

Complaints or req ests for re ie and correction of information shall be in ritten hard cop or electronic form The shall
https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

4/6

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality

Complaints or requests for review and correction of information shall be in written hard copy or electronic form. They shall
be sent to the Agency by regular or electronic mail. Letters (written hard copy) should be sent to: 

Office of the Director

5600 Fishers Lane

Mailstop 06E37A

Rockville, MD 20857 

Electronic mail should be sent via the AHRQ website form. Complaints should state "Information Quality Concern" in the
Subject. The body of the complaint shall state that a request for correction of information is being submitted. 

Complaints shall contain a detailed description of the specific material that needs to be corrected including a detailed
citation, e.g., the publication title, date, and publication number, if any, or the Web site and web page address (url).
Complaints should describe the specific reasons for believing the information does not comply with OMB, HHS, or AHRQ
guidelines and is in error and supporting documentation, if any. In addition, they should include specific recommendations
for correcting the information; a description of how the person submitting the complaint is affected by the information error;
and the name, mailing address, telephone number, email address, and organizational affiliation, if any, of the individual or
organization making the complaint. Requesters should be aware that they bear the "burden of proof" with respect to the
necessity for correction as well as with respect to the type of correction they seek. 

In determining whether and how to address a request, the Agency will consider such factors as the Agency's budget,
resources, and priorities, as well as the complexity of the correction task itself. Also, it needs to be understood that it may
not be in the public interest for the Agency to devote significant resources to correcting information where the expenditure
of such resources is not, in the Agency's view, cost effective in light of the significance of the asserted error, the benefits
that are likely to be derived from such a correction, the costs of the correction, and the Agency's more pressing priorities
and obligations.
B. Responsibility of the Agency
Based on a review of the information provided, the Agency will determine whether a correction is warranted and if, so what
action to take. The Agency will respond to the requestor by letter or email. The Agency's response will explain the findings
of the review and the actions that the Agency will take, if any. The response will consider the nature and timeliness of the
information involved and such factors as the significance of the correction on the use of the information and the magnitude
of the correction. The response will describe how the complainant may request reconsideration. The Agency will respond to
all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve,
the Agency will inform the complainant that more time is required and indicate the reason why and an estimated decision
date.
C. Appeals
If the individual submitting the complaint does not agree with the Agency's decision (including the corrective action, if any),
the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the
Agency's decision. The appeal shall state the reasons why the Agency response is insufficient or inadequate.
Complainants shall attach a copy of their original request and the Agency response to it, clearly mark the appeal with the
words, "Information Quality Appeal," and send the appeal to the addresses for hard copy or electronic submissions
identified above.
The Agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60
calendar days to resolve, the Agency will inform the complainant that more time is required and indicate the reason why
and an estimated decision date.
The Agency official who resolved the original complaint will not have responsibility for the appeal.
Return to Contents

VII.  Influential Scientific, Financial and Statistical Information
From time to time, AHRQ disseminates influential information, including statistical information, as defined in the OMB
Guidelines. By "influential," OMB is referring to information "that the agency can reasonably determine...will have or does have
a clear and substantial impact on important public policies or important private sector decisions." In accordance with standard
Agency quality procedures, care is taken to ensure that information disseminated is substantially reproducible and replicable.
This goal is accomplished by using reliable data sources and sound analytical techniques, and by employing a high degree of
transparency about the data, methods, measures, assumptions and limitations used to develop the information in order to
facilitate reproducibility by qualified third parties
https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

5/6

5/5/22, 10:27 AM

Guidelines for Ensuring the Quality of Information Disseminated to the Public | Agency for Healthcare Research and Quality

facilitate reproducibility by qualified third parties.

Because of ethical and feasibility constraints as well as confidentiality and legal obligations to third parties supplying
information, there may be instances where not all of the original or supporting data may be available to the public. In such
instances, AHRQ will promote understanding and transparency by clearly identifying, to the extent practicable, the resulting
research methodologies and assumptions in reference documents.
Return to Contents

Page last reviewed September 2019
Page originally created September 2015

Back to Top

https://www.ahrq.gov/research/publications/ahrq-info-quality-gdlns.html

6/6