Supporting Statement Part A -- Eisenberg Generic Clearance 2-15-2011 Revised

SUPPORTING STATEMENT

Part A

Eisenberg Center Voluntary Customer Survey Generic Clearance

for the Agency for Healthcare Research and Quality

OMB No. 0935-0128

Version: August 4^th, 2010

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

A. Justification 3

1. Circumstances that make the collection of information necessary 3

2. Purpose and use of information 4

3. Use of Improved Information Technology 6

4. Efforts to Identify Duplication 6

5. Involvement of Small Entities 6

6. Consequences if Information Collected Less Frequently 6

7. Special Circumstances 7

8. Consultation outside the Agency 7

9. Payments/Gifts to Respondents 8

10. Assurance of Confidentiality 8

11. Questions of a Sensitive Nature 8

12. Estimates of Total Burden Hours and Costs 8

13. Estimates of Annualized Respondent Capital and Maintenance Costs 10

14. Estimates of Annualized Cost to the Government 10

15. Changes in Hour Burden 11

16. Time Schedule, Publication and Analysis Plans 11

17. Exemption for Display of Expiration Date 12

List of Attachments 12

A. JUSTIFICATION

1. Circumstances that make the collection of information necessary

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. research that develops and presents scientific evidence regarding all aspects of health care; and

2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) renew under the Paperwork Reduction Act of 1995 AHRQ’s Generic Clearance to collect information from users of work products and services initiated by the John M. Eisenberg Clinical Decisions and Communications Science Center (Eisenberg Center).

AHRQ is the lead agency charged with supporting research designed to improve the quality of healthcare, reduce its cost, improve patient safety, decrease medical errors, and broaden access to essential services (see 42 U.S.C. 299). AHRQ’s Eisenberg Center is an innovative effort aimed at improving communication of findings to a variety of audiences (“customers”), including consumers, clinicians, and health care policy makers. The Eisenberg Center compiles research results into a variety of useful formats for customer stakeholders. The Eisenberg Center also conducts its own program of research into effective communication of research findings in order to improve the usability and rapid incorporation of findings into medical practice. The Eisenberg Center is one of three components of AHRQ’s Effective Health Care Program (see 42 U.S.C. 299b-7). For the period 2005 until September 2008, the Eisenberg Center was operated through a contractual arrangement with the Oregon Health and Science University (OHSU), Department of Medicine, located in Portland, Oregon. In September 2008, the contract for operation of the Eisenberg Center was awarded to Baylor College of Medicine (BCM), located in Houston Texas.

The collections proposed under this clearance include activities to assist in the development of materials to be disseminated through the Eisenberg Center and to provide feedback to AHRQ on the extent to which these products meet customer needs. These materials include Summary Guides that summarize and translate the findings of comparative effectiveness reviews (CER) and research reports for purposes of summarizing research findings for various decision-making audiences, such as consumers, clinicians, or policymakers. The guides are designed to help these decision makers use research evidence to maximize the benefits of health care, minimize harm, and optimize the use of health care resources. In addition, each year of the project the Eisenberg Center will develop one computerized, interactive decision aid for those clinical problems identified from selected CERs. The intent is for the decision aid to increase the patient/consumer’s knowledge of the health condition, options, and risk/benefits, lead to greater assurance in making a decision, increase the congruence between values and choices, and enhance involvement in the decision making process. Information collections conducted under this generic clearance are not required by regulation and will not be used to regulate or sanction customers. Surveys will be entirely voluntary, and information provided by respondents will be combined and summarized so that no individually identifiable information will be released.

The Eisenberg Center will produce from 17 to a maximum of 33 Summary Guides per audience (i.e., clinician, policymaker, consumer) per year, depending on the information needed for each product with each audience. See Attachment B for the major research steps required for product development and assessment for each topic assigned to the Eisenberg Center.

In accordance with OMB guidelines for generic clearances for voluntary customer surveys and Executive Order 12862, AHRQ has established an independent review process to assure the development, implementation, and analysis of high quality customer surveys within AHRQ. Specifically, AHRQ understands that each activity conducted must be submitted to OMB with a supporting statement and accompanying instruments. Information collection may not proceed until approved by OMB.

2. Purpose and Use of Information

Information collections conducted under this clearance will be collected via the following methods:

• Focus Groups. Focus groups may include clinical professionals, patients or other health care consumers, or health policy makers. They will be used to provide input regarding the needs for products and for the development of Decision Aids and Summary Guides. Focus groups may also be used to test draft products to determine if intended information and messages are being delivered through products that are produced and disseminated through the Eisenberg Center.

• In-person or Telephone Interviews. Interviews will be conducted with individuals from one or more of the three groups identified above. The purpose of these interviews is to 1) to provide input regarding the development of Decision Aids and Summary Guides, 2) to determine if intended information and messages are being delivered effectively through products that are produced and disseminated through the Eisenberg Center, and 3) to engage the subject in cognitive testing to a) determine if changes in topical knowledge levels can be identified following exposure to Eisenberg Center informational or instructional products, and b) identify strengths and weaknesses in products and services for purposes of making improvements that are practical and feasible.

• Customer Satisfaction Survey for the Decision Aids. Baseline survey data will be collected on both clinician and patient characteristics, characteristics of the health care condition, and selected outcome measures such as knowledge and decisional self-efficacy. Following delivery of the decision aid, a user survey will be completed to explore subjects’ impressions of the tool, including ease of use, clarity of presentation, length, balance of information, rating of interactive features, and overall satisfaction. Both clinicians and patients/consumers will be surveyed. For patients, the customer satisfaction survey will include decisional outcome measures (e.g., decisional conflict, desire for involvement in decision-making), measures of attitudes and self-efficacy, and indicators of choice intention or actual choice made. If the aid is evaluated within a clinical context, measures of physician-patient interaction will also be considered. Additionally, clinicians may be interviewed about the impact of the aid on clinical flow.

• Customer Satisfaction Surveys for the Summary Guides. These surveys will be offered to health care professionals, consumers, and policy makers that use the online Summary Guides. Respondents will report via Likert-type or numerical response scales how specific informational or educational products or materials influenced health care or clinical practice behaviors.

• Follow-up CME Surveys. Continuing Medical Education (CME) credit will be offered to physicians who wish to participate in online activities developed around the Summary Guides for clinicians. Three months after completing the educational activity, physicians will be asked to complete a follow-up survey to assess realized changes in clinical practice, barriers to making change, and self-assessed impacts on patient care.

• Solicited Topic Nominations. Visitors to the Website will have the opportunity to provide information about suggested topics that might be addressed through the research and dissemination efforts of the EHC program.

• Website Registration. Visitors to the Website will be able to register personal contact information (e.g., name, email address) if wishing to receive updated information and materials as they become available.

• Glossary Feedback Survey. Visitors to the Website who access the health care glossary will be asked to suggest missing terms and provide additional comments on definitions or usage sentences, if desired.
  

This information will be used to develop, improve and/or maintain high quality products and services to lay and health professional publics. Data collection instruments from the first and only project approved under the previous clearance are included with this clearance as examples of possible future instruments (see Attachments C to J).

3. Use of Improved Information Technology

Improved electronic technology (e.g., Web-based materials) will be used whenever possible to reduce the burden on the public. In some instances, however, the most appropriate methodology will involve written or oral responses to brief questionnaires, interviews, and focus groups. Individuals may also be asked to interact with computer and/or Internet technologies to assess functionality and ease of use. In some cases, respondents will be asked to review and rate and/or comment on materials prepared using text and graphics to deliver messages about one or more therapies of interest.

4. Efforts to Identify Duplication

Each survey or other data collection instrument will be designed to reflect the specifics of the customer population served. During the development of these voluntary instruments and other data gathering tools, groups within and outside of AHRQ will be consulted. Plans to conduct information gathering will be reviewed prior to implementation, and any potential duplication will be identified in the review and approval process.

5. Involvement of Small Entities

The survey instruments and procedures for completing the instruments will be designed to minimize the burden on all respondents and will not have a significant impact on small businesses or other small entities. The burden is entirely voluntary.

6. Consequences if Information Collected Less Frequently

Information collections are appropriate vehicles to examine customer experiences and perceptions with regard to products and services developed by the Eisenberg Center and its ability to communicate effectively with a variety of audiences. Collection of data on a less frequent basis would reduce the practical utility of the information and would inhibit the Eisenberg Center’s ability to: 1) determine how well its products and services are meeting customers’ current and anticipated needs; 2) identify problem areas with existing products and services and determine what improvements should be made to improve these products and services; and 3) identify and develop new products and services. The importance of frequent interaction and data collection with targeted audiences was reflected in the request for applications issued in the contracting process, which specifically mandated formative, operational, and assessment data collection efforts related to the Eisenberg Center and its products.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), notice was published in the Federal Register on May 20^th, 2010 for 60 days (see Attachment X). One comment was received; the entire comment and AHRQ’s response is shown below.

Public comment:

From: jean public [mailto:jeanpublic@yahoo.com]
Sent: Thursday, May 20, 2010 9:20 AM
To: Lefkowitz, Doris C. (AHRQ); americanvoices@mail.house.gov; comments@whitehouse.gov
Cc: info@theteaparty.org; info@taxpayer.net; media@cagw.org
Subject: public comment on federarl register - shut down this useless agency

 

this agency exists to do nothing more than take surveys. that is all they do. and then they talk to only certain groups like profiteers and those who agree with their policies. never do they do any research on the highly intelligent families who are trying to raise autistic children harmed as a result of their vaccines. never. they just make life hard for these families. this agency is useless and needs to be shut down. we are pyaing huge bureuacratic bloated salaries to these survey takers. shut down the whole agency. it is useless to the average american.

jean public 8 winterberry court whitehouse station nj 08889

AHRQ’s response:

The agency thanks the reviewer for the comments provided on the proposed information collections. AHRQ, through the Effective Health Care Program, will continue our commitment to engage public participation in our research activities to ensure our work is informed by the public and that it appropriately addresses the public’s stated needs and values. We will also continue to seek feedback on our activities and developed products to ensure they are readily accessible and useful to all decision makers, including patients, clinicians and policymakers.

8.b. Outside Consultations

The Eisenberg Center will consult with AHRQ’s in-house statistical staff, other Federal agencies, and other organizations, which have conducted, or may conduct, similar surveys to identify areas of interest and concern to customers. As appropriate, panels of outside experts may be established to assist in design and implementation of the surveys.

9. Payments/Gifts to Respondents

No remuneration to respondents for written, telephone, web, or other forms of surveys or interviews will be given. On a case-by-case basis, consideration will be given for modest remuneration for participants in focus groups and structured interviews. This remuneration is meant to reimburse the participants for their time and travel. In such cases, the remuneration may range from $50 up to $250 per individual; however, the same remuneration will be offered to all persons participating in a specific activity. Factors influencing the rate of remuneration will include: 1) the participant’s qualifications/demonstrated expertise; 2) projected time allocation and any preparations required for the activity; 3) urgency of gathering data and the implications with regard to incentivizing participants; and 4) projected likelihood of loss of income or other costs to participants (e.g., child care costs, travel expenses) that are associated with participation. Maximum honoraria amounts of $75 will be paid to consumers participating in focus groups, while honoraria amounts of up to $250 will be paid to expert physicians and policymakers for performing complex testing processes. These individuals have limited time availability and are accustomed to receiving similar levels of recompense for their valuable input, which is essential to effective product development and testing. Remuneration for focus groups and other activities demanding participant time is a recognized standard industry practice, without which it would be difficult to achieve appropriate and adequate participation. Specific proposed remuneration will be included in the supporting statement for activities specified under this generic clearance.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

Respondents will be advised that surveys and/or other data collection activities in which they may be asked to participate are entirely voluntary, and that any information they provide will be combined and summarized with information provided by others and no individually identifiable information will be released. In instances where respondent identifiers are needed (e.g., registrations for updates, continuing medical education [CME] follow-up), information collection will fully comply with all respects of the Privacy Act.

11. Questions of a Sensitive Nature

No questions of a sensitive nature are anticipated under this generic clearance.

12. Estimates of Total Burden Hours and Costs

Exhibit 1 shows the estimated total burden for the respondents' time to participate in this research. These estimates assume a maximum of 99 Summary Guides over 3 years and separate Guides for clinicians, policy makers and consumers and are thus slight overestimates.

Focus groups will be used for needs assessment and will be conducted with clinicians and consumers for development of the Summary Guides, and additionally with policymakers for those Guides in which policy recommendations are applicable. Focus groups will be conducted with no more than 3,168 persons over 3 years and will last about 1 ½ hours.

Once the Summary Guides are developed they will be subjected to in-person or telephone interviews for purposes of usability and product testing with clinicians, policy makers and consumers. In-person/telephone interviews will be conducted twice with about 4,158 persons over 3 years and will take about 66 minutes on average. As depicted in Attachment B, two rounds of interviews will be conducted with all consumer representatives during product development, with a second round of interviews conducted occasionally with clinicians and policy makers, as needed.

Customer satisfaction surveys for the Summary Guides will be conducted with approximately 19,800 representatives from the audience to be targeted by the Summary Guides over 3 years (i.e., clinician, policymaker or consumer) and will take 5 minutes to complete.

Customer satisfaction surveys will also be administered to approximately 150 clinicians and 1500 patients in evaluating the Decision Aid. These surveys will take about 10 minutes to complete, and will be administered before and after implementation of the Decision Aid in the study populations.

Clinicians that have completed CME accrediting requirements and are requesting CME credit will be asked to complete the follow-up CME Survey three months following completion of the online activity. This data collection will be completed with about 3,960 clinicians over 3 years and will require 5 minutes to complete.

Approximately 7,500 solicited topic nomination forms will be completed over 3 years by healthcare professional and consumer visitors to the Website and will require about 5 minutes to complete. Website Registration will be completed by all persons wanting to stay up-to-date with the latest information from the Eisenberg Center, about 18,000 over 3 years, and requires about 5 minutes to complete. The Glossary Feedback Survey will be completed by about 600 persons that access the glossary over a three year period and takes 5 minutes to complete. The total burden hours are estimated to be 18,605 over three years.

Exhibit 2 shows the estimated total cost burden associated with the respondent's time to participate in this research. The cost burden is estimated to be $865,829 annually.

Exhibit 1.  Estimated total burden hours over 3 years

Type of Data Collection	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Focus Groups	3,168	1	1.5	4,752
In-person/Telephone Interviews	4,158	2	1.1	9,148
Customer Satisfaction Surveys for the Decision Aid	1,650	2	10/60	550
Customer Satisfaction Surveys for the Summary Guides	19,800	1	5/60	1,650
Follow-up CME Surveys	3,960	1	5/60	330
Solicited Topic Nominations	7,500	1	5/60	625
Website Registration	18,000	1	5/60	1,500
Glossary Feedback Survey	600	1	5/60	50
Total	58,836	na	na	18,605

Exhibit 2. Estimated total cost burden over 3 years

Type of Data Collection	Number of respondents	Total burden hours	Average hourly wage rate*	Total cost burden
Focus Groups	3,168	4,752	$46.71	$221,966
In-person/Telephone Interviews	4,158	9,148	$53.17	$486,399
Customer Satisfaction Surveys for the Decision Aid	1,650	550	$24.50	$13,475
Customer Satisfaction Surveys for the Summary Guides	19,800	1,650	$46.71	$77,072
Follow-up CME Surveys	3,960	330	$73.86	$24,374
Solicited Topic Nominations	7,500	625	$19.56	$12,225
Website Registration	18,000	1,500	$19.56	$29,340
Glossary Feedback Survey	600	50	$19.56	$978
Total	58,836	18,605	na	$865,829

*Based upon the mean and weighted mean wages for clinicians (29-1062 family and general practitioners), policy makers (11-0000 management occupations, 11-3041 compensation & benefits managers, 13-1072 compensation, benefits & job analysis specialists, 11-9111 medical and health service managers, 13-2053 insurance underwriters and 15-2011 actuaries) and consumers (00-0000 all occupations). Focus groups include 528 clinicians ($77.64/hr) and 528 consumers ($20.32/hr); in-person/telephone interviews includes 528 clinicians, 330 policy makers ($39.91/hr) and 528 consumers; customer satisfaction surveys for the decision aid includes 50 clinicians and 500 consumers; customer satisfaction surveys for the summary guides includes 1,650 clinicians, 1,650 policy makers and 3,300 consumers; follow-up CME surveys includes 1,320 clinicians; solicited topic nominations include 1,125 clinicians, 250 policy makers and 1,125 consumers; website registration includes 2,700 clinicians, 600 policy makers and 2,700 consumers; glossary feedback survey includes 90 clinicians, 20 policy makers and 90 consumers, National Compensation Survey: Occupational wages in the United States May 2008, “U.S. Department of Labor, Bureau of Labor Statistics.”

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

The maximum cost to the Federal Government is estimated to be $1,439,003 annually. Exhibit 3 shows the total and annualized cost by the major cost components.

Exhibit 3.  Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	$1,019,970	$339,990
Data Collection Activities	$735,405	$245,135
Data Processing and Analysis	$1,889,505	$629,835
Project Management	$557,380	$185,793
Overhead	$114,750	$38,250
Total	$4,317,010	$1,439,003

15. Changes in Hour Burden

The number of respondents and burden hours have been significantly increased in this renewal clearance. The number of respondents and burden hours in the previous generic clearance were found to be inadequate for the type of data collections conducted by the Eisenberg Center due primarily to the substantial increase in the number of topics for which products are to be produced annually under the 2009 contract expansion.

16. Time Schedule, Publication and Analysis Plans

The purpose of the activities described is to gather both quantitative and qualitative information on Eisenberg Center products and delivery formats and vehicles, including information on their value and utility to the audiences for whom they are intended. Information from analyses of data gathered during formative phases of product development will be used to revise and refine the products and delivery systems prior to dissemination to targeted audiences. Information from analyses of data gathered from users of the information distributed via the Web and through other means (e.g., presentations at professional meetings) will be used to: a) determine if further revisions or refinements would enhance value or utility of materials currently available for the Eisenberg Center; and b) characterize the quality and appropriateness of efforts to use the Center and its resources effectively in supporting delivery of high quality health care. This latter function will be of special importance in guiding decisions by AHRQ regarding future efforts related to the Eisenberg Center and the nature and scope of support committed to it.

The analyses will be descriptive, and it is unlikely that the results can be generalized to the larger population. The results of these findings are primarily for internal use but may be shared with key government policy and management officials, AHRQ staff, public and private health providers, and members of the general public.

Focus groups: Participants will be selected purposively, so that no generalizations to the population will be possible. The analyses will be qualitative and consist mostly of narrative summaries of the discussions.

In-person/telephone interviews for usability/product testing: Interviews to aid in product development and early discovery of problems with draft information products will be conducted in person or remotely via telephone or the Web. When feasible, assessments will be done in controlled environments for purposes of evaluating systematically different products (e.g., comparative effectiveness reports, key messages) in order to refine and enhance their readability, comprehension, and usefulness. Characteristics of respondents needed for product testing will be specified. Participants typically come to a designated location where they are presented with instructions and are then exposed to the study material and asked to respond to a series of questions that will allow the investigators to assess product features or conditions. These technology-facilitated assessments allow the investigator to vary features of reports and decision aids in a systematic way to facilitate evaluation of those features and their use under varying conditions. By conducting one-on-one interviews, individuals are afforded the privacy and flexibility needed to allow them to discuss reactions, reveal interpretations, and communicate how they might use the material and other resources in decision making.

Customer surveys: Electronic technology may be used for this type of information collection. The surveys will be accessible through the Eisenberg Center Website. Visitors to the Website will have the option of responding voluntarily to the electronic form posted on the site. In addition to summarizing responses to questions, basic demographic information will be collected and summarized. On occasion, similar information may be collected from customers over the phone to ensure that we obtain information from customers who do not use the Web frequently.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

Attachment A: Healthcare Research and Quality Act of 1999

Attachment B: Research process for product development and assessment

Attachment C: Oral meds for type 2 diabetes consumer guide -- feedback questionnaire for people with type 2 diabetes

Attachment D: Oral meds for type 2 diabetes consumer guide -- feedback questionnaire for health care providers

Attachment E: Oral meds for type 2 diabetes clinician guide -- feedback questionnaire for health care providers

Attachment F: Oral meds for type 2 diabetes clinician guide -- feedback questionnaire for policymakers/administrators

Attachment G: Screen shots of Osteoporosis Decision Aid

Attachment H: Osteoporosis Decision Aid – feedback questionnaire

Attachment I: Glossary feedback questionnaire

Attachment J: Example of instrument used in gathering follow-up information from participants in CME/CE activities.

Attachment K – Federal Register Notice

13