rting Statement Part A -- Assessing the Feasibility of Disseminating EHC Products Through ED Activities 5-25-2012

SUPPORTING STATEMENT

Part A

Assessing the Feasibility of Disseminating Effective Health

Center Products through Educational Activities Planned

and Implemented in Partnership with the

Society of Academic Continuing Medical Education

Version: May 17, 2011

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

A. Justification 3

1. Circumstances that make the collection of information necessary 3

2. Purpose and Use of Information 6

3. Use of Improved Information Technology 6

4. Efforts to Identify Duplication 6

5. Involvement of Small Entities 7

6. Consequences if Information Collected Less Frequently 7

7. Special Circumstances 7

8. Consultation Outside the Agency 8

9. Payments/Gifts to Respondents 8

10. Assurance of Confidentiality 8

11. Questions of a Sensitive Nature 8

12. Estimates of Annualized Burden Hours and Costs 9

13. Estimates of Annualized Respondent Capital and Maintenance Costs 10

14. Estimates of Annualized Cost to the Government 10

15. Changes in Hour Burden 10

16. Time Schedule, Publication and Analysis Plans 10

17. Exemption for Display of Expiration Date 12

List of Attachments 12

A. JUSTIFICATION

1. Circumstances that make the collection of information necessary

The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. research that develops and presents scientific evidence regarding all aspects of health care;

2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

Request for information collection approval. The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) approve under the Paperwork Reduction Act of 1995 this Project Clearance to collect information from users of products provided by the John M. Eisenberg Clinical Decisions and Communications Science Center (Eisenberg Center). Information collected consists of feedback from managers, instructors, and learners about these health care guides and other products presented as part of Continuing Medical Education activities.

Background on AHRQ’s Eisenberg Center (EC) and Effective Health Care (EHC) Program. AHRQ’s Eisenberg Center (EC) aims at improving communication of research findings to a variety of audiences (“customers”), including consumers, clinicians, and health care policymakers. The EC compiles research results into useful formats for customer stakeholders and conducts investigations into effective communication of research findings in order to improve the usability and rapid incorporation of findings into medical practice. It is one of three components of AHRQ’s Effective Health Care (EHC) Program. From 2005 until September 2008, the EC operated through a contract with the Oregon Health and Science University (EC-OHSU) Department of Medicine in Portland, Oregon. In September 2008, the contract to operate the EC was awarded to Baylor College of Medicine (EC-BCM), in Houston, Texas.

Rationale for the information collection. As had been the case with the OHSU, the primary focus of the EC-BCM is to translate results from systematic reviews of evidence comparing the effectiveness of two or more clinical care processes into information that can be used to support clinical decision-making. The major products of such efforts are brief guides designed for clinicians, patients, and policymakers that summarize the evidence concerning the effectiveness of various diagnostic and treatment processes. Effective dissemination of these products supports AHRQ's mandate to improve the quality of healthcare, reduce its cost, improve patient safety, decrease medical errors, and broaden access to essential services.

For health care professionals, the guides are prepared in formats using professional-level language, with appropriate references and sophisticated graphics to illustrate scientific and/or clinical care processes. For patients, these materials are typically prepared in English and Spanish at about the seventh-grade reading level. For some products, versions of the guides are prepared for persons who have even lower level reading skills, with information conveyed in simple graphical formats and limited text. Other products might include decision aids that clinicians can use to present key clinical information about different care options, including information on risks and benefits associated with different treatment choices presented in ways to help patients make choices that are best for them. All of the guides are designed to help decision makers, including clinicians and health care consumers, use research evidence to maximize the benefits of health care, minimize harm, and optimize the use of health care resources.

The collections proposed under this project include activities to assess the feasibility of disseminating EHC products through Continuing Medical Education (CME) activities, specifically those planned and implemented by member organizations of the Society of Academic Continuing Medical Education (SACME). SACME is an organization with members in both the U.S. and Canada formed in 1976 to “promote the research, scholarship, evaluation and development of CME and Continuing Professional Development (CPD) that helps to enhance the performance of physicians and other healthcare professionals practicing in the United States, Canada, and elsewhere for purposes of improving individual and population health.”

For this project, the EC-BCM is working with six organizations selected from applications submitted by SACME members that had been invited to compete for funding. Organizations selected for participation in the feasibility study have committed to specific activities designed to disseminate EHC Program summary guides to physicians, other clinicians, instructional faculty, and clinical researchers who participate in CME activities. Another partner in these efforts is the Association of American Medical Colleges (AAMC), which is assisting the project through access to MedEdPORTAL and CME4docs, two recently launched initiatives that are designed to encourage use of high quality CME resources by medical school faculty and others involved in development and delivery of CME.

This research has the following goals:

1) Identify critical factors that enhance or impede integration of EHC products into CME activities;

2) Assess strategies to remove, overcome, or work around barriers to integration of EHC products into CME programming with selected audiences;

3) Confirm approaches that can be used in whole or in part to create and deliver effective CME instruction about EHC products (e.g., clinician guides, consumer guides, faculty slide sets); and

4) Review early educational program outcomes associated with integration of EHC products into CME activities.

To achieve the goals of this project the following data collections will be implemented:

1) Interviews with CME Project Directors—Semi-structured interviews will be conducted with the representative of each participating CME institution leading the development and implementation of the educational activities associated with the study. The director is typically, but not always, an expert physician. The interviews will be designed to: a) assess perceived feasibility and obtain feedback on strategies used to integrate EHC products into their planned CME activities involving varied content, instructional methods, and delivery formats; and b) characterize barriers and facilitators to the integration of EHC products into specific CME activities. The interview guide for use with project directors is in Attachment A.

2) Focus Group with CME Project Directors – A focus group will also be convened with the CME Project Directors described above near the midpoint of the project to: a) obtain feedback on the perceived usefulness, currency and quality of the EHC products; and b) explore the overall implications concerning CME activities as an avenue for disseminating EHC products. The focus group topic guide is in Attachment B.

3) Interviews with Faculty Members— Semi-structured interviews will be conducted with clinicians who served as faculty in the CME activities associated with this study to: a) obtain perspectives on the quality, relevancy, and utility of the resources that they accessed and integrated into their CME activities; b) identify obstacles to the integration of EHC products into specific CME activities and contexts; and c) identify additional tools or resources that could facilitate the integration of EHC content into CME activities. The faculty interview guide is included in Attachment C.

This study is being conducted by AHRQ through its contractor, EC-BMC, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).

2. Purpose and Use of Information

The collected data will be used to explore the feasibility of: a) including EHC products (i.e., clinician guides, consumer guides, faculty slide sets) in CME activities that employ varied delivery modalities; and b) initiating additional studies to identify factors that promote effective integration of evidence-based content into educational activities. The data gathered from physicians and other clinical professionals who are participating in CME activities will foster understanding of the current state of awareness of and willingness to learn about results from comparative effectiveness research studies. The planned assessment approaches will promote better understanding of strategies that are most appropriate for use in incorporating comparativeness effectiveness research findings into CME activities, as well as understanding which strategies produce desired educational outcomes and are most acceptable to targeted learners—in this case clinical professionals. The information generated will be used in designing learning programs for delivery through the Eisenberg Center for Clinical Decisions and Communications Science and will be shared with others in the CME community through journal articles, Web-based publications, and scientific presentations.

3. Use of Improved Information Technology

Improved electronic technology (e.g., Web-based materials) will be used whenever possible to reduce the burden on the public.

4. Efforts to Identify Duplication

Through August of 2011, no reports could be identified in the literature that described data-gathering activities related to integration of products and findings from AHRQ’s Effective Health Care (EHC) Program into varied modes of CME. AHRQ has completed literature searches using both the PubMed search capabilities operated through the National Library of Medicine and using the Google search engine to determine if any data similar to this has been reported in the literature. Based on reviews of the literature, it is clear that the work proposed through the project described, including the data gathering with the participating organizations, represents a new area of research that in no way duplicates previous efforts.

5. Involvement of Small Entities

The survey instruments independently developed and distributed by each participating institution have been designed to minimize the burden on all respondents and will not have a significant impact on small businesses or other small entities. The methods are very familiar to developers and faculty of academic CME programs. Clinicians, including those required to obtain CME credit to maintain specialty certification or licensure, are accustomed to completing instruments of the type that will be used.

6. Consequences if Information Collected Less Frequently

The proposed data collection activities are one-time efforts designed to guide future development of materials for use by academic health science centers in disseminating EHC products to academic audiences. There are no plans to gather this information again, but the results will guide preparation of future data collection efforts associated with development, delivery, and assessment of other CME offerings that integrate EHC products.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), notice was published in the Federal Register on (date and page number of 60 day notice) for 60 days (see attachment X).

8.b. Outside Consultations

To achieve its evaluation goals, the EC-BCM works closely with a three-member panel of CME thought leaders (i.e., individuals who have gained national recognition for their knowledge of advancing CME through research and dissemination ), supplemented by two EC-BCM members who have high levels of expertise and experience in evaluation methodology and applied research. This Evaluation Subcommittee has provided guidance in designing the instruments and assessment methodologies and will continue to be engaged throughout the course of the evaluation activities. No other external entities will be engaged for consultation.

9. Payments/Gifts to Respondents

No remuneration will be given to respondents for written, web, or other forms of surveys, though consideration will be given to providing modest remuneration to interviewed faculty members who are clinicians. In such cases, the remuneration amount will not exceed $250 per individual, with the same remuneration offered to all clinicians participating in a specific activity. Clinicians have limited time availability and are accustomed to receiving similar levels of recompense for their input. In a 2005 paper examining payment practices used in 467 studies involving patients and non-patients, including physicians, in over half (55.8%) of the studies, payments ranged from $100 to $500, and in 14.8% of studies, payment for participation exceeded $500. Even with the reimbursement levels reported, the study authors questioned “whether people receive enough for the contribution they are making to research” (Grady C, Dickert N., Jawetz T., Gensler G., Emanuel E. "An analysis of U.S. practices of paying research participants," Contemporary Clinincal Trials, June 2005; 26(3): 365-375). Remuneration for interviews and other activities demanding clinician time is a recognized standard industry practice, without which it would be difficult to achieve appropriate and adequate participation.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

Respondents will be advised that the data collection activities in which they may be asked to participate are entirely voluntary, any information they provide will be combined and summarized with information provided by others, and no individually identifiable information will be released. In instances where respondent identifiers are needed, information collection will fully comply with all requirements of the Privacy Act.

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature. Although some data gathering may deal with specific health conditions as an educational topic, the focus of the assessment is on knowledge gained of the topic and the method of delivery of the information, rather than on any personal issues around a health condition or how it is managed.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden for the respondents' time to participate in this research. Interviews will be conducted with each CME Project Director and will last about 30 minutes, while the focus group will last about 90 minutes. A maximum of 30 interviews will be conducted with CME faculty members. These are estimated to take 30 minutes to complete.

Exhibit 1.  Estimated annualized burden hours

Type of Data Collection	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Interviews with CME Project Directors	10	1	30/60	5
Focus Group with CME Project Directors	10	1	1 and 30/60	15
Interviews with Faculty Members	30	1	30/60	15
Total	50	na	na	35

Exhibit 2 shows the estimated annualized cost burden associated with the respondent’s time to participate in this research. The total annual cost burden is estimated to be $2,541.

Exhibit 2. Estimated annualized cost burden

Type of Data Collection	Number of respondents	Total burden hours	Average hourly wage rate	Total cost burden
Interviews with CME Project Directors	10	5	$64.31†	$322
Focus Group with CME Project Directors	10	15	$64.31†	$965
Interviews with Faculty Members	30	15	$83.59‡	$1,254
Total	50	35	na	$2,541

†Based upon the mean wages for clinicians (29-1062 family and general practitioners) and medical and health services managers (11-9111), National Compensation Survey: Occupational wages in the United States May 2010 “U.S. Department of Labor, Bureau of Labor Statistics.” http://www.bls.gov/oes/current/oes_nat.htm

‡Based upon the mean wages for clinicians (29-1062 family and general practitioners), National Compensation Survey: Occupational wages in the United States May 2010 “U.S. Department of Labor, Bureau of Labor Statistics.” http://www.bls.gov/oes/current/oes_nat.htm

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

The maximum cost to the Federal Government is estimated to be $166,417 annually. Exhibit 3 shows the total and annualized cost by the major cost components.

Exhibit 3.  Estimated Total and Annualized Cost

Cost Component	Total Cost	Annualized Cost
Project Development	$110,846	$55,423
Data Collection Activities	$47,563	$23,781
Data Processing and Analysis	$38,250	$19,125
Project Management	$73,675	$36,838
Overhead	$62,500	$31,250
Total	$332,834	$166,417

15. Changes in Hour Burden

This is a new collection of information.

16. Time Schedule, Publication and Analysis Plans

Exhibit 4.  Approximate Timeline for Data Gathering Activities

Timeline for Data Gathering to Assess the Feasibility of Integrating Effective Health Care (EHC) Program Products in Academic Continuing Medical Education (CME) Activities

Major Data Gathering Tasks to Be Completed

Pre-OMB Clearance (months)

Post-OMB Clearance (months)

CME activities completed by partner academic organizations in staggered fashion

Submit initial OMB clearance request to include information on selected test sites

Work with OMB representatives in revising and refining clearance request

Complete the public comment period required for OMB clearance

Conduct interviews with faculty of CME programs

Administer initial survey assessments to CME learners

Administer follow up survey assessments to CME learners

Conduct focus group with CME Project Directors

Conduct interviews with CME Project Directors

Analyze data and prepare final report on feasibility

Publish results via the EHC Program Web site and journal article(s)

Qualitative data collected using focus groups and interviews will be analyzed to identify themes, patterns and possible explanations. Constant comparison method will be used to group answers to common questions and analyze different perspectives on central issues. The analysis of the qualitative data will identify a) critical factors that enhance or impede the integration of EHC evidence into CME activities; b) types of CME activities (e.g., live program activities, interactive workshops using case-based materials, enduring materials, regularly scheduled conferences) for which EHC products can be used effectively; and c) modifications and support structures necessary to facilitate the integration of EHC evidence into educational programming.

Limited quantitative data will be collected via the questionnaire surveys of learners. The questionnaire data will be summarized using descriptive statistics and limited inferential statistics as applicable. The importance of findings will be informed not only by statistical significance, but also using estimated effect sizes and appraisal of the “clinical significance” or impact of results on practice. Nonparametric techniques will be used if distributional or small sample conditions warrant. Moderating variables, such as identified barriers to implementing change, may be investigated to inform understanding of any non-significant effects observed.

It is expected that at least two types of publications will be developed from results of this project. The first will be a summary prepared for publication on the Effective Health Care Program Web Site. This summary will describe the project methodologies and results including information on the target audience of the CME; the topical content and the methods employed in the CME (e.g., online case studies, live presentations, academic detailing); the assessment tools used to determine educational outcomes; and findings on ways to integrate EHC products successfully into CME activities. In addition, data from follow-up activities will be prepared for submission to at least one peer-reviewed professional journal, such as the Journal of Continuing Education in the Health Professions, Advances in Health Sciences Education, Medical Education, or others.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

Attachment A: Interview guide for CME project directors

Attachment B: Focus group topic guide for CME project directors

Attachment C: Interview guide for faculty members

13