Supporting Material 1

Supporting Statement for Paperwork Reduction Act Submissions

4/24/07 Revised

Office of Women’s Health

The Heart Truth Professional Materials

Program Evaluation

1. JUSTIFICATION

1. Circumstances that make collection of information necessary

The Heart Truth Campaign was launched by the National Heart, Lung, and Blood Institute (NHLBI) in September 2002 to increase women's awareness of heart disease.  The Heart Truth joins together leaders in women's health—along with corporate and media partners—to create a national movement aimed at delivering an urgent wake-up call to women about heart disease.

As part of NHLBI's The Heart Truth Campaign, The Office on Women’s Health’s (OWH) National Centers of Excellence in Women's Health (CoEs) and National Community Centers of Excellence in Women's Health (CCOEs) worked jointly to develop professional education materials for health care professionals. Using these materials, the CoEs and CCOEs are involved in dissemination activities that will educate health professionals on the prevention of heart disease in women.

The campaign materials include:

• Training/curriculum materials for medical students and allied health professional students,

• Slide presentations for cardiologists, primary care physicians, and allied health professionals,

• Web-based interactive multiple-unit, case-based learning modules for training and self study, and

• An online location The Heart Truth Professional Education Web Site housing all the above materials (www.womenshealth.gov/hearttruth).

OWH is interested in evaluating the effectiveness of these materials in terms of the following two outcomes:

• To what extent were the CoE/CCOE teams successful in persuading healthcare institutions and practitioners to obtain and actively use the education and training materials; and

• To what extent did the use of professional education/training materials lead to changes in practice by healthcare providers who were exposed to them?

2. How, by whom, and for what purpose is the information used

HOW: This information will be used by the Office on Women’s Health in meeting the requirements of PART.

BY WHOM: OWH will both use this evaluation.

FOR WHAT PURPOSE: OWH will use the results of this evaluation in two ways. First, evaluating success in dissemination of materials will inform future dissemination efforts undertaken by both institutions. Second, an evaluation of the success of the program overall will serve to guide future distribution of the program and its materials to health care facilities across the country and continue to heighten awareness of women’s heart health issues.

3. Extent of automated information collection

Respondents will be allowed to take the survey on-line. We estimate that between 90% and 95% of respondents will use the on-line option. The Heart Truth participants will be given the option of providing an email address or a mailing address. It is estimated that the on-line option will reduce the burden for those taking the survey (in contrast to paper survey mode). The contractor will mail a survey to any dissemination participant who requests it.

3. Efforts to avoid duplication

No outcomes-based evaluation has been conducted with The Heart Truth materials. There is an education/knowledge retention evaluation of The Heart Truth materials that is being administered by the Program Coordinating Center at the UCLA CoE. However, that evaluation focuses on the extent that professionals learn the information that was presented to them. The PCC is using a data collection instrument administered directly after receiving instruction in the materials. (The OMB reference for that study is OMB Control Number 0900-0288.) The survey proposed here will contact participants 2 to 4 months after the learning event and ask if they have incorporated the learned material into their health care practice.

Part of the PCC evaluation is to also assess pretest and posttest knowledge for an online presentation of these materials for Continuing Medical Education/Continuing Education units (CME/CEU). OMB clearance has been obtained for this evaluation as well (OMB No. 0990-0288).

3. Efforts to minimize the burden on small businesses

It is not anticipated that small businesses will participate extensively in this survey (less than 10% of total). The majority of respondents will be from medical schools. However, the survey is open for medical professionals and therefore, a physician may be eligible. To minimize burden to these physician’s the following steps were taken:

1. Respondents were told that they would be contacted by Gallup at a future time.

2. Physicians who download the materials for themselves, they will be the only one taking the survey and therefore, their entire burden will be about 5 minutes.

3. All contact will be thru electronic mediums so the physician will be able to participate at a time that is most convenient for them.

.

3. Impact of less frequent collection of information

This data collection is a one time assessment specifically designed for The Heart Truth materials.

3. Special Circumstances

There are no special circumstances for this data collection.

3. Compliance with 5 CFR 1320.8

A 60-day Federal Register Notice was published in the Federal Register on September 21, 2006 Vol 71, No 183, pp 55202. There were no public comments received

The study methodology was reviewed by Ana Nunez, MD at the Drexel University CoE, Karen Freund MD at the Boston University CoE, and Wilma Tilson, PhD at ASPE.

3. Payments or Gifts to Respondents

There are neither payments nor gifts to respondents.

3. Assurance of Confidentiality

This survey is a web survey. Gallup has assured the respondents that their e-mail address will be destroyed and not shared with anyone.

3. Justification for data collection of a sensitive nature

There is no sensitive data being collected in this effort.

3. Estimate of burden hours

Form name	Number of Respondents	Number of responses per respondent	Hours per response	Total Burden Hours
Online Form	400	1	0.1	40
TOTAL	400	---	---	40

3. Estimate of cost burden

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Physicians*	10	$83.21	$830.21
Respondents (medical school students)++	60	$41.61	$2,496.60
Total	70		$3,326.81

*salary estimate taken from Bureau.

*salary estimate taken from Bureau of Labor Statistics-General Internists salary.

++Medical student’s salary conservatively considered to be 50% of physician’s salary

12. Estimate of cost to the Federal Government

OWH has contracted for development of the evaluation design, creation of instruments, data collection, analysis and reporting. The total cost of data collection will be $85,000.

13. Explanation of program changes or adjustments

This is a new collection of information. There are no program changes or adjustments.

13. Publication of results of data collection

PLANS FOR PUBLICATION

The contractor will write a final report based upon the requirements of the contract. The report will contain a background section and scope, design, and methodology sections. The main body of the report will describe and interpret the key findings, which will include final analytical tables. The final section of the report will have a conclusion and report recommendations based on the survey outcomes.

The study report will be available upon request. In addition, the study results will be reported to all institutions that request a copy.

PLANS FOR TABULATION

The analysis of this data will be focused on the three outcome questions asked in the survey (the remainder are demographic questions). Tables will by the type of professional exposed to the materials (i.e. medical students vs. practitioners) or by type of professional exposed (i.e. cardiologist vs. internal medicine).

Once thoroughly reviewed and revised as necessary, the analytical tables will be converted from raw statistical output to a format that will be easy for reviewers to read and interpret. This will include incorporating long titles for tables and full descriptive names for all variables and classification categories. Variable construction and table generation will be done with SPSS software.

Exhibit 1: Scheduling of Tasks

ITEM	DUE DATE
Develop draft Survey Instrument	Completed
Finalize survey instrument	Completed
OMB Clearance Package submitted	Completed
IRB approval from each participating hospital	As needed
Receive OMB clearance	In progress
Begin data collection (of those who have been exposed w/in the past 3 months)	May 2007
Analyze data	July 2007
Produce data tables	July 2007
Draft Report on Study Results	August 2007
Final Report	September 2007
Conduct First Briefing	October 2007

13. OMB approval for not displaying OMB date

The OMB approval expiration date will be displayed on all survey materials.

13. Exceptions to the certificate statement

There are no exceptions to the certification statement.

B: COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe potential respondent universe and any sampling selection method to be used

This research project employs a survey instrument, in Web mode, administered 2-4 months after the respondent was exposed to The Heart Truth materials. The population will be medical students and practitioners who are exposed to The Heart Truth materials (i.e., participants who were presented this information).

Respondents will be drawn from three locations:

1. Practitioners or others who download The Heart Truth materials from the OWH website are asked at that time if they would be willing to participate in a follow-up survey. If they respond ‘yes’ the contractor will work with these practitioners to understand when they will be presenting the information. The follow-up will occur 2 to 4 months following that date, the contractor will forward an e-mail to the practitioner to forward on to their audience members. The e-mail will contain a link to a survey that will ask the outcome questions of interest. Because the practitioner will be forwarding the e-mail to the audience, the contractor will not offer any confidentiality pledges.

All audience members of a given practitioner will be given the same access code in order to analyze data by practitioner to understand if there are outcome differences between presentation styles or presentation manners (i.e. grand rounds vs. classroom).

2. Participating CoEs and CCOEs will be asked about their dissemination efforts. For those CoEs and CCOEs who intend to disseminate via classroom, professors/practitioners will be asked for their assistance with student follow-up. For those who intend to disseminate materials through grand rounds, lecturers/practitioners will be asked for assistance in sending e-mails to those who attended the grand round.

The follow-up will proceed as in #1 above.

3. MEDSCAPE is a continuing medical education provider. Practitioners can go on-line and get continuing credit for completing modules on various medical subjects. MEDSCAPE will have 5 The Heart Truth modules which will enable users to earn Continuing Medical Education/Continuing Education units (CME/CEU) at no cost through the University of Wisconsin. At the completion of the module. Practitioners will be told that Gallup will follow up with a survey in the future. The contractor will be forwarded their name and e-mail address. The follow-up will be to send those respondents a link to the survey directly.

This is a convenience sample. The contractor will continue to collect responses from the three groups until 400 responses are obtained as outlined below. The only response rate to be calculated will be the response rate of the number of requests sent out and the number of returns received by the group of exposed professionals. The contractor will look at response rates by time frames. Specifically, the analysis will examine if there was a difference in response from potential respondents two, three, or four months out from viewing the Heart Truth materials (see potential respondent chart below).

POTENTIAL RESPONDENT CHART

SAMPLE FRAME NUMBER OF RESPONDENTS

MEDSCAPE (CME provider)	200
Presentations	150
Direct from website	50

2. Describe procedures for collecting information, including statistical methodology for stratification and sample selection, estimation procedures, degree of accuracy needed, and less than annual periodic data cycles.

Statistical methodology for stratification and sample selection: There is no stratification of the data. There is no “selection” of sample either. The contractor will attempt to contact anyone who completed the assessment of The Heart Truth materials and is contacted 2-4 months after being exposed to the materials. However, it is hoped that the evaluation will be able to get responses from at least 400 exposed professionals to complete this evaluation process.

Estimation procedures: There is no estimation in this process as the population of exposed professionals is unknown.

Degree of accuracy: There is no degree of accuracy with this survey as the population of exposed professionals is unknown.

Unusual problems requiring specialized sampling: Sampling for this survey requires permissions at two steps of the dissemination process: The process requires that The Heart Truth materials be disseminated to organizations that will host presentations and that the materials then presented to an audience of professionals. This survey is focused on the professionals who will be presented the materials in the second step. However, a representative of the host organization must first agree to participate in the study (the practitioner). Although we will know the population that decides to download the materials from the website, and know which CoEs and CCOEs obtained the materials, it is unknown who actually then receives the materials in a lecture. Because the universe of exposed professionals is unknown, no statistical sampling will occur. Those students/practitioners who participate in the MEDSCAPE module will be contacted.

Type I and Type II error: Statistical testing of this type is not permitted for this type of data collection as the population is unknown.

Use of periodic data collection: This is a one time data collection effort.

3. Methods to Maximize Response Rates

The survey will use several methods to get as many respondents as possible (response rates are not the focus here, rather getting 400 completes is the focus).

• All participants who completed the CME assessment will be contacted within two weeks to set up a process to again contact them 2-4 months later to complete the survey.

• OWH hosted a National Heart Truth dissemination conference with representatives from the CoEs and CCOEs. At the conference, we formally asked the CoEs and CCOEs to participate in the study. The contractor will follow-up with each CCOE and CoE to get their information on who is presenting the materials and set up a process for including them.

• Every downloader will be sent the survey to follow up with their exposed practitioners within 2-4 months.

• Therefore, each of the three potential inputs into the data collection process will be touched at least once before the actual survey is sent out.

4. Test of Procedures or Methods

The only testing of procedures will be between those who are exposed to the materials through the OWH affiliated CCOEs or CoEs compared to those who are exposed to the materials via downloading from the website (i.e. no specific affiliation with OWH). No other tests of procedures or methods will be undertaken.

5. Contact Names

Dr. Alison Simon, Project Director, The Gallup Organization, 202-715-3030

Dr. Manas Chattopadhyay, Chief Statistician, The Gallup Organization, 202-715-3030

Dr. James Wells, Methodologist, The Gallup Organization, 202.715.3030

8