EvaluationofHIV Prevention

B. DESCRIPTION OF INFORMATION COLLECTION

B.1 Respondent Universe and Sampling Methods

There are four respondent types participating in this evaluation project. They are program directors, program staff, peer educators, and students (participants). No sampling methods are used for any respondent type. Instead a convenience sample will be used since data is being collected from all grantee staff and program participants. Moreover, program directors, program staff, and peer educators of funded minority institutions are predetermined based on awards made in 2007 by DHHS/OWH. Therefore, the entire universe of respondents for these groups consists of the program personnel at the minority institutions that received grants for this project.

The respondent universe for the students consists of those young women attending minority institutions who are recruited for and participate in the institution’s HIV prevention education program. It is expected that annually 660 young women will be recruited across the 12 participating minority institutions. It is expected that 90-100% of program participants (participating students) will participate in the pre and post-test data collection and 80% in the online follow-up data collection. This estimate is based on the experience of reported by the funded minority institutions on their success in using online surveys with students. The degree of success appears to be a function of student access to personal and student email accounts. Because of the sensitive nature of the information requested tracking of IP addresses will not be saved as students respond to the follow-up survey. In addition, GEARS, the contractor collecting follow-up data will not have access to identifying information on participants at any time during the data collection. Consequently, strategies that increase response rate that depend on knowing a respondent’s email and IP address and consistent contact will not be able to be used. Grantees will be able to send out “general” reminders to participants to complete their online follow-up survey.

B.2 Procedures for the Collection of Information

There are several data collection activities included in this project. Appendix J provides a description of the information collected, rationale and sources/references for each data collection used in this project. The following presents data collection procedures for the evaluation project:

1. Verify IRB approval from all institutions. GEARS will collect IRB approval letters from all institutions participating in the evaluation.

2. Obtain OMB clearance.

3. Finalize all forms with the OMB clearance number printed on forms.

4. The day OMB clearance is received, GEARS will send an email to each program to inform them that: 1) OMB clearance has been obtained and 2) that they will be receiving data collection forms and software via email express mail within the next three business days; and 3) their GEARS program coordinator will contact them to begin scheduling site visits.

5. Within two-three business days after receiving OMB clearance, GEARS will send each program copies of data collection forms and an electronic EXCEL spreadsheet for them to input data required for the evaluation.

6. Two weeks after receiving OMB clearance, begin training sessions for GEARS staff on details of the administration of the process evaluation interviews. This two-hour training will review the purpose of the process evaluation and how the interview fits within the evaluation framework. Each process interview question will be discussed and questions answered. This training will be led by this projects’ Evaluation and Data Manager, Dr. Deborah Brome, Vice President, GEARS. GEARS staff is trained in interviewing administration and have conducted interviewing as part of GEARS’ research and organizational development activities.

7. One month after OMB clearance begin conducting monthly conference call in order to monitor and answer questions about data collection procedures.

8. Approximately one to two months after OMB clearance, GEARS will attend the OWH Grantee Meeting and will train the grantee (minority institution) staff on data collection from students. Emphasis will be placed on the consenting process that has been approved by their IRB’s and utilizing the script developed for creating the unique identifier. (Appendix J)

9. Two months after OMB clearance is received, GEARS will schedule and conduct site visits to the funded minority institutions. Our goal is to complete all site visits within four months of obtaining OMB clearance. During this visit, GEARS staff will administer the initial intake form (process evaluation form with program directors, program staff and peer educators (Appendices C, D and E, respectively). These interviews will be conducted in person during the site visit. GEARS staff will also provide additional training on the Prevention Education Questionnaires for College Women with program staff if needed.

10. On a quarterly basis throughout the project period, respondents (program staff) will enter the data into Excel spreadsheet and transmit results to GEARS. Program staff will also use submitted data as part of their local evaluations. (Refer to Appendices G, H, and I.)

11. Within six months after clearance is obtained, follow-up emails will be sent to students three months post completion of the minority institution’s HIV prevention program. These emails will be sent to students by the program to email addresses provided by students. This email will ask students to participate in a follow-up survey and will provide an email address for them to access. Each minority institution will have a specific site. At this site students will sign in using a unique identifier and their IP addresses will not be saved. GEARS will download completed surveys on a monthly basis and will provide minority institutions with a copy of the data in EXCEL format.

Submission of Data

Data collected for this evaluation are designed as much as possible to fit within existing program data collection activities. GEARS Program Coordinators will make a site visit of all programs and conduct training on evaluation data collection forms and software after OMB clearance is obtained. MIS information as well as data markers required for the evaluation will be entered into a user friendly Excel database and electronically transmitted to GEARS on a quarterly basis.

Process evaluation data will be collected twice a year. These data will be collected by personal interview during the first part of the grant year and by telephone during the second half of the grant year by GEARS staff. GEARS staff will ask the respondent permission to audiotape the Process evaluation interview. Audio-taping is requested in order to ensure that the detailed information provided by the respondent in this interview is accurately captured in the evaluation data. Specifically, GEARS staff will check the accuracy of the recorded interview responses against the audio-taped interview. All audio-tapes will be kept in a locked filed in GEARS offices and will be destroyed at the conclusion of the evaluation project, when all data and data coding have been finalized.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

The follow-up, on-line survey is the only data collection that is not being collected as part of program activities. In order to increase response rates for the online survey, minority institution programs will email their participants three months post program completion and request their assistance in the evaluation. (See Appendix L for email.) They will provide a web link, indicate the ways in which the data collected will be used, and provide instructions on how to create their unique identifier. This email will only be sent twice, two weeks apart so that students do not feel harassed or pressured to participate in the data collection.

One hundred percent participation is expected from Minority Institutions in collecting process evaluation data. This data will be collected as part of the yearly site visit.

B.4 Test of Procedures or Methods to be Undertaken

The majority of the Instruments and items chosen for this evaluation have been selected from standardized instruments previously used by federal organizations, previously OMB approved evaluations, research groups, or academic institutions with either community-based populations or organizations and tested for their cultural and linguistic appropriateness.

Additionally, two sets of respondents (program directors and program staff) had opportunities to input on development of the instruments and submit copies of instruments they currently use in their local evaluations to guide development of instruments used in this project. Every effort was made to ensure that data collected for OWH was also data needed by grantees for their local evaluation. Items in the instruments have been reviewed by five minority college students to further assess culture and language appropriateness and to estimate length of time needed to complete instruments.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Program Development Contact

Adrienne M. Smith, PhD, MS, CHES

Public Health Advisor

U.S. Department of Health and Human Services

Office on Women's Health

202-690-5884

Adrienne.Smith@hhs.gov

Data Collection/Analysis and Statistical Contact

Deborah Brome, Ph.D.

Vice President and Director, Evaluation & Applied Research

Global Evaluation & Applied Research Solutions (GEARS Inc.)

617-328-5141

dbrome@getingears.com

Michael Milburn, Ph.D.

Professor of Psychology

University of Massachusetts, Boston

100 Morrissey Boulevard

Boston, Massachusetts

617-287-6386

michael.milburn@umb.edu

 

http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=105_cong_public_laws&docid=f:publ277.105.pdf

OMB Clearance Supporting Statement – DRAFT 7.1.08

Evaluation of OWH HIV Prevention Program at Minority Institutions Page 4