Monitoring Outcomes of the Enhanced Comprehensive HIV Prevention Planning (ECHPP) Project
0920-New
Supporting Statement A
Contact Information:
Project Officer:
Dr. Holly H. Fisher
Behavioral Scientist, Evaluation Studies Team,
Program Evaluation Branch
National Center for HIV, Hepatitis, STD, and TB Prevention
Center for Infectious Diseases
Centers for Disease Control and Prevention
1600 Clifton Rd, NE, MS E-59
Atlanta, Georgia 30333
Phone: (404) 639-1940
Fax: (404) 639-2029
E-mail: hfisher@cdc.gov
Date: December 9, 2011
Table of Contents
Section
A. Justification
Circumstances Making the Collection of Information Necessary
Purpose and Use of the Information Collection
Use of Improved Information Technology and Burden Reduction
Efforts to Identify Duplication and Use of Similar Information
Impact on Small Businesses or Other Small Entities
Consequences of Collecting the Information Less frequently
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
Explanation of Any Payment or Gift to Respondents
Assurance of Confidentiality Provided to Respondents
Justification for Sensitive Questions
Estimates of Annualized Burden Hours and Costs
Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
Annualized Cost to the Government
Explanation for Program Changes or Adjustments
Plans for Tabulation and Publication and Project Time Schedule
Reason(s) Display of OMB Expiration Date is Inappropriate
Exceptions to Certification for Paperwork Reduction Act Submissions
Table A-1-1 OMB Control Number and Expiration Date of Existing CDC
Data Sources for ECHPP Evaluation
Table A-1-2 2011 HHS Poverty Guidelines Income Limits
Table A-12-1 Estimate of Annualized Burden Hours
Table A-12-2 Estimate of Annualized Burden Costs
Table A-14-1 Estimate of Annualized Cost to the Federal Government
Table A-16-1 Time Schedule for ECHPP Activities With Respect to OMB Approval
ATTACHMENTS
Attachment 1 Authorizing Legislation
1a. Section 301 of the Public Health Service Act
1b. Section 308(d) of the Public Health Service Act
Attachment 2 60-day Federal Register Notice
Attachment 3 Data Collection Forms
3a. Eligibility Screener for Community Survey
3b. Community Survey
3c. Eligibility Screener for Clinic Survey
3d. Clinic Survey
Attachment 4 Information Sheet
Attachment 5 Consent Scripts
5a. Community Survey Consent Script
5b. Clinic Survey Consent Script
Attachment 6 Assurances of Confidentiality and Pledge Form
6a. Guidelines for HIV/AIDS Surveillance – Security
and Confidentiality
6b. Rules of Behavior for CDC Staff and Contractors
6c. Contractor Pledge Form for 308d Confidentiality
Attachment 7 Clinic Director and Venue Owner Scripts
7a. Clinic Director Script
7b. Venue Owner/Manager Script
7c. Clinic Staff Script – Provision of Patient Loads
7d. Clinic Staff Script – Approaching Clients
7e. Project Orientation
Attachment 9 Interviewer Scripts
9a. Community Survey Interviewer Script
9b. Clinic Survey Interviewer Script
Attachment 10 Consultant List
Supporting Statement
A. Justification
Circumstances Making the Collection of Information Necessary
The Centers for Disease Control and Prevention (CDC) requests approval for a new data collection called “Monitoring Outcomes of the Enhanced Comprehensive HIV Prevention Planning (ECHPP) Project”.
Background
In September 2010, twelve U.S. metropolitan statistical areas (MSAs) received CDC funding through a cooperative agreement, with their local/municipal health departments, to conduct the Enhanced Comprehensive HIV Prevention Planning (ECHPP) project.
ECHPP funding provides money to health departments to enhance their existing HIV prevention services and provide a best mix of evidence-based behavioral, biomedical, and structural interventions to have maximum impact on the HIV/AIDS epidemic. No data collection was associated with this activity and therefore, the project did not require OMB PRA review and approval.
According to the CDC HIV Surveillance Report, these twelve MSAs (Atlanta, GA; Baltimore, MD; Chicago, IL; Dallas, TX; District of Columbia; Houston, TX; Los Angeles, CA; Miami, FL; New York City, NY; Philadelphia, PA; San Francisco, CA; and San Juan, Puerto Rico) have the highest AIDS prevalence rates in the U.S. and represent 44% of all U.S. AIDS cases (CDC, 2010).
The Enhanced Comprehensive HIV Prevention Planning (ECHPP) Project is a 3-year project funded by CDC's Division of HIV/AIDS Prevention (DHAP) for the 12 municipalities with the highest number of people living with AIDS in the United States. The ECHPP project attempts to improve program planning and implementation to:
Reduce new HIV infections
Link people with HIV to care and treatment and improve health outcomes,
Reduce HIV-related health disparities, and
Achieve a more coordinated national response to the HIV epidemic in the United States.
Lessons learned from ECHPP will inform how CDC can best work with health departments, other US government agencies and communities to reach the National HIV/AIDS Strategy (NHAS)goals across the country.
The goals of ECHPP are consistent with the Division of HIV/AIDS Prevention Strategic Plan for HIV Prevention and with the National HIV/AIDS Strategy. The NHAS was announced by the White House and the Office of National AIDS Policy in 2010 (ONAP, 2010). ECHPP activities specifically target priority populations identified in the National HIV/AIDS Strategy with particular emphasis on gay and bisexual men and transgender persons, Black Americans, Latino Americans, and substance users.
CDC plans to monitor ECHPP implementation using data collection activities described in this Information Collection Request entitled “Monitoring Outcomes of the Enhanced Comprehensive Prevention Planning (ECHPP) Project”. Most of the data that will be used to monitor ECHPP outcomes will come from existing CDC data sources (e.g., case surveillance and supplemental surveillance systems that CDC uses routinely to monitor the HIV/AIDS epidemic nationally). This information collection request will address data gaps that exist for specific high-risk target populations in high-prevalence areas (i.e., where data for particular populations are not routinely collected by CDC). Data gathered via this information collection request will be analyzed together with data from existing sources as part of the overall ECHPP monitoring and evaluation plan. Additionally, workgroups have been formed with other federal agencies who are implementing NHAS-related projects; data sharing opportunities have been identified and, in some cases, MSA-level data that will inform ECHPP have already been obtained (e.g., HUD data).
Existing CDC data sources that will be used for the evaluation are listed below, along with relevant OMB control numbers and expiration dates.
Table A-1-1: OMB Control Number and Expiration Date of Existing CDC Data Sources for ECHPP Evaluation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A new data collection activity is requested through this ICR to assist with the ECHPP evaluation. This new data collection activity will occur in six of the twelve ECHPP MSAs (Houston, TX; District of Columbia; Los Angeles, CA; Miami, FL; New York City, NY; and San Francisco, CA). (Limited funding only allowed data collection to be funded in six cities.) These six cities were chosen based on rankings for number of people living with AIDS (i.e., AIDS burden in the community) and availability of data from other data sources that will be used in overall monitoring and evaluation activities. This new data collection activity is designed to monitor community-level outcomes of ECHPP and supplement existing HIV surveillance data already being collected via CDC data systems in these cities. Information about behavioral risk, access of HIV-related services and programs, and exposure to HIV prevention messages will be collected from three populations at increased risk for HIV infection or transmission:
Injection drug users
Heterosexuals at increased risk of HIV infection
HIV-positive individuals who access HIV medical care from clinics that provide HIV services
These three populations are also targeted through the following ongoing information collections: the National HIV Behavioral Surveillance System (NHBS, OMB Control #0920-0770, Expiration DATE 6/1/2014) (injection drug users and high-risk heterosexuals) in all twelve of these cities and the Medical Monitoring Project (MMP; OMB Control #0920-0740, Expiration DATE 5/31/2012) (HIV-positive individuals who access clinic care) in nine cities. However, additional data must be collected to evaluate ECHPP for three reasons. First, NHBS data are collected in three-year cycles whereby, each year, data from only one of three risk groups is collected (injection drug users, heterosexuals at increased risk of HIV infection, or men who have sex with men). NHBS data for injection drug users and high-risk heterosexuals will not be available during the time frame necessary to monitor ECHPP outcomes in these two populations in any of the twelve MSAs. Thus, a new data collection targeting these groups is needed. Second, MMP sets goals for data collection at the state level (i.e., MMP staff aim to interview a specific number of HIV-positive people for an entire state during a particular cycle). Sample sizes at the level of the MSA are typically very low (less than 100 people) and, thus, additional interviews among HIV-positive people are needed to monitor and evaluate the impact of ECHPP at the community-level.
Third, data from populations at risk for HIV infection and transmission must be collected at two points in time, at the beginning (in winter 2012, after OMB approval) and end (2013) of the ECHPP time period, to determine whether any community-level changes occurred in risk behavior, uptake of services and programs, and exposure to HIV prevention messages. Synthesizing information from this new data collection activity with information from existing CDC data sources will allow CDC staff to assess the contribution of ECHPP in these communities and make a broad statement about its contribution to outcomes in these high prevalence MSAs. While no longitudinal data are collected that will allow client-level change over time to be monitored, analyses involving community-level trends in service access and risk behaviors over time will be used to understand the ways in which the ECHPP project was associated with any positive outcomes.
The CDC awarded a contract in September 2010 to the contractor, SciMetrika, to lead this survey data collection, once OMB approval has been granted. The first year of this contract has been spent planning the data collection and analysis activities that are scheduled to begin in winter 2012. The contractor will also produce a final ECHPP evaluation report in 2015 that interprets findings from all data obtained from existing and new data collections. Surveys will be administered by SciMetrika staff in the field at pre-specified venues using electronic handheld devices. The data transfer methodology is compliant with the guidelines set forth in OMB memorandum M-04-04 (E-Authentication Guidance for Federal Agencies) as well as with OMB, HHS, and CDC Certification and Accreditation Guidelines outlined in NIST SP 800-37 (Guide for Applying the Risk Management Framework to Federal Information Systems; A Security Life Cycle Approach). In addition to the technical requirements listed above, data management processes are in compliance with The Guidelines for HIV/AIDS Surveillance – Security and Confidentiality (Attachment 6a).
More than 25 years into the HIV epidemic, there remains a critical need to understand HIV related risk behaviors and the reach of prevention to groups at high risk. The rate of new HIV infections continues to be high: an estimated 56,300 Americans become infected with HIV each year. In order to target HIV prevention programs to populations most affected by HIV, CDC must continue to monitor the front line of the epidemic and assess the extent to which national-level projects such as ECHPP have an impact on local HIV/AIDS epidemics.
Privacy Impact Assessment
CDC will not receive any personally identifiable information from respondents. Name, phone number, and e-mail address may be collected from respondents for the purposes of scheduling an interview. However, this information will only be stored on paper, it will not be submitted to CDC, and it will be destroyed after that person’s survey is complete. No personally identifiable information will be stored in the data collection system, including respondent contact information, date of birth, or information regarding the consent process.
Overview of the Data Collection System
Data will be collected in six MSAs: Houston, TX; District of Columbia; Los Angeles, CA; Miami, FL; New York City, NY; and San Francisco, CA. These MSAs are among the top nine U.S. cities with the highest AIDS prevalence in 2007. The contractor SciMetrika has been funded for five years (October 2010 to October 2015) to collect and analyze these data (in addition to other activities related to the evaluation such as report writing). During the first year of the contract, there was no data collection. The first year of the contract was spent planning data collection activities and data analysis. Data collection is anticipated to begin in winter 2012, upon OMB approval.
Data collection will begin after OMB approval has been received during two specific time periods: at the beginning of the ECHPP implementation period (in 2012, after OMB approval) and at the end of the ECHPP implementation period (2013). Each data collection period will last up to 12 months. For each data collection period, data will be collected from three populations at risk for HIV infection or transmission: injection drug users, heterosexuals at increased risk, and HIV-positive individuals seeking HIV medical care. Injection drug users and high-risk heterosexuals will complete the community survey; HIV-positive individuals who are seeking HIV medical care will complete the clinic survey. All data will be collected via venue-based, convenience sampling through interviewer-administered, face-to-face, computer-assisted behavioral assessment.
For each person recruited to participate, a short, computer-based, eligibility screener will be administered by an interviewer to assess eligibility and collect limited demographic information (Attachments 3a and 3c). If the respondent is eligible for the survey and consents to an interview, the interviewer will administer the appropriate survey (Attachments 3b and 3d). (Eligibility for a specific risk population is described in the next section.)
Data will be collected by trained SciMetrika personnel through face-to-face interaction with participants. An interviewer script (Attachments 9a and 9b) will be used when interviewers first approach potential participants to tell them about the project. Individuals who are interested in participating will be screened for eligibility (Attachments 3a and 3c). Verbal consent will be obtained from eligible participants and recorded in the survey. The consent process will involve the interviewer reading a consent form script to the potential respondent (Attachments 5a and 5b) and documenting whether consent was provided by the individual (yes or no) on the survey. All instruments are interviewer-administered. Interviewers will collect the screener and survey data using a software application loaded onto handheld computerized devices.
Response data will be encrypted and the electronic devices used for data collection (e.g., iPad or PalmPilot) will be password-protected so that unauthorized users will be unable to view, export, or modify collected data. Survey data will be stored at SciMetrika for the duration of the contract (until 2015). At the end of the contract, all final data sets will be submitted to CDC. No names or phone numbers will be collected or recorded on the survey.
Venues and Clinical Facilities: SciMetrika will establish and coordinate a process to identify and assess the location for recruiting and administering the community and clinic surveys in consultation with local health department staff, community advisors, and the local clinic staff. Selection of venues (for the community survey) and clinics (for the clinic survey) will include a standard assessment process and application of specific criteria to identify appropriate locations in each city.
Venues: Identifying local health department staff and community consultants who are involved with the local HIV Prevention Community Planning Group will be critical to successful venue identification and selection. SciMetrika will also consult with NHBS staff (OMB Control #0920-0770, EXp. DATE 6/1/2014) to identify and prioritize potential venues for data collection.
Venues eligible for consideration during recruitment of injection drug user and high-heterosexual participants will include bars, dance clubs, retail businesses, cafes and restaurants, health clubs, social and religious organizations, adult bookstores, high-traffic street locations, parks, and beaches. Survey administration will take place in these same venues.
To recruit participants for the community survey, SciMetrika staff will visit the types of venues listed above, as advised by local health department staff and community advisors. Interviewers will use a convenience, venue-based sampling method where community surveys are administered to any eligible person who is willing to participate while the interviewer is visiting the venue. The SciMetrika interviewer will collaborate with local health department staff and community advisors to determine the best strategy for approaching individuals about the project. Strategies may vary across venues. Interviewers will bring copies of an information sheet (Attachment 4) that describes the project to the venue in case individuals who are approached decline to take the survey at that time but indicate possible interest in the future. The information sheet will contain a project description and a phone number/e-mail address where the interviewer can be reached. An interviewer script (Attachment 9a) will be used by all interviewers when approaching potential participants and describing the project. Oral consent will be obtained from potential respondents. The consent process will involve the interviewer reading a consent form script to the potential respondent (Attachment 5a) and documenting whether consent was provided by the individual (yes or no) on the survey.
The community survey will be administered in a private or semi-private area at the venue. The interviewer will prioritize the safety and privacy of the respondent when determining where to administer the survey. No added burden is placed on the venue for the space used to administer the survey.
Clinical Facilities: HIV care facilities will be identified using methods similar to those used for MMP (OMB Control #0920-0740, EXP. 5/31/2012). A clinical facility is defined as any clinic, health care institution, private or group physician practice that shares common medical records or a medical record system. Thus, a facility is defined in terms of medical record storage, not in terms of a physical location (address) or the names of individual practitioners. Clinic facilities that participate in the current MMP cycle, whereby data collection is taking place at the same time, will be excluded where possible.
The availability of a private area for interviewing will be a required criterion for participating clinics. Alternate locations for conducting the clinic survey will not be needed. No added burden is placed on the clinical facilities for the space used to administer the survey.
Candidate clinic facilities are those that provide HIV care. An HIV care facility is operationally defined as a facility conducting CD4 or HIV viral load testing or providing prescriptions for antiretroviral medications in the context of treating and managing a patient’s HIV disease. Thus, facilities providing HIV care could include outpatient facilities such as hospital-affiliated clinics, free-standing clinics or private physician offices.
The following facility types will not be selected for this project:
Facilities that do not provide medical care (e.g., sites that only conduct HIV counseling and testing)
Facilities where medical providers obtain CD4 counts and HIV viral loads only for referral purposes
Facilities where medical providers only provide antiretroviral refill prescriptions and do not play an active role in managing their patients’ HIV infection
Facilities that provide exclusively inpatient care, including hospices
Emergency departments
Facilities located outside the funded jurisdiction
Federal, state and local correctional and work-release facilities
Tribal facilities
Health facilities located on military installations
Facilities that only provide HIV care only to patients under the age of 18
Veterans Administration (VA) facilities in every MSA will be considered in the facilities selection process.
The purpose of this data collection effort is to monitor the outcomes in ECHPP jurisdictions following implementation of enhanced HIV prevention planning by health departments.
The method used to approach and recruit HIV-positive individuals in clinics will vary by facility based on clinic provider consultations. Some patients may be approached by clinic staff during one of three time points during their medical visit: at check-in, at triage, and during the office visit with the medical provider. Information sheets (Attachment 4) will be distributed by clinic staff within the clinics and they will also be available in the waiting rooms with the permission of clinic staff. This sheet will contain project information and a phone number/e-mail address for the interviewer.
When SciMetrika interviewers first approach potential participants, they will use an interviewer script (Attachment 9b) to describe the project. Oral consent will be obtained from potential respondents. The consent process will involve the interviewer reading a consent form script to the potential respondent (Attachment 5b) and documenting whether consent was provided by the individual (yes or no) on the survey. SciMetrika will consult with clinic staff to determine the best strategy for approaching individuals and each clinic recruitment strategy will be tailored based on staff recommendations.
SciMetrika staff will also establish a schedule whereby they are available on specific days of the week to be onsite at the facility to administer surveys. The schedule will be published in the information sheet and clinic staff will also be informed of the schedule. Where possible, interviews will be conducted in the clinic after the patient finishes with his/her appointment. Interested individuals who would like to participate but would prefer to do so on a different day will be encouraged to return to the facility clinic during the regularly-scheduled SciMetrika times/days.
It is expected that clients will be surveyed upon arrival for services (clinic survey) and when attending specific venues (community survey). However, contact information may be collected if a potential participant selects to participate in the survey at another time. When it is necessary to collect this information, the information will only be used by SciMetrika staff for scheduling purposes. The contact information will be kept separately from all interview materials and survey data. Potential participants will be encouraged to use pseudonyms or first names only when scheduling a day/time to take the survey.
Items of information to be collected
Eligibility Screeners: The following data will be collected in both the community and clinic eligibility screeners: age; race/ethnicity; previous participation in community, clinic, MMP, or NHBS surveys; county of residence; and gender.
The eligibility screener for the community survey (Attachment 3a) is based on criteria used with NHBS, and will include questions about injection drug use (lifetime), vaginal and anal sex with person of opposite gender (previous 12 months), highest level of education, employment status, household income (previous 12 months), and number of people living on this household income. Respondents will be classified as injection drug users if they indicate they injected drugs in the past 12 months (regardless of responses to other eligibility questions). Respondents will be classified as high-risk heterosexual individuals if they indicate they have never injected drugs, they have had sex with a partner of the opposite sex in the previous 12 months, and their answers to the education and income questions qualify them as low socioeconomic status according to the current HHS poverty guidelines (CDC, 2011). These guidelines are updated periodically in the Federal Register by the U.S. Department of Health and Human Services (http://aspe.hhs.gov/poverty/11poverty.shtml). To qualify as low socioeconomic status, an individual’s highest education level must be high school or less, or his/her income must be based on the 2011 HHS Poverty Guidelines (DiNenno et al, in press).
Table A-1-2: 2011 HHS Poverty Guidelines Income Limits
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SOURCE: Federal Register, Vol. 76, No. 13, January 20, 2011, pp. 3637-3638
The eligibility screener for the clinic survey (Attachment 3c) will include a question regarding self-reported HIV status.
The community survey collects data on demographics, sexual behavior, alcohol and drug use history, HIV testing experiences, exposure to HIV prevention messages, and participation in HIV prevention activities (Attachment 3b).
The clinic survey collects data on demographics, HIV care and testing experiences, sources of care, met and unmet needs, HIV treatment and adherence, sexual behavior, alcohol and drug use history, exposure to HIV prevention messages, participation in HIV prevention activities, gynecological and reproductive history, health conditions, preventative therapy, and employment and productivity (Attachment 3d).
No information in identifiable form will be collected for either survey and, thus, individuals cannot be directly or indirectly identified through the survey data.
Data collected through these surveys will be stored and accessed by a survey identification number at SciMetrika. The sensitive information collected will not be linked to any personally identifiable information and cannot be used to reveal the identity of any person. There is no link to any name, either locally or at CDC, and data will not be collected on paper forms.
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age
There will be no websites or internet content directed at children under the age of 13.
Purpose and Use of Information Collection
This data collection activity will collect information about behavioral risk, access of HIV-related testing and services, and exposure to HIV prevention messages from injection drug users, heterosexuals at increased risk of HIV infection, and HIV-positive individuals accessing HIV medical care from clinics that provide HIV services.
The primary objectives of this information collection are to:
1) Monitor community-level outcomes of ECHPP in selected cities `
2) Supplement existing HIV surveillance data already being collected in these cities
A large and geographically diverse sample will provide an important data source for evaluating progress towards national public health objectives, such as the Healthy People 2020 objectives to: Reduce the number of new AIDS cases among adolescents and adults who inject drugs (HIV-7); Increase the proportion of sexually active persons who use condoms (HIV-17); and Increase the proportion of people living with HIV who know their serostatus (HIV-13).
ECHPP outcome monitoring data from this information collection will be useful for documenting the need for prevention resources and the reach of prevention programs targeting persons at highest risk in these MSAs. Data on utilization patterns of prevention resources is critical to determine resource requirements for future funding cycles for prevention programs. ECHPP outcome monitoring data will also be used to answer questions about prevention service reach, gaps, and impact of allocated resources in these MSAs.
At the local level, ECHPP outcome monitoring data may be used for local HIV prevention program planning, including the development of local epidemiologic profiles and responding to data requests. Information about access to and use of HIV prevention services, and also services specifically targeted to people living with HIV/AIDS, can be used to evaluate local prevention services for at-risk subpopulations in these communities.
As this is a program evaluation activity, there are limits to the generalizability of findings from this data collection.
Privacy Impact Assessment Information
The eligibility screener is necessary to ensure that respondents meet minimum criteria for participation in the data collection, including residency in the MSA and age 18 years or older and behavioral and clinical eligibility criteria specific to the surveys. For the community survey, the screener includes questions about behavior to determine which risk group is most appropriate to the participant. Such a screener is necessary in order to ensure that current injection drug users are being interviewed and, also, that sexually active heterosexuals are being interviewed. For the clinic survey, it will be necessary to ask about HIV status to ensure only HIV-positive individuals participate in the survey.
None of the data collected in the surveys will be linked to personally identifiable information. Therefore ECHPP data cannot be used to reveal the identity of any one person.
Both surveys include questions that involve the respondent’s sexual and drug use behaviors that may increase the risk for acquisition or transmission of HIV. Questions about HIV testing and medical care for HIV-positive respondents are also included in both surveys. Although the information requested is sensitive, the goals of this project cannot be accomplished without this collection. Participants will be informed that they may decline to participate without penalty at any time. They will also be told that if they agree to participate, they may refuse to answer any question later on. They will also be informed that the data will be used to improve HIV prevention services for HIV-positive persons and persons at increased risk of HIV in their area, and that aggregated data may be released in published reports.
No information that could directly identify an individual will be collected in the eligibility screeners or surveys. When necessary to collect contact information, for example, if an individual would like to schedule the survey for a different time or day, contact information will only be recorded on paper and will be not be submitted to CDC. This information will be destroyed after that individual has completed the survey.
CDC will collaborate with SciMetrika to identify and implement methods used to protect the security and confidentiality of the information collected. Interviewers will also receive training from SciMetrika management staff regarding how to protect the information collected. All SciMetrika staff including interviewers will complete security and confidentiality training and sign a statement indicating their understanding of security and confidentiality policies (Attachment 6c). SciMetrika staff has been trained on CDC data security and confidentiality guidelines. All field staff who will have access to the survey data will sign the contractor’s non-disclosure agreements and rules of behavior.
Several safety precautions are in place to prevent information from being accessed or potentially connected to a respondent. Data collection is designed so that no data will be stored directly on the electronic collection device and all data is transmitted to a secure website. The web-based software used for the survey supports the ability to encrypt response data and password-protect surveys so that unauthorized users will be unable to view, export, or modify collected data. All interviewers will be trained on confidentiality and data security, password protection, encryption of devices, and controlling access to hardware.
Use of Improved Information Technology and Burden Reduction
Interview data will be collected on password-protected encrypted handheld computers using an electronic data collection device (e.g., iPad or Palm Pilot). Data collection is designed so that no data will be stored directly on the electronic collection device and all data will be transmitted via a secure, 128bit encrypted connection to a secure website. The Web-based software supports the ability to encrypt survey data, and password-protect the surveys, so that unauthorized users will be unable to view, export, or modify collected data. Additional data security will be maintained through training, password protection, encryption of devices, and controlling access to hardware. It is expected that 100% of interviews will be collected using electronic applications. All interviews will be conducted by trained local SciMetrika staff. SciMetrika is licensed to use the software and has previous experience with such data collection systems in the field.
The use of electronic, handheld, interview devices in this project has several advantages over paper data collection methods. First, interview time is typically reduced when an electronic survey is used. Second, skip patterns and error loops programmed into the electronic survey result in fewer inconsistencies, missing data, and errors. Also, the need for data cleaning associated with data entry on paper forms is removed which means there is a reduction in the time between the last interview and the production of a final analysis dataset. Finally, the cost of data collection using handheld devices instead of paper data collection forms is also reduced despite the increased start-up costs associated with purchasing the handheld devices and interview software. CDC/DHAP has implemented the use of handheld devices for other national surveillance systems (e.g., NHBS and MMP).
Computer-assisted personal interviews conducted by an interviewer also reduce burden for the respondent because comprehension may be improved (compared with a self-administered questionnaire) which may improve response time. The computer “assists” by customizing the question wording for each respondent (based on previous answers), allowing the interviewer to focus on explaining complex terms or definitions, giving instructions, and ensuring that answers are relevant and entered accurately.
SciMetrika will conduct centralized and site-specific training and site visits with local survey interviewers in each MSA to provide instructions and technical assistance on how to use the survey software, administer the surveys, and submit the data to a centralized database. CDC will provide technical assistance, as needed, to SciMetrika during the training of data collection specialists and the data collection process, including participating in site visits to the MSAs.
CDC will also provide assistance as needed to SciMetrika in the development of standardized and site-specific protocols involving participant recruitment, data collection, and data management. CDC will require that SciMetrika staff provide supervision and monitor interviewers regularly.
Training of project interviewers to assess interviewing skills and data collection procedures will be conducted, and regular on-going assessment of interviewers and collected data will be done to assure high quality execution of process and activities.
SciMetrika will convene lessons-learned meetings with data collection specialists to identify and resolve the problems that can occur with the software and hardware that is used for conducting the interviews. All problems and challenges experienced during data collection will be documented and resolved appropriately, with CDC collaboration as needed.
4. Efforts to Identify Duplication and Use of Similar Information
Within CDC, there are some complementary systems already in place that contain similar data elements to this information collection request:
National HIV Behavioral Surveillance System (NHBS) (OMB # 0920-0770, exp. 6/1/2014)
Medical Monitoring Project (MMP) (OMB #0920-0740, exp. 5/31/2012)
However, there is information unique to the ECHPP community and clinic surveys that are necessary to evaluate outcomes of ECHPP. Additionally, although some data collected from NHBS and MMP in the six MSAs may be used in the ECHPP evaluation, more data are needed to increase the volume of data required for analyses, specifically with regard to injection drug users (IDU)and high-risk heterosexuals HET populations. Furthermore, the existing NHBS and MMP data collections cannot be modified to satisfy the needs of the proposed project..
We reviewed other currently-funded programs and did not identify potential areas of duplication.
5. Impact on Small Businesses or Other Small Entities
The impact on small businesses in this data collection activity will be minimal. Initially, clinic directors, including providers that are small businesses, will be contacted to find out if they are interested in participating in the clinic survey data collection. A script will be used by field staff when initially contacting clinic directors (via e-mail) and explaining the project (Attachment 7a). It is estimated that this initial contact will consist of the clinic director reading the email and providing consent for the clinic to participate.
For clinics who agree to participate, a brief meeting will be scheduled with an identified clinic staff member for orientation to the project (Attachment 7e). The SciMetrika field supervisor will describe the project and clinic staff involvement in the project. Clinic staff will be asked to provide estimated patient loads on a routine basis to SciMetrika so interviewers can determine when to visit the clinic. The SciMetrika field supervisor and clinic contact will discuss suitable private areas within the facility where interviewing may take place. The SciMetrika field supervisor will also describe how clinic staff should approach potential participants who visit the clinic. Finally, the SciMetrika field supervisor and clinic contact will discuss strategies for preferred communication throughout data collection (e.g., e-mail, phone) and the SciMetrika field supervisor will answer any questions or concerns that the clinic contact may have. This orientation is estimated to take up to 30 minutes.
During data collection, participating providers will be asked to provide estimated patient loads on a routine basis to SciMetrika so interviewers can plan their interview schedules. This will involve the identified clinic contact communicating the clinic’s estimated patient load (via email or phone) to the SciMetrika field supervisor on a weekly basis (Attachment 7c). It is estimated that provision of patient loads will take approximately 5 minutes.
It is also estimated that it will take clinic staff an average of 5 minutes per client to: mention the project during the client visit (e.g., at check-in, during triage, and/or during their appointment) and provide the client with an information sheet (Attachment 4) about the survey (Attachment 7d). The amount of time spent per client, by clinic staff, will be the same at all clinics, regardless of facility size. Scheduling and administration of the survey will be coordinated between the client and SciMetrika field staff, without the involvement or coordination of clinic staff.
Regarding the community survey data collection, there is minimal time burden on the owners of commercial/private venues beyond getting their permission to recruit individuals who visit the venue to participate in the project. Commercial/private venue owners and managers will be contacted in-person by field staff supervisors at their venues during normal operating hours. SciMetrika field supervisors will describe the project to the venue owners/managers and request permission for SciMetrika staff to approach venue patrons to ask them if they are interested in participating in the community survey. Verbal confirmation from venue owners/managers will be sufficient for SciMetrika to recruit in a given venue. Venue owners/managers may communicate preferences regarding SciMetrika’s presence in their venue if desired (e.g., optimal recruitment times, space usage within the venue) which will inform future field staff actions. A script will be used by field staff when contacting venue owners/managers and explaining the project (Attachment 7b). Consent will be requested to administer the survey to venue patrons. Following initial communication with venue owners/managers, no other engagement involving them is required.
Consequences of Collecting the Information Less Frequently
ECHPP outcome monitoring data collection activities will occur during two time periods. The first time period will last up to 12 months and will begin in winter 2012, after OMB approval has been received. The second data collection time period will last up to 12 months and will take place from 2013 to 2014. The purpose of the data collection activities is to assess if any community-level changes occurred in risk behavior, uptake of services and programs, and exposure to HIV prevention messages during the ECHPP implementation period. Both the community and clinic surveys will be administered during both time periods.
As described in Section A.1 (page 6), this information collection is necessary to monitor outcomes among high-risk populations in ECHPP areas that are not currently monitored through surveillance systems (during the same time period). Also, in order to assess the community-level contribution of ECHPP over time, it will be necessary to collect data from the target populations at the beginning and again at the end of the ECHPP project period (via serial, cross-sectional surveys). If the data collection time periods, or reporting frequency, were reduced, we would not have enough information necessary to assess the community-level impact of ECHPP programs in these cities among all target populations (e.g., we would not have data from all high-risk populations). Furthermore, we would not be able to evaluate changes in behavioral and clinical outcomes in the community by comparing data collected early in the ECHPP implementation period to data collected at the end of the ECHPP implementation period. A pre/post comparison is necessary to assess community-level changes over time.
There are no legal obstacles to reduce the burden.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request fully complies with the regulation 5 CFR 1320.5
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A 60-day notice to solicit public comments was published in the Federal Register on 04/04/2011 (Volume 76, Number 64, pages 18554-18555). The Federal Register notice has been included in this package (Attachment 2). One non-substantive comment was received on April 4, 2011 in response to the notice. Due to the nature of the comment, no action was taken.
No persons outside the agency were consulted to obtain their views on the availability of data, frequency of collection, the clarity of instructions and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported. Local NHBS and MMP staff (in the ECHPP cities) were consulted when determining where data collection should occur and what type and amount of tokens of appreciation should be used. All names, affiliations, and contact information for these consultants are provided in Attachment 10.
Explanation of any Payment or Gift to Respondents
The community survey will take approximately 25 minutes to complete, the clinic survey will take approximately 40 minutes to complete. Participants will receive $25 (community survey) or $40 (clinic survey) for their interview. The exact amount for survey incentives was determined after discussions with local NHBS and MMP staff and community stakeholders. Within a city, all community survey participants will receive the same token amount and all clinic survey participants will receive the same amount. Tokens will be in the form of VISA gift cards, gift certificates, bus or subway tokens, or cash equivalents. Due to ethnical considerations associated with providing cash to self-identified injection drug users, SciMetrika will only offer gift cards to IDU participants. The tracking system for remuneration will be kept separate from the actual survey data.
Facilities that agree to participate in the clinic data collection will be given tokens of appreciation of $25 (for each 12-month data collection period). Facility staff will be asked to help recruit participants, for example, by distributing information sheets that describe the project or by verbally mentioning the project to clients. Data collection will occur in designated areas in the facility itself. Privately-owned or commercial venues (e.g., bars, clubs, restaurants, gyms, Laundromats) that are used for project recruitment and data collection activities of the community survey will be also be given tokens of appreciation for $25 (for each 12-month data collection period).
Tokens of appreciation have been used in other similar CDC data collection efforts that seek to conduct surveys with hard-to-reach and highly selective populations, to ask them highly sensitive questions about issues such as sexual behavior and substance use, and that have a similar length of time for completing the behavioral assessment (NHBS, OMB 0920-0770, exp. 6/14/2014; MMP, OMB 0920-0740, exp. 5/31/2012; Transgender HIV Behavioral Survey, OMB 0920-0794 exp. 12/31/2010). Studies have found that tokens of appreciation can improve response rates in mail, telephone, and face-to-face surveys (Singer et al., 1999; Whiteman et al. 2003). Without incentives, it is likely that participation in the community and clinic surveys would be reduced (McKnight, 2006; Stueve, 2001; Valleroy, 2000).
Assurance of Confidentiality Provided to Respondents
Certificates or Assurance of Confidentiality do not apply for this project.
These surveys do not collect name, social security number, or other personally identifying information. Only oral consent will be obtained from potential respondents. Data collected through this information collection request, both locally and at CDC, are stored and accessed by a survey identification number. Other data collected, while sensitive, are not personally identifying. Survey questions are listed in (Attachments 3b and 3d).
IRB Approval
It has been determined that this activity is not human subjects research and that the primary intent is to evaluate a public health program. Data collected through this project will be used for program monitoring and evaluation purposes only. Because this project does not involve human subjects research, the protocol will not be reviewed by CDC’s IRB.
Privacy Impact Assessment Information
This submission has been reviewed by ICRO, who determined that the Privacy Act does not apply. Individually identifiable information (IIF) will not be collected on the surveys. If a potential participant selects to participate in the survey at another time and provides personal contact information for scheduling purposes, they will be told that any individually identifiable information collected for scheduling purposes will not be submitted to CDC.
The Health Insurance Portability and Accountability Act (HIPAA) regulates how covered entities (including most health care delivery organizations) use and disclose certain individually identifiable information called protected health information (PHI). None of the data collected in the surveys will be linked to personally identifiable information. Therefore, the “Monitoring Outcomes of the Enhanced Comprehensive HIV Prevention Planning (ECHPP) Project”, data cannot be used to reveal the identity of any one person. The surveys will be administered by trained SciMetrika staff in a private location where the questions and responses cannot be overhead by others.
B. Describe how information will be secured, addressing relevant technical, physical, and administrative safeguards.
ECHPP monitoring outcome data will be transmitted to a secure web-based SciMetrika server. Encryption security for all data must meet the current National Institute of Standards and Technology (NIST) Federal Information Processing Standards (FIPS), which meet or exceed Advanced Encryption Standards (AES). See the document “Technical Guidance for HIV/AIDS surveillance Programs, Volume III: Security and Confidentiality Guidelines” for further information (www.cdc.gov/hiv/surveillance.htm).
The handheld devices used in this project are solely used for the data collection activity and no other activities. Project data will at no time be stored on the hardware device. A web-based application will be used to transfer data in real-time (page-by-page) to a secure, central database. The handheld device will be protected by using a coded password only known by authorized SciMetrika staff.
Data collection is designed so that no data will be stored directly on the electronic collection device and all data is transmitted via a secure 128bit encrypted connection to the secure website. The Web-based software used for provider survey supports the ability to encrypt response data and password-protect surveys so that unauthorized users will be unable to view, export, or modify collected data. Additional data security will be maintained through training, password protection, encryption of devices, and controlling access to hardware.
The Apple iPad device, one of the devices that will be used, provides a source of security due to the capabilities it provides for prohibiting access to hardware and any data residing on the device. Security capabilities that SciMetrika will employ to secure data include:
Lock-down of the device to permit access to project specific certain applications
Strict password rules and enforcement for accessing the device, application, and web-server
Auto-lock of the device if it remains inactive for a set period of time, requiring user to log back into encoding hardware-based encryption.
Ability to activate a remote “device wipe” by IT administrators to remove project information if device is lost or stolen.
Configuration of device to automatically “wipe” its data after a certain number of failed password attempts.
In the event that an Internet connection is not available, the survey administrator will complete the survey manually using a paper copy of the survey instrument. The paper copies that will be administered if an Internet connection is not available are provided in Attachment 3b (community survey) and Attachment 3d (clinic survey). The completed paper survey will be secured temporarily in a locked satchel that will be transported back to the SciMetrika field office by the next work day close of business. If Internet service is not re-established, no further paper surveys will be administered that day. Data from the paper survey will be entered manually into the Web survey application within 24 hours.
The data collection devices will be kept with staff at all times while in the field. The devices will be collected and secured by the field supervisor after the last interview each day. When not in use in the field, the devices will be kept in a locked cabinet in a locked office to which only the field supervisor has access.
CDC’s Procurement and Grants Office requires the inclusion of 308(d) clauses in any HIV/AIDS support services work done by contractors (e.g., data analysis, computer programming, LAN support). All CDC permanent employees and their contractors will be required to attend annual confidentiality training, to sign a nondisclosure agreement, and to update their confidentiality agreements annually. Contractors must sign a “Contractor’s Pledge of Confidentiality” (Attachment 6c). Access to HIV/AIDS data maintained at CDC is restricted to authorized personnel who have signed the contractor’s pledge and received the appropriate training.
C. Describe opportunities for obtaining respondent consent, if any.
Participation in this project is strictly voluntary.
The informed consent process for respondents will be fulfilled by obtaining oral consent. The Community Survey Interviewer Script (Attachment 9a) and Clinic Survey Interviewer Script (Attachment 9b) will be used by all interviewers to inform potential respondents of project prior to obtaining oral consent. All sites must obtain oral consent from respondents and document it in the data collection form on the handheld computer. The surveys will only be administered to eligible individuals who provide oral consent. Respondents will be informed that data collected from them will be kept secure and that the data will be reported in aggregate format.
D. Indicate whether respondents are informed about the voluntary or mandatory nature of their response.
Participation in the “Monitoring Outcomes of the Enhanced Comprehensive Prevention Planning (ECHPP) Project” is strictly voluntary. The consent form clearly indicates that participation is voluntary and that there are no mandatory requirements, beyond eligibility, for participating in the project. Participants will be informed that they may decline to participate without penalty at any time.
Justification for Sensitive Questions
The collection of information about HIV status, HIV-related risk behaviors, and HIV-related service access and treatment regimens is sensitive because of stigma and discrimination associated with HIV infection. The modes of transmission of HIV (through sexual contact and the sharing of HIV-contaminated needles and syringes) necessitate the collection of sensitive data regarding sexual practices and drug use. In keeping with the purpose of this data collection, other sensitive data are collected about specific behaviors, experiences or conditions that have been shown to be associated with HIV infection. This includes the collection of STD and HIV diagnosis and testing information, Hepatitis diagnosis and vaccination information, history of incarceration in the past 12 months alcohol use, and income.
Geographic information such as county of residence is collected for the purposes of spatial analysis of the data to understand the geographic distribution of disease and risk (and coverage of health services that are accessed by survey participants). Questions about race and ethnicity, which some consider to be sensitive, will also be asked using OMB’s two-question format. These questions will be used to report on racial and ethnic disparities that have been well-documented in other research on HIV risk and risk behaviors.
Although the information requested from participants is sensitive, the purposes of the ECHPP outcome monitoring project cannot be accomplished without their collection. Collection of the data will be used to understand barriers to engaging in protective behaviors and to accessing HIV prevention services. These data will also be used to enhance HIV prevention programs designed to reduce high-risk behaviors in persons most likely to acquire or transmit HIV. In order to successfully conduct an outcome monitoring project, it is necessary to include questions about sexual activity and substance use as they pertain to HIV transmission risk.
The context in which questions are asked helps to overcome their potential sensitivity.
Estimates of Annualized Burden Hours and Costs
ECHPP monitoring outcome data collection will occur over three years.
Year 1 –winter 2012 (after OMB approval) to summer 2012
Collect data for both surveys in all cities
Year 2 – fall 2012 to fall 2013
No data collection
Year 3 – fall 2013 to summer 2014
Collect data for both surveys in all cities
For clinics that agree to participate, project orientation to the identified clinic contact is estimated to take up to 30 minutes. The time taken for clinic staff to provide upcoming estimated patient loads to SciMetrika is approximately 5 minutes (per week). (The estimate in the table is for clinic staff to provide estimates to SciMetrika a total of 600 times-24 times per clinic per city.) The time taken by clinic staff to approach each potential participant about the clinic survey will take approximately 5 minutes.
Across all six MSAs, 1400 HIV-positive individuals who visit HIV clinics (age > 18 yrs) will be screened for eligibility per data collection year in the clinic survey. Across all six MSAs, 1200 HIV-positive individuals who visit HIV clinics (age > 18 yrs) will consent to participate and will be administered the clinic survey per data collection year. The clinic survey (Attachment 3d), will take approximately 40 minutes to administer.
For the community survey (Attachment 3b), the targeted population for this project includes 750 injection drug users (age > 18 yrs) and 750 high-risk heterosexuals (age 18 to 60 yrs) who will be screened per data collection year for eligibility in the community survey combined for all six MSAs. Across all six MSAs, 600 injection drug users (age > 18 yrs) and 600 high-risk heterosexuals (age 18 to 60 yrs) will consent to participate and will be administered the community survey per data collection year. The community survey will take approximately 25 minutes to administer.
Each eligibility screener (Attachment 3a and 3c) will take approximately 5 minutes to administer.
The time taken to complete either survey will vary widely based on level of risk reported (e.g., if a person does not report recent sexual activity, he/she will not be asked detailed questions about intercourse) and other skip patterns due to HIV treatment (e.g., detailed questions about medications will only be asked for the specific medications the respondent is currently taking). There is no cost to respondents other than their time.
Table A-12-1: Estimate of Annualized Burden Hours
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
B. Estimated Annualized Cost to Respondents
Annualized cost to respondents for the burden hours is provided in Table A.12.2. The mean hourly rates were determined by using information obtained from the latest government statistics from U.S. Department of Labor, Bureau of Labor Statistics (for all occupations), May 2010 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm#31-0000, [for the general public (all occupations), healthcare practitioners, and for the healthcare support and business/financial operations occupations].
Estimates for annualized burden costs are calculated based on per year for all cities (i.e., for two data collection years). These per year estimates are calculated in the same way as the previous section.
Table A-12-2: Estimate of Annualized Burden Costs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Source: The latest government statistics from U.S. Department of Labor, Bureau of Labor Statistics (for all occupations), May 2010 National Occupational Employment and Wage Estimates. http://www.bls.gov/oes/current/oes_nat.htm#31-0000
Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no other costs to respondents associated with this proposed collection of information.
Annualized Cost to the Federal Government
The cost of this project for the three years is estimated to be $2,754,823. The annualized cost is summarized in Table A-14-1.
Table A-14-1: Estimated Annualized Cost to the Federal Government
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The personnel assigned to this project include three behavioral scientists (at the GS 13 and 14 levels) and a Manila public health analyst (GS-12 equivalent). The contractor SciMetrika is funded for a five-year contract to perform project-related tasks, including this data collection activity. CDC and Manila travel is related to providing technical assistance to SciMetrika and interviewers, and conducting site visits to the SciMetrika office and to the six MSAs.
ECHPP outcome monitoring data will be compiled by SciMetrika staff in the six MSAs and sent via a secure network to a central processing location. SciMetrika will analyze all data from the six MSAs and produce a clean, final data set for use by CDC.
SciMetrika staff will work with the CDC team to create data tables to be displayed in surveillance reports and other products.
Explanation for Program Changes or Adjustments
This is a new data collection.
Plans for Tabulation and Publication and Project Time Schedule
Data will be collected during two time periods and will be collected from three target populations. OMB approval is requested through October 2014. Below is a brief overview of this new data collection activity.
Table A-16-1: Time Schedule for ECHPP Activities With Respect to OMB Approval
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Data from the ECHPP outcome monitoring information collection will inform prevention programs and services and increase existing knowledge in the behaviors that lead to acquisition of HIV infection.
Most of the results are expected to be useful at the local level, while other results will be more meaningful aggregated across sites. These data will be disseminated to the participating agencies, CDC staff, policy makers and other stakeholders through presentations at local, national and international conferences, publications in peer reviewed journals, and presentations at forums. Furthermore, CDC regularly publishes surveillance reports using data collected annually; depending on publication schedules, these reports have been published within 16 months - 2 years of the end of each cycle of data collection.
National data results will be released through national publications and presentations at conferences. Local data results will be reported back to the community through means such as local publications, Epidemiologic Profile reports, and presentations to local AIDS Service Organizations and community planning bodies and at local conferences and workshops.
Analyses related to this project will focus on the following key behavioral outcomes (but analyses are not limited to these outcomes only) during and after ECHPP implementation period:
Prevalence of HIV risk behaviors (e.g., unprotected vaginal and anal sex, injection drug use) among target populations
Risk reduction among target populations
Access of, and participation in, ECHPP-related services and programs among target populations
Met and unmet HIV prevention among target populations
These data will be synthesized with other data collected through existing CDC data sources (see Table A-1-1) for the ECHPP evaluation to make a broad statement about the contribution of ECHPP.
Reason(s) Display of OMB Expiration Date is Inappropriate
The OMB expiration date will be displayed.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification.
References
Centers for Disease Control and Prevention. Characteristics associated with HIV infection among heterosexuals in urban areas with high AIDS prevalence—24 cities, United States, 2006-2007. MMWR 2011;60:1045-9.
Centers for Disease Control and Prevention. HIV Surveillance Report, 2008; vol. 20. http://www.cdc.gov/hiv/ topics/surveillance/resources/reports/. Published June 2010. Accessed Sept. 6, 2011.
DiNenno EA, Oster AM, Sionean et al. Piloting a system for behavioral surveillance among heterosexuals at increased risk of HIV in the United States. Open AIDS Journal. In press.
McKnight C, Des Jarlais D, Bramson H et al. Respondent-driven sampling in a study of drug users in New York City: Notes from the field. Journal of Urban Health 2006; 83(7):i54-i59.
Office of National AIDS Policy. (2010). National HIV/AIDS Strategy. http://www.whitehouse.gov/administration /eop/onap/nhas. Accessed Sept. 6, 2011.
Singer E, Gebler N, Raghunathan T, Hoewyk JV, and McGonagle K. 1999. The effect of incentives in interviewer-mediated surveys. Journal of Official Statistics 15:217-30.
Stueve A, O'Donnell L, Duran R, San Doval A, Blome J. Time-space sampling in minority communities: Results with young Latino men who have sex with men. American Journal of Public Health 2001;91(6):922-926.
United States Department of Labor. (2010). Bureau of Labor Statistics. May 2010 National Occupational Employment and Wage Estimates, United States. http://www.bls.gov/oes/current/oes_nat.htm#31-0000. Accessed Nov. 7, 2011.
Valleroy L, Mackellar D, Karon J et al. HIV prevalence and associated risks in young men who have sex with men. JAMA 2000; 284(2):198-204.
Whiteman M, Langenberg P, Kjerulff K, McCarter R, and Flaws JA. A randomized trial of incentives to improve response rates to a mailed women's health questionnaire. Journal of Women's Health 2003;12(8): 821-828.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |