Web based survey_Part B

Web-Based HIV Behavioral Survey among Men who have Sex with Men

0920-New

Supporting Statement

Part B

November 29, 2011

Project Officer:

Kristina Bowles, MPH

Behavioral Scientist, Behavioral Surveillance Team

National Center for HIV, STD, and TB Prevention

Office of Infectious Disease

Centers for Disease Control and Prevention

1600 Clifton Rd, NE, MS E-46

Atlanta, Georgia 30333

Phone: (404) 639-1815

Fax: (404) 639-8640

E-mail: KBowles@cdc.gov

TABLE OF Contents

Section B Justification

1 Respondent Universe and Sampling Method

2 Procedures for the Collection of Information

3 Methods to Maximize Response Rates and Minimize Nonresponse

4 Tests of Procedures or Methods to be Undertaken

5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Table

Table 1 Estimated Numbers of Participants

LIST OF ATTACHMENTS

Attachment Number		Document Description
1a		Section 301 of the Public Health Service Act
1b		Section 306 of the Public Health Service Act
1c		Section 308d of the Public Health Service Act
2a		Eligibility Screener
2b		Behavioral Assessment Survey
3a		Eligibility Screener (Spanish)
3b		Behavioral Assessment Survey (Spanish)
4a		Federal Register Notice (30 day)
4b		Federal Register Notice (60 day)
5		Comments received on Federal Register Notice
6		Assurance of Confidentiality for HIV/AIDS Surveillance
7		Model Consent Form
8		CDC Project Determination

1. Respondent Universe and Sampling Method

Target Populations

The respondent universe for the proposed information collection, men who have sex with men (MSM), is estimated to be approximately 4.0% of the U.S. population (Purcell, Johnson et al. 2010). It is not possible to create a sampling frame of this population.

In order to provide yearly data on HIV-related sexual risk behavior among men who have sex with men (MSM), the proposed information collection targets internet using MSM in 56 U.S. jurisdictions (all 50 U.S. states, the District of Columbia, Puerto Rico, American Samoa, Guam, the Northern Mariana Islands, and the U.S. Virgin Islands). In addition, 21 metropolitan statistical areas (MSAs) with high AIDS prevalence will be oversampled (New York Division; Los Angeles Division; Washington Division; Chicago Division; Miami Division; Philadelphia Division; Atlanta-Sandy Springs-Marietta, Georgia; Houston-Baytown-Sugar Land, Texas; San Francisco Division, Baltimore-Towson, Maryland; San Juan-Caguas-Guaynabo, Puerto Rico; Dallas Division; San Diego MSA; Denver MSA; New Orleans MSA; Boston Division; Detroit Division; St Louis MSA; Nassau Division; Newark Division; Seattle Division).

MSM are disproportionately affected by the U.S. HIV epidemic (CDC 2011). HIV incidence has been increasing among MSM since 2000 (Hall, Song et al. 2008). The web-based behavioral assessment focuses on this at-risk group. Persons are eligible for the proposed information collection if they are at least 18 years old; are male; have ever had sex with another man; reside in the United States of America or territory thereof; are able to complete the interview in English or Spanish; and have not already participated in the proposed information collection during the same annual data collection cycle. The definition of MSM is based on simple behavioral criteria because HIV prevalence among this group is high and thus anyone engaging in the behaviors could reasonably be considered at risk for HIV infection.

Respondent eligibility criteria

Participant inclusion criteria

To be eligible, potential participants must:

• Be at least 18 years old;

• Be male;

• Have had sex with another man;

• Be a resident of the United States of America or territory thereof;

• Be able to read and understand either English or Spanish;

• Not have already participated during the same annual data collection cycle.

Selection of Respondents

A target sample of 170,000 eligible respondents, 84,000 from the 21 priority MSAs and 86,000 from the 56 jurisdictions, will complete the interview online each year. Oversampling in the 21 priority MSAs will occur to: 1) focus surveillance resources in areas with high AIDS burden, 2) compliment and allow comparison to ongoing surveillance activities, and 3) provide sufficient data for comparison between these priority areas and the other U.S. juridictions. Recruitment methods for the proposed information collection were chosen based on multiple consultations with sampling methodologists, those with expertise conducting research or behavioral surveillance activities with MSM, and public health practitioners who provide services to this population, as described in Section A8. The selection of appropriate methods to recruit samples of participants is complicated by the fact that population-based samples of MSM are not feasible, as MSM cannot be easily identified or enumerated for sampling purposes. Consequently MSM are considered a “hidden” or “hard-to-reach” population. Several guiding principles determined the selection of methods to conduct the proposed annual web-based behavioral assessment of MSM. These principles included the selection of methods that would 1) result in the most representative sample possible, 2) be feasible for annual implementation, 3) allow for trend analysis across time periods, and 4) provide data that are comparable and complementary to, but distinct from, data already collected in ongoing survey efforts.

The proposed information collection will utilize online direct marketing (ODM), a systematic sampling methodology targeting internet-using MSM. Internet-using MSM will be recruited through a direct marketing method that utilizes selective placement of banner advertisements on non-profit and privately owned websites, a method which has been used successfully to recruit MSM throughout the United States (Bull, Vallejos et al. 2008; Rosser, Miner et al. 2009; Rosser, Oakes et al. 2009; Khosropour, Luisi et al. 2010; Raymond, Rebchook et al. 2010; Sullivan 2010; Sullivan and Khosropour 2010). Recruitment will occur on a monthly basis during each calendar year. Findings from market research will guide the optimal sampling strategy, but we anticipate that banner ads will be displayed based upon an algorithm that randomizes both the day and the time when the banner advertisement will be displayed. The frequency with which the banner advertisement will be displayed at random times will be consistent across the targeted websites to ensure that the advertisements are not displayed more frequently on some websites than on others. The algorithm will be implemented each month until the targeted sample size has been reached.

The proposed sampling methodology will be conducted in three steps. First, websites from which to recruit MSM will be identified. Websites will be assessed for the number and characteristics of visiting MSM and logistics and feasibility of recruiting. A variety of websites will be chosen, including social networking sites, dating sites, sex-seeking sites, and sites offering health and support services to MSM. In the second component, a series of advertisements will be developed. Advertisements will include a broad range of models representing different race groups and ethnicities. When possible, the themes of advertisements will be tailored to specific websites or to the appeal of underrepresented subgroups should any be identified during current or previous data collection cycles. In the third component, MSM will be recruited through advertisements placed on the websites identified in the first component.

The proposed sampling method is considered a convenience sampling methodology because it does not utilize an active, statistically based recruitment strategy. However, the method has produced general population data comparable to those collected with offline methods (Mustanski 2001; Gosling, Vazire et al. 2004) and samples recruited online have been equated to mail-in survey samples (Couper, Traugott et al. 2001).

The web-based instruments for the proposed information collection will not elicit specific identifiers (e.g., name, address, social security number). All behavioral assessment data will be stored on a secure server. Paper surveys will not be used.

Sample size

Sample size considerations for the proposed convenience sampling differ from other methods for several reasons. First, convenience samples are generally less efficient than probability samples and require larger sample sizes to generate credible estimates. Second, sampling cost is concentrated in development of advertisements and advertising time so that the cost of recruiting someone who is discovered to be ineligible is the same as the cost of a fully completed interview. Finally, the automated nature of online surveys minimizes effort needed to recruit each additional participant (incremental cost).

During each year of data collection, a 170,000 eligible men will be recruited and interviewed, representing 1.2% of the estimated United States MSM population (Purcell, Johnson et al. 2010). This sample size should provide sufficient sample size to evaluate most behaviors of interest (e.g. sexual risk) by major demographic variables (e.g. race) and geographic areas, including priority MSAs. The numbers in the table represent target sample sizes for the proposed information collection by region.

		Screened	Interviewed
TOTAL		309,090	170,000
Northeast		28,727	15,800
Midwest		32,000	17,600
South		63,273	34,800
West		27,636	15,200
Other Jurisdictions*		4,727	2,600
21 Priority MSAs		152,727	84,000

*Includes U.S. Territories, Alaska, and Hawaii

Expected response rates

ODM recruitment of men differs from more common recruitment methods in that potential participants are not directly approached or solicited to participate. Instead, participants are given the opportunity to participate through banner advertisements that appear on websites frequented by the target population. The standard response rate definition: the percent of men approached who agree to participate, does not apply. Instead, response is evaluated with two measures: the click-through rate on the banner ads and the interview participation rate.

The click-through rate is defined as the total number of banner advertisement clicks, divided by the total number of exposures to the advertisements, without adjustment for repeated viewing.

The number of exposures (number of times the ad appeared on an open web page) is routinely provided to advertisers by websites and can be thought of as the number of passive recruitment attempts. The click-through rate can be considered the screening rate and tends to be less than 0.5%. Based on experience with previous MSM studies conducted by web-based survey researchers, we anticipate a click-through rate of 0.2% (Rosser, Miner et al. 2009; Rosser, Oakes et al. 2009; Sullivan, Salazar et al. 2009; Sullivan and Khosropour 2010).

The participation rate is defined as the number of eligible persons who start the behavioral assessment divided by the number of persons who complete the eligibility screener. Recent studies conducted by the web-based survey researchers have achieved participation rates above 50% (Khosropour, Luisi et al. 2010; Sullivan 2010; Sullivan and Khosropour 2010). A target of 309,090 persons will be screened.

Further details and calculations are provided in Section B3 below.

2. Procedures for the Collection of Information

Main steps in data collection

For the proposed information collection, data will be collected through a self-administered, web-based interview. Data collection will begin with completion of an online eligibility screener (Attachment 2a). If the person completing the screener is not eligible, he will be thanked for his time and interest in the project. If he is eligible, he will be asked to give his informed consent to participate (Attachment 7); his response will be documented on an online form. Persons who consent will be automatically directed to the interview (Attachment 2b). Those who do not consent to participate will be thanked for their time and interest in the project.

Quality Control

Data quality will be ensured by the use of self-administered web-based interviews, which improve data quality in several ways:

1. Interview errors are reduced because individuals do not have to follow complex routing instructions; the computer skips automatically to the next appropriate question.

2. Participant errors are also reduced. Checks for consistent responses are programmed into the behavioral assessment so that participants who provide inconsistent answers are automatically asked for clarification.

3. Self-administered questionnaires provide privacy for the participant, ideally minimizing lack of willingness to report information about sensitive topics, such as sexual risk behaviors, that sometimes affects responses in face-to-face interviews; reduced pressure to provide socially desirable (but personally inaccurate) responses is another benefit of self-administration of the questionnaire, vis à vis face-to-face administration. (Tourangeau and Yan 2007).

4. Use of a computer-based behavioral assessment also reduces coding and data entry errors, which makes possible more accurate and efficient preparation of the data for analysis.

In addition to the checks provided through the computer-based behavioral assessment, monthly recruitment monitoring reports will allow for identification and correction of errors in the online questionnaire application.

3. Methods to Maximize Response Rates and Deal with Non response

Online, direct marketing recruitment

Response rate expectations of more than 80% for surveys employing probability sampling are not applicable to the proposed information collection for multiple reasons. First, probability sampling cannot be used because of the difficulty of assembling a sampling frame for MSM. ODM recruitment methods for MSM are in line with other methods designed to sample hard-to-reach populations that utilize the same resources that members of the populations rely on for population-specific activities. Second, the standard notion of a response rate is predicated on an active recruitment strategy involving solicitation of participation. In such studies, the response rate is calculated as the percent of those approached who participate. For the proposed web-based behavioral assessment, no individuals will be directly approached (rather, exposure to the advertisements soliciting participation is self-directed), precluding estimation of the denominator for a standard response rate.

Instead of the standard measure of response, two measures, defined above in section B1, are used to estimate response associated with ODM recruitment: the click-through rate and the participation rate. Published accounts of response rates from web-based surveys vary drastically due to differences in sampling and design (Ekman and Litton 2007). In two recent studies, methods like those proposed for this information collection request have consistently achieved response rates among MSM that are among the highest in published literature: a 0.37% click-through rate and a 55% participation rate (Khosropour, Luisi et al. 2010; Sullivan and Khosropour 2010); 0.13% click-through rate, 76% participation rate (Sullivan 2010).

Methods to maximize response rates

Refusal rates for the proposed information collection behavioral assessment may be adversely affected by the passive recruitment method used and the sensitive nature of the topics covered. The Contractor will monitor response in real time through the project’s secure knowledge transfer site. CDC will receive monthly summary reports and statistics allowing for identification, investigation, and adaptation to changes in response rates in real time.

Research indicates that click-through rates are associated with the ability of a banner advertisement to attract and hold the interest of website visitors. Participation and completion rates are associated with the ability of a survey to keep participants engaged and interested in following through to completion. Based on previous market research, for the proposed information collection, banner ads, user interfaces, and graphics, including animation, will be specifically tailored to appeal to and attract MSM. When user profile information is available from websites, banner advertisements specifically tailored to demographic characteristics of users will be employed to increase response rates. Population-specific themes will be developed and maintained throughout the interview to appeal to participants' extant motivations (e.g. altruism). Interviews will be short, user-friendly, and visually attractive to reduce drop outs.

It is likely that certain websites may generate higher click-through rates than others.  This desirable characteristic must be balanced, however, with concerns about representativeness.  For example, sex-seeking websites may have higher or lower click-through rates than social networking sites that are not sexually oriented.  However, it would not be desirable for the proposed web-based behavioral assessment to emphasize such sites for recruitment because the men who would be referred from those sites would tend to be different (likely have a higher risk of HIV infection) than men referred from banner advertisements on general social networking sites.

Generalizability

All methods to recruit samples of MSM are expected to result in samples that have unknown generalizability.  No sampling frame for men who have sex with men exists, and the difficulties of assembling a sampling frame for this relatively hidden population have been described. Men recruited for the proposed information collection will constitute a convenience sample that may not be generalizable to the greater population of MSM. However, through internet recruitment, the proposed information collection will reach larger numbers and a geographically broader group of MSM than would otherwise be possible, which may help to compensate for the limitations of convenience sampling.

To meet the primary purpose of the web-based behavioral assessment, it is important both to characterize biases when possible and to address whether there are changes in these biases over time which might affect the interpretation of trends. Investigators will compare demographic and behavioral characteristics of men recruited for the proposed information collection to men recruited through the NHBS, which collects triennial MSM data from the 21 priority MSAs (the same ones that will be oversampled for the proposed web-based behavioral assessment) using venue-based sampling, an offline recruitment methodology for hidden or hard-to-reach populations.

Evaluation

An evaluation of each annual cycle will be conducted at its conclusion. This evaluation will encompass survey procedures and results. Relevant findings from these evaluations will be incorporated into the design of the next round of the web-based behavioral assessment. Specific findings related to analysis of the data collected will be made available to users of the data.

4. Tests of Procedures or Methods to be Undertaken

The data collection instruments were developed using questions from previous CDC surveillance projects, and with input from expert advisors (listed in Supporting Statement A, Table 8-A-1) .As most questions comprising the data collection instruments have been previously tested and used, only internal testing by CDC and contract staff will be needed. CDC and contract staff will test the skip patterns and responses in the automated, web-based behavioral assessment that will be used for the proposed information collection.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Consultants on Statistical Aspects

Chris Johnson, MS Statistician, Centers for Disease Control and Prevention 1600 Clifton Rd, NE MS E-48 Atlanta, GA 30333 Phone: (404) 639-2989 Email: CJohnson3@cdc.gov	Stephen Tregear, PhD Project Director Manila Consulting Group, Inc. 1420 Beverly Road, Ste 220 McLean, VA 22101 Phone: (571) 633-9400 Email:stregear@manilaconsulting.net
Patrick Sullivan, PhD, DVM Associate Professor, Department of Epidemiology Emory University 201 Dowman Drive Atlanta, GA 30322 Phone: 404-727-2038 e-mail: pssulli@emory.edu

Contractor

In October 2010, CDC awarded a contract to Manila Consulting, Inc. to develop and implement the proposed information collection. Contact information for the Project Director is listed below.

Stephen Tregear, PhD

Project Director

Manila Consulting Group, Inc.

1420 Beverly Road, Ste 220

McLean, VA 22101

Phone: (571) 633-9400

Email:stregear@manilaconsulting.net

CDC Project Staff

All CDC project staff can be reached at the following address and phone number:

Behavioral and Clinical Surveillance Branch

Division of HIV/AIDS Prevention

Centers for Disease Control and Prevention

1600 Clifton Rd, NE MS E-46

Atlanta, GA 30333

Phone: (404) 639-2090

Elizabeth DiNenno, PhD

Acting Team Leader

Email: EDiNenno@cdc.gov

Kristina Bowles, MPH

Epidemiologist

Email: KBowles@cdc.gov

Alexa Oster, MD

Medical Epidemiologist

Email: AOster@cdc.gov

Cyprian Wejnert, PhD

Epidemiologist

Email: CWejnert@cdc.gov

Amanda Smith, MPH

Epidemiologist

Email: ASmith3@cdc.gov

Pollyanna Chavez, PhD

Epidemiologist

Email: PChavez@cdc.gov

Kathy Hageman, PhD, MPH

Behavioral Scientist

Email: KHageman@cdc.gov

Jeanne Bertolli, PhD

Associate Director for Science, Behavioral and Clinical Surveillance Branch

Email: JBertolli@cdc.gov

James Heffelfinger, MD

Chief, Behavioral and Clinical Surveillance Branch

Email: JHeffelfinger@cdc.gov

References

Bull, S. S., D. Vallejos, et al. (2008). "Improving recruitment and retention for an online randomized controlled trial: experience from the Youthnet study." AIDS Care 20(8): 887-893.

CDC (2011) "HIV Surveillance Report, 2009." 21.

Couper, M. P., M. W. Traugott, et al. (2001). "Web survey design and administration." Public Opinion Quarterly 65(2): 230-253.

Ekman, A. and J. E. Litton (2007). "New times, new needs; e-epidemiology." European Journal of Epidemiology 22(5): 285-292.

Gosling, S. D., S. Vazire, et al. (2004). "Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires." American Psychologist 59(2): 93-104.

Hall, H. I., R. Song, et al. (2008). "Estimation of HIV incidence in the United States." JAMA 300(5): 520-529.

Khosropour, C. M., N. Luisi, et al. (2010). Challenges in Engaging Black Men in Online Internet Research. United States Conference on AIDS.

Mustanski, B. S. (2001). "Getting wired: Exploiting the Internet for the collection of valid sexuality data." Journal of Sex Research 38(4): 292-301.

Purcell, D. W., C. H. Johnson, et al. (2010). Estimating the National Population Size of Men Who Have Sex with Men to Develop HIV/AIDS and Syphilis Rates. Atlanta, GA, CDC.

Raymond, H., G. Rebchook, et al. (2010). "Comparing Internet-Based and Venue-Based Methods to Sample MSM in the San Francisco Bay Area." AIDS and Behavior 14(1): 218-224.

Rosser, B. R., M. H. Miner, et al. (2009). "HIV risk and the internet: results of the Men's INTernet Sex (MINTS) Study." AIDS Behav 13(4): 746-756.

Rosser, B. R., J. M. Oakes, et al. (2009). "HIV sexual risk behavior by men who use the Internet to seek sex with men: results of the Men's INTernet Sex Study-II (MINTS-II)." AIDS Behav 13(3): 488-498.

Sullivan, P. S. (2010). CVCT (Couples Voluntary Counseling & Testing) online scale validation data.

Sullivan, P. S. and C. M. Khosropour (2010). The BOPR Study: Methods to improve the enrollment of racial/ethnic minority MSM in online HIV prevention research. Sex: Tech Conference.

Sullivan, P. S., L. Salazar, et al. (2009). "Estimating the proportion of HIV transmissions from main sex partners among men who have sex with men in five US cities." Aids 23(9): 1153-1162.

Tourangeau, R. and T. Yan (2007). "Sensitive questions in surveys." Psychological Bulletin 133(5): 859-883.



16