SS Part B

Information Collection #14:

National Tobacco Prevention and Control Public Education Campaign:

Rough Cut Testing of Spanish, Chinese, Korean, and Vietnamese Language Print Advertisements for the 2015 Tips Campaign

Submitted for approval under CDC generic approval #0920-0910

Message Testing for Tobacco Communication Activities

Submission of this GenIC has been approved by

HHS/Assistant Secretary for Planning and Evaluation (ASPE)

November 4, 2014

Supporting Statement: Part B

­­­­__________________________________________

Data Collection Instruments

• Attachment 1a-S. Screener Questionnaire in Spanish

• Attachment 1a-C. Screener Questionnaire in Chinese

• Attachment 1a-K. Screener Questionnaire in Korean

• Attachment 1a-V. Screener Questionnaire in Vietnamese

• Attachment 1b-S. Main Questionnaire in Spanish

• Attachment 1b-C. Main Questionnaire in Chinese

• Attachment 1b-K. Main Questionnaire in Korean

• Attachment 1a-V. Main Questionnaire in Vietnamese

Other Attachments

• Attachment 2-S. Email to Potential Respondents (Initial Email Invitation) in Spanish

• Attachment 2-C. Email to Potential Respondents (Initial Email Invitation) in Chinese

• Attachment 2-K. Email to Potential Respondents (Initial Email Invitation) in Korean

• Attachment 2-V. Email to Potential Respondents (Initial Email Invitation) in Vietnamese

• Attachment 3-S. Toluna Panelist Privacy Policies in Spanish

• Attachment 3-C. Toluna Panelist Privacy Policies in Chinese

• Attachment 3-K. Toluna Panelist Privacy Policies in Korean

• Attachment 3-V. Toluna Panelist Privacy Policies in Vietnamese

• Attachment 4-S. Toluna Panelist Terms and Conditions in Spanish

• Attachment 4-C. Toluna Panelist Terms and Conditions in Chinese

• Attachment 4-K. Toluna Panelist Terms and Conditions in Korean

• Attachment 4-V. Toluna Panelist Terms and Conditions in Vietnamese

• Attachment 5-S. Screen Shots (annotated) in Spanish

• Attachment 5-C. Screen Shots (annotated) in Chinese

• Attachment 5-K. Screen Shots (annotated) in Korean

• Attachment 5-V. Screen Shots (annotated) in Vietnamese

Notes on Excluded Attachments

In this GenIC, CDC outlines a plan to test 10 draft advertisements in print form with content that may be considered sensitive. The draft materials are not included in the attachments for this GenIC because:

The advertisements have not been approved for public distribution by HHS/Assistant Secretary for Public Affairs (ASPA).

The untested advertisements could be perceived by the public as ineffective or offensive (testing is designed to identify potential problems).

Release of the advertisements must be coordinated with the launch of a comprehensive HHS/CDC campaign. Unauthorized release could jeopardize the evaluation strategy for the campaign.

To support adequate review of this GenIC by OMB, CDC requests permission to provide OMB with a secure link to the draft materials.

­­­­__________________________________________

B.1 Respondent Universe and Sampling Methods

CDC’s data collection contractor, Toluna, will recruit respondents from the opt-in Toluna Panel, a multimillion-member panel of cooperative online respondents. It is one of the largest databases of individual opt-in respondents for market research in the world. Individuals in the Toluna Panel have agreed to periodically participate in online surveys. Toluna’s panel is highly profiled with both behavioral and demographic characteristics. Panelists have joined the Toluna Panel through hundreds of different sources. Many diverse methods are leveraged to acquire panelists, including co-registration offers on partners’ Websites, targeted emails sent by online partners to their audiences, graphical and text banner placement on partners’ Websites, trade show presentations, targeted postal mail invitations, TV advertisements, member referrals, and telephone recruitment of targeted populations. Toluna has developed a proprietary panel across the globe. Since it is known that people who want to join an online panel are often similar to one another, Toluna improves the quality and representative nature of its online sample by incorporating participants from online communities, social networks, and Websites of all types. Participants are recruited to the panel via banners, invitations and messaging of all types. Then they go through rigorous quality controls before being included in any sample or on any panel. In addition, Toluna can potentially access anyone online via a network of relationships with websites, panels, communities and social media groups. Toluna’s goal is to provide access to people to give their opinions wherever they are in the way that best suits the needs of the research project. For Terms of Use and Conditions for Toluna, please see Attachments 4-S, 4-C, 4-K, and 4-V for the Spanish, Chinese, Korean, and Vietnamese versions, respectively.

The sample for respondents in this survey will be drawn from the Toluna panel based on the populations of interest. The proposed information collection will involve testing of print ads among smokers and nonsmokers ages 18-54 who speak Spanish, Chinese, Korean or Vietnamese. The legal age of consent is 18 years in all states in the United States except for Alabama and Nebraska, where the age of majority is 19. Toluna’s opt-in process for this survey is designed to screen out minors and anyone less than 18 of age, except in Alabama and Nebraska. In Alabama and Nebraska, anyone aged less than 19 will not be able to participate. In all cases, the screeners will also screen out individuals older than 54. The sample plan is essentially a convenience sample but will be based on demographic variables to ensure a reasonable degree of diversity in key demographic characteristics, such as age, gender, region of residence, race/ethnicity, education, and income. As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of messages under test.

In Gen IC #13, CDC obtained OMB approval to test 19 English language advertisements that are also part of the 2015 Tips campaign. In the current information collection request, CDC seeks OMB approval to test 10 print ads that have been developed in the languages of Spanish, Chinese, Korean, and Vietnamese.

• Four print ads are in the Spanish language. These ads correspond to four English language ads described in GenIC #13.

• Advertisement entitled “Julia’s Tip,” colorectal cancer ( Spanish ad #1)

• Advertisement entitled “Mark’s Tip,” colorectal cancer ( Spanish ad #2)

• Advertisement entitled “Marlene’s Tip,” macular degeneration ( Spanish ad #3)

• Advertisement entitled “Dual Use/Heart Attack Tip,” dual-use ( Spanish ad #4)

Each ad test will be conducted with approximately 200 respondents (total of 800 Spanish language respondents; each respondent will view participate in only one ad test).

• Six print ads are based on two new message platforms. Each platform will be tested in three languages: Chinese, Korean, and Vietnamese (resulting in six rough cut print ads). These ads do not correspond to English language print ads described in GenIC #13.

• Advertisement entitled “Health Effects/Shawn”

(Chinese ad #5, Korean ad #6, and Vietnamese ad #7)

• Advertisement entitled “Shawn’s Tip”

(Chinese ad #8, Korean ad #9, and Vietnamese ad #10)

Each ad test will be conducted with approximately 100 respondents (total of 200 Chinese language respondents, 200 Korean language respondents, and 200 Vietnamese language respondents).

Each respondent will view one of the 10 draft ads that are part of this specific information collection. The sample size recommendation is based on a calculation that determines number of respondents needed to assess differences between ads. Approximately 4,648 potential respondents will be screened in order to obtain completed questionnaires from 1,400 respondents in the target age range of 18-54 years along with other identifying characteristics. Note that since 10 ads are being tested in this information collection request, approximately 200 respondents will view each Spanish advertisement, and approximately 100 respondents will view each Asian language advertisement.

Smokers and nonsmokers will be interviewed. Approximately 75% of the total sample will be smokers and 25% will be nonsmokers. Toluna has profiled its panel in terms of smoking behavior and as such can target and identify respondents who are pre-identified smokers, of legal age and younger than 55, to the survey. Other profiled characteristics of the Toluna panel include demographics such as gender and ethnicity. A small percentage (1-2%) is anticipated to decide to opt-out of the survey once started.

For these ads, when a respondent is identified as meeting the criteria of the specified ad, they will be randomly assigned and routed to the portion of the survey to view one of the ads for which they qualify. For example, if a respondent is identified as a smoker and speaks Spanish, he or she will randomly be assigned to one of the 4 Spanish-specific ads. In addition to smokers and nonsmokers, four specific subpopulations will be monitored for a specific quota of respondents. The quota for these additional subpopulations is listed in the following table:

Subpopulations of Interest	Number of Responses
Spanish language speakers	800
Chinese language speakers	200
Korean language speakers	200
Vietnamese language speakers	200
Total	1,400

During the data collection period, we will review the distribution of the qualified respondents who have participated and select additional panel members, as needed, who will receive targeted email invitations to ensure the appropriate balance of respondents.

B.2 Procedures for the Collection of Information

The survey will be hosted on Toluna’s server farm, managing their 4.7-million member panel. All interviews will be conducted using a self-administered, online questionnaire via proprietary, Web-assisted interviewing software. The selected panelists will receive an initial invitation that indicates they have been invited to participate in a new survey (see Attachments 2-S, 2-C, 2-K, and 2-V for the Spanish, Chinese, Korean, and Vietnamese translations, respectively). The email invitations will also state the length of the survey and incentive they receive if they qualify and complete the survey.

The list of study procedures is as follows:

1. Respondents are recruited from Toluna’s existing online panel, using an email invitation (see Attachment 2, available in Spanish, Chinese, Korean, and Vietnamese) sent from Toluna’s sample management system. The sample selection is discussed above in Section A1.

2. The invitation includes a link behind a “Start” button, with the link going to a web page containing the screening survey.

3. If the potential respondent consents to participating in the study, he or she clicks the “Start” button.

4. If the respondent passes the screening questions, he or she progresses to the main questionnaire.

5. If the respondent does not pass the screening questions, he or she is routed to a page that says “Thank you for your participation in this research study. Unfortunately, your responses indicate that you are do not fit the specific criteria needed for this particular study, or that we have already reached our required quota of responses from participants similar to you. We appreciate your enthusiasm for our research study and hope you will join us on future surveys!”

Once the invitation is opened, respondents then will link to the survey URL, with an individual, unique and secure link, and complete the Screener (Attachment 1a, available in Spanish, Chinese, Korean, and Vietnamese) and Main Questionnaire (see Attachment 1b, available in Spanish, Chinese, Korean, and Vietnamese). Due to identity protection technology, it will not be possible for anyone to enter the survey who has not been recruited, or for a respondent to complete the survey more than once. In addition, the same-worded invitation will be sent at regular intervals after the original invitation is sent to those respondents who have not yet responded.

The routing described above happens automatically within the programmed survey. In this manner, the random assignment of specific ads to specific subpopulations occurs. Once a quota is met for a specific subpopulation and ad, the ad will be removed from the random assignment for that specific subpopulation. Once all quotas are filled, the survey will be completed. Screen shots are provided as Attachments 5, available in Spanish, Chinese, Korean, and Vietnamese. The screenshot file is annotated with comments that explain minor differences in questionnaire routing and wording.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

The project methodology attempts to maximize response rates at two points in the data collection process. The survey invitation is targeted at pre-identified individuals who share the screening characteristics. The survey invitation itself is developed to elicit a broad response to maximize the number of respondents that “click” on the survey link. The survey will be available over desktop Internet devices. The respondent email invites will include information regarding the survey length, the incentive for participation and a secure and protected link to the survey. Response rates are closely monitored during the field period and, if needed, multiple reminders and new replicates will be sent to potential survey respondents. The invitations currently being utilized have been developed based on continual testing of the content, including but not limited to types of subject line, topic description, survey details, incentive description, and format (html vs. text) that elicits the most favorable response rates.

B.4 Test of Procedures or Methods to be Undertaken

None.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Carol Sue Haney

Toluna USA

21 River Road

Wilton, CT 06897

Phone (203) 846-5838

Email: carol.haney@toluna.com

John Bremer

Toluna USA

21 River Road

Wilton, CT 06897

Phone (203) 846-5838

Email: john.bremer@toluna.com

David L. Vannette, PhD Candidate

Stanford University

Stanford, CA 94305

Phone: (616) 502-4828

E-mail: vannette@stanford.edu

6