Information Collection #11:
National
Tobacco Prevention and Control Public Education Campaign:
Message
Platform Testing for Development of Future Advertising
Submitted for approval under CDC generic approval #0920-0910
Message Testing for Tobacco Communication Activities
Submission of this GenIC has been approved by
HHS/Assistant Secretary for Planning and Evaluation (ASPE)
July 23, 2014
Supporting Statement: Part B
__________________________________________
Data Collection Instruments
Other Attachments
Notes on Excluded Attachments
A draft message outline will be provided to OMB under separate cover.
__________________________________________
B.1 Respondent Universe and Sampling Methods
The target audience for this information collection is adults, ages 18 – 54 years, who are current smokers or former smokers who also use, or have used, electronic cigarettes (known as e-cigarettes), chewing tobacco, snus, cigars, cigarillos or little cigars. Eighteen years of age is the legal age to purchase tobacco cigarettes in most states, except for Alabama and Nebraska, where the legal age is 19. The data collection effort applies screening techniques to detect respondents who meet the legal age in Alabama and Nebraska. The screener will also screen out individuals older than 54.
CDC’s data collection contractor, Toluna, will recruit respondents from the opt-in Toluna Panel, a multimillion-member panel of cooperative online respondents. It is one of the largest databases of individual opt-in respondents for market research in the world. Individuals in the Toluna Panel have agreed to periodically participate in online surveys. Toluna’s panel is highly profiled with both behavioral and demographic characteristics. Panelists have joined the Toluna Panel through hundreds of different sources. Many diverse methods are leveraged to acquire panelists, including co-registration offers on partners’ Websites, targeted emails sent by online partners to their audiences, graphical and text banner placement on partners’ Websites, trade show presentations, targeted postal mail invitations, TV advertisements, member referrals, and telephone recruitment of targeted populations. Toluna has developed multiple proprietary panels across the globe. Since it is known that people who want to join an online panel are often similar to one another, Toluna improves the quality and representative nature of its online sample by incorporating participants from online communities, social networks, and Websites of all types. Participants are recruited to Toluna’s sample via banners, invitations and messaging of all types, and then they go through rigorous quality controls before being included in any sample or on any panel.
In addition, Toluna can potentially access anyone online via a network of relationships with Websites, panels, communities and social media groups. Toluna’s goal is to provide access to people to give their opinions wherever they are in the way that best suits the needs of the research project. For Terms of Use and Conditions for Toluna, please see Attachment 4.
The sample for respondents in this survey will be drawn from the Toluna panel based on the populations of interest – the proposed information collection will involve testing of messages among smokers or former smokers who have used electronic cigarettes (known as e-cigarettes), chewing tobacco, snus, cigars, cigarillos or little cigars, about their opinions of messages that emphasize the negative health effects of cigarette smoking in conjunction with use of these other products. The sample plan is essentially a convenience sample but will be based on demographic variables to ensure a reasonable degree of diversity in key demographic characteristics, such as age, gender, and region of residence, race/ethnicity, and income. As this study is considered part of formative testing for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the effectiveness of messages that inform dual tobacco product users about the harmful effects of continued cigarette smoking. The rationale for this specific creative target is that from previous formative research, we know that current smokers believe that by smoking cigarettes in combination with other products, such as electronic cigarettes (known as e-cigarettes), chewing tobacco, snus, cigars, cigarillos or little cigars, they’re doing something healthier for their bodies. This is incorrect in that smoking any amount of cigarettes still puts them at risk for tobacco-related disease. We want to position messaging to this specific audience so that it will resonate and encourage them to quit smoking cigarettes.
Each respondent will
view only one of the five draft messages. The sample size
recommendation is based on a calculation that determines the number
of respondents needed to assess differences between groups. We will
need to screen approximately 4,590 individuals who are smokers, or
former smokers, and are using e-cigarettes, chewing tobacco, snus,
cigars, cigarillos, or little cigars, from the Toluna panel to obtain
3,600 complete responses to the main survey. Each of the 3,600
respondents will view one message, resulting in an average of 720
total views per message. Each message has a variable number of
variants and audiences, so each message variant (12 in total) will be
viewed 300 times. To adjust for order effects within each group, we
will also balance the presentation order of the messages by
randomization of the messages, thus possibly resulting in a small
difference of base size per ad.
Subpopulation |
Msg
1 |
Msg 2 (2 variants, 2 audiences) |
Msg 3 (2 audiences) |
Msg 4 (2 variants) |
Msg 5 (2 variants) |
total |
Those who use e-cigarettes or chewing tobacco, in conjunction with cigarettes |
600 |
0 |
600 |
0 |
0 |
1,200 |
Those who use e-cigarettes or chewing tobacco, in conjunction with cigarettes or those who use cigars, cigarillos, or little cigars, in conjunction with cigarettes or singly |
0 |
1,200 |
0 |
0 |
0 |
1,200 |
Those who use cigars, cigarillos, or little cigars, in conjunction with cigarettes or singly |
0 |
0 |
0 |
600 |
600 |
1,200 |
subtotal: |
600 |
1,200 |
600 |
600 |
600 |
3,600 |
As this is a message test, we need a certain number of people to view each message to compare receptivity across messages. In order for us to test differences between messages by different behavioral and demographic groups, it is important that the smallest cell size for a specific attribute (such as ages 18-24, for example) to be at minimum of 50. Given the number of response categories of some items range up to 6, having an average of 300 respondents view each message will allow, for specific dependent variables, a minimum cell size of 50, if those response categories are grouped and summarized. This will allow for a level of aggregation acceptable for this specific message test.
Toluna has profiled its panel in terms of e-cigarette and certain non-combustible tobacco product usage as well as smoking behavior and as such can target and identify respondents who are pre-identified with these screening criteria, of legal age, to the survey. It is anticipated that the likelihood of respondents who do not qualify will be in the 25-28% range. A small percentage (2%) is anticipated to decide to opt-out of the survey once started.
During the data collection period, we will review the distribution of the qualified respondents who have participated and select additional panel members, as needed, to receive targeted email invitations to ensure the appropriate balance of respondents.
B.2 Procedures for the Collection of Information
The survey will be hosted on Toluna’s server farm, managing Toluna’s 4.7-million member panel. All interviews will be conducted using a self-administered, online questionnaire via proprietary, Web-assisted interviewing software. The screening questions are presented in Attachment 1a and the main questionnaire is included in Attachment 1b. The selected panelists will receive an initial invitation that indicates they have been invited to participate in a new survey (see Attachment 2). The email invitations will also state the length of the survey and incentive they receive if they qualify and complete the survey. The invitations that are currently utilized have been developed based on continual testing of the content, including but not limited to types of subject line, topic description, survey details, incentive description and format (html vs. text) that elicits the most favorable response rates.
Once the invitation is opened, respondents then will link to the survey URL, with an individual, unique and secure link, and complete the survey. The programmed survey is in the attached “Attachment 5” for review. Due to identity protection technology, it will not be possible for anyone to enter the survey without being recruited, or for a respondent to complete the survey more than once. In addition, the same-worded invitation will be sent at regular intervals after the original invitation is sent to those respondents who have not yet responded.
During the quantitative data collection process, approximately 120 respondents will be asked to participate in follow-up interviews. Of these respondents it is anticipated that 30 will agree to participate in a one-on-one interview with a moderator, over the telephone or web, for 30 minutes in length. Respondents will be permitted to schedule the one-on-one in-depth interview (IDI) at their own convenience by making an appointment with the moderator, or going immediatelyinto the interview using a browser-based telephonic software. A pre-loaded moderator’s guide is programmed for use by the moderator, and the moderator’s guide can be modified during the course of the study. IDI respondents will selected to ensure participation from each of the five audience segments involved in this message testing activity.
Project managers will have the ability to communicate live with the moderator and send private prompts during the interview, from their locations, to review specific responses as deemed necessary. A thorough overview of the research will be available to all participants as an online informed consent form, and online technical support will be available to participants throughout the course of the research, should the need arise. Full transcripts of the interviews will be available to project managers immediately after data collection is completed.
B.3 Methods to Maximize Response Rates and Deal with Nonresponse
The project methodology attempts to maximize response rates with two different strategies during the data collection process. The first strategy is that the survey invitation is targeted at pre-identified individuals (identification collected during the Toluna panel registration and profile process) who share the targeted screening characteristics. The second strategy is that the survey invitation itself is developed to elicit a broad response to maximize the number of respondents that “click” on the survey link. The survey will be available over mobile (smartphone and tablet) and desktop Internet devices. The follow-up in-depth interview will be available over desktop Internet devices. The respondent email invites will include information regarding the survey length, the incentive for participation and a secure and protected link to the survey.
Participants in the IDI will receive an honorarium of $20.
Response rates are closely monitored during the field period and, if needed, multiple reminders and new replicates are also sent to potential survey respondents.
B.4 Test of Procedures or Methods to be Undertaken
None.
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
George Terhanian, PhD
Toluna USA
Chief Strategy Officer
21 River Road
Wilton, CT 06897
Phone (646) 430-3420
Email: george.terhanian@toluna.com
David L. Vannette, PhD Candidate
Stanford University
Methodologist
Stanford, CA 94305
Phone: (616) 502-4828
E-mail: vannette@stanford.edu
Carol Sue Haney
Toluna USA
Survey Director
21 River Road
Wilton, CT 06897
Phone (203) 846-5838
Email: carol.haney@toluna.com
John Bremer
Toluna USA
Chief Research Officer
21 River Road
Wilton, CT 06897
Phone (203) 846-5838
Email: john.bremer@toluna.com
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | bkf4 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |