Message Testing for Tobacco Communication Activities (MTTCA)
(OMB No. 0920-0910)
Supporting
Statement: Part B
Program Official/Contact
Michelle O’Hegarty, PhD
Office on Smoking and Health
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
4770 Buford Highway, NE MS F-79
Atlanta, Georgia 30341
770-488-5582
FAX:770-488-5939
Email: mohegarty@cdc.gov
February 20, 2018
TABLE OF CONTENTS
B. STATISTICAL METHODS
Respondent Universe and Sampling Methods
Procedures for the Collection of Information
Methods to Maximize Response Rates and Deal with No response
Tests of Procedures or Methods to be Undertaken
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
LIST OF ATTACHMENTS
Attachment 1: Public Health Service Act
Attachment 2: Overview of Planned Information Collections
Attachment 3a: Example Data Collection Instrument – Rough Cut Testing Survey Screener
Attachment 3b: Example Data Collection Instrument – Rough Cut Testing Survey
Attachment 3c: Example Data Collection Instrument – Creative Concepts Survey Screener
Attachment 3d: Example Data Collection Instrument – Creative Concepts Survey
Attachment 3e: Example Data Collection Instrument – Creative Concepts Focus Group Screener
Attachment 3f: Example Data Collection Instrument – Creative Concepts Focus Group
Moderator’s Guide
Attachment 4: Example Informed Consent Form
Attachment 5: 60 Day Published FRN
Attachment 6: Institutional Review Board Approval Notification or exemption determination
B.1. Respondent Universe and Sampling Methods
The Centers for Disease Control and Prevention’s Office on Smoking and Health (CDC/OSH) requests OMB approval for an extension for Message Testing for Tobacco Communication Activities (MTTCA), OMB No. 0920-0910, exp. 3/31/2018. The purpose of this generic clearance is to collect information about smokers’ and nonsmokers’ attitudes and perceptions, and to pretest draft messages and materials for clarity, salience, appeal, and persuasiveness. Information collected will be used to develop tobacco-related health messages, including messages related to the National Tobacco Education Campaign (NTEC).
The primary NTEC audience is cigarette smokers ages 18 through 54. The existing clearance also includes youth ages 13-17 years old. There are no proposed changes to the populations of interest from the last revision request to MTTCA. Respondents will not include members of the state or local government.
This package is a three-year extension request to the 0920-0910 MTTCA clearance to encompass the proposed activities. No modification is requested for information collection activities, methodology, respondents, or burden.
As these information collections are considered part of formative work for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. A variety of qualitative and quantitative information collection strategies are supported through this mechanism, including in-depth interviews; in-person focus groups; online focus groups; computer-assisted, in-person, or telephone interviews; and online surveys.
Qualitative: Qualitative methods are not intended to yield results that are statistically projectable or used to derive quantitative estimates. However, these methods allow tobacco communication messages and/or programs to be designed and marketed with specific audiences in mind (e.g., 18-54 year old cigarette smokers). In qualitative studies, quota sampling is often used to select a convenience sample of individuals who meet certain qualifications that reflect characteristics typical of the target audience. Response rate is not applicable to quota sampling because this type of sampling results in a nonprobability sample which is not representative of the population. In qualitative studies, respondents can be initially contacted by telephone, through the mail, or online; over-recruiting is done to compensate for non-respondents.
Quantitative: Where quantitative methods are used, information collection activities will target particular audiences with statistical sampling procedures employed to identify potential survey respondents. Online surveys for message testing will seek a convenience sample (e.g. through an online panel) that nonetheless has an acceptable degree of diversity in key demographic characteristics such as age, gender, education, and race/ethnicity. OSH does not intend to generate nationally representative results or precise estimates of population parameters using these surveys.
OSH will request Office of Management and Budget (OMB) approval for each data collection by submitting an Information Collection Request that describes project purpose, use, and methodology. Sampling methods will vary based on the target audiences and methodology for each round of data collection.
B.2. Procedures for the Collection of Information
The methodologies planned for use in this submission will follow standard state-of-the-art approaches adapted from marketing and communications research. In this context, the term pretesting refers to testing messages, strategies, and communication materials before they are finalized and fielded. Questions in all pretesting methodologies include standard measures of communications that are designed to assess to what degree the message was successful in communicating information, including perceived effectiveness, main idea recall, comprehension, believability, personal relevance, motivation to quit smoking, and likes and dislikes. Additional questions may be added to address any specific concerns regarding a message or advertisement, such as how a respondent views a particular logo or caption related to the ad. The following describes examples of the types of methodologies to be used.
Individual Interviews: Individual in-depth interviews are generally conducted in-person or online at a designated internet location. These interviews are used to collect information from key informants to elicit attitudes and perceptions that offer insight into critical influences on individual’s belief structures or for pretesting message concepts, draft materials, and communication strategies. Individual in-depth interviews are ideal when the information in question requires in-depth probing or when individual, rather than group, responses are considered more appropriate. This methodology is appropriate for determining target audience attitudes, beliefs, and feelings.
Focus Groups: Focus groups or group interviews can also be conducted in-person or online. Focus groups are used to obtain insights into target audience perceptions, beliefs, and attitudes in the early stages of the communication process (i.e., in concept, strategy and materials development) and to understand how individuals discuss a message or advertisement with each other. Focus groups are usually composed of 8 - 12 people who have characteristics similar to the target audience or subgroups of the target audience. The groups are conducted by a professional moderator who keeps the session on track while allowing respondents to talk openly and spontaneously. The moderator uses a structured discussion outline. Focus groups are valuable in exploring consumer reactions to message concepts before additional resources are put into their development.
Bulletin Boards: Online Bulletin Boards are a form of qualitative data collection used in formative projects and in pretesting to aid in identifying messages and themes that resonate the most with audience members, as well as to identify gaps for the development of new creative materials. Online Bulletin Boards may be composed of different target audience members (e.g., a general population sample of 18-54 year-olds from diverse media markets). Bulletin board sessions typically run over a two- to three-day period, and respondents can respond at their convenience during the session, while a trained live moderator will monitor responses throughout the duration of the bulletin board.
Surveys: Surveys can be conducted using self-administered online, telephone (CATI), or paper-and-pencil questionnaires. A sample of consenting participants is recruited from the target audience; respondents will be asked to respond to questions regarding their reactions to messaging or ads with respect to the main message, believability, comprehension, perceived effectiveness, and whether it would impact their behavioral intentions regarding tobacco use or secondhand smoke. This method of formative testing is not designed to generate nationally representative results or estimates of population parameters but rather is used to test whether messages or ads are credible, comprehensible and persuasive. Surveys can also be conducted for recruitment of real people to appear in advertisements used in future national tobacco education campaigns.
For all methodologies, professionally recognized procedures will be followed in each information collection activity to ensure high quality data. Some examples of these procedures include the following:
Training sessions, supervision and monitoring will be conducted for all data collection efforts in which moderators/interviewers interact with respondents (e.g., focus groups).
Observers will monitor focus groups, focus group proceedings will be recorded, and online technical support will be made available, should the need arise.
An institutional review board approval will be requested, when needed.
Online survey procedures require that respondents must respond or explicitly decline each presented question before moving forward, which ensures that as much information pertinent to the research is provided.
Data submitted through online surveys will be subjected to statistical validation techniques (such as disallowing out-of-range values).
All data collection and analysis will be performed in compliance with OMB, Privacy Act, and Protection of Human Subjects requirements.
B.3. Methods to Maximize Response Rates and Deal with Non-response
In the case of data collection activities that involve interviews or surveys conducted in-person, online, or on the telephone, several procedures can be used to increase responses. Below are a few examples.
Interviewers will participate in thorough training sessions. Training topics will include study objectives, question-by-question reviews of data collection instruments, strategies for engaging respondents, role playing, and techniques for fostering respondent cooperation and survey completion.
Experienced, highly-trained staff will moderate all focus groups and bulletin board sessions in-person and online.
After an initial online qualifying screener, all respondents will be rescreened and confirmed via telephone. Prior rounds of data collection indicate that this is a very effective method for keeping high response and show rates for bulletin boards and focus groups.
For bulletin boards, if a respondent has not joined the discussion, they will typically be sent a follow-up email at the start of Day 2.
Potential respondents will be informed through a variety of methods, such as email messages, about the importance of these projects and encouraged to participate.
Content and layout of email invitations will be assessed for clarity to ensure robust response rates.
After the original invitation, respondents who have not completed the survey after 48 hours may receive a reminder invitation.
Creative and attractive graphics will be used to attract the attention of respondents, where relevant.
CDC will be identified as the agency of record, since this agency is credible and serves the public good.
Participants will be given incentives, which can include cash, prepaid gift cards, or points that can be redeemed for other items, such as Amazon gift cards; they will be made aware of the type and amount of incentive prior to participating in the study.
B.4. Tests of Procedures or Methods to be Undertaken
This project involves the collection of quantitative and qualitative information. This package is requesting an extension of a previously-approved generic clearance package. Similar procedures have already been used to conduct testing through this clearance. Additionally, contractors or OSH staff may pretest or pilot test the instrument(s) and method of data collection, when time permits and if deemed necessary. Lessons from the pretest or pilot test will be identified, and changes, as necessary, will be incorporated into the instrument and method. Pretests and pilot tests will typically involve no more than a minimal number of individuals unless OMB clearance is sought for a larger number.
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Primary responsibility for data collection and analysis through this clearance will be conducted by Carol Haney and Steven Snell from Qualtrics, and Lisa John, Amanda Berger, and Robert Alexander from Battelle, whose information is listed below.
Carol Sue Haney
Senior Research and Data Scientist
Qualtrics
400 Qualtrics Drive
Provo, UT 84604
Phone (802) 258-0518
Email: carolh@qualtrics.com
Steven Snell, PhD
Principal Research Scientist
Qualtrics
333 West River Park Drive
Provo, UT 84604
Phone: (616) 502-4828
Email: ssnell@qualtrics.com
Lisa V. John, PhD, PMP
Project Manager
Battelle
Public Health Research & Translational Science
5712 Oakland Ave.
St. Louis, MO 63110
Office: 314-880-3652
Email: johnl@battelle.org
Amanda Berger, PhD
Principal Research Scientist
Battelle
Public Health Research & Translational Science
6115 Falls Rd, Suite 200
Baltimore, MD 21209
Phone: (410) 372-2757
Email: bergera@battelle.org
Robert L. Alexander Jr., PhD, MPH, CHES
Director
Public Health Research & Translational Science
Battelle
2987 Clairmont Road, Suite 450
Atlanta, GA 30329
Office: 404-460-1462
Email: alexander@battelle.org
Individuals consulted at CDC on the study design are listed below.
Centers
for Disease Control and Prevention 4770 Buford Highway, N.E MS F-79 Atlanta, GA 30341 |
||
Israel Agaku |
Deputy Associate Director for Science |
Phone: 770.488.5138 E-mail: wgn9@cdc.gov |
Diane Beistle |
Chief, Health Communication Branch |
Phone: 770.488.5066 E-mail: zvg1@cdc.gov |
Lauren Boyle-Estheimer |
Scientist |
Phone: 404.498.2283 Email: yjw7@cdc.gov |
Lindsey McCarter |
Health Communication Specialist |
Phone: 770.488.4239 Email: lpq4@cdc.gov |
Michelle O’Hegarty |
Health Communication Specialist
|
Phone: 770.488.5582 E-mail: mohegarty@cdc.gov |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |