SS Part B Experimental Study on Graphic Health Warning Statements

Experimental Study on Warning Statements for Cigarette Graphic Health Warnings

0910-NEW

SUPPORTING STATEMENT PART B

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe for this study is (1) adolescent current cigarette smokers aged 13-17 years old, adolescents who are susceptible to initiation of cigarette smoking aged 13-17 years old; (2) young adult current cigarette smokers aged 18 to 24 years old; and (3) older adult current cigarette smokers aged 25 years old and older.

Study participants will be recruited from a national online panel of adults managed by Lightspeed. Lightspeed panel members will receive an email inviting them to participate in the study. Adolescent children of adult panel participants will be invited to complete the survey through an email invitation to their parents asking for their consent to solicit their child’s opinions. Panel members and children of panelists who choose to participate will complete the questionnaire. Completed interviews will be monitored to ensure samples are diverse in terms of age, gender, education, and ethnicity/race and roughly approximate distributions of these demographics in the U.S. population. We estimate a total of 2,500 respondents will complete the main data collection.

The Agency does not intend to generate nationally representative results or precise estimates of population parameters from the experimental study; generating a representative sample of the size necessary for this study, using Random Digital Dialing or other similar method, would be cost prohibitive. The study will use convenience samples rather than probability samples. Despite the attempt to match the study’s sample and the respondent universe in four demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics. Despite best efforts to have the study population reflect the demographic makeup of the larger population, the nature of convenience samples still limits the generalizability of the results from this study. Thus, the conclusions drawn from this sample, while representative of the sample, may not necessarily generalize to other populations not included in the study. These limitations in generalizability do not affect the internal validity of the study. Additional limitations are that conclusions can only be drawn based on the stimuli presented, and conclusions about warnings in general or warnings when paired with images cannot be made from the present study. Rather, the results of this study will inform the Agency’s future research efforts to test a new set of cigarette graphic health warnings (textual warning statements plus graphics) in fulfilment of the Agency’s statutory obligation under section 201 of the Tobacco Control Act. Such limitations will be noted in the context of describing the results of the study.

2. Procedures for the Collection of Information

For the information collection, Lightspeed will send email invitations to the target audiences using their market research panel. Adult (age 18+) panel members will be sent an email directly inviting their participation in the study and instructions for accessing the secure website for the survey. Adolescents will be invited to participate through an email invitation to an adult panel member who has indicated in their panel profile that they have a child in the eligible age range. Parents or guardians will be asked to provide permission before allowing their child to participate. Once a panel member or child of a panel member enters the secure web site, they will access the screener to determine eligibility based on the study inclusion and exclusion criteria. Those respondents who are determined to be eligible to participate will be presented with a brief introduction informing the participant of the confidential and voluntary nature of the study. Those respondents who are determined to be eligible and provide consent to participate will then be randomly assigned to a condition and complete the study. As data collection progresses, study staff will check the distribution profile of those completing the study. If that distribution does not roughly approximate distributions of these demographics in the U.S. population, Lightspeed may adjust the targeting of invitations to those in the research panel to better ensure a final sample that is more in line with population demographics.

This experimental study will be conducted using an Internet panel and a questionnaire designed to measure responses to textual warning statements that explain the negative health consequences of cigarette smoking—specifically, if the textual warnings statements, which have been revised from those enumerated in section 201 of the Tobacco Control Act (which amends section 4 of the Federal Cigarette Labeling and Advertising Act (FCLAA) (15 U.S.C. 1333)), promote greater public understanding of the risks associated with the use of cigarettes compared to those outlined in section 201 of the Tobacco Control Act. In addition, demographics and tobacco use states will be collected from the respondent.

In the first part of the study, respondents will be randomly assigned to one of 17 conditions. In each condition, respondents will view 9 warning statements, one at a time, centered on the screen presented in 17-point font using black text on a white background, consistent with the provisions of section 201 of the Tobacco Control Act.

Respondents randomized to the control condition will view all 9 of the warning statements listed in section 201 of the Tobacco Control Act, henceforth referred to as the “statutory warning statements”:

S1: WARNING: Cigarettes are addictive.

S2: WARNING: Tobacco smoke can harm your children.

S3: WARNING: Cigarettes cause fatal lung disease.

S4: WARNING: Cigarettes cause cancer.

S5: WARNING: Cigarettes cause strokes and heart disease.

S6: WARNING: Smoking during pregnancy can harm your baby.

S7: WARNING: Smoking can kill you.

S8: WARNING: Tobacco smoke causes fatal lung disease in nonsmokers.

S9: WARNING: Quitting smoking now greatly reduces serious risks to your health.

Respondents randomized to one of the 16 experimental conditions will view 8 of the warning statements listed above plus 1 statement that is a revised version of a statutory text warning henceforth referred to as the “Revised Warning Statements.” In 4 of the 16 conditions (Conditions 12-16), a randomly selected statutory warning statement will be replaced by the revised warning statement; for the other 12 experimental conditions, the statutory warning statement to be replaced is indicated in Table 1 below.

The revised warning statements being tested in this study are:

R1a. WARNING: Smoking causes mouth and throat cancer.

R1b. WARNING: Smoking causes head and neck cancer.

R1c. WARNING: Smoking causes bladder cancer, which can lead to bloody urine.

R2a. WARNING: Smoking during pregnancy causes premature birth.

R2b. WARNING: Smoking during pregnancy stunts fetal growth.

R2c. WARNING: Smoking during pregnancy causes premature birth and low birth weight.

R3a. WARNING: Secondhand smoke causes respiratory illnesses in children, like pneumonia.

R4a. WARNING: Smoking can cause heart disease and strokes by clogging arteries.

R5a. WARNING: Smoking causes COPD, a lung disease that can be fatal.

R5b. WARNING: Smoking causes serious lung diseases like emphysema and chronic bronchitis.

R6a. WARNING: Smoking reduces blood flow, which can cause erectile dysfunction.

R6b. WARNING: Smoking reduces blood flow to the limbs, which can require amputation.

R7a. WARNING: Smoking causes type 2 diabetes, which raises blood sugar.

R8a. WARNING: Smoking causes age-related macular degeneration, which can lead to blindness.

R8b. WARNING: Smoking causes cataracts, which can lead to blindness.

After viewing each statement, all respondents will complete a set of measures assessing prior exposure to the information contained in the warning statement, learning as a result of exposure to the warning statement, and beliefs about the content of the warning statement.

After viewing each of the 9 statements all respondents will then complete measures assessing knowledge of the negative health consequences of cigarette smoking contained in the warning statements. Analyses involve a comparison of each experimental condition to the control condition to assess the influence of each revised statement on the outcomes of interest compared to the original statements.

Table 3. Condition Assignment and Sample Size by Study Population

Condition	Statutory warning statement excluded	Revised warning statement included	Adolescents Respondents	Young Adult Respondents	Older Adults Respondents	Total
Control	None	N/A	183	183	182	548
1	S4	R1a	41	40	41	122
2	S4	R1b	40	41	41	122
3	S4	R1c	41	41	40	122
4	S6	R2a	41	40	41	122
5	S6	R2b	40	41	41	122
6	S6	R2c	41	41	40	122
7	S2	R3a	41	40	41	122
8	S5	R4a	40	41	41	122
9	S8	R5a	41	41	40	122
10	S3	R5a	41	40	41	122
11	S3	R5b	40	41	41	122
12	Random	R6a	41	41	40	122
13	Random	R6b	41	40	41	122
14	Random	R7a	40	41	41	122
15	Random	R8a	41	41	40	122
16	Random	R8b	41	40	41	122
Total			834	833	834	2501

In the next part of the study, respondents view a set of warning statements, rather than individual statements. Respondents in the experimental condition will view a set of 9 warnings comprised only of revised warning statements. This set of 9 revised warnings statements will be presented simultaneously as a group. Respondents in the control condition will view the same 9 statutory warning statements they previously viewed a second time, also presented simultaneously as a group, in order to control for time spent completing the study and exposure to two rounds of warning statements. Respondents in the experimental conditions will view one randomly selected statement per topic area (identified by different numbers in the list above), with the exception of statements focus on cancer (revised statement R1a-c above), for which they will view two of the three randomly selected statements. After viewing the set of 9 warning statements, all respondents will complete a final set of measures assessing knowledge and beliefs about the health consequences of smoking contained in the warning statements. Finally, respondents will complete measures assessing demographics and health literacy. Analyses for this part of the study will be conducted at the level of the set of statements and will involve a comparison of the outcomes between the experimental and control conditions.

RTI, the contract research organization implementing the study, will analyze the data collected from this study, the results of which will inform the Agency’s efforts to implement section 201 of the Tobacco Control Act, which requires FDA issue regulations to implements textual warning statements accompanied by color graphics depicting the negative health consequences of smoking.

Power

To determine statistical power to detect effects, we used Dunnett’s multiple comparison procedure to adjust the alpha level for multiple comparisons for tests at the statement level for both 2- and 1-sided tests. Dunnett’s test is specially made for situations involving multiple comparisons of a control versus the experimental condition. Dunnett’s test takes into account the dependencies arising because each comparison has the control level in common. Dunnett’s power calculations and minimum detectable difference was computed using 500 simulations. We optimized the distribution of sample between experimental and control conditions, and our sample allocation for the study is based on this analysis. Power calculations suggest an optimal allocation of 538 in the control group and 122 in each of the 16 experimental groups. This allocation gives us adequate power for 2- and 1-sided tests using the full sample of n=2,500. As the key outcomes of this study are focused on changes in knowledge of the negative health consequences of cigarette smoking and learning of such information, the power analysis was conducted considering those endpoints. Estimates of effect sizes used in the power analysis were drawn from previously conducted studies with similar methodologies and outcomes as the present study. Given this sample size, the allocation between experimental and control conditions, the adjusted alpha, and a power of 0.8, the minimum detectable difference is 0.38 and 0.35 respectively for 2- and 1-sided tests (medium effect size). Similarly, for analyses done at the level of the set of statements we have adequate power to detect medium effect sizes.

3. Methods to Maximize Response Rates and Deal with Non-response

Experience with online experimental studies suggests that about 10% of adults and 6% of adolescents who are sent survey invitations will complete a study. FDA will implement several procedures to maximize participation. We will conduct a pretest to help ensure understandability of the questionnaire, to reduce participant burden, and to enhance interview administration. We will keep the study questionnaire at a reasonable length to minimize break-offs. Additionally, the following procedures will be used to maximize cooperation and to achieve the desired response rates:

• A brief introductory paragraph will identify FDA as the sponsor of the study, state the purpose of the study, and encourage participation.

• Lightspeed will provide toll-free telephone numbers to all sampled individuals and invite them to call with any questions or concerns about any aspect of the study. RTI will provide a toll-free telephone number for a RTI project member and a toll-free telephone number for the RTI IRB hotline should participants have any questions about the study or their rights as a study participant.

• Lightspeed data collection staff will conduct ongoing monitoring of response levels and drop-off rates, and will work with RTI project staff to address any problems that arise throughout the course of the collection of information.

• Nonrespondents will receive an initial email invitation and up to two email reminders from Lightspeed requesting their participation in the survey.

• In recruiting panelists, Lightspeed uses a double opt-in registration process whereby panelists are invited to participate and then must sign up through an opt-in confirmation email. This process protects against fraudulent account registrations and ensures that panelists are actively motivated to participate in surveys.

• As part of the registration process, panelists provide information about a range of sociodemographic characteristics, including age and smoking status, which can be used to target particular groups in survey recruitment and maximize eligible responses. Lightspeed actively manages panelist profiles, requesting updated information on an ongoing basis to ensure that profile information is up to date.

As with any study conducted using opt-in online panels, this study may be subject to

several threats to external validity that limit the generalizability of study results. Panelists are recruited into the online panel using convenience sampling methods, and thus do not have a known probability of selection into the panel. Recruitment of the study sample from the online panel is also subject to bias resulting from potential differences between survey responders (i.e., panelists who received the invitation and opted to participate in our study) and non-responders (i.e., panelists who were invited but chose not to participate) in characteristics that may be associated with key study outcomes. Because of these limitations, the relationship between treatment and outcomes we find in our study may not generalize to the broader U.S. population. Nevertheless, the experimental design of the study, including random assignment to condition, enhances the internal validity of the study, i.e., the ability to establish a causal relationship between treatment and outcomes. While random assignment does not rule out every threat to internal validity, it does rule out or minimize most. In our case, the internal validity threat that potentially remains is differential attrition between treatment and control groups (i.e., differences between treatment and control groups in the frequency of respondents who begin the survey but drop out before completing). However, there is no reason to believe that our design would lead to those in the treatment groups dropping out of the survey at a different frequency than those in the control group. In addition to randomization, our design, measures, power, and analysis plans are appropriate to ensure that we can draw valid statistical conclusions about the relationship between treatment and outcomes.

We do not anticipate significant item non-response and thus have no plans to utilize imputation procedures. All analyses will be conducted using Stata 14.1 with specific estimators determined by the measurement of the outcome variable and model used.

4. Test of Procedures or Methods to be Undertaken

Measures included are drawn from previously used and/or validated instruments to ensure that instruments are not ambiguous, burdensome, or confusing. Stimuli used in the study have been previously tested in a study conducted between May and June 2015, entitled “Qualitative Study on Cigarettes and Smoking: Knowledge, Beliefs, and Misperceptions,” OMB #0910-0674, and revised based on feedback from respondents in the same groups as will be included in this study. Additionally, RTI will conduct two pretests with survey panelists from Lightspeed to thoroughly test the programmed questionnaire. At the conclusion of the pretest, all strategies, algorithms, and programs for sampling, survey administration, and data compilation will be tested, validated, and readied for launch of the main data collection. The questionnaire and study protocol will be revised, if necessary, based on the pretest findings.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

RTI International will manage the information collection on behalf of FDA. Dr. James Nonnemaker is the project director at RTI. RTI will subcontract to Lightspeed to collect the data. Karl Berner is the project manager at Lightspeed.

Analysis and dissemination of the data will be led by Dr. David Portnoy at FDA’s Center for Tobacco Products and Dr. James Nonnemaker at RTI International.

David B. Portnoy, Ph.D., MPH

Food and Drug Administration

Center for Tobacco Products

Document Control Center

Building 71, Room G335

10903 New Hampshire Avenue

Silver Spring, MD 20993-0002

(301) 796-9298

david.portnoy@fda.hhs.gov

James M. Nonnemaker, Ph.D.

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709-2194

919-541-7064

jnonnemaker@rti.org

Karl Berner

Lightspeed

501 Boylston Street, Suite 6101

Boston, MA 02116

(508) 503-3396

Karl.Berner@lightspeedresearch.com

6