Part B Supporting Statement Experimenatal Study Consumer HPHC list-091812

Experimental Study on the Presentation of Harmful and Potentially Harmful Tobacco Constituents

0910-NEW

SUPPORTING STATEMENT

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe for the experimental study is (1) current adult smokers aged 25 years old and older, (2) young adult smokers aged 18 to 24 years old, (3) youth smokers aged 13 to 17 years old, (4) adult smokeless tobacco users aged 18 and older, and (5) youth age 13 to 17 years old who may be susceptible to initiation of smoking. The five separate quota samples will be selected from the e-Rewards online member panel, a national opt-in email list sample. Sampled panel members will receive an email inviting them to participate in the study. Panel members who choose to participate will complete the questionnaire (see Appendix). Completed surveys will be monitored to ensure samples are diverse in terms of age, gender, education, and ethnicity/race. We estimate 3,150 respondents will complete a survey, and the response rate will be 70%.

The agency does not intend to generate nationally representative results or precise estimates of population parameters from the experimental study; generating a representative sample of the size necessary for this study, using Random Digital Dialing (RDD) or other similar method, would be cost prohibitive. The study will use convenience samples rather than probability samples. Despite the attempt to match the study’s sample to the respondent universe in five demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics.

FDA does not intend to draw a sample of youth that use smokeless tobacco products exclusively. According to the National Survey on Drug Use and Health, fewer than 3% of youth have used smokeless tobacco in the past 30 days. Of those, approximately half have also used cigarettes. (See http://www.oas.samhsa.gov/2k9/smokelessTobacco/smokelessTobacco.pdf). FDA intends to screen for both smokeless and cigarette use and include all youth that indicate smokeless use and supplement the sample with those that use cigarettes. FDA has revised the sample grid to reflect that the sample of youth viewing the smokeless HPHC lists will be “Youth Tobacco Users”.

FDA will use quota sampling in an attempt to achieve specific response rates for certain groups, such as low education (High School diploma or less), African Americans, and Hispanic youth.

2. Procedures for the Collection of Information

For the information collection, e-Rewards will send email invitations to the target audiences using their market research panel. Each invitation will contain the survey title, the length of the survey, e-Rewards incentive amount provided for successful completion of the survey, and instructions for accessing the secure website for the survey. Once a panel member enters the secure web site, a brief introduction will be presented informing the panel member of the private and voluntary nature of the survey (see attached). Individuals who consent to participate in the survey will be able to access the survey by clicking on the link to the survey URL. Respondents who access the questionnaire will be randomly assigned to an experimental condition.

This experimental study will be conducted using an Internet panel and a questionnaire designed to measure responses to the HPHC list formats and collect demographic and smoking status information from the participant. Participants will be randomly assigned to an exposure or control condition. Participants in the exposure condition will view one of 6 HPHC list formats (see Appendix B) for a hypothetical brand of cigarettes, smokeless tobacco product, or roll-your-own tobacco product. RTI will analyze information collected from the study, the results of which will inform FDA’s implementation of a publicly displayed list of HPHCs by brand and subbrand as required by the Tobacco Control Act.

The proposed design implements the experimental study with each of five target groups: (1) current adult smokers aged 25 years old and older, (2) young adult smokers aged 18 to 24 years old, (3) youth smokers aged 13 to 17 years old, (4) adult smokeless tobacco users aged 18 and older, and (5) youth age 13 to 17 years old who may be susceptible to initiation of smoking. In the experiment, we are testing 6 HPHC list formats and 1 control condition (no HPHC list seen) for either a hypothetical cigarette, smokeless tobacco product, and roll-you own tobacco product for a total of 63 cells. Half of the formats will contain additional written information and half will not (see Appendix B), Each cell will contain 50 respondents for a total sample size of 3,150 (See Table 1).

It is the practice of FDA to place an OMB burden statement and control number on all study materials viewed by participants once OMB has approved the package and provided FDA the OMB control number.

Table 1

Product	Population	Information	Format 1 (Full List)	Format 2 (Tested/ not tested)	Format 3 (Tested only)	No List (control)
Cigarettes	Adult smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Young adult smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Youth smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Youth at risk for initiation (n=350)	Present	50	50	50	50
		Not present	50	50	50
Smokeless	Adult smokeless (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Youth smoker (n=300)	Present	50	50	50	50
		Not present	50	50	50
RYO	Adult smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Young adult smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
	Youth smoker (n=350)	Present	50	50	50	50
		Not present	50	50	50
N=3,150			900	900	900	450

Summary of Protocol

Survey:

• Survey screener – confirm eligibility.

• Random assignment to treatment or control.

• Treatment groups are exposed to one of 6 HPHC list formats for either a cigarette, smokeless tobacco, or roll-your-own tobacco product; Control groups are not exposed to a list.

• Respondents answer questions while being exposed to a HPHC list. The questions assess general comprehension about HPHCs in tobacco products. Additional questions assess susceptibility to initiation of tobacco use (youth high risk non-smokers), motivation and confidence to quit tobacco use, and risk perceptions about tobacco use.

Measures

Key Outcomes

• Comprehension - total correct responses to questions assessing the communication objectives

• Risk Perception

  • Perceptions of harmfulness of the stimulus product (i.e. brand X cigarette)

  • Perceptions of harmfulness of tobacco use

• Unintended consequences

  • Quit intentions

  • Openness to smoking (youth)

Covariates:

Age, gender, race, SES (income and education), health literacy

Analysis plan

Primary analyses

1. Tests of treatment effects on comprehension:

Hypotheses 1: Exposure to the list of HPHCs will improve comprehension of communication objectives, and thus treatment groups will have higher comprehension scores relative to control groups.

Hypothesis 2: The presentation of supplemental information will improve comprehension and thus treatment groups presented with information will have higher comprehension scores relative to groups not presented with information.

2. Contrasts between treatment groups: comparison of list formats to ascertain relative effectiveness.

• For comprehension of communication objectives, contrasts will be made between treatment groups (adjusted for multiple comparisons) to assess group differences in understanding based on list format.

Secondary Analyses

3. Tests for group differences in risk perceptions, quit intentions and susceptibility to initiation, based on list exposure and list format.

4. Test for moderating effects of product type and population: Tests for interaction effects will be used to determine if the impact of list format on key outcomes differs by (a) product type and (b) age group.

The sample design is adequately powered to test the primary research hypotheses:

• Exposure to a list of HPHCs, with supplemental information, will result in better comprehension of the communication objectives.

The experimental design includes three list format conditions (with and without additional information) and a control group, with a total of 3,150 participants. Each list format condition (with and without additional information) includes 900 participants, for a total of 2700 exposure participants, and the control group includes 450 participants. Our primary test compares exposure participants to control participants.

For the purpose of sample size calculations, the proportion of participants who comprehend communication objectives will serve as the outcome. Our calculations assume comprehension is assessed as a binary outcome. Data will be analyzed using a logistic regression model with no covariates. Based on these assumptions, the test statistic will have 80% statistical power to observe a difference of 8.2 percentage points or larger if the comprehension rate among the control group is low (e.g., 50%). If the comprehension rate among the control group is higher (e.g., 75%), the test statistic will have 80% statistical power to observe a difference of 6.7 percentage points or larger. This power analysis also applies to examples of measures that employ Likert scale ratings, as these measures will be dichotomized prior to analysis, e.g., assigning high ratings a value of 1 and low ratings a value of 0. For outcomes based on a continuous measure, the sample design is powered to detect statistically significant differences of 0.17 standard deviation units or greater. For a scale with a range of 0 – 100, a mean value of 50, and a standard deviation of 15, the test will have 80% statistical power can detect a significant difference between the exposure and control conditions of 2.55 points or greater.

Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are involved.

Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time survey data collection effort.

3. Methods to Maximize Response Rates

Experience with online experimental studies suggests that about 15% of those who are sent survey invitations will complete a study. FDA will implement several procedures to maximize participation. We will conduct cognitive interviews and pretests to help improve understandability of the questionnaire, to reduce participant burden, and to enhance interview administration. We will keep the study questionnaire at a reasonable length to minimize break- offs. Additionally, the following procedures will be used to maximize cooperation and to achieve the desired response rates:

• A brief introductory paragraph will identify FDA as the sponsor of the study, state the purpose of the study, and encourage participation.

• e-Rewards will provide toll-free telephone numbers to all sampled individuals and invite them to call with any questions or concerns about any aspect of the study. RTI will provide a toll-free telephone number for a RTI project member and a toll-free telephone number for the RTI IRB hotline should participants have any questions about the study or their rights as a study participant.

• e-Rewards data collection staff will work with RTI project staff to address any problems that arise throughout the course of the collection of information.

• Nonrespondents will receive one e-mail reminder from e-Rewards requesting their participation in the survey.

4. Tests of Procedures or Methods

RTI will conduct 9 cognitive interviews with a mix of adult smokers and youth smokers and nonsmokers to evaluate and refine the draft questionnaire (see Appendix A). The cognitive interviews will help to identify areas where the instrument was ambiguous, burdensome, or confusing for respondents and the survey will be revised accordingly.

Additionally, we will conduct a pretest with individuals who are Federal employees to thoroughly test the programmed questionnaire. This pretest will be used solely to identify any errors in programming and assess questionnaire flow. At the conclusion of the pretest, all strategies, algorithms, and programs for sampling, survey administration and data compilation will be tested, validated, and readied for launch of the Internet experimental survey. The questionnaire may be revised based on the pretest findings.

5. Individuals Involved in Statistical Consultation and Information Collection

RTI International will manage the information collection on behalf of FDA. Jon Blitstein is the project director at RTI. RTI will subcontract to e-Rewards to collect the data. Bethany Moffett is the project manager at e-Rewards. Analysis and dissemination of the data will be led by Laura Shay at FDA’s Center for Tobacco Products.

Conrad J. Choinière, Ph.D.

Food and Drug Administration, Center for Tobacco Products

9200 Corporate Blvd

Rockville, MD 20850

(301) 796-9228

conrad.choiniere@fda.hhs.gov

Laura Shay, Ph.D., R.N., C-NP

Food and Drug Administration, Center for Tobacco Products

9200 Corporate Blvd

Rockville, MD 20850

(301) 796-0994

laura.shay@fda.hhs.gov

Jon Blitstein, Ph.D.

RTI International

3040 Cornwallis Road

PO Box 12194

Research Triangle Park, NC. 27709-2194

(919) 541-7064

jnonnemaker@rti.org

 

Bethany D. Moffett
e-Rewards Market Research
8401 N. Central Expressway, Suite 900
Dallas, TX  75225

(214) 365-5024

bmoffett@e-rewards.com

6