OMB Final Supporting Statement Part B 3-31-2016

Quantitative Information in Direct-to-Consumer Television Advertisements

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

The study will be conducted with Research Now, a national market research firm. The sample will come from Research Now’s e-Rewards® Opinion Panel. There are approximately three million panel members in the e-Rewards panel.

The target population is the adult, noninstitutionalized population in the United States aged 60 and older who have access to the Internet. The sample will not be representative of the population, but the goal will be to recruit a sample that has equal proportions of males and females, at least 10% African American, and equal distributions of participants with these education levels: 50% completed high school or less than high school; 50% completed some college or higher.

Adult panel members 60 years of age and older will be invited to participate by receiving an e-mail invitation (Appendix D) and, if interested, can click on a hyperlink within the e-mail and gain access to the screener (Appendix E). The screener will ask participants to confirm their age and other demographics (level of educational attainment, gender, ethnicity, and race). After participants are screened, those who are eligible will be randomly assigned to conditions.

2. Procedures for the Collection of Information

Design Overview

In Study 1, we plan to examine experimentally the presence and complexity of quantitative benefit and risk information in DTC television ads (Table 4). In Study 2, we plan to examine experimentally the presence of quantitative benefit information and how the ad visually represents efficacy (by having no images, images that accurately reflect the improvement in health that could be expected with treatment, or images that overstate the improvement in health that could be expected with treatment; Table 5).

Table 4. Study 1 Design.
		Quantitative Risk Claim
		No	Yes: General (e.g., Side effects that occur in 10% or less of people who take Drug X include…)	Yes: Specific (e.g., Side effects that occur in [6-10%, 1-5%, and less than 1%] of people who take Drug X include…)
Quantitative Efficacy Claim	No
	Yes: Single outcome (e.g., 52% of people with cataracts improved their vision to 20/40 while taking Drug X, compared to 23% without Drug X. [starting at an average baseline of 20/70])
	Yes: Multiple outcomes (e.g., 52% of people with cataracts improved their vision to 20/40 while taking Drug X, compared to 23% without Drug X. [starting at an average baseline of 20/70]. With Drug X, people could see an average of 85 letters on a 100-letter eye chart, compared to 73 letters without Drug X.)

Table 5. Study 2 Design.
		Images of Improvement
		None	Accurate improvement in health conveyed in images	Overstated improvement in health conveyed in images
Quantitative Benefit Claim	No
	Yes (Single outcome)

Procedure

We plan to conduct two 20-minute studies of 900 participants each. The studies will be conducted using an Internet panel.

In both studies, participants will be randomly assigned to one experimental condition and view the corresponding television ad. The ad will be for a fictitious drug to treat cataracts. The ads will be created and pretested to ensure that consumers perceive different levels of complexity across the ads in Study 1 and different levels of image accuracy in Study 2 (“Pretests for a Study on Quantitative Information in Direct-to-Consumer Television Advertisements” was submitted under OMB Control Number 0910-0695). After viewing the ad twice, participants will complete a questionnaire that assesses consumers’ understanding of the drug information, their retention of the information, and their perceptions of the drug (Appendices B and C). We will also measure covariates such as demographics and numeracy.

Participants

All participants will be Internet panel members 60 years of age or older. We will exclude individuals who work in healthcare or marketing. We selected a sample of participants 60 years and older to increase the likelihood that participants will be interested in the fictitious study drug and therefore motivated to pay attention to the ad during the study. Panel members can only participate in one of the studies and participants cannot have participated in the pretests for these studies.

Hypotheses

In Study 1, we hypothesize that, replicating past studies, adding simple quantitative information about benefits and risks will lead to increased understanding among consumers. We will test whether adding complex quantitative information results in the same outcomes as simple quantitative information or whether it is too much quantitative information for consumers to process. In Study 2, we hypothesize that overstated images of improvement will lead consumers to overestimate the drug’s efficacy; however, adding a quantitative claim may moderate this effect.

Analysis Plan

We will conduct ANOVAs (for continuous variables) and logistic regressions (for dichotomous variables) with interaction terms and planned comparisons to test the hypotheses outline above. If a main effect is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another, with p-values adjusted for multiple comparisons.

Power

The following tables show the power calculations for the main studies. The assumptions made in deriving the sample size for each study were: 1) 0.90 power, 2) 0.05 alpha and 3) an effect size between small and medium.

Table 6. – Study 1: A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, main effects, and interactions) to achieve power of 0.90 (Faul et al., 2007).
	Main effects			Post-hoc comparisons among conditions
Effect size f*	0.10	0.15	0.20	0.10	0.15	0.20
α error probability	0.05	0.05	0.05	.001	.001	.001
Power (1 – β error probability)	0.90	0.90	0.90	0.90	0.90	0.90
Numerator df	2	2	2	1	1	1
Number of groups	9	9	9	2	2	2
Total Sample Size	1269	566	320	2389	1066	603

*An effect size of 0.10 is traditionally considered small, whereas an effect size of 0.25 is considered medium (Cohen, 1988).¹ Here we have shown three different effect sizes centering around small to medium effects.

With a total of 900 participants (100 participants per cell) in Study 1, we will be able to detect small effects in the test of the main effects and small-to-medium effects in post-hoc comparisons.

Table 7. – Study 2: A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, main effects, and interactions) to achieve power of 0.90 (Faul et al., 2007).2
	Main effects			Post-hoc comparisons among conditions
Effect size f*	0.10	0.15	0.20	0.10	0.15	0.20
α error probability	0.05	0.05	0.05	.003	.003	.003
Power (1 – β error probability)	0.90	0.90	0.90	0.90	0.90	0.90
Numerator df	2	2	2	1	1	1
Number of groups	6	6	6	2	2	2
Total Sample Size	1269	566	320	2088	931	527

*An effect size of 0.10 is traditionally considered small, whereas an effect size of 0.25 is considered medium (Cohen, 1988).³ Here we have shown three different effect sizes centering around small to medium effects.

With a total of 900 participants (150 participants per cell) in Study 2, we will be able to detect small effects in the test of the main effects and small-to-medium effects in post-hoc comparisons.

3. Methods to Maximize Response Rates and Deal with Non-response

This experimental study will use an existing research panel to draw a sample.  The panel comprises individuals who have signed up to participate in online studies.  To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the experiment over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Email a reminder to the respondents who do not complete the protocol after the original invitation to participate is sent.

4. Test of Procedures or Methods to be Undertaken

In a separate data collection (OMB Control Number 0910-0695; “Pretests for a Study on Quantitative Information in Direct-to-Consumer Television Advertisements”) we will create and test the stimuli (DTC television ads) to be used in this study.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, RTI, will collect the data on behalf of FDA as a task order under Contract HHSF223201400474G. Linda Squiers, Ph.D., 919-597-5128, is RTI’s Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Helen W. Sullivan, Ph.D., M.P.H., 301-796-4188, and Amie C. O’Donoghue, Ph.D., 301-796-0574.

1 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

2 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A, (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

3 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

5