Rationale for Conducting Media Tracking

ATTACHMENT 4: RATIONALE FOR CONDUCTING MEDIA TRACKING

Mass media campaign evaluation routinely relies on tracking studies to measure the reach and receptivity to campaign advertisements among their target audiences. Media tracking surveys are a critical tool for both campaign planners and evaluators to assess the immediate influence of media buys. Two important tracking measures—campaign awareness and perceived ad effectiveness—will be measured across the planned campaigns. Awareness of campaign advertising is a CDC recommended (CDC, 2007) and commonly used measure of campaign exposure, because it is associated with cumulative and weekly GRPs and can be linked with other individual-level survey data, including beliefs, attitudes and behaviors (Southwell, 2002; Niederdeppe, 2005). Youth awareness of the campaigns, as measured by self-reported recall and recognition of campaign advertisements, is the first critical step necessary in order to influence tobacco initiation. For this reason, it is important to measure and document levels of awareness within the first year of the campaign. In fact, the CDC suggests that new campaigns that reach 75% to 85% of the target audience can expect to produce attitude and behavior change within the following two years (CDC, 2007). Further, research suggests that the mere presence of ads does not guarantee that target audiences have engaged with the messages in a meaningful way (Southwell et al., 2002). Messages must not only be viewed and remembered but must also be attended to, understood, and perceived as persuasive. Perceived effectiveness measures in the tracking survey will gauge the persuasiveness of campaign messages, perceptions about the salience of its messages, and other general impressions about the campaign (e.g. Davis et al., 2011).

Because attitudinal and behavioral changes in regard to tobacco use among youth are, to a certain extent, a function of the level of cognitive processing that occurs in response to a campaign message, campaign awareness and perceived effectiveness serve as early indicators of the success of the media strategies and may allow for early corrections as necessary. Importantly, campaign planners can understand the extent to which the campaign’s media strategy is reaching the target audience with salient messages among key groups of interest after the first few months on air (see Attachment 2a for more detail on the expectations of campaign effects). For example, evaluators of the truth campaign used media tracking surveys to assess the relationship between campaign exposure and changes in beliefs about smoking within 9 months of launch; these data were valuable in that they: 1) served as an indicator of future behavior change, and 2) contributed to our understanding of the mechanisms by which the campaign ultimately changed youth prevalence (Farrelly, 2002). Media tracking data were also used to understand truth campaign impact among racial/ethnic minorities; they showed that different advertisements appealed to youth based on their race/ethnicity (Cowell, 2009). In addition, continued and consistent tracking of campaign awareness and receptivity to campaign messages is necessary because campaign effects may diminish based on media purchase patterns (Wakefield et al., 20011).

Table 1-1 describes the key evaluation questions to be addressed in the media tracking survey.

Table ‎1-1. Campaign Key Evaluation Questions, Media Tracking Survey

Campaign Awareness and Receptivity

What proportion of 12- to 17-year-old youth is aware of specific campaign ads by media channel?

What proportion of 12- to 17-year-old youth is aware of ANY campaign ad overall and by media channel?

How does awareness vary by demographic subgroups, such as race/ethnicity, gender, population density, and/or by tobacco use?

What are youth’s reactions to campaign media?

How do youth’s reactions to campaign media vary by subgroups?

To address the evaluation questions, we will conduct three Web-based, cross-sectional media tracking surveys, each with a unique sample of 4,000 youth. Surveys will be conducted approximately 4 months after campaign launch and at 8-month intervals throughout the evaluation period. Power calculations were conducted to determine the appropriate sample size to understand campaign awareness and receptivity among several key subpopulation groups. The sample size is calculated based on CTP requirements to assess campaign awareness and ad-level awareness by gender, age (aged 12 to 14 and aged 15 to 17), geographic area (rural and non-rural), and tobacco use susceptibility. Each media tracking survey will provide approximately 250 youth in each of sixteen segments. For the purposes of estimating statistical power, we assume that the test statistic evaluating subpopulation differences in campaign awareness will involve a two-tailed hypothesis test with a Type I error rate of 0.05 and a Type II error rate of 0.020, yielding 80% statistical power. Analyses of the media tracking will consist primarily of summary statistics on awareness of and reactions to campaign advertisements within the sixteen subpopulations noted.

RTI will conduct this study using a sample of youth purchased from the digital data collection company Global Market Insite, Inc. (GMI). GMI will distribute a link to these RTI-programmed, RTI-housed Web-based surveys to approximately 40,000 members of their youth panel and track the number of completed surveys remotely using redirect links until we reach our target of 4,000 for each survey. Youth will be advised of the privacy of their data and be asked to provide their assent to participate before encountering the first survey question. All data will be disassociated with names, addresses, and other identifying information to ensure respondent privacy to the fullest extent of the law, and all data will be entered directly into secure RTI servers.

The data collected are not a nationally representative sample and are not intended to draw conclusions at the population-level. The primary outcome evaluation will be the source for all public information about the campaign. The media tracking surveys allow FDA to track campaign awareness and perceived ad effectiveness for internal planning purposes only.

While measures of campaign awareness and perceived effectiveness are included in the outcome evaluation survey, the addition of surveys early after launch and at additional time points to the current outcome evaluation study design would be harmful to the evaluation. Because youth are exposed to campaign advertisements during the survey, additional surveys would artificially increase campaign awareness, especially at later waves. Also, the attrition introduced by these additional surveys would substantially increase the baseline sample sizes required as well as increase the likelihood of survey response bias over time.

Table 1-2 provides the survey items enumerated by type and a description of how the various survey measures will be used in analyses.

Table 1-2. Survey Items by Type and Intended in Analysis, Media Tracking Survey

Type of Item	Survey Items	Intended Use in Analysis of Media Tracking Data
Demographics	S1, C1 – C5	stratification variable
Campaign Awareness	A3 – A9	outcome variable
Campaign Receptivity	A10 – A15	outcome variable
Tobacco-related Attitudes, Beliefs, Risk Perceptions, and Social Norms	B11 – B21	control variable
Ever Use of Tobacco	B1 – B2, B7	stratification variable
Tobacco-Related Intentions Behaviors	B3 – B6, B8 - 10
Media Use	A1 – A2, A16 – A19	control variable
Youth Environment	C6 – C12	control variable

REFERENCES

Centers for Disease Control and Prevention. Best Practices for Comprehensive Tobacco Control Programs—2007. . Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health;2007.

Cowell AJ, Farrelly MC, Chou R, Vallone DM. Assessing the impact of the national 'truth' antismoking campaign on beliefs, attitudes, and intent to smoke by race/ethnicity. Ethnicity & health. Feb 2009;14(1):75-91.

Davis, K. C., Nonnemaker, J. M., Farrelly, M. C., & Niederdeppe, J. (2011). Exploring differences in smokers’ perceptions of the effectiveness of cessation media messages. Tobacco Control. 20, 26–33.

Farrelly MC, Healton CG, Davis KC, Messeri P, Hersey JC, Haviland ML. Getting to the truth: evaluating national tobacco countermarketing campaigns. American journal of public health. Jun 2002;92(6):901-907.

Southwell, B. G., Barmada, C. H., Hornik, R. C., & Maklin, D. M. (2002). Can we measure encoded exposure? Evidence from a national campaign. Journal of Health Communication 7, 445–453.

Niederdeppe J. Assessing the validity of confirmed AD recall measures for public health communication campaign evaluation. Journal of health communication. Oct-Nov 2005;10(7):635-650.

Wakefield, M. A., Spittal, M. J., Yong, H-H., Durkin, S. J., & Borland, R. (2011). Effects of mass media campaign exposure intensity and durability on quit attempts in a population-based cohort study. Health Education Research, 26(6), 988–997.