0915_Supporting Statement_A The Real Cost Campaign Outcomes Evaluation Study_10-2024

U.S. Food and Drug Administration

Center for Tobacco Products

The Real Cost Campaign Outcomes Evaluation Study: Cohort 3 (Outcomes Study)

OMB Control No. 0910-0915

Supporting Statement Part A

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

This information collection supports the U.S. Food and Drug Administration’s (FDA) efforts to assess campaign effectiveness. The Center for Tobacco Products (CTP) is conducting a quantitative study of the effects of FDA’s The Real Cost campaign on youth in the U.S. In 2019, 27.5% of high school students reported vaping and 31.2 % reported use of any tobacco product in the past 30 days (Wang et al., 2019). From 2017 to 2018, reported prevalence of current and daily vaping among young adults ages 18 – 24 years increased by 46.2% (Dai & Leventhal, 2019) and 38% of young adults were current users of a tobacco product in 2013 and 2014 (Kasza et al., 2017). As a way to reduce the enormous public health burden of tobacco use, the Family Smoking Prevention and Tobacco Control Act has given the FDA the authority to take action to protect children, encourage smokers to quit, and reduce tobacco-related disease and death. The law also enables FDA to educate the public, especially young people, about the dangers of tobacco products. Research shows that public education mass media campaigns can be used to change attitudes and beliefs about tobacco use and reduce smoking prevalence which is supported by the Centers for Disease Control and Prevention (CDC) who considers mass media campaigns to be a “best practice” for tobacco control.

FDA launched “The Real Cost” in February 2014, seeking to reduce tobacco use among at-risk youth ages 12–17 in the United States who are open to smoking cigarettes and/or using electronic nicotine delivery systems (ENDS) products, or have already experimented with cigarettes and/or ENDS products. Complementary evaluation studies, including the "Evaluation of FDA’s Public Education Campaign on Teen Tobacco (ExPECTT)," were designed and implemented to measure awareness of and exposure to “The Real Cost” paid media campaign among youth ages 12–17 in targeted areas of the United States. The first cohort (ExPECTT: Cohort 1) assessed the campaign’s impact on outcome variables of interest from November 2013 – November 2016. The second cohort (ExPECTT: Cohort 2) has been assessing the campaign’s impact on outcome variables of interest from June 2018 and will run through September 2022. To continue assessing the impact of “The Real Cost” campaign, FDA will implement The Real Cost Campaign Outcomes Evaluation Study: Cohort 3. The main study will consist of a baseline survey and five follow-up (FU) surveys – FU1, FU2, FU3, FU4, and FU5. Baseline surveys will be administered to youth ages 11-20. Surveys of youth will be conducted in the United States to measure the effectiveness of FDA’s “The Real Cost” campaign. In addition to the main data collection for the Outcomes Study, we intend to supplement the data collection with a convenience sample of youth ages 14-20 who identify as LGBTQ+ or who have a mental health disorder utilizing social media recruitment. These subpopulations have been shown to be at higher risk of initiating use of cigarettes and ENDS products.

The purpose of FDA’s The Real Cost Campaign Outcomes Evaluation Study: Cohort 3 is to evaluate whether changes in key outcomes can be attributed to campaign exposure. We intend to measure self-reported campaign exposure to media advertising, which among many things, will enable FDA to assess its relationship with market-level delivery.

The study will be conducted using web-based surveys that are self-administered on personal computers or web enabled mobile devices. We will survey 6,000 youth at baseline and maintain five follow-up waves with sample sizes of at least 4,800 respondents. The survey will take approximately 30 minutes to complete per participant. This survey will include questions on youth’s awareness of campaign advertisements, tobacco-related beliefs, and psychosocial predictors and precursors to tobacco-use behavior. For the supplemental data collection, RTI will administer the online surveys with 1,500 participants in subpopulations shown to be at higher risk of initiating use of cigarettes and electronic nicotine delivery systems (ENDS) products, such as youth who identify as LGBTQ+, and youth who have a mental health disorder recruited through social media (e.g., Facebook, Instagram). All respondents who complete the supplemental baseline survey will be invited to participate in each of the follow-up surveys, which will occur approximately every six months, in conjunction with the main survey, over a five-year period.

2. Purpose and Use of the Information

The information obtained from this data collection will be used to inform FDA, policy makers in the United States, prevention practitioners, and researchers about: (a) the extent of youth’s exposure to campaign messages, and (b) the extent to which this exposure is associated with changes in intended outcomes. While not exhaustive, the list below illustrates a range of purposes and uses for the information collection:

• Provide critical data on the reach of the campaign among youth in the United States; particularly estimates of the proportion of the population that was exposed to the campaign.

• Understand the influence of the campaign on specific beliefs targeted by messages (message-targeted beliefs).

• Understand the impact of the campaign on psychosocial predictors and precursors of tobacco use behavior, such as:

• Health and addiction risk perceptions 

• Perceived loss of control or threat to freedom expected from tobacco use 

• Anticipated guilt, shame, and regret from tobacco use 

• Tobacco use susceptibility  

• Intention or willingness to use tobacco 

• Intention to quit and/or reduce daily consumption 

We are using survey testing and subject matter expert (SME) input to aid in survey clarity (e.g., address typos and decrease confusing wording) and skip pattern testing. Based on survey testing and SME input of follow-up waves, some items may be updated non-substantively for clarity or for accuracy given the evolving science around the health harms of e-cigarette use (e.g., changing “will” to “may” when stating possible health harms of e-cigarette use). Based on skip pattern testing and instrument length, some items may be excluded.

3. Use of Information Technology and Burden Reduction

The main and supplemental study survey data collection will be web-based. Respondents can take the survey on a personal computer, smartphone, or tablet at a time and location of their choosing. This type of data collection allows the respondent to be candid with their responses. It also increases accuracy of the data because respondents tend to provide more honest responses with this method, as compared to other types of data collection methods, especially when it is clear that the answers will remain private. In addition, using a web-based survey will allow for more participants to respond in a cost-effective and timely manner compared to in-person data collection. The self-administered, web-based survey permits greater expediency with respect to data processing and analysis (e.g., several back-end processing steps, including coding and data entry are automatic instead of manually processed) because data are transmitted electronically, rather than by mail. These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. An added benefit is increased data protection by limiting the amount of personally identifiable information (PII) collected from participants, reducing the risk of data security issues. Finally, as noted above, this technology permits respondents to complete the survey in privacy. The use of a more private data collection method makes reporting on potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) feel less threatening and enhances response validity and response rates.

4. Efforts to Identify Duplication and Use of Similar Information

FDA’s “The Real Cost” campaign includes specific messages delivered through a variety of advertisements. There are no existing data sources that contain measures on awareness of and exposure to these campaign messages, a requirement in order to determine the association between campaign exposure and intended campaign outcomes. In designing the proposed data collection activities, we took several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets would address the proposed study questions. We carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address FDA’s need for evaluation data on the campaign. Data sources we examined for this purpose include ongoing national surveillance systems such as the National Youth Tobacco Survey (NYTS), the Youth Risk Behavior Surveillance System (YRBSS), the National Health Interview Survey (NHIS), and the Population Assessment of Tobacco and Health (PATH). We also reviewed data collected to evaluate other national tobacco-focused media campaigns such as CDC’s Tips From Former Smokers campaign. We concluded that these data sources do not include the measures, or frequency of data collection, needed to evaluate the potential effects of the campaign. This information collection therefore does not duplicate previous efforts.

5. Impact on Small Businesses or Other Small Entities

Respondents in this study will be members of the general public, specific subpopulations or specific professions, not business entities. No impact on small businesses or other small entities is anticipated.

6. Consequence of Collecting the Information Less Frequently

Participants in this evaluation study are surveyed on approximately a six-month basis. While there are no legal obstacles to reduce burden, lack of information needed to evaluate the FDA’s youth tobacco education campaign may impede the federal government’s efforts to improve public health. Without the information collection requested for this evaluation study, it would be difficult to determine the value or impact of the campaign on the lives of the people they are intended to serve. Failure to collect these data could reduce effective use of FDA’s program resources to benefit youth in the United States. Careful consideration has been given to how frequently the campaign’s intended audience should be surveyed for evaluation purposes. We believe the longitudinal evaluation design will provide sufficient data to evaluate the campaign effectively.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2). The message testing activities fully comply with the guidelines in 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice for public comment in the Federal Register of July 26, 2022 (87 FR 44409). FDA received one PRA related comment.

(Comment) The commentor does not support this data collection and expressed concerns with collecting data from those who identify as LGBTQ+. The rational for not collecting these data is because those who identify as LGBTQ+ are at risk for privacy and security concerns by asking them to report their sexual orientation or gender identification. The commentor believes this type of questioning is invasive and may expose LGBTQ+ members to further bias and discrimination. Further, the commentor believes that FDA’s proposal to target LGBTQ+ youth aged 11-17 is concerning as youth can be particularly vulnerable to exploitation for two reasons: (1) their minds are still developing, and (2) “function creep” occurs when data is collected for one reason and can then be utilized for other, non-intended purposes.

(Response) FDA appreciates the comment in response to the 60-day. We provide more information below about why this is an important opportunity to support LGBTQ+ youth populations and how the FDA is proposing to carry out in a manner that minimizes risks, while building credible and useful evidence about LGBTQ+ youth populations. This data will be used to inform tobacco public education campaigns that aim to reduce tobacco use disparities, including among LGBTQ+ populations. Recent data from the 2021 National Youth Tobacco Survey demonstrates that teens who are sexual or gender minorities have higher rates of cigarette and e-cigarette use compared to heterosexual teens. For example, 6.0% of heterosexual teens reported ever experimenting with cigarettes, compared to 10.9% of gay or lesbian teens, 15.6% of bisexual male teens, 14.0% of bisexual female teens, and 11.2% of teens who are transgender. Furthermore, 17.9% of heterosexual teens reported ever using e-cigarettes, compared to 27.3% of bisexual male teens, 29.6% of bisexual female teens, and 30.7% of teens who are transgender. This is credible evidence as to why LGBTQ+ youth are priority populations when it comes to minimizing health disparities.

The cited negative impact raised by the commentor, in which data collected are misused to the detriment of LGBTQ+ youth, is mitigated by the extensive, specific, and efficacious measures and practices put in place by the FDA and its contractors to secure data privacy and avoid individual harm. This is not a broad data collection effort but rather data collection limited in nature solely for the purpose of collecting data to answer a circumscribed set of questions that will support the FDA’s mission of Protecting and Promoting Public Health which includes LGBTQ+ youth populations.

Advancing Health Equity.

The FDA is committed to advancing health equity for LGBTQ+ youth. In accordance with the White House Executive Order on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals, this evaluation and its proposed data collection is one way in which FDA can meet its mission to protect youth, improve public health, and advance health equity for all Americans. Collecting data on the health and health behaviors (e.g., tobacco use) of LGBTQ+ youth is crucial to advancing health equity and reducing health disparities in these populations. The rationale for oversampling LGBTQ+ youth is supported by decades of research showing significant disparities in tobacco use by gender identity (e.g., Johnson et al, 2019; Delahanty et al, 2019), and sexual orientation (e.g., Johnson et al, 2019; McCabe et al., 2018). In order to ensure that future tobacco public education campaigns effectively reach youth demographic groups experiencing tobacco use disparities (e.g., LGBTQ+ youth), it is necessary to collect data on these populations. Below we provide more information on how the FDA is taking the necessary precautions to maximize data privacy, security, and limiting data use in order to advance scientific knowledge of LGBTQ+ youth populations.

Federal Priority to Advance LGBTQ+ Equality.

Recently, the White House issued the Executive Order on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals, which outlines priority areas for federal agencies to collect sexual orientation and gender identity (SOGI) data. There are two areas in the order that makes the collection of SOGI data critical: (1) “advancing equity and full inclusion for LGBTQI+ individuals requires that the Federal Government use evidence and data to measure and address the disparities that LGBTQI+ individuals, families, and households face.” and (2) the order mandates Federal Agencies to, “describe disparities faced by LGBTQI+ individuals that could be better understood through Federal statistics and data collection” (White House Executive Order #14075, 2022). The White House further issued a Presidential Memorandum on Restoring Trust in Government Through Scientific Integrity and Evidence-Based Policymaking. This memo outlines the importance of evaluating government programs among demographic groups that are at risk to assess evaluation effects on critical constructs such as social equity and justice (The White House, 2021). Thus, the collection of SOGI data via this evaluation is not only advised at the Executive level, but also necessary to understand the evaluation effects on at-risk populations. By restricting the collection of these data, the issue of health disparities among LGBTQ+ youth will remain and be critically misunderstood, effectively rendering the group invisible for the purposes of resource allocation and support from the federal government.

Data Gaps.

By not addressing the gaps that exist in the data, this evaluation will be underpowered to support the needs of LGBTQ+ youth populations. The Institute of Medicine (IOM) stated the following, “the ability to address these [health] disparities [among LGBTQ+ youth] is hampered by our lack of knowledge about LGBT youth...efforts to survey young people about their sexual orientation have been fraught with difficulties at both the institutional review board and community levels. These barriers have impeded important developmental research” (Institute of Medicine, 2011, pg. 42). A barrier remains in the health communication field to develop interventions aimed at reducing disparities because thoughtful and properly collected data on this at-risk population remains sparse. The FDA cannot support LGBTQ+ youth populations via its programming and resources (i.e., public education campaigns) if it is not aware of how risk factors (such as attitudes, beliefs, and behaviors) may be associated with tobacco use among LGBTQ+ youth.

LGBTQ+ Population Visibility.

In addition to federal requirements to collect SOGI data, LGBTQ+ community advocacy organizations are also encouraging the collection of these data from youth by routinely conducting surveys on the health of LGBTQ+ youth. The intent of these data are to serve, support, and advocate for LGBTQ+ populations. For example, The Human Rights Campaign LGBTQ+ youth report (The Human Rights Campaign Foundation, 2018) and The Trevor Project National Survey on LGBTQ Youth Mental Health (The Trevor Project, 2020) are two community advocacy driven data collections that aim to support the needs of the LGBTQ+ population. Using these data collections, researchers and advocates for LGBTQ+ advocates are trying to gain important insights into the disparities that these youth experience and identify opportunities to address these disparities. Not collecting data from LGBTQ+ in federal data collections would mean being out of touch with groups and people who advocate for this vulnerable population.

Experience Collecting Sensitive Data.

Collecting SOGI data is a commonplace procedure in federal surveys. Several federal surveys of youth populations, including the Population Assessment of Tobacco use and Health (PATH), National Youth Tobacco Survey (NYTS), and Youth Risk Behavior Surveillance System (YRBSS) routinely collect SOGI data (U.S. Department of Health and Human Services, 2022; Centers for Disease Control and Prevention, 2021a; Centers for Disease Control and Prevention, 2021b). Furthermore, several other federal surveys of youth collect information that is equally or more sensitive than collecting information on SOGI. For example, the aforementioned PATH, NYTS, and YRBSS surveys collect information on behaviors for which it is illegal for youth to engage, such as tobacco use, alcohol use, and marijuana use (U.S. Department of Health and Human Services, 2022; Centers for Disease Control and Prevention, 2021a; Centers for Disease Control and Prevention, 2021b). Furthermore, the YRBSS collects comprehensive and sensitive data on sexual behavior among youth in 9th through 12th grades (approximately 14 to 18 years of age) (Centers for Disease Control and Prevention, 2021b). Federal public health surveys routinely collect information on mental health, including suicidality. These are also highly stigmatized conditions/identities, but federal surveys of youth routinely collect these data from youth. For example, the YRBSS asks youth about both mental health concerns and suicidality, and the PATH survey asks about specific mental health diagnoses (Centers for Disease Control and Prevention, 2021b; U.S. Department of Health and Human Services, 2022). Given the frequency with which the federal government conducts surveys that contain SOGI, the FDA and its research partners bring extensive experience in the successful collection, management, and reporting of highly sensitive data to build credible and actionable evidence.

Data Protections and Privacy

Privacy Protection Procedures. Furthermore, the federal government’s data and privacy protection procedures are well-established and reviewed and approved by experts in data privacy. Both HHS and FDA privacy reviewed and approved a Privacy Impact Assessment for this study. We discuss these areas in further detail below.

Data security procedures are of the highest standard and actively implemented to ensure data are used as intended. Data are collected through a web-based survey hosted on two separate platforms; Blaise for mail to web recruitment and Qualtrics for social media recruitment. Both platforms are used widely for statistical and scientific research and are secure. As described in more detail below, there are numerous security measures put in place to guard against breaches of data. To date, no known breaches of data stored on these two platforms have occurred in RTI International (RTI) studies.

The Blaise survey will be hosted on RTI’s secure servers. Data are encrypted using https protocols and stored on secure Structured Query Language (SQL) databases. RTI will use a unique alphanumeric variable (i.e., Case ID) to connect screener data (that contain Personally Identifiable Information (PII)) and survey data (that do not contain PII) and determine if a participant has completed the survey. All data files will be stored together in Blaise within RTI servers. RTI servers are restricted, requiring credentials to access. Each project team is granted as-needed access only to the projects they are assigned and only RTI project team staff who need access to the data (for programming, sampling, recruitment, or analysis) will be granted access within project folders. FDA will not have access to survey data files during data collection. At the completion of data collection, the databases will be deleted from RTI’s Blaise account and remain only on RTI’s secure shared drive and RTI’s Federal Information Processing Standards (FIPS) 199 moderate server. The data on the FIPS 199 moderate server can only be accessed by connecting to a virtual desktop at a physical RTI location. Data files containing PII will be stored on the FIPS 199 moderate server for five years after the project has ended.

The social media recruitment will be hosted on Qualtrics’ secured servers. RTI will use a unique alphanumeric variable (i.e., Case ID) to connect screener data (that contain Personally Identifiable Information (PII)) and survey data (that do not contain PII) and to determine if a participant has completed the survey. Screener data and survey data will be stored separately on Qualtrics servers and encrypted at rest. RTI will use a Secure Sockets Layer (SSL) connection to download data from Qualtrics servers to RTI servers. Once data are downloaded to RTI servers, RTI staff will follow the same protocol described for data collected through Blaise.

Data management procedures are carried out so that PII is stripped from individual records; it is not possible to disclose individual youth identities and experiences. No respondent identifiers will be contained in reports to FDA; results will only be presented in aggregate form and all data will be deidentified before delivery to FDA. Any data made available for public use will undergo additional scrutiny to remove small cell sizes with data, or combinations of data, that could be used to deduce the identity of participants.

FDA's design of this evaluation and associated data collection is meant to be limited in scope to collect enough data to get sufficient information to inform understanding of FDA efforts. FDA has carefully weighed the benefit of collecting these data with potential risks. While there are risks with any research, FDA and its research partners (contractors like RTI) have demonstrated a track record of implementing high quality data collection efforts to support program improvement and ensuring that public resources are most effectively utilized.

Survey respondents have the option to only answer the items that they select to answer. They may end the survey at their discretion. We take every measure to ensure that respondents are aware of their choices and provide assent or consent to participate.

Before requesting any information from participants, RTI will provide information about data security and ways to request data removal from RTI databases. RTI will also inform participants that they can select “prefer not to answer” to any question and will recommend that participants complete the survey where no one can see their answers. RTI provides study information through informed consent and concludes the survey immediately for participants who state that they do not want to take the survey after reading the study information.

Data Suppression Techniques.

Data suppression techniques will be used in addition to protect any PII data from collected from respondents in the evaluation. Data suppression is a tool to help understand in which cases data should not be reported in order to mitigate disclosure of a participant’s identity or in cases where the estimates are deemed to be unstable (i.e., low precision). Guidelines provided from the Center for Tobacco Products (CTP) Office of Science (OS) guidelines, as well as the National Center for Health Statistics (NCHS) will be applied and enforcement of data suppression will occur if any of the following conditions are met: (1) The coefficient of variation of the proportion or estimate is > 30% and/or (2) n < 50, where n is the unweighted sample size in the denominator of the estimated proportion or the denominator used for calculating the estimate. Additional CTP/OS guidelines will be followed, in order to protect sensitive data: (1) Each estimate (or table cell) must be generated based on a numerator of 3, including means, total, numerators of proportions, all table cell counts, and marginal counts. If an estimate is based on a numerator of 1 or 2, it will be combined with another category; (2) table differencing (i.e., calculating the sample size of a small cell from cells of another related table) will be mitigated and will ensure this by using consistent categories across tables; and (3) continuous/ordered variables will be presented so that extreme values pertaining to an individual are not evident.

The following individuals inside the agency have been consulted on the design of the study, instrument development, or intra-agency coordination of information collection efforts:

Lindsay Pitzer

Senior Scientist

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-620-9526

E-mail: lindsay.pitzer@fda.hhs.gov

Hibist Astatke

Social Scientist

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Ave

Silver Spring, MD 20993

Phone: 301-796-1038

E-mail: Hibist.Astatke@fda.hhs.gov

Debra Mekos

Social Scientist

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Ave

Silver Spring, MD 20993

Phone: 301-796-8754

E-mail: Debra.Mekos@fda.hhs.gov

FDA collaborates with other federal government agencies that sponsor or endorse health communication projects, such as the CDC’s, Office on Smoking and Health (CDC/OSH), and the National Institutes of Health National Cancer Institute (NIH/NCI). These affiliations serve as information channels, help prevent redundancy, and promote use of consistent measures of effectiveness. Coordination activities include:

• Review of proposed messages for advertisements;

• Review of questionnaires for testing purposes;

• Sharing data; and

• Standardizing survey tools where at all possible.

The following individuals outside the agency have been consulted on survey development. Additionally, input has been solicited and received from FDA on the design of this study, including participation by FDA in meetings with OMB.

Anna MacMonegle

Public Health Manager

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-990-8427

E-mail: amacmonegle@rti.org

Nathaniel Taylor

Public Health Program Manager

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: 919-316-3523

E-mail: ntaylor@rti.org

Laurel Curry

Research Public Health Analyst

RTI International

3040 E. Cornwallis Road

Research Triangle Park, NC 27709

Phone: 202-728-2086

E-mail: lcurry@rti.org

LeTonya Chapman

Research Public Health Analyst

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Tel: 770-407-4928

E-mail: lchapman@rti.org

Patty LeBaron

Survey Methodologist

RTI International

3040 E. Cornwallis Road

Research Triangle Park, NC 27709

Phone: 312-777-5204

E-mail: plebaron@rti.org

9. Explanation of Any Payment or Gift to Respondents

At baseline, each household that receives a package of screening materials will receive a $1 pre-incentive for completing the screening questionnaire. At baseline and each wave of follow-up data collection, youth respondents who participate in the main data collection will be offered a $25 incentive for an estimated average time of 30 minutes for taking the survey. If a respondent completes the survey during the approximately three-week early release period, they will receive a $5 bonus (Table 1).

Respondents will be offered a $25 incentive to complete the survey after the early release period and for the supplemental baseline data collection. We estimate that the follow-up surveys for the supplemental data collection will also take 30 minutes to complete. Although youth in the supplemental baseline data collection will be recruited through social media, they will participate in the main data collection at follow-up (i.e., the supplemental sample will be contacted with the main sample and complete the same questionnaire). Participants in the supplemental sample will receive a $25 incentive. If the survey is completed within the first 3 weeks of the start of the data collection wave, they will receive a $5 bonus. In addition, the study will use procedures designed to maximize respondent participation, such as e-mail, text, and mail (paper letter) reminders, encouraging messages, humorous attention check items, and a landing page to encourage engagement. The youth’s parent or guardian does not receive an incentive for completing the screener and household roster interview. The proposed incentive allows us to treat participants justly and with respect by acknowledging competing demands for their time and the effort they spend participating (Gelinas et al., 2018).

RTI International, the contractor acting on behalf of FDA, has experience conducting over 35 different recruitment campaigns specifically with youth, and they have surveyed more than 15,000 participants using social media data collection. Their observations indicate that incentives minimize non-response bias, help to complete data collection goals in a timely manner, reduce overall burden, and reduce costs. Based on their experience, older youth are more likely to complete at higher incentive amounts, which is reflected in the literature (Martinson et al., 2000; Festinger et al., 2005). We are seeking to retain as much of our cohort as possible in follow-up waves and to maximize the national representativeness of our sample. Other studies have used similar incentive amounts to the proposed incentive for this study. OMB Control Number 0930-0110, National Study for Drug Use and Health (NSDUH), currently uses an incentive (including cash or gift card) for youth ages 12 to 17. The study experienced an increase in the weighted overall response rate (screening * interviewing) from 67% to 71% from 2001 to 2002 when the incentive was modified to $30.00. OMB Control Number 0910-0753, provided similar incentives for the first two cohorts of The Real Cost evaluation. In 2013, a $20 incentive was offered for the first cohort which, adjusted for inflation, is $25.35 in 2022 USD (U.S. Bureau of Labor Statistics, 2022) and in 2020, an increased incentive of $25 (and a $5 bonus for responding during the early release period) was offered for the second cohort of the study.

In order to recruit a diverse sample, we want to ensure that we can reach socially disadvantaged groups, including people with low socioeconomic status. People with low socioeconomic status are consistently underrepresented in all types of research studies (Bonevsk et al., 2014). When applied in a reasonable manner, incentives are not an unjust inducement—they are an approach that acknowledges participants for their participation (Halpern et al., 2004).

In general, empirical studies show that incentives can increase response rates and reduce attrition in longitudinal surveys within some respondent populations (Scharff et al., 2010; Becker et al., 2019; Pejtersen, 2020) and cash incentives are more effective than non-cash incentives (Martinson et al., 2000; Festinger et al., 2005; 2008; Becker et al., 2019). Although most of the published research on this topic is based on mail, telephone, or in-person surveys, there are now several studies on the effects of incentives within the context of a web-based survey. For example, a 2006 meta-analysis of 32 studies indicates that incentives increase the odds that potential respondents will begin a web survey, and a second meta-analysis of 26 studies shows that incentives increase the odds of completing a web survey once respondents have begun it (Göritz et al., 2006).

Table 1. Incentive Schedule

Type of Incentive	Participant	Amount/Value	Total Amount for Completing Waves
Parent pre-incentive	All baseline households	$1/household	$1
Youth Survey incentive- Early Release Period: Completion during the initial three weeks of data collection	All longitudinal panel members at baseline and in 5 follow-up waves	$30/survey	$180
Youth Survey incentive-Completion after Early Release Period expires	All longitudinal panel members at baseline and in 5 follow-up waves	$25/survey	$150
Youth Survey—Supplemental	All longitudinal panel members at baseline	$25	$25

10. Assurance of Confidentiality Provided to Respondents

Privacy Analysis & Design

In developing this study, CTP consulted FDA’s Privacy Office to identify potential risks to the privacy of participants and other individuals whose information may be handled by or on behalf of FDA in the performance of this study. FDA designed the study to minimize privacy risks in keeping with the Fair Information Practice Principles (FIPPs) and applying controls selected from the National Institute of Standards and Technology (NIST), Special Publication 800-53, Security and Privacy Controls for Federal Information Systems and Organizations. CTP also identified privacy compliance requirements and coordinated with FDA’s Privacy Office to ensure responsible offices in CTP satisfy all in accordance with law and policy. A privacy threshold analysis was approved by HHS and assigned PTA ID 2060996.

PII Collection

As part of this study, RTI International, the contractor acting on behalf of FDA, is collecting and maintaining personally identifiable information (PII) about participants who complete the online screener and online surveys for the main and supplemental data collections. As part of the online screener process to confirm eligibility, parents are asked to provide the following PII: parent’s first and last name, e-mail address and phone number. The following non-PII is also collected from parents: education level, gender, race/ethnicity, and household income. For the main data collection, the only PII we will be collecting from parents is parent’s first and last name, email address, IP address, and birthdate. The PII we will be collecting from youth participants are: first and last name, date of birth, gender, race/ethnicity, grade in school, home address, and phone number as part of the baseline and follow-up surveys. Youth participants who are 18 years old (19 in AL/NE) and complete the follow-up surveys online are also asked to provide their email address, physical address and phone number so the study team can send them a virtual gift card or Visa gift card incentive for completing the survey. The following non-PII is also collected from youth: media use, sexual orientation, gender identity, grade level. This information will be stored separately from each other and from survey responses (except for 24 hours after download when the fraud detection procedures are completed).

For the main data collection, all PII (address, e-mail address, birthdate, zip code, name) will be collected using Blaise (survey programming software) hosted on RTI’s secure servers. Data are encrypted using https protocols and stored on secure Structured Query Language (SQL) databases. RTI is using a unique alphanumeric variable to connect screener data and survey data and to determine if a participant has completed the survey. The unique alphanumeric variable, participant responses to the screener, and responses to the body of the survey will be stored by RTI in a Blaise server. As needed, RTI analysts will extract the data to a secure project network location for processing. All of the data files (PII, screener data and survey data (unique variable but no other PII)) will be stored together in Blaise (which has restricted access and requires credentials to access). At the completion of data collection, the databases will be deleted from RTI’s Blaise account and remain only on RTI’s Federal Information Processing Standards (FIPS) -moderate network server. Data files containing PII will be stored on the FIPS-moderate network server for five years after the project has ended. The FIPS moderate network is not accessible through the internet. It requires specified users with username and password to access a Hosted Virtual Network to access the data.

For the supplemental data collection, datasets of screener and survey responses stored in Qualtrics will not contain any PII. All PII (name, mailing address, e-mail address, birthdate, zip code) will be collected separately by Qualtrics, downloaded by RTI, and stored by Qualtrics as one file that contains only PII and an RTI-assigned unique study ID. IP addresses will be collected by Qualtrics, downloaded by RTI, and stored by Qualtrics and RTI as a second file that only contains IP address, the unique ID, and the participant responses to the screener (excluding PII). Responses to the body of the survey will be collected by RTI and stored as a third file. IP address and PII (e-mail address, birthdate, state, and zip code) will not be collected in the same file. All data on Qualtrics servers are encrypted at rest. RTI will use a Secure Sockets Layer (SSL) connection to download data from Qualtrics servers to RTI servers. We are not collecting any Protected Health Information, defined as “Personally identifiable information that relates to a person's health, medical treatment or payment, and which was obtained from a "covered entity" (health care provider, health plan, or healthcare clearinghouse), as defined by HIPAA (Health Insurance Portability and Accountability Act) regulations.”

This study is funded by the FDA, a Department of Health and Human Services supported agency, and is covered by a Certificate of Confidentiality (CoC). Section 2012 of the 21st Century Cures Act that includes significant amendments to the previous statutory authority for such protections to enhance privacy protections for individuals who are the subjects of federally funded research, under subsection 301(d) of the Public Health Service Act (42 U.S.C. 241). Specifically, the amended authority requires the FDA to issue a CoC to investigators or institutions engaged in research funded by the Federal government to protect the privacy of individuals who are subjects of this research. We will notify participants in the consent form of the protections that the Certificate provides.

Privacy Act Applicability

The information collection is not subject to the Privacy Act of 1974. Hence, no Privacy Act Statement is required to be displayed on the form, website, mobile application or other point at which individuals submit their information.

Data Minimization

The PII collected for this study is limited to the minimum necessary to achieve the authorized purpose and produce a valid study. The purpose of the study is to evaluate The Real Cost public education campaign to reduce and prevent tobacco use being conducted by CTP in support of its mandate to positively impact public health. The PII is necessary in order to determine household eligibility, contact parents for parental permission, invite youth participants to take the baseline and follow-up surveys, conduct quality control checks, and distribute incentives.

Likewise, any potentially sensitive information gathered from participants in association with their PII is limited to that which is essential for the study, such as tobacco use and home tobacco environment. Items such as media use and sensation seeking are collected because they are established risk factors for tobacco use in youth.

FDA has minimized the risk of unnecessary access, disclosure, use or proliferation of PII about participants. FDA and other parties involved in the study maintain study records containing PII only as long as required (until 3 years after the project has ended). RTI International uses a unique case identification number to identify participants. Access to PII is restricted by role to personnel who must access this information. Sensitive records are kept in a secure location until destruction occurs. RTI has in place standard operating procedures based on RTI Policy to ensure the security and privacy of recorded information during all phases of the destruction process, including pickup and transport of records from RTI’s locations to the destruction site. All PII, including electronic PII, will be destroyed as stipulated in the PIA. Non-identifiable or de-identified data (i.e., responses to the study, but without any PII) will be sent by RTI to FDA. No PII will be sent to or be accessible by FDA at any time.

Parents of the youth participants who complete the online survey provide their phone number to receive text reminders, their email address to receive email reminders, and their mailing address so that their child can receive an incentive for participation. RTI study staff will provide an encrypted file with the participants’ mailing addresses to the incentive provider group at RTI (Division of Research Services Respondent Incentive Group) so that incentives can be distributed via mail. RTI does not share this information with FDA. RTI shares Case ID, password, parent first and last names, youth first name, and household mailing addresses with the print vendor for the invitation letters, reminder letters, and postcard reminders for follow-up waves of the survey. This information is sent to the printer vendor via encrypted files. RTI does not share this information with FDA. The print vendor does not have access to any other PII or non-PII from the study. RTI International will not share PII gathered via this collection with any other individuals or entities.

Notice and Transparency

All participants are provided notice regarding the collection and use of the information they provide. The purpose of the study and intended use of the information collected is described on the first page of the web screeners in the baseline data collection. In the web screener, parents are told that the information collected would determine their household’s eligibility for the study and must provide their permission for their child to complete the baseline survey. Youth participants who complete the baseline survey and the follow-up surveys must first read an electronic informed assent form and provide their acceptance before they can complete the survey. Youth participants who turn 18 during the course of the study must read an electronic informed consent form and provide their acceptance. All study materials and website pages clearly state that the study is being sponsored by the FDA.

Individual Participation and Control

Participation in The Real Cost Campaign Outcomes Evaluation Study: Cohort 3 is entirely voluntary. Participants may choose to not join the study and are free to withdraw at any time without incurring any negative consequences. For all parent permission, youth assent and youth consent forms, affirmative assent or consent is obtained electronically by clicking an “accept” radial button below the electronic assent text on their personal mobile device, tablet, or laptop.

Third-Party Accountability

RTI is held accountable for complying with privacy and security procedures (including reporting data breaches) by its contract with FDA, which requires that RTI complies with 45 CFR part 46 and with the contractor’s current Federal-wide Assurance (FWA) on file with the Office for Human Research Protections (OHRP), Department of Health and Human Services. RTI agrees to provide certification at least annually that the Institutional Review Board has reviewed and approved the procedures, which involve human subject’s protections in accordance with 45 CFR part 46 and the Assurance of Compliance. RTI also has an established protocol in place for privacy breaches that includes the Project Director notifying RTI’s IRB and CTP, who, in turn, notifies FDA’s IRB. In addition, RTI has an Incident Response and Breach Notification Plan in place that activates first responders when an incident occurs, and as required by law, a breach notification policy with respect to protected health information. RTI subcontractors are accountable via contract terms for all data that it handles, uses, shares and maintains as part of this survey.

Data Security

RTI International’s data security procedures for the FIPS moderate network, which is the RTI network on which the data from the evaluation will be stored, have been reviewed by a FedRAMP certified Third Party Organization and deemed acceptable. This organization issued an Authorization to Operate (ATO) for the FIPS moderate network. PII will remain on the FIPS-moderate network following the end of data collection and for 3 years after the project has ended.

Advarra’s Institutional Review Board (IRB) has reviewed and approved the study protocol and permission, consent, and assent forms (Attachments 4, 5a, 5b, 6, 7, 8) for the Outcomes Evaluation Study. These forms include language for parental permission and youth assent (under age 18) or consent (18 or older). The IRB’s primary concern is protecting respondents’ rights, one of which is maintaining the privacy of respondent information to the fullest extent of the law.

Concern for privacy and protection of respondents’ rights plays a central part in the implementation of The Real Cost Campaign Outcomes Evaluation Study: Cohort 3 and will receive the utmost emphasis. All consenting documents include an explanation of the Certificate of Confidentiality (CoC). This text explains that the CoC provides legal protection for respondent information, and outlines contexts in which youth information may or may not be shared. The text specifically notes that the CoC does not affect federal, state or local reporting requirements such as reporting of child abuse, communicable diseases, and threats to harm self or others. The text also explains that Personally Identifiable Information (PII) will not be disclosed. Parental permission is obtained from the youth’s parent or guardian for those that are ages 11 to 13; subsequently, youth assent is requested. RTI has received a waiver of parental permission from Advarra IRB for youth ages 14 to 17. Youth who turn 18 during the course of the study provide their own consent. Signed consent and assent are waived in this study.

Names, email addresses, phone numbers, and mailing addresses are never transmitted to FDA/CTP. Only authorized RTI staff will have access to this information on a need-to-know basis. Security for youth participants who complete the online baseline and follow-up surveys is assured in a number of ways: (1) parental permission is required for all eligible youth ages 11 to 13 prior to completing the follow-up survey; (2) participants log onto the study’s secure server hosted by RTI using a unique identifier and password; (3) participants are provided with information about the privacy of their data before they encounter the first survey item; (4) respondents are asked to provide their assent or consent to participate before they encounter the first survey item; and (5) participants have the option to decline to respond to any item in the survey for any reason. All study staff who handle or analyze data are required to adhere to RTI’s standard data security policies.

To ensure data security, all RTI project staff are required to adhere to strict standards. RTI maintains restricted access to all data preparation areas (i.e., receipt and coding). All data files on multi-user systems are under the control of a database manager, with access limited to project staff on a “need-to-know” basis. No respondent identifiers will be contained in reports to FDA, and results will only be presented in aggregate form.

Implementation of data security systems and processes occur as part of the survey data collection. Data security provisions involve the following:

• All data collection activities are conducted in full compliance with FDA regulations to maintain the privacy of data obtained from respondents and to protect the rights and welfare of human research subjects as contained in their regulations. Respondents receive information about privacy protections as part of the informed consent process.

• All data entered via the study’s web-based survey is encrypted, as the responses will be on a website with an SSL certificate applied. Data are passed through a firewall at RTI and then collected and stored on a protected network share on the RTI Network. Only authorized RTI project staff members have access to the data on the secure network share.

• Participants access the online baseline and follow-up surveys with a unique Case ID and password and complete the survey on a secure server online.

All respondents are assured that the information they provide is maintained in a secure manner and will be used only for the purpose of this research. Respondents are assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Respondents are told that the information obtained from all surveys will be combined into a summary report so that details of individual surveys cannot be linked to a specific participant. The data will be kept private to the extent allowed by law.

Respondents participate on a voluntary basis. The voluntary nature of the information collection is described in the key information section of the parent permission form (Attachment 4), assent/consent forms (Attachments 5a, 5b, 6, 7, 8) and the lead letter (Attachment 11a and 11b).

Data Suppression Techniques

An additional approach to secure sensitive data will be to employ data suppression techniques to protect any PII data from survey respondents in the evaluation. Data suppression is a readily applied technique where estimates are not reported if they could result in disclosure of a participant’s identity or are deemed to be unstable (i.e., low precision).

Based on well-established guidelines followed by the Center for Tobacco Products (CTP) Office of Science (OS) guidelines, as well as the National Center for Health Statistics (NCHS), data suppression will be used if any of the following conditions are met: (1) The coefficient of variation of the proportion or estimate is > 30% and/or (2) n < 50, where n is the unweighted sample size in the denominator of the estimated proportion or the denominator used for calculating the estimate.

To further reduce disclosure, we will follow further established guidance from CTP/OS:

• Each estimate (or table cell) must be generated based on a numerator of 3. This includes means, total, numerators of proportions, all table cell counts, and marginal counts. If an estimate is based on a numerator of 1 or 2, it will be combined with another category.

• We will work to ensure table differencing (i.e., calculating the sample size of a small cell from cells of another related table) does not occur by using consistent categories across tables.

• Continuous/ordered variables will be presented so that extreme values pertaining to an individual are not evident.

11. Justification for Sensitive Questions

The majority of questions asked will not be of a sensitive nature. However, it will be necessary to ask some questions that may be considered sensitive to assess specific health behaviors such as tobacco product use. These questions are essential to the objectives of this data collection. Although we do not anticipate any risks from these questions, some participants may perceive them to be sensitive. Additionally, some demographic information, such as race and ethnicity, could also be considered sensitive. Collection of detailed demographic information, including race/ethnicity, sexual orientation, and gender identity, are necessary in order to assess disparities in tobacco use and possible differences in campaign impact across different populations.

Decades of research has shown significant disparities in tobacco use by race/ethnicity (e.g., Harlow et al., 2019; Odani et al, 2018), gender identity (e.g., Johnson et al, 2019; Delahanty et al, 2019), and sexual orientation (e.g., Johnson et al, 2019; McCabe et al., 2018). Therefore, collecting detailed information on these demographic characteristics will allow us to measure these differences with the goal of reducing these disparities. Multiple studies of youth and young adults have reported approximately 12-15% of the sample identified as gender non-conforming/non-binary (e.g., The Human Rights Campaign 2018 LGBTQ+ Youth Report; The Trevor Project 2020 National Survey on LGBT Youth Mental Health; CTP’s evaluation of This Free Life campaign), included gender non-conforming/non-binary response options is necessary to identify and assess tobacco use and campaign effectiveness among this population. Gender identity questions with gender queer/gender non-conforming/non-binary response options have been approved by OMB for ExPECTT (0910-0753), RESPECT (0910-0808), and Formative Research Support: Outcomes and Awareness Measurement Research (0910-0810).

Furthermore, data from the 2021 National Youth Tobacco Survey demonstrates that teens who are sexual or gender minorities have higher rates of cigarette and e-cigarette use compared to heterosexual teens. For example, 6.0% of heterosexual teens reported ever experimenting with cigarettes, compared to 10.9% of gay or lesbian teens, 15.6% of bisexual male teens, 14.0% of bisexual female teens, and 11.2% of teens who are transgender. Furthermore, 17.9% of heterosexual teens reported ever using e-cigarettes, compared to 27.3% of bisexual male teens, 29.6% of bisexual female teens, and 30.7% of teens who are transgender. Including survey items on sexual orientation and gender identity is necessary to identify our target audience for education activities and address disparities in cigarette and e-cigarette use.

Along with the extensive and increasing body of literature showing tobacco use disparities among LGBTQ+ populations, the White House issued the Executive Order on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals, which includes obligations for federal agencies to collect SOGI data. The order states that, “advancing equity and full inclusion for LGBTQI+ individuals requires that the Federal Government use evidence and data to measure and address the disparities that LGBTQI+ individuals, families, and households face.” It also states that federal agencies must “describe disparities faced by LGBTQI+ individuals that could be better understood through Federal statistics and data collection” (White House, 2022). The Office of the Chief Statistician of the United States developed a report based on Executive Order 14075 (“Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals”) to “...provide recommendations for Federal agencies on the current best practices for the collection of self-reported sexual orientation and gender identity (SOGI) data on Federal statistical surveys.” The report states that, “Measuring Sexual and gender minority (SGM) populations in Federal surveys improves understanding of SGM populations and supports evidence-based policymaking”. Based on the best practices outlined in this document, any SOGI questions asked are scientifically based and appropriate for youth participants.

Data gaps need to be addressed in order to support the needs of LGBTQ+ youth populations. In a 2011 report, the Institute of Medicine asserted that, “the ability to address these [health] disparities [among LGBTQ+ youth] is hampered by our lack of knowledge about LGBT youth...efforts to survey young people about their sexual orientation have been fraught with difficulties at both the institutional review board and community levels. These barriers have impeded important developmental research” (Institute of Medicine, 2011, pg. 42). The lack of data that was cited in 2011 continues to be a barrier to developing health communication interventions aimed at reducing disparities in tobacco use behaviors among this particularly vulnerable population. The FDA cannot support LGBTQ+ youth populations via its programming and resources (i.e., public education campaigns) if it is not aware of how risk factors (such as attitudes, beliefs, and behaviors) may be associated with tobacco use among LGBTQ+ youth to support LGBTQ+ youth populations to be healthy and safe, the FDA needs to be knowledgeable about their specific needs.

Given that teens who are sexual or gender minorities have higher use of cigarettes and e-cigarettes compared to heterosexual and cisgender teens, it’s important for us to capture both sexual and gender identity information to accurately identify our target audience for education activities and address disparities in cigarette and e-cigarette product use. Other federal surveys on youth tobacco use also collect SOGI data from youth, including the NYTS (ages 9-19) and the Population Assessment of Tobacco and Health (PATH; youth ages 13-17). The project team will not conduct or report on statistical analysis for demographic groups for which there is insufficient statistical power. As outlined in the data security section of this document, data are securely maintained in areas that are FedRAMP certified and are functioning under an Authorization to Operate (ATO). The participants privacy and rights are monitored by an external Institutional Review Board (IRB). The primary concern of the IRB is protecting respondents’ rights, including maintaining the privacy of respondent information to the fullest extent of the law. Also, parent or guardian permission is required for youth who are ages 11 to 13.

To address any concerns about inadvertent disclosure of sensitive information, participants will be fully informed of the applicable privacy safeguards. The informed consent protocol will notify participants that these topics will be covered in the survey. This study includes a number of procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including the following:

• Participants will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.

• Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses.

• Participants will be provided with a phone number and email address for the Principal Investigator should they have any questions or concerns about the study.

12. Estimates of Annualized Burden Hours and Costs

12 a. Annualized Hour Burden Estimate

An estimated one-time reporting burden for this collection will be approximately 154,921 hours (Table 2). This includes the time burden associated with the recruitment materials, parent and youth screeners, household roster, informed consent, and survey. Our goal is to obtain a sample size of 7,500 youth at baseline that will include participants from the main data collection (n=6,000) and supplemental data collection (n=1,500).

The main data collection will include a baseline survey and five follow-up surveys. The recruitment sample for the main data collection is youth aged 11 to 20. We intend to replenish the longitudinal sample starting at follow-up 2 to obtain a total of 6,000 youth respondents in order to maintain at least 4,800 longitudinal respondents at each wave. We expect the screening process to yield a 100 to one ratio of eligible responding households. For the baseline data collection, we mailed approximately 330,000 recruitment/study material packages (9 minutes per response) in order to receive at least 200,000 completed screeners (5 minutes per response) by adults within households. For households identified as eligible for the study during the screening process (i.e., the presence of one or more youth aged 11 to 20), we will ask the parent/guardian or an adult household member to list all eligible youth in their households for study selection, a process called rostering (5 minutes per response).

For the main data collection at baseline, we plan to collect data from approximately 6,000 youth respondents from the 4,000 eligible households identified through screening. More than one eligible youth per household may be recruited for the study. These 6,000 youth respondents will provide baseline assent (5 minutes per response) and complete the survey (30 minutes per response). For these youth respondents, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study.

We will send invitations and study materials to all 7,500 Baseline sample respondents (main data collection n=6,000; supplemental data collection n=1,500) to complete the survey at Follow-up 1 (FU1; 11 minutes). These changes do not necessitate additional burden hours beyond what was approved in the original submission for this collection. We estimate that we will retain 80% of the sample from baseline and collect data from 4,800 respondents (5 minutes per response) at FU1. These 4,800 youth respondents are estimated to provide assent (5 minutes per response) for follow-up 1 and complete the survey (30 minutes per response). For these youth respondents, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study. We do not intend to replenish the sample at FU1.

We will send invitations and study materials to all 7,500 Baseline sample respondents to complete the survey at FU2 (11 minutes). We estimate that we will retain 80% of the sample from FU1 resulting in 3,840 respondents at FU2. To replenish the longitudinal sample at FU2, we will send additional “baseline” screeners to new households. We intend to send recruitment/study material packages to an additional 145,000 households (10 minutes per response) to receive an estimated 72,500 completed screeners (5 minutes per response). For households identified as eligible for the study during the screening process (i.e., the presence of 1 or more youth aged 11 to 20), we will ask an adult household member to list all eligible youth in their households for study selection (5 minutes per response). Replenishing the sample will allow us to obtain 6,000 youth respondents at FU2 (3,840 from the original sample, and 2,160 from the replenishment sample) and maintain a minimum study sample of 4,800 respondent at all study waves. These 6,000 youth respondents are estimated to provide assent (5 minutes per response) for follow-up 2 and complete the survey (30 minutes per response). For these youth respondents, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study.

We will send invitations and study materials to all 7,500 Baseline sample respondents as well as the additional 2,160 sample members recruited at FU2 to complete the survey at FU3 (11 minutes). We estimate that we will retain 80% of the sample from FU2 and collect data from 4,800 respondents at FU3. We do not intend to replenish the sample at FU3. These 4,800 youth respondents are estimated to provide assent (5 minutes per response) for follow-up 3 and complete the survey (30 minutes per response). For these youth respondents, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study.

For FU4, we will replenish the sample in order to obtain a sample size of 6,000 youth. To achieve this, first, we will send invitations and study materials to all 9,660 sample respondents (7,500 Baseline sample respondents and 2,160 sample members recruited at FU2) to complete the survey at FU4 (11 minutes). We estimate that we will retain 80% of the sample from FU3 and collect data from 3,840 respondents at FU4. Next, to replenish the longitudinal sample at FU4, we will send additional “baseline” screeners to new households. We intend to send recruitment/study material packages to an additional 145,000 households (9 minutes per response) to receive an estimated 72,500 completed screeners (5 minutes per response). For households identified as eligible for the study during the screening process (i.e., the presence of 1 or more youth aged 11 to 20), we will ask an adult member to list all eligible youth in their households for study selection (5 minutes per response). Replenishing the sample will allow us to obtain 6,000 respondents at FU4 and maintain a minimum study sample of 4,800 respondent at all study waves. These 6,000 respondents are estimated to provide assent (5 minutes per response) for FU4 and complete the survey (30 minutes per response). Where required, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study.

We will send invitations and study materials to all 7,500 Baseline sample respondents as well as the additional 2,160 sample members recruited at FU2 and the 2,160 sample members recruited at FU4 to complete the survey at FU5 (11 minutes). We estimate that we will retain 80% of the sample from FU4 and collect data from 4,800 respondents at FU5. We do not intend to replenish the sample at FU5. These 4,800 youth respondents are estimated to provide assent (5 minutes per response) for follow-up 5 and complete the survey (30 minutes per response). For these youth respondents, we will ask the parent/guardian to provide permission (5 minutes per response) for the youth to participate in the study.

In addition to the main data collection, we intend to collect data from subpopulations shown to be at higher risk of initiating use of cigarettes and ENDS products, such as youth who identify as LGBTQ+ and youth who have a mental illness. Data collection will consist of online self-administered surveys of participants recruited through social media advertisements. The recruitment sample for this data collection will be youth ages 14 to 20 who meet the subpopulation criteria. We intend to collect data at baseline from 1,500 respondents. We anticipate that we will need to screen 5,000 respondents (5 minutes per response) to obtain a baseline sample of 1,500 respondents who meet the subpopulation criteria. At baseline, we plan to collect data from approximately 1,500 respondents identified as eligible through screening. These 1,500 youth respondents are estimated to provide assent (5 minutes per response) and complete the survey (30 minutes per response). We estimate that we will lose approximately 20 percent of the original baseline sample at each FU wave; therefore, estimating 1,200 respondents at FU1, 960 respondents at FU2, 768 respondents at FU3, 614 respondents at FU4, and 492 at FU5. For the FU samples, youth will provide assent (5 minutes per response) and complete the survey (30 minutes per response).

The attachments are provided in both English and Spanish. We will not be recruiting separate English-speaking and Spanish-speaking samples for this study. We are simply providing Spanish-language consent/assent forms and surveys for participants who prefer to complete them over the English-language versions. Regardless of what language the respondents complete the consent/assent and surveys in, the estimated burden hours are identical.

Table 2.--Estimated Annual Reporting Burden

Respondent/Activity	No. of Respondents	No. of Responses per Respondent	Total Annual Responses	Average Burden per Response	Total Hours
Recruitment Study Materials--Main: Baseline Recruitment & Follow-up Replenishments	620,000	1	620,000	.15 (9 mins)	93,000
Recruitment Study Materials--Main: Follow-up waves for Longitudinal respondents	46,140	1	46,140	.19 (11 mins)	8,767
Parent Screener--Main: Baseline & Follow-up Replenishments	345,000	1	345,000	.08 (5 mins)	27,600
Household Roster--Main: Baseline & Follow-up Replenishments	7,000	1	7,000	.08 (5 mins)	560
Parent Permission--Main: Baseline & Follow-up waves	32,400	1	32,400	.08 (5 mins)	2,592
Youth Assent--Main: Baseline & Follow-up waves	32,400	1	32,400	.08 (5 mins)	2,592
Youth Survey--Main: Baseline & Follow-ups	32,400	1	32,400	.50 (30 mins)	16,200
Youth Screener—Supplemental	5,000	1	5,000	.08 (5 mins)	400
Youth Assent--Supplemental: Baseline & Follow-up waves	5,534	1	5,534	.08 (5 mins)	443
Youth Survey--Supplemental: Baseline & Follow-up waves	5,534	1	5,534	.50 (30 mins)	2,767
Total	635,320				154,921

12b. Annualized Cost Burden Estimate

Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. To calculate the estimated annual cost, the mean hourly wage of $7.25 was used for youth and $26.65 was used for young adults. The youth costs represent the minimum wage, and the young adult costs represent the mean hourly wage for all other Life, Physical, and Social Science occupation earnings from the U.S. Department of Labor Bureau of Labor Statistics (https://www.bls.gov/oes/current/oes_nat.htm#19-0000 May 2021 data). There are no direct costs to respondents associated with participation in this information collection. RTI has conducted many smoking-related surveys of similar length among youth and adults. We have examined diagnostic data from each of these prior surveys and estimate that data collection for this study will take, on average, 5 minutes per respondent for screening, 5 minutes per respondent for assenting/consenting, and approximately 30 minutes per respondent for the online surveys. Thus, assuming an average hourly wage of $7.25 and $26.65 (youth and adult) and doubling this to account for benefits and overhead, yielding an hourly wage rate of $14.50 for youth and young adult $53.30, the estimated one-time cost to participants is estimated to be $7,381,668.20. The estimated value of respondents’ time for participating in the information collection is summarized below.

Annualized Cost Burden
Type of Respondent	Activity	Annual Burden Hours	Hourly Wage Rate	Total Cost1
Parents of Youth aged 11-17	Parent Recruitment Study Materials	101,767	$53.30	$5,424,181.10
	Parent Screening	27,600	$53.30	$1,471,080.00
	Household Rostering	560	$53.30	$29,848.00
	Parent Permission	2,592	$53.30	$138,153.60
Youth aged 11–20
	Youth Assent	2,592	$14.50	$37,584.00
	Online Survey	16,200	$14.50	$234,900.00
	Youth Assent--Supplemental	400	$14.50	$5,800.00
	Online Survey-Supplemental	2,767	$14.50	$40,121.50
Total				$7,381,668.20

¹ Cost was rounded up to the next dollar.

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital, start-up, operating, or maintenance costs associated with this data collection.

14. Annualized Cost to the Federal Government

This information collection is funded through a contract with RTI. The estimated costs attributable to this data collection are $753,587 per year. There are additional contract-funded activities occurring before and after this data collection that include project planning and data analysis. Other activities outside this data collection include coordination with FDA, instrument development, reporting, Advarra IRB, project management and progress reporting. This information collection will occur for two years after OMB approval.

Annualized Cost to the Federal Government
Government Personnel	Time Commitment	Average Annual Salary	Total1
GS-12	5%	$104,808	$5,240
GS-13	20%	$121,065	$24,213
GS-13	20%	$121,065	$24,213
GS-14	10%	$143,064	$14,306
		Total Annual Salary Costs	$67,972
Annual Contract Cost			$2,212,572
Total Annual Cost			$2,280,544

¹ Costs were rounded up to the next dollar.

15. Explanation for Program Changes or Adjustments

We are submitting this change request to:

1. Add two additional follow up waves of data collection – FU4 and FU5. We have been asked by our Center leadership to prioritize our assessment of the impact of The Real Cost campaign on tobacco use behavior. Because behavior change happens over time, often many years, we plan to add two additional follow-up waves of data collection to allow us to follow our longitudinal sample for more time to assess for campaign effects on tobacco use initiation and other tobacco use behaviors.

2. Add a replenishment sample at FU4 (ages 11-20 years) to ensure we have sufficient sample sizes (and therefore, sufficient statistical power) to detect campaign effects of tobacco use behavior. The replenishment sample will also ensure we have a sufficient number of youth in the sample (the intended audience for The Real Cost) given that participants are aging up over time.

3. Adjustments to estimated burden hours and estimated annualized costs based on the above changes. Based on these adjustments, we estimate a net increase of 39,522 burden hours for a new total of 154,921.

16. Plans for Reporting and Project Time Schedule

Data from this information collection will be used to estimate awareness of the campaign among youth and model the effects of exposure to the campaign on a variety of intended psychosocial outcomes and predictors of tobacco use behaviors. Estimates will take the form of descriptive data on survey items such as self-reported ad recognition and recall that assess basic exposure as well as frequency of ad exposure. Data will also be used to examine statistical associations between exposure to the campaign and pre-post changes in specific outcomes of interest. This will be accomplished with the use of multivariate models that estimate follow-up measures of each relevant outcome as a function of prior self-reported exposure to the campaign, controlling for individual and other characteristics that may confound the relationship between campaign exposure and changes in outcomes. We hypothesize that there should be larger changes in intended outcomes among individuals who report greater exposure to the campaign (i.e., dose-response effects).

The reporting and dissemination mechanism will consist of three primary components after each survey wave: (1) summary statistics produced from analytic data files on individual awareness of and campaign outcomes, (2) data analyses summarized in a report format (e.g., PowerPoint presentations, data dashboards, etc.), and (3) report writing for various audiences, which could include briefings, written reports, and/or peer-reviewed journal articles that document the relationships between campaign exposure and changes in the aforementioned outcomes of interest. The key events after each round of data collection are listed below.

Approximate Project Schedule

Project Activity	Date
Surveys	April 2023 to April 2027 (Approximate)
Preparation of analytic data files	Approximately 6 weeks after completion of each wave of data collection
Data Analysis	Approximately 12 weeks after completion of each wave data collection
Report Writing	Approximately 16 weeks after completion of each wave of data collection

17. Reason(s) Display of OMB Expiration Date is Inappropriate

All data collection instruments will display the OMB approved expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

These information collection activities involve no exception to the Certification for Paperwork Reduction Act Submissions.

References

Becker, R., Moser, S., & Glauser, D. (2019). Cash vs. vouchers vs. gifts in web surveys of a mature panel study – main effects in a long-term incentives experiment across three panel waves. Social Science Research. 81. 221-234.

Bonevski, B., Randell, M., Paul, C. et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol 14, 42 (2014). https://doi.org/10.1186/1471-2288-14-42

Dai, H., & Leventhal, A. (2019). Prevalence of e-cigarette use among adults in the United States, 2014-2018. JAMA. 322(18), 1824-1827.

Delahanty, J., Ganz, O., Hoffman, L., Guillory, J., Crankshaw, E., & Farrelly, M. (2019).

Tobacco use among lesbian, gay, bisexual and transgender young adults varies by sexual and gender identity. Drug and alcohol dependence, 201, 161-170.

Festinger, D.S., Marlowe, D.B., Dugosh, K.L., et al. (2005). Do research payments precipitate

drug use or coerce participation? Drug and Alcohol Dependence. 78:275-281.

Festinger, D.S., Marlow, D.B., Croft, J.R., et al. (2008). Higher magnitude cash payments improve research follow-up rates without increasing drug use or perceived coercion. Drug and Alcohol Dependence. 86:128-135.

Gelinas L, Largent EA, Cohen IG, Kornetsky S, Bierer BE, Fernandez Lynch H. A Framework

for Ethical Payment to Research Participants. N Engl J Med. 2018;378(8):766-771.

doi:10.1056/NEJMsb1710591

Göritz AS. (2006). Incentives in web studies: methodological issues and a review. Int J Internet Science. 1(1), 58-70.

Halpern, SD., Karlawish, JH., Casarett, D., Berlin, JA., Asch, DA. Empirical assessment of whether moderate payments are undue or unjust inducements for participation in clinical trials. Archives of Internal Medicine. 2004; 164(7), 80l-803.

Harlow, A. F., Stokes, A., & Brooks, D. R. (2019). Socioeconomic and racial/ethnic differences

in e-cigarette uptake among cigarette smokers: longitudinal analysis of the population assessment of tobacco and health (PATH) study. Nicotine and Tobacco Research, 21(10), 1385-1393.

Institute of Medicine. 2011. The Health of Lesbian, Gay, Bisexual, and Transgender People:

Building a Foundation for Better Understanding. Washington, DC: The National Academies Press. https://doi.org/10.17226/13128.

Johnson SE, O'Brien EK, Coleman B, Tessman GK, Hoffman L, Delahanty J. Sexual and gender

minority U.S. youth tobacco use: Population Assessment of Tobacco and Health (PATH) Study Wave 3, 2015-2016. Am J Prev Med. 2019;57(2):256-61. Available from: https://www.ncbi.nlm.nih.gov/pubmed/31326009. DOI: 10.1016/j.amepre.2019.03.021

Kasza, K., Ambrose, B., Conway, K., et al. (2017). Tobacco-product use by adults and youths in the United States in 2013 and 2014. N. Engl J. Med. 376(4), 342-353.

Martinson, et al. (2000). Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking. Preventive Medicine, 31, 706-713.

McCabe, S. E., Matthews, A. K., Lee, J. G., Veliz, P., Hughes, T. L., & Boyd, C. J. (2018).

Tobacco use and sexual orientation in a national cross-sectional study: age, race/ethnicity, and sexual identity–attraction differences. American journal of preventive medicine, 54(6), 736-745.

Odani, S., Armour, B. S., & Agaku, I. T. (2018). Racial/ethnic disparities in tobacco product use

among middle and high school students—United States, 2014–2017. Morbidity and Mortality Weekly Report, 67(34), 952.

Pejtersen, J. H. (2020). The effect of monetary incentive on survey response for vulnerable children and youths: A randomized controlled trial. PLoS One, 15, e0233025.

Scharff DP, Mathews KJ, Jackson P, Hoffsuemmer J, Martin E, Edwards D. More than Tuskegee: Understanding Mistrust About Research Participation. Journal of health care for the poor and underserved. 2010; 21(3):879-897.

U.S. Bureau of Labor Statistics. (2022). All items in U.S. city average, all urban consumers, not seasonally adjusted. https://www.bls.gov/data/inflation_calculator.htm. Retrieved October 26, 2022.

Wang, T., Gentzke, A., Creamer, M., et al. (2019). Tobacco product use and associated factors among middle and high school students – United States, 2019. Morbidity and Mortality Weekly Report. 68(12).

The White House. (2022). Executive Order on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals. Retrieved from https://www.whitehouse.gov/briefing-room/presidential-actions/2022/06/15/executive-order-on-advancing-equality-for-lesbian-gay-bisexual-transgender-queer-and-intersex-individuals/

1