0621 NYTS SSA_Final 01.13.2021

Information Collection Request

Revision

NATIONAL YOUTH TOBACCO SURVEY, 2021 - 2023

OMB No. 0920-0621, expires 04/30/2021

SUPPORTING STATEMENT: PART A

Submitted by:

David Homa, PhD, MPH

Centers for Disease Control and Prevention

Office on Smoking and Health

Epidemiology Branch

4770 Buford Highway NE, MS-S107-7

Atlanta, GA 30341

Phone: 770-488-3626

Fax: 770-488-5848

E-mail: DHoma@cdc.gov

July 29, 2020

TABLE OF CONTENTS

A. JUSTIFICATION

A.1. Circumstances Making the Collection of Information Necessary

A.2. Purpose and Use of Information Collection

A.3. Use of Improved Information Technology and Burden Reduction

A.4. Efforts to Identify Duplication and Use of Similar Information

A.5. Impact on Small Businesses or Other Small Entities

A.6. Consequences of Collecting the Information Less Frequently

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.9. Explanation of Any Payment or Gift to Respondents

A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents.

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions

A.12. Estimates of Annualized Burden Hours and Costs

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

A.14. Annualized Cost to the Government

A.15. Explanation for Program Changes or Adjustments

A.16. Plans for Tabulation and Publication and Project Time Schedule

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

REFERENCES

LIST OF ATTACHMENTS

A1 Authorizing Legislation

A2 Federal Register Notice

A3 Federal; Register Notice Comments and Agency Responses

B. State Tobacco Control Reports that Cite National Youth Tobacco Survey Data

C. Publications from Prior Cycles of the National Youth Tobacco Survey

D1. State-level Recruitment Script for the National Youth Tobacco Survey

D2. State-level Recruitment Script for the National Youth Tobacco Survey Supplemental Document – State Letter of Invitation

E1. District-level Recruitment Script for the National Youth Tobacco Survey

E2. District-level Recruitment Script for the National Youth Tobacco Survey Supplemental Document – District Letter of Invitation

F1. School-level Recruitment Script for the National Youth Tobacco Survey

F2. School-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents – School Letter of Invitation and NYTS Fact Sheet for Schools

F3. School-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents – Letter to Agreeing Schools

G1. Data Collection Checklist for the National Youth Tobacco Survey

G2. Data Collection Checklist for the National Youth Tobacco Survey Supplemental Documents – Letter to Teachers in Participating Schools

H1. National Youth Tobacco Survey Questionnaire

H2. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form Distribution Script

H3. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form (Active) and Fact Sheet (English Version)

H4. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form (Passive) and Fact Sheet (English Version)

H5. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form (Active) and Fact Sheet (Spanish Version)

H6. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form (Active) and Fact Sheet (Spanish Version)

H7. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form Reminder Notice (English Version)

H8. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form Reminder Notice (Spanish Version)

H9. National Youth Tobacco Survey Questionnaire Supplemental Documents – Questionnaire Administration Script

I Refusal Conversion and Nonresponse Cover Memos

J1. CDC IRB Approval Letter

J2. Contractor IRB Approval Letter

K. Sample Table Shells

L. Detailed Sampling and Weighting Plan

M1 Dear Teacher Letter, On-Campus Instruction, Active Permission

M2 Dear Teacher Letter, On-Campus Instruction, Passive Permission

M3 Dear Teacher Letter, Virtual/Distance Instruction, Active Permission

M4 Dear Teacher Letter, Virtual/Distance Instruction, Passive Permission

N NYTS Student and Teacher Video Scripts

O NYTS Online Teacher Landing Page-Class Enrollment Form

P Summary of School Arrangements Forms

LIST OF TABLES

Table A.8: Consultants for 2021-2023 NYTS

Table A.10: Access Controls

Table A.12a: Estimated Annualized Burden Hours

Table A.12b: Annualized Estimated Cost to Respondents

Table A.14: Annualized Study Cost

Table A.15: Annualized Estimates of Respondents and Burden, 2021-23 NYTS

Table A.16: Schedule of Activities for 2021 NYTS

Table B.1: Distribution of Schools by Urban Status and School Type

Tables B.2: Major Means of Quality Control

• Goal of the study
  
  

The study is to design, conduct, and report on the school-based National Youth Tobacco Survey (NYTS) among 6th through 12th grade students in 2021, 2022, and 2023. The purpose of the survey is to assess student use of tobacco product in a variety of forms; their knowledge of and attitudes toward tobacco products; their exposure to secondhand tobacco smoke; and their exposure to influences that promote or discourage tobacco product use, such as portrayals of tobacco products in advertising and mass media, enforcement of age restrictions in the sales of tobacco products to minors, provision of school- and community-based interventions, and access to supports in attempting to stop using tobacco products.

• Intended use of the resulting data
  
  

The NYTS data will be used to inform the National Comprehensive Tobacco Control Program; inform progress towards achieving Healthy People 2030 objectives related to tobacco product use and youth; provide data to inform the Department of Health and Human Service’s Tobacco Control Strategic Action Plan, and provide national benchmark data for state-level Youth Tobacco Surveys and for comparison with the international community through the Global Youth Tobacco Survey.

• Methods to be used to collect
  
  

Data for the NYTS has been collected on school campus using a digitally based self-administered questionnaire on a tablet. Given the unprecedented circumstances presented by the COVID-19 pandemic, 2021 NYTS will be administered as a virtual (web-based) survey. For adequate participation during the COVID-19 pandemic, the NYTS sample was augmented in an effort to secure an estimated participation of 24,000 students from approximately 509 sampled schools.

• The subpopulation to be studied
  
  

Contingent upon final analytic sample size, NYTS will provide data among subpopulations of youth, by race/ethnicity, sexual orientation, gender identity, mental health (depression/anxiety), and socio-economic status.

• How data will be analyzed

The NYTS data will be weighted to provide nationally representative estimates. Data sets and documentation for the NYTS available online. National trends and patterns of the distribution and determinants of tobacco product use behaviors among youth enrolled in grades 6-12 can be compared by demographics and across years.

While virtual data collection during the COVID-19 pandemic in 2021 will provide useful information on youth use of tobacco products, the 2021 NYTS data will not be compared to the results from 2021 to previous or subsequent years of data collection that primarily collect data on school campuses. Comparability could be compromised by the change in methods of survey administration and data collection. Additional information is available under section “A.16. Plans for Tabulation and Publication and Project Time Schedule.”

OVERVIEW

CDC requests OMB approval to conduct the NYTS in 2021, 2022, and 2023 (OMB No. 0920-0621; exp. 4/30/2021). The survey instrument has been developed to include items that are relevant to the present circumstances in tobacco prevention and control efforts among youth. Burden allocation for instrument development and testing has been calculated to allow for cognitive testing related to potential questionnaire revisions that might occur after 2021. However, the estimated burden per response to complete the actual survey has not changed.

A. JUSTIFICATION

A.1. CIRCUMSTANCES MAKING THE COLLECTION OF INFORMATION NECESSARY

Collection of data on tobacco product use among youth reflects a critical public health priority. Tobacco prevention and control remains one of the strategic goals of the U.S. Department of Health and Human Services for advancing the health, safety, and well-being of the American People. Given that most tobacco products initiation begins in adolescence, monitoring of tobacco use among youth is important. The NYTS is the only nationally representative survey of middle and high school students that focuses exclusively on tobacco product use patterns and associated factors.

The NYTS measures short-term outcomes (such as increased knowledge about the negative health consequences of tobacco product use and exposure to secondhand smoke), intermediate-term outcomes (such as reduced access to tobacco products), and long-term outcomes (such as reduced tobacco product use prevalence) (CDC, 2012; CDC, 2014).

The justification for the implementation of the NYTS is based on three factors: (1) public health implications of tobacco product use; (2) the burden of tobacco product use; and (3) mandates to monitor, reduce, and alter attitudes toward tobacco product use and reduce exposure to pro-tobacco influences found in Section 301 of the Public Health Service Act (42 USC 241) (Attachment A1). Specifically, the following factors make it necessary for the CDC to conduct the NYTS:

• The NYTS provides data to support several strategic planning priorities for the U.S. Department of Health and Human Services (DHHS), including CDC’s Budget Request Summary for FY 2019-2021 (CDC, 2018b) on selected Government Performance and Results Act (GPRA) measures; DHHS’s Tobacco Control Strategic Action Plan (USDHHS, 2012), and activities mandated by the Family Smoking Prevention and Tobacco Control Act. The NYTS is also the data source for 7 Healthy People 2030 objectives related to reducing adolescent prevalence of: current use of any tobacco products; current use of e-cigarettes; current use of cigarettes; current use of cigars, cigarillos, and little cigars; current use of flavored tobacco products; current use of smokeless tobacco products; and exposure to tobacco product marketing.

The annual administration of NYTS has helped in the identification of emerging trends, such as the increased use of electronic cigarettes (e-cigarettes) from 2011 through 2018 (CDC, 2013; CDC, 2015; CDC, 2019), and allows for the development and inclusion of specific measures relevant to national objectives for tobacco prevention and control among youth. The 2018 NYTS showed a dramatic increase in current e-cigarette use among both middle school and high school students; notably, prevalence increased by 78% among high school students during 2017-2018 (from 11.7 % to 20.8%) (FDA, 2018; CDC, 2019a). Driven by the increase in e-cigarette prevalence, the prevalence of any tobacco product use increased among both middle school and high school students, erasing the decline in youth tobacco product use that had occurred in previous years (CDC, 2019). In 2019, NYTS showed the prevalence of e-cigarette use further increased, with 1 in 10 middle school students and over 1 in 4 high school students having used e-cigarettes in the past 30 days (FDA-CDC, 2019; CDC, 2019b). More recently, the 2020 NYTS findings indicated a decline in current e-cigarette use since 2019, however 3.6 million U.S. youth still currently used e-cigarettes in 2020. Moreover, the use of disposable e-cigarettes increased, and menthol e-cigarette use was prominent (CDC, 2020). Publications based on past cycles of the NYTS are listed in Attachment C.

• The NYTS serves as a national benchmark against which states can measure their progress in tobacco control and prevention. Many states conduct a Youth Tobacco Survey (YTS) using comparable methodology to the NYTS. States therefore can measure their program’s progress relative to national trends.

Rationale for Modifications to the Existing Data Collection Procedures for 2021 due to COVID-19 Pandemic Precautions

In order to respond to the current environment in schools as a result of the pandemic, the 2021 NYTS data collection will be done via a web-based URL to allow for participation by eligible students from sampled schools and classrooms that are currently engaged in virtual/distance learning. This virtual data collection model is a proactive approach designed to anticipate the remote/distance learning shifts within schools as they address the COVID-19 pandemic, allow for schools to participate without undue burden on the teacher or school, and ensure that study protocols are followed. Data collectors will not be utilized; however, Technical Assistance Providers (TAPs) will be available remotely during fielding to handle IT requests and/or questions from teachers. Verbal instructions typically provided by data collectors will be substituted by a brief, high-quality instructional video after student login and prior to any questions displayed to screen. A brief, high-quality video will also be produced for teachers that provides instructions on how to accurately record their classroom enrollment and participation information via an online version of the data collection checklist. To maintain a survey environment conducive to taking the survey, students will be provided earbuds, so that they can hear the audio associated with the pre-survey instructional video. Teachers will also be provided with earbuds for viewing/listening to teacher instructional video.

A.2 PURPOSE AND USE OF INFORMATION COLLECTION

NYTS collects information on the use of tobacco products; knowledge of and attitudes toward tobacco; exposure to secondhand smoke; and, exposure to pro- and anti-tobacco influences such as portrayals of tobacco products in advertising and mass media, provision of school- and community-based interventions, and enforcement of minors’ access laws. Data collected through the NYTS can: (1) inform the development of health policy and guidelines that protect nonsmokers from secondhand smoke; (2) help researchers and policy makers to better understand youth exposure to pro-tobacco influences; (3) provide comprehensive tobacco product use data to support tobacco control programs; and (4) inform the implementation of other key evidence-based policies that will prevent youth initiation, decrease the number of tobacco product users, and save lives.

Despite the COVID-19 pandemic, tobacco product use in youth remains a priority for CDC and other federal and state agencies. Annually, NYTS data will be used, not only by CDC, but also by several other Federal agencies, including FDA. Additionally, the NYTS data can be used by state and local governments, nongovernmental organizations, academic institutions, and others in the private sector.

• CDC: uses NYTS data for evaluation of comprehensive tobacco control policies; measuring progress made in reaching national objectives (e.g., Healthy People 2030 objectives); policy and program development; research synthesis; and technical assistance to state, local, and other partners.

• FDA: uses the NYTS data over time to inform and monitor its regulatory authority over the manufacture, distribution, and marketing of tobacco products.

• Health Resources and Services Administration (HRSA): uses NYTS data to support HRSA, Maternal and Child Health Bureau (MCHB), and the American Academy of Pediatrics’ Bright Futures Health Supervision Guidelines formulate specific risk-reduction recommendations to prevent and assess tobacco product use and exposure for children, and adolescents.

• National Cancer Institute (NCI): uses NYTS data to help inform its research, educational efforts, and demonstration projects focused on youth tobacco product use prevention and the determinants of cessation.

• Office of the Surgeon General: uses and references the NYTS results to assess the need for focused use of resources for tobacco prevention and control efforts targeting youth. NYTS data have figured prominently in recently released reports. Based on findings from the 2018 NYTS (FDA, 2018; CDC, 2019), the U.S. Surgeon General issued an advisory on e-cigarette use among youth, declaring the increased use as epidemic. (OSG, 2018).

• State and local governments: use NYTS data as an index against which state and local health and education agencies can compare their state YTS results. Attachment B lists state tobacco control reports that cite NYTS data. State and local law enforcement officials also can use findings from the NYTS to determine national compliance with the Synar Amendment, which bans the sale of tobacco products to youth aged <18 years.

• Nongovernmental organizations, foundations, and academic institutions (e.g. American Cancer Society; The Robert Wood Johnson Foundation; The California Cancer Research Fund for the University of California; the Truth Initiative; the American Medical Association; and The Institute of Medicine): have extensively used NYTS data in official reports, white papers, and fact sheets. Additionally, academic researchers use data from NYTS for research and surveillance.

Thus, in order to maintain NYTS data collection for use by CDC and other federal partners, state and local governments, nongovernmental organizations, academic institutions, and others, CDC will conduct the 2021 NYTS as an entirely virtual (web-based) survey to enable continued data collection despite challenges and restrictions imposed by necessary environmental changes due to the COVID-19 pandemic. While entirely virtual data collection during the COVID-19 pandemic in 2021 will provide useful information on youth tobacco use behavior, adequate caution should be used when comparing the results from 2021 to previous or subsequent years of data collection that primarily collect data on school campuses. Additional information is available under section “A.16. Plans for Tabulation and Publication and Project Time Schedule.”

A.3 USE OF IMPROVED INFORMATION TECHNOLOGY AND BURDEN REDUCTION

From 1999-2018, the NYTS was administered via a paper-and-pencil questionnaire. However, the NYTS fully transitioned to a digital-based survey mode in 2019. Participants were provided a tablet to complete the survey in the classroom; data were collected offline using a programmed survey application. Students absent on the day of survey administration could complete a make-up survey using a web-based version of the survey programmed to mimic the tablet-based application.

This transition to digital-based allowed for the programming of skip instructions to tailor the questionnaire to the individual tobacco product use status of respondents. In addition to improving both the overall detail and validity of responses, the transition resulted in reduced burden time, as individuals are not asked to read through and answer questions that are not applicable to their current tobacco product use behaviors. Preliminary estimates of response time burden from administration of the 2019 survey suggested that the average time to complete the survey (104 questions, with programmed skip instructions) was about 12.5 minutes. However, the allotted burden time for the 2019 digital-based survey remained the same as when the NYTS was administered by paper-and-pencil, allowing for one class period (up to 45 minutes) for survey completion. Time to complete the 2020 survey was an average of 14 minutes and this is expected to remain the same for the 2021-2023 surveys; all respondents will be allowed up to one 45-minute class period to complete the survey. Thus, overall respondent burden has been kept at the same level for the current 2021-2023 cycle.

Prior experience with administering NYTS as a web-based survey for the facilitation of make-up surveys allowed for the modification of existing data collection procedures and programs to develop an entirely virtual survey administration for the 2021 NYTS that will mimic the student experience of the tablet-based application typically used for on-campus data collection. CDC expects time needed to complete the virtual survey to be comparable to the tablet-based survey; hence no change in response burden for student participants using the virtual survey is anticipated.

A.4 EFFORTS TO IDENTIFY DUPLICATION AND USE OF SIMILAR INFORMATION

In order to minimize duplication of data collection and the burden on survey participants, the CDC and the Food and Drug Administration (FDA) have collaborated to leverage the NYTS as a single data source to inform national objectives for tobacco product use prevention and control among youth.

The target population of NYTS (grades 6-12) makes it inherently distinct from other school-based surveys such as the national Youth Risk Behavior Survey (grades 9-12); and Monitoring The Future (grades 8, 10, and 12). While other multi-purpose household surveys also sample youth (such as the National Survey on Drug Use and Health (NSDUH) and the National Health and Nutrition Examination Survey (NHANES)) and contain some tobacco-related content, the scope of these tobacco-specific questions cannot meet the needs specific to the evaluation of tobacco prevention and control activities at the national level.

In addition to the CDC-FDA collaboration specific to the NYTS, enhanced review procedures were instituted in 2013 to promote overall efficiency and quality in federally-sponsored data collection relating to tobacco product use and control. An inter-agency workgroup was established under the HHS Data Council with representatives from HHS OPDIVS and programs collecting tobacco related data. The role of the group is to build infrastructure and connections to facilitate coordination and communication during the developmental stage of survey design to reduce duplication, improve response rates, reduce respondent burden, and promote standardization of estimates, where feasible. Federal agencies consulted through this process include NCHS, NIH/NCI, NIH/NIDA, and SAMSHA.

A.5 IMPACT ON SMALL BUSINESSES OR OTHER SMALL ENTITIES

The planned data collection does not involve small businesses or other small entities.

A.6 CONSEQUENCES OF COLLECTING THE INFORMATION LESS FREQUENTLY

NYTS currently is conducted annually. Conducting it less than annually will adversely impact the ability to assess emerging trends. This will be particularly important for assessing progress toward reaching Healthy People 2030 national targets for tobacco control. Furthermore, the tobacco product environment is rapidly evolving; CDC and other public health agencies need annual data to identify and track emerging products and issues and to inform public health policies and actions. For example, after increasing from 2011-2015, youth e-cigarette use declined during 2016 and 2017, followed by a rapid 1-year increase from 2017 to 2018, from 11.7% to 20.8% among high school students (FDA, 2018; CDC, 2019). The prevalence of e-cigarette use among both middle school and high school students increased further in 2019, with 27.5% of high school students and 10.5% of middle school students reporting past 30-day use (FDA-CDC, JAMA, 2019). However, recent data from the 2020 NYTS has shown that, for the first time in several years, e-cigarette use among U.S. middle and high school students declined; e-cigarette use was reported by 19.6% of high school students and 4.7% of middle school students in 2020 (CDC, 2020). Despite the COVID-19 pandemic, annual information on youth tobacco product use and related knowledge and behaviors remains important to inform legislation, health policy and regulatory policies. A less frequent NYTS administration would not have been able to track these nuanced changes in e-cigarette use and resulting changes in overall tobacco product use among youth. Furthermore, FDA also requires annual monitoring of youth tobacco product use behavior to inform and evaluate tobacco regulatory policies. The collection of annual data has been particularly important in the early years following FDA's regulatory authority as many regulations are being implemented in a short time frame. For example, in December 2019, the federal minimum age of sale of all tobacco product types increased from 18 to 21 years (FDA, 2019). Furthermore, under the authority of the 2009 Family Smoking Prevention and Tobacco Control Act, FDA issued guidance in January 2020 to prioritize the enforcement against certain flavored e-cigarette products that appeal to youths, including mint and fruit flavors (FDA, 2020). The annual NYTS monitors tobacco product use among the nation’s youth and collects key information that will assist both CDC and FDA in ensuring that both agencies are protecting the public’s health. The collaboration between CDC and FDA in administering the NYTS annually will help both federal agencies, as well as other stakeholders whose mission it is to reduce tobacco product use. Maintaining the 2021 NYTS data collection will maintain continuity in supporting this mission.

A.7 SPECIAL CIRCUMSTANCES RELATING TO THE GUIDELINE OF 5 CFR 1320.5

The data collection will be implemented in a manner consistent with 5 CFR 1320.5. No special circumstances are applicable to this proposed survey.

A.8 COMMENTS IN RESPONSE TO THE FEDERAL REGISTER NOTICE AND EFFORTS TO CONSULT OUTSIDE THE AGENCY

Federal Register Announcement

A 60-day Federal Register Notice (Attachment A2) was published in the Federal Register on January 23, 2020, Docket no. CDC-2019-0117, volume 85, no 15, document no. 2020-01042, pages 3916-3918.

Six comments were received through the 60-day FRN. CDC’s responses to these comments are provided in Attachment A3.

Consultations

Since the 2015 NYTS, the CDC and FDA established a working group to obtain guidance and suggestions for new items on the questionnaire that would help facilitate the measurement of key data needed to address the missions of both agencies.

Consultations on the design, instrumentation, products, and statistical aspects of the NYTS have been made with these experts to ensure the technical soundness and user relevance of survey results; to verify the importance, relevance, and accessibility of the information sought in the survey; to assess the clarity of instructions; and to minimize respondent burden. Some of these experts are shown in in Table A.8 below.

Table A.8: Consultants for 2021-2023 NYTS

	Office on Smoking and Health, Centers for Disease Control and Prevention 4770 Buford Highway NE, Atlanta GA 30341
	Linda J. Neff, Ph.D., M.S.P.H. Chief, Epidemiology Branch Phone: 770-488-8647 E-mail: LNeff@cdc.gov	David Homa, Ph.D., M.P.H. Senior Science Advisor, Epidemiology Branch Phone: 770-488-3626 E-mail: DHoma@cdc.gov
	Brian A. King, Ph.D., M.P.H. Deputy Director for Research Translation, Office of the Director Phone: 770-488-5107 E-mail: BaKing@cdc.gov	Ahmed Jamal, M.B.B.S., M.P.H. Surveillance Team Lead, Epidemiology Branch Phone: 770-488-5077 E-mail: AJamal@cdc.gov
	Katrina Trivers, Ph.D., M.S.P.H Research Team Lead, Epidemiology Branch Phone: 404-498-6861 Email: KTrivers@cdc.gov	Sean Hu, MD, Dr.P.H. Senior Epidemiologist, Epidemiology Branch Phone: 770-488-5845 E-mail:SHu@cdc.gov
Andrea Gentzke, Ph.D., M.S. Health Scientist, Epidemiology Branch Phone: 404-498-1795 Email: AGentzke@cdc.gov		Teresa Wang, Ph.D., M.S. Epidemiologist, Epidemiology Branch Phone: 404-498-1502 E-mail: TWWang@cdc.gov
Center for Tobacco Products, Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993
Bridget Ambrose, Ph.D., M.P.H. Deputy Director, Division of Population Health Science Phone: 301-796-4235 E-mail: Bridget.Ambrose@fda.hhs.gov		Benjamin Apelberg, Ph.D., M.H.S. Director, Division of Population Health Science Phone: 301-796-8869 E-mail: Benjamin.Apelberg@fda.hhs.gov
Karen Cullen, Ph.D., M.P.H. Chief, Epidemiology Branch 2 Phone: 240-402-4513 E-mail: Karen.Cullen@fda.hhs.gov		Eunice Park-Lee, Ph.D., M.S. Survey Statistician, Division of Population Health Science Phone: 301-837-7342 E-mail: Eunice.Park-Lee@fda.hhs.gov
Michael Sawdey, Ph.D., M.P.H. Epidemiologist, Epidemiology Branch 2 Phone: 301-796-3452 Email: Michael.Sawdey@fda.hhs.gov		Kimberly Snyder, M.P.H. Social Scientist, Evaluation Branch Phone: 240-402-2216 E-mail: Kimberly.Snyder@fda.hhs.gov

A.9 EXPLANATION OF ANY PAYMENT OR GIFT TO RESPONDENTS

Schools will be given $500 in appreciation for their participation in NYTS, which is consistent with previous years of NYTS implementation (2011-2020). No payments will be offered or made to student respondents. OMB first suggested that CDC offer school incentives on school-based surveys as a means of improving school response rates and, thereby, improving the generalizability of results. Increasingly in recent years, school-based data collections, most of which do not fall under OMB review, have offered financial incentives to increase or maintain school participation rates. CDC believes that offering school incentives helps maintain, or slightly increase, school participation rates despite the growing number of competing, non-instructional demands placed on schools, including standardized testing.

A.10 PROTECTION OF THE PRIVACY AND CONFIDENTIALITY OF INFORMATION PROVIDED BY RESPONDENTS

The CIO’s Information Systems Security Officer reviewed this submission and determined that the Privacy Act does not apply. This determination is based on the fact that the information that will be collected within NYTS is not considered a “record” as defined by the Privacy Act: it will not include individuals’ financial transactions, medical history, criminal or employment history, name, or the identifying number, symbol, or other identifier assigned to any individual, such as a finger or voice print or a photograph. No individually identifiable information is collected on the NYTS survey; therefore, there is no way to connect students’ names to their response data. Participation in the NYTS should pose little or no effect on the respondent’s privacy. Participation is voluntary and respondents will be assured that there is no penalty if they decide not to respond, either to the information collection as a whole or to any particular question. Participants can choose to leave the study at any point. Participants also can choose to skip any questions they find uncomfortable.

The NYTS does not collect any student-level personal identifiers. Furthermore, school-level identifiers and sub-national level identifiers (e.g., state code, month and date of survey) are not included in the final analytic dataset to protect the privacy and confidentiality of individual respondents. The sampling variables in the dataset, required for use in analysis using complex sampling procedures, have been modified so that users cannot identify locations based on these variables. The Primary Sampling Unit (PSU) ID is created using an algorithm seeded by a random number. SAS is used to create the new PSU ID as a function of the Original PSU. A PSU crosswalk, not available in the public use data release, is provided by the contractor to CDC OSH.

All selected schools, students, and their parents will be informed that anonymity will be maintained throughout data collection, that all data will be safeguarded closely, and that no institutional or individual identifiers will be used in study reports. Anonymity and protection of privacy are promised to students on parental permission forms (Attachments H3, H4, H5, H6). Students will be reminded that their responses are anonymous at the start of the survey administration session by a professionally trained NYTS data collector. Access controls used to secure and protect collected data are listed in Table A.10.

In the 2021 administration, protection of participant confidentially and privacy will be maintained under virtual administration of the survey. On the day of scheduled survey administration, the teacher will provide students with the survey URL and a unique access code. Students participating in the classroom will be provided with a randomly distributed code. Students participating outside of the classroom (hybrid or exclusively distance learning) will be provided with a classroom-level access code. For those using a classroom-level access code, the system will automatically assign a unique student-level access code to each student upon logging in; students will be instructed to write this code down in case they need to log off and resume the survey at a later time. Students will take the survey using an internet connected device provided by the school or by the student.

Student access codes cannot be linked back to individuals. Those participating on-campus are provided with a randomly distributed access code and students do not provide their name on any study materials. Use of the classroom-level access code for those participating outside the classroom avoids linking unique student-level access codes to individual students (e.g., through email or other electronic forms of communication). If a student loses internet access during the survey or if they need to step away from the survey prior to submitting their responses, they can log out of the web survey application and log back in later using their unique 5 -digit student access code. This ensures that the student’s responses remain private and that they can resume the survey where they left off to complete it. Participants maintain the same rights regarding participating in the virtual survey as they do for the tablet-based survey. Access controls (Table A.10) remain identical to those of the tablet-based survey.

Table A.10: Access Controls

Technical Controls	Physical Controls	Administrative Controls
User identification Passwords Firewall Virtual Private Network (VPN)	Guards/Security Officers 24-hour maintenance of Video/Audio of all data centers and all offices Identification badges Key Cards	No directly identifying information will be collected (thus, the Privacy Act does not apply). Methods will be in place to ensure least privilege. Data and all identifying information about respondents will be handled in ways that prevent unauthorized access at any point during the study. All contractor staff involved with the project are required to sign a non-disclosure, intellectual property, non-competition and non-solicitation agreement which is a statement of personal commitment to safeguard data obtained. NYTS data are currently stored on the CDC network in an access-restricted CDC shared directory folder (\\cdc.gov\project\NCCD_OSH_NYTS). Two versions of this shared directory have been created, one version providing read-only access to files and the other version providing full access. The NYTS project leader, Dr. Ahmed Jamal, manages access to the shared directory. Currently, these files containing the raw and final data for each year of the NYTS is only accessible to OSH Epidemiology branch leadership and the NYTS implementation team members.

A.11 INSTITUTIONAL REVIEW BOARD (IRB) AND JUSTIFICATION FOR SENSITIVE QUESTIONS

All procedures have been developed in accordance with federal, state, and local guidelines to ensure that the rights and privacy of participants are protected and maintained. This data collection has received IRB approval from the CDC Human Research Protection Office. This approval is noted on the parental permission forms. Current NYTS IRB Approval Letters are in Attachments J1 and J2.

Sensitive Questions

Although unlikely, certain questions asked during the survey about tobacco product use could be considered by some individuals to be sensitive, although tobacco product use behaviors would not generally be considered highly sensitive. Of note, the 2020 NYTS survey was approved to begin collecting information related to sexual orientation. The data collection instrument for 2021 also has proposed asking questions related to gender identity, depression and anxiety, and socio-economic status, all of which are associated with higher tobacco product use. As with other questions on the NYTS, participants may choose to skip any question they are not comfortable answering. Furthermore, no protected personal information is being collected in this study that could trace responses back to individual students.

A.12 ESTIMATES OF ANNUALIZED BURDEN HOURS AND COSTS

Federal tobacco control and surveillance activities must adapt to a dynamic product environment. From time to time, CDC may modify instrument content to reflect changes in the federal government’s need for information to inform public health and regulatory activities. These modifications will be submitted to OMB through the Change Request mechanism.

Before requesting OMB approval of changes to the NYTS questionnaire, CDC also may conduct (i) cognitive testing of new questions or proposed changes in the wording of, or response options associated with individual questions, and/or (ii) pre-testing of the NYTS instrument as a whole, to ensure that burden per response remains compatible with administration in one class period. Detailed descriptions of these information collections also will be submitted to OMB under the Change Request mechanism.

Estimated Burden Hours

The estimated burden for this information collection is based on over 20 years of experience conducting the NYTS. The planned information collection involves administration of the NYTS questionnaire (Attachment H1) to independent samples of students in the spring of 2021. Respondents include state-level, district-level, and school-level administrators who provide information in the Recruitment Scripts for the NYTS (Attachments D1, E1, and F1), teachers who complete the Data Collection Checklist for the NYTS (Attachment G1). For the 2021 cycle of data collection, the total estimated number of respondents, by type, will include: state-level administrators (n=37), district-level administrators (n=304), and school-level administrators (n=509) who provide information in the Recruitment Script for the NYTS; teachers (n=1,177) who complete the Data Collection Checklist for the NYTS; and students (n=24,000) who receive instructions for and complete the NYTS questionnaire. There are no costs to respondents except their time.

Burden estimates are based on expected sample sizes and budget under the current contract for conducting the 2018-2020 NYTS cycle. Due to changes in the relevant product environment, patterns of tobacco product use, or other factors, testing may be needed to assess new questions, changes in the wording of existing questions, or the response options associated with individual questions. The burden table includes an additional allocation of 75 annualized burden hours for instrument testing activities, resulting in 153 total annualized burden hours for these testing activities (up from 78 in previous package). The estimate of 153 burden hours per year was developed as follows. Cognitive testing of questionnaire content will typically be conducted in semi-structured interviews of two hours or less (40 interviews per year @ two hours per interview = 80 burden hours). In addition, CDC may conduct pre-tests to ensure that each year’s NYTS questionnaire can be completed within one class period (30 tests per year @ 45 minutes/test = 23 hours). Finally, the allocation for testing includes screening of up to 300 youth prior to participation in these testing activities (300 youth @ 10 minutes/response = 50 hours). Such respondent screening is needed to ensure that testing is conducted with individuals whose characteristics are similar to the NYTS target population of youth in grades 6-12. The configuration of testing activities may vary from year to year. For purposes of burden estimation, no more than 300 respondents will be included in screening activities (10 minutes a response).  Of those 300 respondents, no more than 40 may participate in cognitive testing (120 minutes a response) and no more than 30 may participate in survey pretesting activities (45 minutes a response). Each testing activity will be submitted to OMB as a Change Request.

Due to the augmented sampling increasing the number of schools contacted for 2021, there is a small increase in burden for recruitment, compared to the tablet-based survey. However, burden for teachers and student participants remains the same. Students also still have up to one 45-minute class period to complete the survey. Burden for cognitive testing activities also remains unchanged.

The total burden estimated for the NYTS and associated support activities is 18,874 hours. These totals for this cycle are provided in Table A.12a.

Estimated Cost to Respondents

There are no direct costs to the respondents themselves or to participating schools. However, the cost for administrators, teachers, and students can be calculated in terms of their time. In each category, the estimated respondent burden hours have been multiplied by an estimated average hourly salary for persons in that category. Wages are based on May 2018 national data on occupational employment and wages published by the U.S. Bureau of Labor Statistics (USBLS 2019). The estimated burden cost in terms of the value of time students spend in responding are based on a minimum wage for students aged less than 20 years of $4.25/hour. The total estimated respondent burden cost for conducting the 2021 NYTS is $114,999 (Table A.12b). This estimate has been increased for the 2021 administration due to augmentation of the sample to account for anticipated lower school participation as schools adjust to changing learning environments due to the COVID-19 pandemic.

Table A.12a: Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden Per Response (In Hours)	Total Burden (In Hours)
State Administrators	State-level Recruitment Script for the National Youth Tobacco Survey	37	1	30/60	19
District Administrators	District-level Recruitment Script for the National Youth Tobacco Survey	304	1	30/60	152
School Administrators	School-level Recruitment Script for the National Youth Tobacco Survey	509	1	30/60	255
Teachers	Data Collection Checklist for the National Youth Tobacco Survey	1,177	1	15/60	295
Students	National Youth Tobacco Survey	24,000	1	45/60	18,000
	Cognitive Testing	40	1	120/60	80
	Survey Pre-tests	30	1	45/60	23
	Testing Activities	300	1	10/60	50
	Total				18,874

Table A.12b: Annualized Estimated Cost to Respondents

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden Per Response (In Hours)	Hourly Wage Rate	Total Respondent Costs
State Administrators	State-level Recruitment Script for the National Youth Tobacco Survey	37	1	30/60	$60.23	$1,114
District Administrators	District-level Recruitment Script for the National Youth Tobacco Survey	304	1	30/60	$70.23	$10,675
School Administrators	School-level Recruitment Script for the National Youth Tobacco Survey	509	1	30/60	$63.38	$16,130
Teachers	Data Collection Checklist for the National Youth Tobacco Survey	1,177	1	15/60	$33.75	$9,931
Students	National Youth Tobacco Survey	24,000	1	45/60	$4.25	$76,500
	Cognitive Testing	40	1	120/60	$4.25	$340
	Survey Pre-tests	30	1	45/60	$4.25	$96
	Testing Activities	300	1	10/60	$4.25	$213
Total						$114,999

A.13 ESTIMATES OF OTHER TOTAL ANNUAL COST BURDEN TO RESPONDENTS OR RECORD KEEPERS

There will be no respondent capital and maintenance costs.

A.14 ANNUALIZED COSTS TO THE GOVERNMENT

The NYTS currently is funded through 2022 under Contract No. 200-2017-F-96232. The total contract award to ICF (Rockville, MD) to conduct the 2021 and 2022 NYTS is $5,777,520. The estimated cost of the contract, annualized over two of the three years of this clearance request, is $2,691,993. These costs cover the activities in Table A.14 below. Of note, there is an overall price reduction due to the 2021 NYTS not utilizing traditional field staff (i.e. personnel who travel throughout the fielding period to participating schools to administer the survey). Technical assistance Providers, used in lieu of field staff, will provide virtual support before, during, and after data collection. Some activities will be conducted during the pre-clearance period and others will occur post-clearance. The contract will be up for rebid for 2023 and forward; study costs are expected to be comparable.

Additional costs will be incurred indirectly by the government in personnel costs of staff involved in oversight of the study and in conducting data analysis. It is estimated that three CDC employees will be involved for approximately 40% of time (one at a salary of $59.86 per hour and two at salaries of $42.60 per hour) and one CDC employee 70% of time at a salary of $50.66 (based on 2019 General Schedule Locality Pay Tables for Atlanta, GA; for federal personnel 100% time = 2,080 hours annually). The direct annual costs in CDC staff time will be approximately $120,690 + $73,761 = $194,451 annually. The total estimated annualized cost for the study, including the contract cost and federal government personnel cost, is $2,886,444‬.

Table A.14: Estimated Annualized Study Cost

Activity	Cost
Contract Costs
Design and plan	$386,611
Programming and developing	$206,238
Recruitment and preparation	$1,223,784
Printing and distribution	$0
Recruiting and training	$92,184
Collection of data	$416,248
Processing, cleaning, weighing and developing data files	$261,878
Dissemination and reporting of results	$105,050
Subtotal	$2,691,993
Federal Employee Time Cost
40% time for three FTEs	$120,690
70% time for one FTE	$73,761
Subtotal	$194,451
Total Estimated Annualized Cost to the Federal Government	$2,886,444

*Components may not sum to this figure due to rounding.

A.15 EXPLANATION FOR PROGRAM CHANGES OR ADJUSTMENTS

The 2018-2020 NYTS instruments were revised through non-substantive change requests to maintain valid surveillance of both traditional and emerging tobacco products as well as obtain other data relevant to the youth tobacco product use environment. The content of the 2021 instrument remains largely consistent with the past NYTS surveys. For 2021-2023, there are no changes to the estimated burden per response, the frequency of data collection for the survey instrument, the recruitment scripts, or the checklist used by teachers. For this approval period, we are including an allocation of 55 new burden hours per year to allow for instrument testing activities (133 burden hours, total).

We have increased the number of school districts contacted, and thus the number of schools in the sample, in an effort to adjust further for nonresponse and to achieve a sample of approximately 24,000 middle school and high school students. This therefore increases the overall burden estimate slightly relative to the previous cycle. The 2018-2020 NYTS approval was based on 25,614 annualized responses and 18,537 annualized burden hours. Current estimates for the 2021-2023 cycles of survey administration are based on estimates of 26,397 annualized responses and 18,874 annualized burden hours for 2021. The increase in burden hours is due to further augmentation of the sample size in anticipation of a potential reduction in school participation rates as schools adjust to changing educational environments due to the COVID-19 pandemic.

Table A.15: Annualized Estimates of Respondents and Burden, 2021-23 NYTS

Type of Respondent	Form Name	No. of Respondents	Change from 2018-20	No. of Responses per Respondent	Average Burden Per Response (In Hours)	Total Burden (In Hours)	Change from 2018-20
State Administrators	State-level Recruitment Script for the National Youth Tobacco Survey	37	-1	1	30/60	19	0
District Administrators	District-level Recruitment Script for the National Youth Tobacco Survey	304	+151	1	30/60	152	+75
School Administrators	School-level Recruitment Script for the National Youth Tobacco Survey	509	+269	1	30/60	255	+135
Teachers	Data Collection Checklist for the National Youth Tobacco Survey	1,177	+204	1	15/60	295	+52
Students	National Youth Tobacco Survey	24,000	0	1	45/60	18,000	0
	Cognitive Testing	40	+10	1	120/60	80	+50
	Survey Pre-tests	30	0	1	45/60	23	0
	Testing Activities	300	+150	1	10/60	50	+25
	Total	26,397	+783			18,874	+337

A.16 PLANS FOR TABULATION AND PUBLICATION AND PROJECT TIME SCHEDULE

Tabulation Plans

The results from 2021 NYTS should not be directly compared to previous or subsequent years of data collection that primarily collect data on school campuses. Comparability will be compromised by the changes in methods of survey administration and data collection. The circumstances of the pandemic do not allow for a bridge study or other methodological research techniques typically used to assess the impact of a major change in survey methods.

For example, school closures due to the COVID-19 pandemic could potentially pose a digital disparity, particularly among students from lower socioeconomic status (SES). For example, students lacking consistent and reliable access to internet, even for virtual learning, may skew the data and/or survey response rate. Analyses will be conducted to determine the demographic distribution of who took the survey and a non-response analysis will be conducted to determine the potential impact of the change to web-based administration of the survey.

Data will be tabulated in ways that will address the principal research purposes outlined in A.2. Some of the planned analyses and the sample table shells are shown in Attachment K. Data will be summarized using descriptive analyses, including percentages, means, and interquartile ranges. Within-group comparisons will be made using chi-squared tests, ANOVA, and F-statistic, as appropriate. Multivariable analyses will be done using regression models.

1. Estimate the prevalence of tobacco product use behaviors and behavioral determinants among middle and high school students overall and by sex, grade in school, and race/ethnicity--Descriptive statistics (percentages and confidence intervals) will be calculated to address this objective.

2. Assess whether tobacco product use behaviors and behavioral determinants vary by sex, grade in school, and race/ethnicity--Cross tabulations, chi-squared analyses, and regression analysis initially will be conducted to address this objective.

3. Determine the associations between tobacco product use behaviors and behavioral determinants –chi-squared and logistic regression analyses will be used.

4. Describe trends in tobacco product use behaviors and behavioral determinants among middle and high school students overall and by sex, grade in school, and race/ethnicity--Multiple regression analyses that controls for sex, grade in school, and race/ethnicity and that simultaneously assesses linear and higher order time effects will be used.

5. Examine the effects of schools and local areas (school districts or PSUs) in estimating the prevalence of tobacco product use-- multilevel models will be used.

Examples of the table shells that will be completed through analysis of the data are in Attachment L.

Publication Plans

This information will be used to inform the development of policy briefs, official reports, and peer reviewed scientific papers for publication in journals. Annually, the official estimates of national youth tobacco product use typically are featured in MMWR; beginning with the 2019 cycle, the annual estimates were published as a MMWR Surveillance Summary (CDC, 2019b). However, other NYTS findings have been published in high profile peer review journals such as the Journal of the American Medical Association (JAMA), American Journal of Public Health, the American Journal of Preventive Medicine, and JAMA Pediatrics. Additionally, NYTS results and a public use data set are available on the CDC web site at: http://www.cdc.gov/tobacco/data_statistics/surveys/NYTS/index.htm.

Time Schedule for the Project

The following represents our proposed schedule of activities for the NYTS, in terms of months after receipt of OMB clearance. The end date for data collection is constrained by the dates on which schools close for the summer. In addition, given that some twelfth-grade students may be absent during the final weeks of the school year, it is highly desirable to complete data collection one months before schools close for the summer.

Key project dates will occur during the following time periods for the 2021 data collection:

Table A.16: Schedule of Activities for 2021 NYTS

Activity	Time Period
Recruit and schedule schools	1 to 3 months after OMB clearance
Program digital survey	<1 to 2 months after OMB clearance
Train field data collectors	2 months after OMB clearance
Collect data	2 to 5 months after OMB clearance
Process data	3 to 6 months after OMB clearance
Weight/clean data	7 to 8 months after OMB clearance
Produce data file with documentation	9 months after OMB clearance
Analyze data	10 to 11 months after OMB clearance
Publish results	15 to 17 months after OMB clearance

Data collection is currently scheduled to occur during February through May 2021. The time schedule for the 2022 and 2023 data collections will be analogous to that of the 2021 data collection. Results will be published by early 2022 initially in MMWR, and subsequently, in other publications.

A.17 REASON(S) DISPLAY OF OMB EXPIRATION DATE IS INAPPROPRIATE

The expiration date of OMB approval of the data collection will be displayed.

A.18 EXCEPTIONS TO CERTIFICATION FOR PAPERWORK REDUCTION ACT SUBMISSIONS

There are no exceptions to the certificate.

REFERENCES

CDC (2001). Youth Tobacco Surveillance–United States, 2000. MMWR; 50(SS-4).

CDC (2012). National Youth Tobacco Survey. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention. Available at http://www.cdc.gov/tobacco/data_statistics/surveys/nyts.

CDC (2013). Arrazola RA, Singh T, Corey CG, et al. Tobacco Use Among Middle and High School Students — United States, 2011–2014. MMWR Morb Mortal Wkly Rep 2015 / 64(14);381-385.

CDC (2014). Best Practices for Comprehensive Tobacco Control Programs – 2014. Atlanta, GA: U.S. Department of Health and Human Services, CDC, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.

CDC (2015). Arrazola RA, Singh T, Corey CG, et al. Tobacco Use Among Middle and High School Students — United States, 2011–2014. MMWR Morb Mortal Wkly Rep, 2015: 64(14);381-385.

CDC (2016). Centers for Disease Control and Prevention. CDC Winnable Battles: final report. Available at https://www.cdc.gov/winnablebattles/report/index.html.

CDC (2018a). Centers for Disease Control and Prevention. CDC's 6|18 initiative: accelerating evidence into action. Available at https://www.cdc.gov/sixeighteen/index.html.

CDC (2018b). Centers for Disease Control and Prevention. Fiscal year 2019: Justification of estimates for appropriation committees. Available at https://www.cdc.gov/budget/documents/fy2019/fy-2019-cdc-congressional-justification.pdf.

CDC (2019a). Gentzke AS, Creamer M, Cullen KA, et al. Vital Signs: Tobacco product use among middle and high school students — United States, 2011–2018. MMWR Morb Mortal Wkly Rep 2019;68:157–164. DOI: http://dx.doi.org/10.15585/mmwr.mm6806e1

CDC (2019b). Wang TW, Gentzke AS, Creamer MR, et al. Tobacco Product Use and Associated Factors Among Middle and High School Students — United States, 2019. MMWR Surveill Summ 2019;68(No. SS-12):1–22. DOI: http://dx.doi.org/10.15585/mmwr.ss6812a1

CDC (2020). Wang TW, Neff LJ, Park-Lee E, et al. E-cigarette Use Among Middle and High School Students – United States, 2020. MMWR Morb Mortal Wkly Rep 2020;69:1310-1312. DOI: http://dx.doi.org/10.15585/mmwr.mm6937e1

FDA (2014). FDA Proposes to Extend Its Tobacco Authority to Additional Tobacco Products, including e-cigarettes. FDA NEWS RELEASE. N.p., 24 Apr. 2014. Web. 9 May 2014. http://www.fda.gov/newsevents/newsroom/pressannouncements/ucm394667.htm

FDA (2018). Cullen KA, Ambrose BK, Gentzke AS, et al. Notes from the Field: Use of Electronic Cigarettes and Any Tobacco Product Among Middle and High School Students — United States, 2011–2018. MMWR Morb Mortal Wkly Rep 2018;67:1276–1277. DOI: http://dx.doi.org/10.15585/mmwr.mm6745a5

FDA (2019). Food and Drug Administration. newly signed legislation raises federal minimum age of sale of tobacco products to 21. Silver Spring, MD: US Department of Health and Human Services, Food and Drug Administration; 2019. https://www.fda.gov/tobacco-products/ctp-newsroom/newly-signed-legislation-raises-federal-minimum-age-sale-tobacco-products-21.

FDA-CDC (2019). Cullen KA, Gentzke AS, Sawdey MD, et al. E-cigarette use among youth in the United States, 2016-2019. JAMA 2019;322(21):2095-2103. Published online November 05, 2019. doi:10.1001/jama.2019.18387

FDA (2020). Center for Tobacco Products. Enforcement priorities for Electronic Nicotine Delivery Systems (ENDS) and other deemed products on the market without premarket authorization (revised). Silver Spring, MD: US Department of Health and Human Services, Food and Drug Administration; 2020. https://www.fda.gov/media/133880/download.

Hu SS, Gentzke A, Jamal A, et al. Feasibility of Administering an Electronic Version of the National Youth Tobacco Survey in a Classroom Setting. Prev Chronic Dis 2020;17:190294. DOI: https://doi.org/10.5888/pcd17.190294.

National Institute on Drug Abuse (2014). Monitoring the Future national results on drug use: 1975-2013: Overview, Key Findings on Adolescent Drug Use. National Institute on Drug Abuse, National Institutes of Health. Ann Arbor, MI: Institute for Social Research, The University of Michigan.

OSG (2018). Office of the Surgeon General. Surgeon General’s advisory on e-cigarette use among youth. December 18, 2018. Available at https://e-cigarettes.surgeongeneral.gov/documents/surgeon-generals-advisory-on-e-cigarette-use-among-youth-2018.pdf.

USBLS 2018. U.S. Bureau of Labor Statistics (2019). May 2018 National Occupational Employment and Wage Estimates, United States. Available at http://www.bls.gov/oes/current/oes_nat.htm

USDHHS (2010). U.S. Department of Health and Human Services. Healthy People 2020. Washington, D.C.: Available at: http://healthypeople.gov/2020/default.aspx

USDHHS (2012). US Department of Health and Human Services, Ending the Tobacco Epidemic: Progress toward a Healthier Nation. Washington, DC: Office of the Assistant Secretary for Health.

USDHHS (2019). U.S. Department of Health and Human Services. Development of the national health promotion and disease prevention objectives for 2030. Available at https://www.healthypeople.gov/2020/About-Healthy-People/Development-Healthy-People-2030.

24