2017 NATIONAL YOUTH TOBACCO SURVEY Computer Based Pilot Survey
OMB No. 0920-16AWN
SUPPORTING STATEMENT: PART B
ICRO’s Desk Officer Review
December 1, 2016
Submitted by:
Ahmed Jamal, MBBS, MPH
Centers for Disease Control and Prevention
Office on Smoking and Health
Epidemiology Branch
4770 Buford Highway NE, MS-F 79
Atlanta, GA 30341 Phone: 770-488-5077
E-mail: AJamal@cdc.gov
Centers for Disease Control and Prevention
Department of Health and Human Services
TABLE OF CONTENTS
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
B.1. Respondent Universe and Sampling Methods
B.2. Procedures for the Collection of Information
B.3. Methods to Maximize Response Rates and Deal with Nonresponse
B.4. Tests of Procedures or Methods to Be Undertaken
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
LIST OF ATTACHMENTS
Authorizing Legislation
B1. 60-Day Federal Register Notice
B2. Public Comments on the 60-Day Federal Register Notice and CDC Response
C. State Tobacco Control Reports that Cite National Youth Tobacco Survey Data
D Publications from Prior Cycles of National Youth Tobacco Survey
E1. State-level Recruitment Scripts for the National Youth Tobacco Survey
E2. State-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents - State Letter of Invitation
F1. District-level Recruitment Scripts for the National Youth Tobacco Survey
F2. District-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents - District Letter of Invitation
G1. School-level Recruitment Scripts for the National Youth Tobacco Survey
G2. School-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents - School Letter of Invitation and NYTS Fact Sheet for Schools
G3. School-level Recruitment Script for the National Youth Tobacco Survey Supplemental Documents - Letter to Agreeing Schools
H1. Data Collection Checklist for the National Youth Tobacco Survey
H2. Data Collection Checklist for the National Youth Tobacco Survey Supplemental Documents - Letter to Teachers in Participating Schools
I1. National Youth Tobacco Survey Questionnaire
I2. National Youth Tobacco Survey Questionnaire Supplemental Documents - Parental Permission Form Distribution Script
I3. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form and Fact Sheet (English Version)
I4. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form and Fact Sheet (Spanish Version)
I5. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form Reminder Notice (English Version)
I6. National Youth Tobacco Survey Questionnaire Supplemental Documents – Parental Permission Form Reminder Notice (Spanish Version)
I7. National Youth Tobacco Survey Questionnaire Supplemental Documents - Questionnaire Administration Guide
I8. Example: Instrument Testing Activity
J. IRB Approval Letter
K. Sample Table Shells
L. Detailed Sampling and Weighting Plan
LIST OF TABLES
Table A.12a - Estimated Annualized Burden Hours
Table A.12b - Annualized Estimated Cost to Respondents
Table A.14a - Annualized Study Cost
Table B.1a - Distribution of Schools by Urban Status and School Type
Table B.3a - Major Means of Quality Control
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
The National Youth Tobacco Survey (NYTS) is currently a paper and pencil survey. The CDC proposes to conduct an evaluation of an electronic version of the survey in 2017 using an electronic tablet device to administer a computer-assisted self-interview (CASI) survey. The pilot evaluation of the electronic mode of delivery will provide information about 1) respondent burden; 2) the reliability and efficiencies of electronic mode data collection; (3) the reliability and validity of survey results obtained from electronic data; (4) the cost-effectiveness of electronic administration; and (5) the length of time between data collection and dissemination of findings.
The primary objectives of the NYTS are to develop estimates of tobacco use behaviors and their correlates among students enrolled in middle school and high school; to identify differences related to demographic characteristics (age, grade, sex, and race/ethnicity); and to determine whether there are time trends in tobacco use behaviors and exposure to influences that promote or discourage tobacco use. Data from the NYTS provide a comprehensive picture of the tobacco use behaviors of adolescents and their exposure to pro- and anti-tobacco influences. Data are reported at the national level only; no school district or regional estimates will be produced. Such information is required to support federal responsibilities in providing technical assistance in the planning, monitoring, and evaluation of national, state, and local tobacco prevention and control programs.
The 2017 NYTS and the NYTS Pilot are a continuation of the NYTS cycles that took place in 1999, 2000, 2002, 2004, 2006, 2009, and 2011-2016. The NYTS employs a repeat cross-sectional design to develop national estimates of tobacco use behaviors and exposure to pro- and anti-tobacco influences among students enrolled in grades 6–12. The general sampling design framework used for the 2016 NYTS will be employed for the 2017 electronic pilot at the primary sampling unit (PSU) and secondary sampling unit (SSU) levels. The sampling frame for schools will be obtained from using the same methods that NYTS has used in prior years of conducting the survey.
Table 3.1 displays the U.S. distribution of eligible schools by urban status and type of school as of 2014. This tabulation was computed over a frame of eligible schools with middle school and/or high school grades prepared using the data filed used for the sampling frame. The figures displayed in this table will be updated to reflect 2016 results in the final methodology report. The CASI Pilot survey will stratify the sampled population by school type and urban status according to the same proportions that will be applied to the paper and pencil interviewing (PAPI) survey.
Table B.1a – Distribution of Schools by Urban Status and School Type
Table of School Type by Urban Status |
|||
School Type |
Urban Status |
||
Frequency |
Rural |
Urban |
Total |
Non-public |
5597 |
7737 |
13334 |
Public |
32060 |
22869 |
54929 |
Total |
37657 |
30606 |
68263 |
For the electronic pilot survey a probability sample will be selected that will support national estimates by grade, public/private school type, and size of school for students enrolled in grades 6-12. The design will further support separate estimates of the prevalence rates of tobacco use among students by school level (middle and high school) type, grade, and size. The procedures for stratification and sample selection are consistent with those from previous cycles of NYTS.
The sample design will impose a school size threshold as an additional criterion for eligibility. By removing from the frame those schools with an aggregate enrollment of less than 25 students across eligible grades, we can improve efficiency and safeguard privacy.
Sampling Frame and Stratification. The 2017 NYTS electronic survey will use the same methods as the PAPI NYTS to create the sampling frame. The PAPI NYTS uses a combination of sources to create the school frame in order to increase school coverage. Along with the MDR dataset, NYTS has used data from the National Center for Edication Statistics (NCES; the Common Core Dataset (CCD) which is a national file of public schools and the Private School Universe Survey Dataset (PSS), a file of national non-public schools. In this iteration of the survey we will consider data available from the School Districts Demographic System (SDDS) and resources available from the National Assessment of Education Progress (NAEP). In combining multiple data sources we can increase the coverage of schools nationally.
The sampling frame representing the 50 U.S. States and the District of Columbia will be stratified by urban status and by racial/ethnic minority concentrations, as is done for the PAPI survey. The definition of urban status strata, distinguishing urban and non-urban areas, will be based on metropolitan statistical area, or Metropolitan Statistical Area (MSA), versus non-MSA areas. The sample will be structured into geographically defined units, called primary sampling units (PSUs), which consist of one county or a group of small, contiguous counties.
PSUs will be selected with probability proportional to the student enrollment in the PSU. The PSUs will be allocated to the urban/non-urban strata in proportion to the total eligible student enrollment in the stratum. This approach will increase the sampling efficiency by generating a nearly self-weighting sample.
Schools will be classified by enrollment size as small, medium or large. Small schools contain one or more grades with less than 25 students per eligible grade. The remaining schools will be classified as medium if they have fewer than 50 students in any of the eligible grades; otherwise, they are considered large schools. Additional schools will be selected to add approximately 5,500-6,000 students to the national sample.
Classes will be selected based on two specific scientific parameters. First, classes have to be selected in such a way that all students in the school have an equal opportunity for selection to participate. Second, all classes must be mutually exclusive so that no student is selected more than once. In each school, once we have determined the type of class or time period from which classes will be selected, we will randomly select the appropriate number of classes within each grade. To maintain acceptable school participation rates, it is essential that each school have input into the decision of which classes will be sampled in their school following one of the above approaches. Examples of class sampling frames that have been used in past cycles include all 2nd period classes or a required physical education class.
Mode of survey administration, in-person computer-assisted self-interview without skip logic or in-person computer-assisted self-interview with skip logic, will be decided after the selection of schools. Of the additional 6,100 student sample, half will be randomly assigned to CASI without skip logic and half will be randomly assigned to CASI with skip logic. More than one survey condition may be applied within schools as is required to determine internal validity.
School districts, schools, or students who refuse to participate in the study will not be replaced in the sample. We will record the characteristics of schools that refuse along with reasons given for their refusal for analysis of potential study biases.
Throughout its history, the NYTS has maintained exceptional student and school response rates, with an average 78% combined (school x student) response rate. The current sampling frame will estimate this at 72% due to some recent trends. At the school and student levels, response rates have been higher. The school participation rate has averaged 86% and student participation rate has averaged approximately 90%. We plan to implement strategies to maintain or exceed these high response rates as we implement the electronic survey administration.
The anticipated total number of participating students is 6,100 from 64 schools, as developed in Attachment L. We will randomly select 10 schools of the 24 large high schools and 10 schools of the 24 large middle schools into the double class sampling group. In other words, we will select two classes per grade in these schools (i.e., six classes in middle schools and eight classes in high schools) to ensure that target precision levels are met for racial/ethnic minority group estimates. Among the remaining large schools, only one class per grade level will be selected 22 high schools and 22 middle schools). Similarly, one class per grade level will be selected in medium schools. In small schools, that is, those that cannot support a full class selection at each grade, all students in all eligible grades are taken into the sample.
Studies examining the effect of survey mode on the prevalence of self-disclosure on sensitive topics including risk-taking and health-related behaviors among youth have shown mixed results (Brener et al. 2006; Kays, et al. 2012). For example, two studies comparing CASI and paper and pencil interviewing (PAPI) modes of administration in schools found no significant differences in risk behavior prevalence between the two modes (Hallfors et al. 2000). However, substance use prevalence has been shown to vary depending on the degree of privacy offered by the survey mode (Aquillino, Wright, and Supple, 2000). Such differences raise important questions among NYTS stakeholders about whether CASI or PAPI survey modes produce similar prevalence estimates or if prevalence estimates of tobacco usage by youth will be significantly different depending on the type of survey mode. During the deployment of this electronic version, an evaluation of the computer based modality will be employed so that any differences in survey accuracy and resources can be detected.
Comparison of results derived from the two survey modes and the effect prevalence estimates will be assessed. Electronic data collection has known advantages over paper and pencil survey mode including, but not limited to, the ability to use complex skip patters, real-time consistency in individual’s responses and increased efficiency in the data cleaning, compiling and analyzing processes. Previous research has shown that programmed skip patterns results in better quality data because questionnaires contain fewer inconsistent responses than questionnaire without skip patterns (Denniston et al., 2010).
NYTS sampling and recruiting of the respondents will be coordinated for the electronic and PAPI survey administrations to reduce the potential for over- or under-coverage of the target population, gaps in sample frame, or differences in the sample design.
Confidence intervals vary depending upon whether an estimate represents the full population or a subset, such as a particular grade, sex, or racial/ethnic group. Within a grouping, they also vary depending on the level of the estimate and the design effect associated with the measure.
Based on the prior NYTS cycles, as well as on precision requirements that have driven the sampling design, we can expect the following subgroup estimates to be within ±5% at 95% precision level:
Estimates by grade, sex, and grade cross-tabulated by sex
Racial/Ethnic minority group estimates for non-Hispanic blacks and Hispanics cross-tabulated by school level
Perceptions of privacy and anonymity, costs, burden on the school and students, and accuracy and reliability of the data will all be evaluated as part of this electronic pilot study. The purpose will be to assess the advantages and disadvantages of the skip-logic enabled CASI survey mode compared to the traditional PAPI method. The evaluation of the CASI survey without skip-logic will focus on internal validity and the effect of the mode. Further, the Deloitte team will compare prevalence estimates on select key indicators to assess any potential variation in estimates between the two survey modes.
The NYTS was initially designed as a biennial survey. However, as witnessed during the 1990s, youth tobacco use can increase or decrease rapidly, making biennial collection less optimal. On June 22, 2009, the Family Smoking Prevention and Tobacco Control Act was enacted, which gave FDA the authority to regulate the manufacturing, marketing, and sale of tobacco products. Under this new authority, a number of regulatory and enforcement actions are underway or will be commencing soon, including the prohibition of certain types of tobacco advertising and promotion, prohibition of the sale of single cigarettes, elimination of flavors in cigarettes (other than menthol), enforcement of youth access restrictions, and the introduction of graphic warning labels on cigarette packs. To assist the FDA in ensuring its shared goal with CDC of protecting young people from tobacco use is met, annual data collection is needed to monitor the impact of FDA's actions on public health as well as to measure potential unintended consequences (such as increased use of currently unregulated tobacco products such as e-cigarettes and little cigars). The collection of annual data is particularly important in the first few years following the new regulatory authority as many regulations are being implemented in a short time frame. Rather than develop a completely new surveillance system to monitor measures critical to FDA regarding youth tobacco use, thereby increasing burden, CDC and FDA entered into a partnership to leverage the CDC’s existing NYTS system to collect annual data that will be useful to both agencies. The collaboration between CDC and FDA in administering the NYTS annually will efficiently allow both agencies to meet their goals.
The NYTS collects data on key short-term, intermediate, and long-term tobacco prevention and control outcome indicators. Some questions may be added to the questionnaire that pertain specifically to students perceptions of the survey mode. This will be used in the evaluation of the Pilot to determine students’ perceptions of privacy and confidentiality when taking an electronic survey. Specific areas covered by the survey included: prevalence of tobacco product use; knowledge of and attitudes toward tobacco use; pro- and anti-tobacco media and advertising; minors’ access to tobacco products; nicotine dependence; cessation attempts; exposure to second-hand smoke; harm perceptions; exposure to tobacco product warnings; and tobacco use prevention school curricula. Half of students assigned to the CASI mode will answer all survey questions in order to evaluate the effect of the mode, at a high level, against the PAPI surveys. Half of students assigned to the CASI mode will complete a questionnaire programmed with skip-logic, and will therefore only respond questions relevant to their individual tobacco use behavior. Survey validation will be programed into the skip logic survey to indicate logical inconsistences in responses and allow students to re-respond and provide a logical response.
The survey will be programmed using Qualtrics, a commercially available software company that allows for complex and customized survey capabilities. Qualtrics has the ability to collect data without internet connection, collect data on a mobile phone or tablet, and program advanced skip logic. Having these features ensures that all schools will be able to participate in the computer based mode. Qualtrics has been used by CDC for past research, and the software will be certified and approved by CDC and used in compliance with Federal security guidelines. CDC will own all data collected using this software, and all data will be password protected and only accessible to those with approved access.
On the day of the survey, the data collector will bring all materials needed to conduct the survey. The data collector will work with the respective classroom teacher to determine which students have completed the necessary parental permission form process (using the Data Collection Checklist), and consequently are eligible to take the survey. Students who have obtained parental permission to participate, and are in classrooms selected to participate, will be asked to report about their tobacco use behaviors and behavioral determinants on the CASI questionnaire.
Data collectors will distribute computer tablets to each student seat. The computer tablets will be pre-installed with Qualtrics ® Insight Platform software which will enable the student to access the NYTS research study survey instrument without relying on any active internet/web-based computer connection.
After the survey is completed, the NYTS research study data collectors will collect the computer-tablets the students used to access the survey. Completed survey data will be uploaded by the NYTS research study trained data collector to the survey database at the end of the day and off of the premises of the school property using the data upload feature of the software.
The following describes the specific steps the data collectors will take to complete the data collection process:
Step 1: Verify all tablets are running properly, load the survey onto all tablets and verify no other application are running or accessible to students.
Step 2: Verify that all assembled students have completed appropriate permission form process required for this school and that nonparticipating students (if any) have an alternate activity.
Step 3: After students are seated, introduce yourself and the survey to the class. Example script:
I’d like to thank each of you for participating in this computer based research survey today. This survey is being conducted on behalf of the Centers for Disease Control and Prevention (also known as CDC). In this survey you will be asked about your attitudes, knowledge, and behaviors related to tobacco usage. Participating in this survey is voluntary and your grade in this class will not be affected, whether or not you answer the questions. You are not required to answer any questions you do not want to answer. However, only a limited number of students like yourselves are participating in this survey in schools all over the nation. The answers you give are very important so the results are accurate.
I would like to emphasize that this is not a test of you or this school. In order to help develop better education programs, educators and health officials must collect comprehensive data on the attitudes, knowledge, and behaviors of middle and high school students (grades 6‑12) with respect to tobacco, and on other influences that might make a youth susceptible to tobacco use in the future. This research survey is part of that effort.
Step 4: Distribute tablets with survey pre-loaded on screen. Emphasize privacy/ anonymity. Example script:
Throughout the entire survey process, we will maintain strict procedures to protect your privacy and allow for your anonymous participation. Your answers are private and we do not want to know your name. Results of this survey will never be reported by names, class, or school. When you finish the survey, select “submit” and leave the tablet on your desk.
Step 5: Instruct the class in filling out questionnaire. Both the data collector and the teacher will be present in the room during survey administration. While students are taking the survey, work with the teacher to complete the Data Collection Checklist. Example script:
Now I would like you to look at the instructions for the questionnaire. Please take a moment to read the instructions on the screen. The screen can be scrolled using your finger to slide the instructions up and down.
For each question on the survey, there is a corresponding set of squares. For each question, choose the answer that best fits what you know, feel, or do, then select the corresponding square by pressing or selecting it with your finger. You can unselect an answer by pressing or selecting it again. If you must change an answer, you may go back to the question by scrolling to the question. Or, if it is on another page of the survey, by using the guide arrows on the survey to navigate to the page on which the question is located.
We have allowed 35 minutes for completing the survey. If you finish before that time, place your tablet screen down on the desk, and stay seated. It is important that you answer the questions based on what you really know, believe, and do. Don’t pick a response just because you think that’s what someone wants you to say. Your teacher and I are not allowed to answer any questions. Simply do the best that you can. Please begin.
Step 6: At the end of class period, thanks participants. Example script:
The CDC would like to thank all of you for participating in this computer based research survey. The information you have provided will be used to support decisions on how best to inform health education programs for students like yourselves all around the nation. If you have any questions related to the topics on the survey, please contact: Mike Knight xxx-xxx-xxxx.
Step 7: When students leave, collect tablets.
Step 8: Thank the teacher.
Step 9: Take inventory of all tablets and other materials brought to the class. Reset the tablets if another class will be taking the survey on the same day. Pack tablets in carrying cases and collect all other materials if you are moving to another class or this is the last survey instance of the day.
Teachers will be asked to remain at the front or back of the classroom and not to walk around the room monitoring the aisles during survey administration because doing so could affect honest responses and compromise anonymity. Teachers also will be asked to identify students with parental consent to participate in the survey and to make sure non-participating students have appropriate alternate activities. The rationale for this is to increase the candor and comfort level of students. The only direct responsibility of teachers in data collection is to distribute and follow up on parental permission forms sent out prior to the scheduled date of data collection in the school. Teachers are provided with a parental permission form distribution script (Attachment I2) to follow when distributing permission forms to students. The Data Collection Checklist (Attachment H1) is completed by teachers to track which students have received parental permission to participate in the data collection. The teachers receive instructions on completing the Data Collection Checklist in the “Letter to Teachers in Participating Schools” (Attachment H2).
In general, our data collection procedures have been designed to ensure that:
Protocol is followed in obtaining access to schools
Everyday school activity schedules are disrupted minimally
Administrative burden placed on teachers is minimal
Parents give informed permission to participate in the survey
Anonymity of student participation is maintained, with no punitive actions against non-participants
Alternative activities are provided for nonparticipants
Control over the quality of data is maintained
Be aligned to and not overlap with the burden of the full PAPI mode of the 2017 NYTS
CDC will conduct recruitment of the schools for the computer based Pilot so that recruitment procedures for schools mimic the PAPI version of the survey. Additionally, this will halt any potential overlap between discussion of the CASI and PAPI mode in the same school districts. During recruitment, districts and schools will be informed that anonymity will be maintained throughout data collection, that all data will be safeguarded closely, and that no institutional or individual identifiers will be used in study reports. Anonymity will be promised to students and their parents on the parental permission forms. Additionally, at the start of the survey administration sessions, professionally trained NYTS data collectors will instruct students to not enter their names anywhere on the survey instrument and remind them that their responses will be treated in an anonymous manner.
The procedures for gaining access to and support from states, districts, and schools will have three major steps:
First, support will be sought from State Education Agencies and State Departments of Health. The initial request will be accompanied by a study fact sheet and a list of all sampled districts and schools in their jurisdiction. States will be asked to provide general guidance on working with the selected school districts and schools and to notify school districts that they may anticipate being contacted about the survey.
Once cleared at the state level, an invitation packet will be sent to sampled school districts in the state. Districts will receive a list of schools sampled from within their district in the invitation packet and will be asked to provide general guidance on working with them and to notify schools that they may anticipate being contacted about the study. Telephone contact will be made with the office comparable to the district office (e.g., diocesan office of education), if there is one.
Once cleared at the school district level, selected schools will be invited to participate. Information previously obtained about the school will be verified. The burden and benefits of participation in the survey will be presented. After a school agrees to participate, a tailor-made plan for collection of data in the school will be developed (e.g., select classes, determine whether the survey will be administered to selected classes sections simultaneously or in serial). Well in advance of the agreed upon survey administration date, schools will receive the appropriate number of parental consent forms and instructions. All materials needed to conduct the survey will be provided by the data collector visiting the school. Contact with schools will be maintained until all data collection activities have been completed.
Prior experience suggests the process of working with each state’s health and education agencies, school districts and schools will have unique features. Communication with each agency will recognize the organizational constraints and prevailing practices of the agency. Scripts for use in guiding these discussions will be provided along with a NYTS Pilot Fact Sheet and paper copy of the survey for schools
Participation in the NYTS Pilot is voluntary at both the school and student levels. At the student level, participation will be anonymous. Schools will use passive consent forms, but active consent forms will be used at discretion to fulfill state and federal No Child Left Behind requirements. In these instances parents may need to be provided with a means to opt out of their child’s participation. Consent form materials will be distributed to students prior to beginning the survey to obtain active consent and passive consent depending on the jurisdiction.
The consent form informs both the student and the parent about an important activity in which the student has the opportunity to participate. By providing adequate information about the activity, it helps ensure that permission will be informed. A copy of the permission form is contained in Appendices G4 (English version) and G5 (Spanish version). In accordance with the No Child Left Behind Act, the permission form indicates that a copy of the questionnaire will be available for review by parents at their child’s school. Active parental permission forms, forms that must be returned with the parent’s signature in order for the child to participate, will be sent out at least 2 weeks prior to the scheduled date of data collection for schools that require active consent.
Table B3 lists the major means of quality control. As shown, the task of collecting quality data begins with a clear and explicit study protocol and ends with procedures for the visual inspection and scanning of collected data. In between these activities, and subsequent to data collector training, measures must be taken to reinforce training, to assist field staff who express/exhibit difficulties completing data collection activities, and to check on data collection techniques. Because the ultimate aim is production of a high quality database and reports, various quality assurance activities will be applied during the data collection phase.
The Pilot study requires a final yield of approximately 6,100 students. It is necessary to draw the Pilot sample from the full NYTS sample that is considerably larger than this target number to compensate for school and student non-participation. On prior cycles of the NYTS, school participation averaged 89%, with a low of 83%; student participation averaged 90% with a low of 88%. For the 2017 NYTS Pilot, we conservatively have combined (school and student) 72% student participation rate due to recent trends. A $500 incentive will be offered to each participating school.
Table B.3a - Major Means of Quality Control
Survey Step |
Quality Control Procedures |
Mailing to Districts and School |
|
Telephone Follow-up Contacts |
|
Previsit Logistics Verification |
|
Data Collector Training and Supervision of School Visits |
|
Receipt Control |
|
Manual Editing |
|
Computer data collection |
|
NYTS participation rates traditionally have been relatively high compared to other federally funded, national, school-based, health-related surveys of high school students. For example, the widely cited Monitoring the Future survey (formerly known as the High School Senior Survey) achieves substantially lower participation rates. The participation rates established by the NYTS are the product of the application of proven and tested procedures for maximizing school and student participation.
As indicated in A.16.c, it is desirable to complete data collection before the final month of school (i.e., by mid-April to mid-May, depending on location). Many schools are very busy at that time with standardized testing and final exams; in addition, attendance can be very unstable, especially among twelfth grade students.
We have identified six potential types of nonresponse problems: refusal to participate by a selected school district, school, teacher, parent, or student; and collection of incomplete information from a student. To minimize refusals at all levels--from school district to student--we will coordinate with the PAPI administrator on a number of techniques, emphasizing the importance of the survey. Given the subject matter is tobacco, we expect that a few school districts or schools will need to place the issue of survey participation before the school board. To increase the likelihood of an affirmative decision, we will: (1) work through the state agencies to communicate its support of the survey; (2) indicate that the survey is being sponsored by CDC; (3) convey to the school district or school that the survey has the endorsement of many key national educational and health associations, such as the National PTA, American Medical Association, National Association of State Boards of Education, Council of Chief State School Officers and the National School Boards Association;(4) maintain both a toll-free hotline and dedicated email account to answer questions from the school board; (5) offer a package of educational products to each participating school, as recommended by OMB in approving the 1998 YRBS in alternative schools (OMB No. 0920-0416, expiration 12/98) and implemented on NYTS ever since; (6) comply with all requirements from school districts in preparing written proposals for survey clearance; (7) convey a willingness to appear in person, if needed, to present the survey before a school board, research committee, or other local entity tasked with reviewing the survey; and (8) offer schools a monetary incentive of $500.
Maximizing responses and dealing with refusals from parents, teachers, and students require different strategies. To maximize responses, we will recommend that schools help to advertise the survey through the principal’s newsletter, PTA meetings, and other established means of communication. Reminders will be sent to parents who have not returned parental permission forms within an agreed upon time period (e.g., three days); those who do not respond to the reminder will be sent a second and final reminder. The permission form will provide a telephone number at CDC that parents may call to have questions answered before agreeing to give permission for their child’s participation. Permission forms will be available in English, Spanish, and any other languages spoken by a large percentage of parents in a given school district. Field staff will be available on location to answer questions from parents who remain uncertain of permission. Bilingual field staff will be used in locations with high Hispanic concentrations (e.g., California, Florida, New York City, and Texas).
Teacher refusals to cooperate with the study are not expected to be a problem because schools will already have agreed to participate. Refusals by students who have parental permission to participate are expected to be minimal. No punitive action will be taken against a nonconsenting student. Nonconsenting students will not be replaced. Data will be analyzed to determine if student nonresponse introduces any biases.
To minimize the likelihood of missing values on the survey, Students will be given technical guidance to ensure they are able to use the computer tablets to complete the survey.
Deloitte will conduct a non-response analysis to detect whether there are systematic differences between individuals who complete the survey and those who do not choose to complete the survey.
The analysis will first considers differential response rates across population subgroups; i.e., bivariate analyses of potential non-response bias. Any variables found significant in the bivariate analyses will then be included in the multivariate analysis. The multivariate analyses examine the independent effect that each school-level characteristic may have on non-response.
The NYTS core questionnaire items–those identified for use both nationally and at the state level–originally were subjected to cognitive analyses by RTI in 1999. This cognitive analysis directly affected the first NYTS questionnaire fielded in 1999. Cognitive analyses of a small number of new questions were conducted in the fall of 2003 to investigate potential sources of error. A limited pretest of the 2004 NYTS questionnaire was also conducted in August 2003. Cognitive testing was undertaken again prior to the 2006 NYTS. Specifically, testing evaluated revisions to certain existing core survey questions and additional new items subsequently under consideration. In April 2005, a pretest of the NYTS 2006 questionnaire was conducted in accord with OMB guidelines. The pretests sharpened the articulation of certain survey questions and confirmed the existing empirical estimate of the survey burden. In 2012, cognitive testing was performed on 26 new questions that were added to the NYTS; while retaining the overall length of the survey to 81 questions. In 2013, another round of cognitive testing was done but this time it was performed on the whole survey. For the 2015 cycle of NYTS, cognitive testing was done on 11 new questions that focused on electronic vapor products (e.g. electronic cigarettes, electronic cigars, vape pens, electronic hookah). The new questions that were tested, including any changes, and final question wording, are provided in Attachment I-8. This attachment also provides a list of questions that were removed to maintain the previous length of 81 items; thus, not increasing the burden on students who will receive the questionnaire.
Testers will confirm that they following aspects of the survey and are fully functional, in that:
The online survey matches the approved questionnaire; this includes that all introductory and explanatory text, question text, choice text, and that the question order is the same as in the written approved questionnaire
The skip logic is correctly programmed and reflects the intent of the questionnaire
No question requires an answer
Mutually exclusive responses are programmed as mutually exclusive
Questions are in the correct format, e.g. a single-punch question does not allow multiple responses
Key indicator questions are programmed correctly so that questions are asked a second time (with a “decline to answer” option) if a respondent tries to move past the question without responding (please note: questions do not require a response but a response is encouraged as these are high priority questions that either collect important demographic information or determine skip patterns)
The survey flow is logical and visually appealing
The survey uses consistent language, grammar, punctuation, and capitalization
Once the Deloitte Team has thoroughly tested the online survey, we will provide a test link to the survey for CDC’s review. The survey will not launch before securing CDC’s approval of the online survey.
.
Statistical aspects of the study have been reviewed by the individuals listed below.
Linda J. Neff, Ph.D., M.S.P.H. Epidemiology Branch Chief Office on Smoking and Health Phone: 770-488-8647 E-mail: LNeff@cdc.gov |
S. Sean Hu, MD Senior Epidemiologist Office on Smoking and Health Phone: 770-488-5845 Email: shu@cdc.gov. |
David Homa, PhD, MPH Senior Science Advisor for the Epidemiology Branch Office on Smoking and Health Phone: 770-488-3626 Email: DHoma@cdc.gov |
Michael S. Knight, Specialist Leader Deloitte Consulting, LLC. Phone: 571-882-7297 Email: miknight@deloitte.com |
Within the agency, the following individual will be responsible for receiving and approving contract deliverables and will have primary responsibility for data analysis:
Ahmed Jamal, MBBS, MPH
Centers for Disease Control and Prevention
Office on Smoking and Health
Epidemiology Branch
4770 Buford Highway NE, MS-F 79
Atlanta, GA 30341
Phone: 770-488-5077
E-mail: AJamal@cdc.gov
The representative of the contractor responsible for conducting the planned data collection is:
Michael S. Knight, Specialist Leader
Deloitte Consulting, LLC.
1919 N. Lynn St.
Arlington, VA 22202
Phone: 571-882-7297
Email: miknight@deloitte.com
REFERENCES
American Cancer Society (2013a). Cancer Prevention & Early Detection: Facts and Figures 2013. Atlanta, GA: American Cancer Society, 2013.
American Cancer Society (2013b). Child and Teen Tobacco Use. Atlanta, GA: American Cancer Society.
American Cancer Society (2013c). Smokeless Tobacco: Who Uses Smokeless Tobacco? Retrieved from http://www.cancer.org/cancer/cancercauses/tobaccocancer/smokeless-tobacco
American Legacy Foundation (2000a). Cigarette Smoking Among Youth: Results from the 1999 National Youth Tobacco Survey. First Look Report 1. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2000b). Pathways to Established Smoking: Results from the 1999 National Youth Tobacco Survey. First Look Report 3. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2000c). The Relationship Between Cigarette Use and Other Tobacco Products: Results from the National Youth Tobacco Survey. First Look Report 4. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2000d). What Youth Think About Smoking: Results from the 1999 National Youth Tobacco Survey. First Look Report 2. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2000e). Youth Access to Cigarettes: Results from the 1999 National Youth Tobacco Survey. First Look Report 5. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2001a). Cigarette Smoking Among Youth: Results from the 2000 National Youth Tobacco Survey. First Look Report 7. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2001b). Youth Exposure to Environmental Tobacco Smoke. First Look Report 6. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2002). Using Multiple Strategies in Tobacco Use Prevention Education. First Look Report 8. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2003a). The Relationship between Cigarette Use and Other Tobacco Products: Results from the 2000 National Youth Tobacco Survey. First Look Report 10. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2003b). Youth Tobacco Cessation: Results from the 2000 National Youth Tobacco Survey. First Look Report 11. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2004). Cigarette Smoking among Youth: Results from the 2002 National Youth Tobacco Survey. First Look Report 13. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2005). Beyond Cigarettes: The Use Of Other Tobacco Products. First Look Report 15. Washington, DC: American Legacy Foundation.
American Legacy Foundation (2012). Tobacco Fact Sheet: Cigars, Cigarillos, and Little Cigars. Retrieved from http://www.legacyforhealth.org/content/download/642/7502/version/2/file/Fact_Sheet-Cigars_Cigarillos_LittleCigars.pdf
American Medical Association (2006). 2006 Annual Tobacco Report. Retried from http://www.ama-assn.org/ama1/pub/upload/mm/471/bot15A06.doc
Appleyard, J., Messeri, P., & Haviland, D. (2001). Smoking among Asian American and Hawaiian/Pacific Islander youth: new data from the 2000 National Youth Tobacco Survey. Asian American Pacific Islander Journal of Health, 9(1), 5-14.
Arrazola, R.A. (2013, November 21). Young Smokers. Centers for Disease Control and Prevention. Podcast retrieved from http://www2c.cdc.gov/podcasts/player.asp?f=8630364
Arrazola, R.A., Kuiper, N.M., & Dube, S.R. (2013). Patterns of Current Use of Tobacco Products Among U.S. High School Students for 2000-2012-Findings From the National Youth Tobacco Survey. Journal of Adolescent Health, 54(1), 54-60. http://dx.doi.org/10.1016/j.jadohealth.2013.08.003.
Asian Pacific Partners for Empowerment and Leadership (2000). Critical Policy Issues on Tobacco Prevention and Control for the Asian American and Pacific Islander Community. The Robert Wood Johnson Foundation.
Asian Pacific Partners for Empowerment and Leadership (2002). Making Tobacco Relevant for Asian American and Pacific Islander Communities. The Robert Wood Johnson Foundation, Centers for Disease Control and Prevention, Office on Smoking and Health, Campaign for Tobacco-Free Kids.
Aslam, N. & Bushra, R. (2010). Active Smoking in Adolescents of Karachi, Pakistan. Oman Medical Journal, 25(2), 142.
CDC (2001). Youth Tobacco Surveillance–United States, 2000. MMWR; 50(SS-4).
CDC (2005). Tobacco Use, Access, and Exposure to Tobacco in Media Among Middle and High Schools Students – United States, 2004. MMWR; 54(12):297-301.
CDC (2009). Cigarette Brand Preference Among Middle and High School Students Who Are Established Smokers- United States, 2004 and 2006. MMWR; 58(05): 112-115.
CDC (2010). Tobacco Use Among Middle and High School Students—United States, 2000-2009. MMWR; 59(33):1063-1068.
CDC (2012a). Current Tobacco Use Among Middle and High School Students – United States, 2011. MMWR; 61(31): 581-585.
CDC (2012b). National Youth Tobacco Survey. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention. Available at http://www.cdc.gov/tobacco/data_statistics/surveys/nyts.
CDC (2013a). Notes from the field: Electronic cigarette use among middle and high school students—United States, 2011-2012. Morbidity and Mortality Weekly Report; 62(35), 729–730.
CDC (2013b). Tobacco Product Use Among Middle and High School Students- United States, 2011 and 2012. Morbidity and Mortality Weekly Report; 62(45), 893-897.
CDC (2013c). Winnable Battles Progress Report- 2010-2015. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention.
CDC (2014a). Best Practices for Comprehensive Tobacco Control Programs – 2014. Atlanta, GA: U.S. Department of Health and Human Services, CDC, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.
CDC (2014b). Budget Request Summary- Fiscal Year 2015. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention.
CDC (2014c). Winnable Battles. Retrieved from http://www.cdc.gov/winnablebattles/
CDC (2014d). Annual Performance Report and Performance Plan- Fiscal Year 2014. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention.
Dutra, L.M. & Glantz, S.A. (2014). Electronic Cigarettes and Conventional Cigarette Use Among US Adolescents: A Cross-sectional Study. JAMA Pediatrics, published online March 06, 2014. doi:10.1001/jamapediatrics.2013.5488.
FDA (2014). FDA Proposes to Extend Its Tobacco Authority to Additional Tobacco Products, including e-cigarettes. FDA NEWS RELEASE. N.p., 24 Apr. 2014. Web. 9 May 2014. http://www.fda.gov/newsevents/newsroom/pressannouncements/ucm394667.htm
Frieden, T.R. (2010). A framework for public health action: The health impact pyramid. American Journal of Public Health, 100(4), 590-595.
Institute of Medicine (2011). Leading Health Indicators for Healthy People 2020: Letter Report. Washington, DC: The National Academies Press.
Lee S., Grana R.A., & Glantz, S.A. (2013). Electronic Cigarette Use Among Korean Adolescents: A Cross-Sectional Study of Market Penetration, Dual Use, and Relationship to Quit Attempts and Former Smoking. Published online November 25, 2013. Journal of Adolescent Health. doi:10.1016/j.jadohealth.2013.11.003.
Martin-Pujol, A., Fernandez, E., Schiaffino, A., Moncada, A., Ariza, C., Blanch, C., Martinez-Sanchez J.M., & the RESPIR-NET research group. (2013). Tobacco smoking, exposure to second-hand smoke, and asthma and wheezing in schoolchildren: a cross-sectional study. Acta Pediatric, 102(7), 305-309.
National Institute on Drug Abuse (2003). Youths’ Opportunities to Experiment Influence Later Use of Illegal Drugs. National Institute on Drug Abuse, 17(5).
National Institute on Drug Abuse (2014). Monitoring the Future national results on drug use: 1975-2013: Overview, Key Findings on Adolescent Drug Use. National Institute on Drug Abuse, National Institutes of Health. Ann Arbor, MI: Institute for Social Research, The University of Michigan.
Neergaard, J., Singh, P., Job, J., & Montgomery, S. (2007). Waterpipe smoking and nicotine exposure: A review of the current evidence. Nicotine and Tobacco Research, 9(10), 987–994.
Richardson, A., He, J.P., Curry, L., & Merikangas, K. (2012). Cigarette smoking and mood disorders in U.S. adolescents: Sex-specific associations with symptoms, diagnoses, impairment and health services use. Journal of Psychosomatic Research, 72(4), 269-275.
Shelley, D., Cantrell, J., Faulkner, D., Haviland, L., Healton, C., & Messeri, P. (2005). Physician and Dentist Tobacco use Counseling and Adolescent Smoking Behavior: Results from the 2000 National Youth Tobacco Survey. Pediatrics, 115(3), 719-725. American Legacy Foundation.
Smith, J.R., Novotny, T.E., Edland, S.D., Hofstetter, C.R., Lindsay, S.P., & Al-Delaimy, W.K. (2011). Determinants of Hookah Use among High School Students. Nicotine and Tobacco Research, (13)7, 565-572.
Starr, G., Rogers, T., Schooley, M., Porter, S., Wiesen, E., & Jamison, N. (2005). Key outcome indicators for evaluating comprehensive tobacco control programs. Atlanta, GA: Centers for Diesease Control and Prevention.
The University of California (2012). Tobacco Use among Asian American, Native Hawaiian and Pacific Islander Communities in California. The California Cancer Research Fund.
U.S. Bureau of Labor Statistics (2014). May 2013 National Occupational Employment and Wage Estimates, United States. Retrieved from http://www.bls.gov/oes/current/oes_nat.htm
Upadhyaya, H.P., Deas, D.D., Brady, K.T., & Kruesi, M. (2002). Cigarette smoking and psychiatric comorbidity in children and adolescents. Journal of the American Academy of Child & Adolescent Psychiatry, 41(11), 1294-1305.
USDHHS (2010a). How Tobacco Smoke Causes Disease: The Biology and Behavioral Basis for Smoking-Attributable Disease: A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Service, Public Health Service, CDC, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 2010.
USDHHS (2010b). Healthy People 2020. Washington, D.C.: U.S. Department of Health and Human Services. Available at: http://healthypeople.gov/2020/default.aspx
USDHHS (2012a). Ending the Tobacco Epidemic: Progress toward a Healthier Nation. Washington, DC: US Department of Health and Human Services, Office of the Assistant Secretary for Health.
USDHHS (2012b). Preventing Tobacco Use Among Youth and Young Adults: A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.
USDHHS (2014). The Health Consequences of Smoking- 50 years of Progress: A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.
USDHHS, HRSA (2004). Promising Practices in MCH Needs Assessment: A Guide Based on a National Study. U.S. Department of Health and Human Services, Health Resources and Services Administration.
USDHHS, NIH, & NCI (2007). NCI’s President’s Cancer Panel 2006-2007 Annual Report: Promoting Healthy Lifestyles: Policy, Program, and Personal Recommendations for Reducing Cancer Risk. U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute.
USDHHS, NIH, & NIDA (2007). Director’s Report to the National Advisory Council on Drug Abuse. U.S. Department of Health and Human Services, National Institutes of Health, National Institute on Drug Abuse.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kim, Taewon |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |