Part B NTPS 2023-24 Preliminary Activities

This request is to contact districts and schools in order to begin preliminary activities for the NTPS 2023-24 collection, including: (a) contacting and seeking research approvals from special handling districts, where applicable, (b) notifying sampled schools of their selection for the survey, and (c) inviting sampled schools to complete a short Screener Survey to verify eligibility for the survey and establish a point of contact at the school. This document describes the preliminary plans for NTPS 2023-24 sample design, estimation details, and recruitment and data collection procedures based on the NTPS 2020-21 design. The NTPS 2023-24 Main Study clearance request, which will be published for public comment in winter 2022-23, will describe the final sample design, recruitment, and data collection plans.

B.1 Universe, Sample Design, and Estimation

Section B.1.1 includes information on the study universe of interest and sample design planned for NTPS 2023-24, based on the NTPS 2020-21 sample design. Section B.1.2 describes the precision requirements and target sample sizes set out for the study.

B.1.1 Universe and Sample Design: Respondent Universe

B.1.1.1 Schools

The respondent universe for NTPS 2023-24 data collection consists of approximately 93,000 public schools and 24,000 private schools in the 50 U.S. states and the District of Columbia (DC) that offer instruction in any of grades 1-12 or the ungraded equivalent. To be eligible for inclusion in the sample, schools must: provide classroom instruction to students; have one or more teachers who provide instruction; serve students in at least one of grades 1-12 or the ungraded equivalent; be located in one or more buildings; and be located in the United States.

The most recent final Common Core of Data (CCD) file available from NCES at the time of sampling in spring 2023 will be used to construct the public school frame.¹ The respondent universe for charter schools will be identified as those public charter schools that meet the NTPS definition of an eligible school found on the CCD. The universe has been adjusted to remove kindergarten-terminal schools, which are not eligible for NTPS.

Table 1 presents the number of public schools on the 2020-21 NTPS public school universe, which are based on the 2017-18 CCD, by urbanicity and school level. The CCD that will be used to construct the sample for NTPS 2023-24 is not yet available at the time of this submittal. The NTPS 2023-24 school sample will be drawn in April-May 2023, and we will begin to contact sampled schools in June 2023.

Table 1. Estimated respondent universe by school level and urbanicity for the proposed NTPS 2023-24 public school sample, based on the 2017-18 Common Core of Data

School level
Urbanicity	Primary	Middle	High	Combined	Total
City	15,265	3,853	5,998	948	26,064
Suburban	17,742	5,495	6,150	801	30,188
Town	5,714	2,392	3,623	507	12,236
Rural	11,794	3,493	6,655	2,824	24,766
Total	50,515	15,233	22,426	5,080	93,254

SOURCE: 2017-18 CCD.

The private school frame will be drawn from the 2021-22 Private School Survey (PSS) frame. Preschools and schools with kindergarten as the highest grade will be excluded. Table 2 presents the number of private schools on the 2017-18 PSS universe by urbanicity and school level, as the 2021-22 PSS is not yet available at the time of this submittal.

Table 2. Respondent universe by school level and urbanicity for the proposed private school sample, based on the 2017-18 Private School Survey (PSS)

School level
Urbanicity	Elementary	Secondary	Combined	Total
City	4,621	1,070	2,555	8,246
Suburban	4,649	854	2,903	8,406
Town	1,234	148	731	2,113
Rural	2,735	447	1,882	5,064
Total	13,239	2,519	8,071	23,829

SOURCE: 2017-18 PSS.

Details of the first-stage sample design of schools are provided in section B.1.2.

B.1.1.2 Teachers

Teachers will be randomly sampled within the second design stage from roster information provided by each participating sampled school. Teachers within the sampled school are classified as ineligible for NTPS if they are a short-term substitute teacher, student teacher, or a teacher’s aide; or if they do not teach any of grades K-12 or comparable ungraded levels. The information that classifies teachers as ineligible is obtained from the Teacher Questionnaire.

B.1.2 Precision Requirements and Sample Sizes

This section details the approximate school sample sizes and precision requirements for the NTPS 2023-24 public and private school samples.

The final NTPS 2023-24 public school sample will include approximately:

9,920 schools and school principals (8,660 traditional public and 1,260 public charter), with the goal of at least 6,700 interviews for each; and
49,250 teachers (43,460 traditional public and 5,790 public charter), with the goal of at least 34,700 interviews.

The final NTPS 2023-24 private school sample will include approximately:

3,000 schools and school principals, with the goal of at least 1,750 interviews for each; and
6,300 teachers, with the goal of at least 4,500 interviews.

Sampling – Public Schools

The 2023-24 NTPS oversampling stratification will be based preliminarily on the following domains:

Charter/Non-charter;
School Level (primary, middle, high, combined);
Urbanicity (city, suburb, town, rural);
State Tier.

The NCES standards for publishability indicate that the coefficient of variation (CV) must be no larger than 50%, and if the CV is between 30% and 50%, the estimates are published with a caveat. For a population proportion of 20%, a CV of 30% corresponds to a standard error of 6%. In order to make sure that we don’t fall below the CV 30% minimum with the uncertainties about response and about exact values of design effects, we set as a target a CV of 25% as a lower bound. This corresponds to an expected standard error of 5%. This considerably reduces the chance of falling below the 30% boundary (if we set 30% itself as the target, we would be below it one-half of the time). Our target goal then for each state is to make sure that the expected standard error is no larger than 5% for a population proportion of 20% (a CV of 25%), at both the school and teacher level. The teacher sample size for a sampled school should be proportional to the product of the final teacher multiplier (based on the expected attrition adjustment factors), final school oversampling factor, and measure of size for the school (square root of the number of full-time teachers).

Table 3 presents a portion of the analysis for public schools by school type, grade level, urbanicity, enrollment, and poverty status. Presented are the anticipated number of responding schools or principals for the NTPS design and the expected precision based on analyses using the NTPS 2020-21 final response rates and CV of 25%.

Table 3. NTPS 2023-24 public school domain expected interviews, standard errors, and design effects with state oversampling to achieve 25% CV or less, based on NTPS 2020-21

Domain	Frame Schools	Expected Sample Size (completed interviews)	Expected Standard Error	Design Effect
All	93,634	6,700	0.63%	1.68
Charter	6,819	771	1.71%	1.41
Non-charter	86,815	5,929	0.67%	1.66
Primary	51,470	3,057	0.88%	1.49
Middle	14,177	1,119	1.43%	1.42
High	20,406	1,714	1.42%	2.17
Combined	7,581	810	1.90%	1.83
City	26,085	1,942	1.17%	1.67
Suburban	30,305	1,990	1.13%	1.58
Town	12,630	1,040	1.61%	1.69
Rural	24,614	1,728	1.29%	1.79
Enrollment < 100	7,946	313	3.54%	2.45
100 <= Enrollment < 300	7,341	462	2.36%	1.60
300 <= Enrollment < 500	36,097	2,392	1.00%	1.49
500 <= Enrollment < 750	23,395	1,653	1.15%	1.38
750 <= Enrollment < 1,000	9,447	798	1.64%	1.34
1,000 <= Enrollment	9,408	1,083	1.36%	1.26
Percent FRPL < 35%	27,165	2,056	1.21%	1.89
35% <= Percent FRPL < 50%	15,870	1,233	1.43%	1.58
50% <= Percent FRPL < 75%	26,578	1,852	1.18%	1.60
75% <= Percent FRPL	24,021	1,559	1.28%	1.59

Table 4 presents the analogous precision analysis for public schools by state.

Table 4. NTPS 2023-24 public school expected interviews, standard errors, and design effects by state with state oversampling to achieve 25% CV or less, based on NTPS 2020-21

State	Frame Schools	Expected Completed Interviews	Expected Standard Error	Design Effect
All	93,634	6,700	0.63%	1.68
AK	510	107	4.96%	1.65
WY	339	84	4.99%	1.30
DC	219	61	4.98%	0.95
VT	315	71	4.99%	1.11
SD	496	98	4.98%	1.52
MT	563	99	4.99%	1.54
DE	218	63	4.98%	0.98
RI	297	67	4.98%	1.04
ND	421	80	4.98%	1.24
HI	292	69	4.99%	1.08
NH	455	77	5.00%	1.20
ID	706	89	4.98%	1.38
ME	613	79	5.00%	1.23
NV	666	94	5.00%	1.47
WV	740	81	4.98%	1.26
NM	835	92	4.98%	1.42
NE	870	93	4.99%	1.45
CT	1,209	121	4.99%	1.88
UT	993	90	4.97%	1.38
OR	1,227	91	4.99%	1.42
MS	1,071	86	4.99%	1.34
IA	1,186	88	5.00%	1.37
AR	960	85	4.99%	1.32
KS	1,257	86	4.99%	1.35
KY	1,462	92	5.00%	1.44
MN	2,066	134	4.97%	2.07
SC	1,231	84	4.92%	1.28
AL	1,508	91	5.00%	1.42
AZ	2,306	145	4.99%	2.26
MD	1,421	87	4.98%	1.35
OK	1,441	92	4.99%	1.44
LA	1,349	89	4.99%	1.39
CO	1,671	103	4.78%	1.47
WA	2,257	109	4.77%	1.54
MA	1,786	105	4.66%	1.43
IN	1,881	105	4.54%	1.35
WI	1,943	109	4.52%	1.40
TN	1,787	107	4.46%	1.32
MO	2,008	125	4.27%	1.42
VA	2,072	132	4.01%	1.32
NJ	2,474	150	3.90%	1.43
GA	2,303	155	3.72%	1.34
MI	3,302	185	3.72%	1.59
NC	2,638	166	3.58%	1.33
PA	3,029	188	3.41%	1.37
OH	3,357	193	3.37%	1.37
IL	3,920	209	3.30%	1.42
FL	4,047	286	2.95%	1.56
NY	4,780	311	2.67%	1.39
TX	8,880	561	2.28%	1.82
CA	10,257	535	2.21%	1.63

Table 5 provides the analogous precision analysis for public school teachers. The expected standard errors were calculated based on analyses using the NTPS 2020-21 final response rates and CV of 25%.

Table 5. NTPS 2023-24 major domain expected public school teacher interviews, standard errors, and design effects with state oversampling to achieve 25% CV or less, based on NTPS 2020-21

Domain	Frame Full-Time Equivalent Teachers (in 1000s)	Expected Teacher Completed Interviews	Expected Standard Error	Design Effect
All	3,137.6	34,722	0.36%	2.74
Charter	160.4	3,609	1.02%	2.35
Non-charter	2977.2	31,113	0.38%	2.77
Primary	1487.2	13,686	0.53%	2.45
Middle	548.3	6,323	0.82%	2.68
High	912.8	10,986	0.66%	3.04
Combined	189.3	3,727	1.03%	2.46
City	928.7	10,389	0.64%	2.68
Suburban	1214.0	11,477	0.61%	2.69
Town	365.0	5,077	0.91%	2.63
Rural	629.8	7,780	0.75%	2.73
Enrollment < 100	38.4	1,013	1.93%	2.37
100 <= Enrollment < 300	90.3	1,653	1.52%	2.38
300 <= Enrollment < 500	863.4	9,424	0.64%	2.42
500 <= Enrollment < 750	864.9	9,544	0.67%	2.71
750 <= Enrollment < 1,000	475.8	4,930	0.95%	2.80
1,000 <= Enrollment	804.7	8,159	0.78%	3.07
Percent FRPL < 35%	978.7	11,160	0.65%	2.91
35% <= Percent FRPL < 50%	547.1	6,421	0.84%	2.86
50% <= Percent FRPL < 75%	870.5	9,521	0.67%	2.68
75% <= Percent FRPL	741.3	7,620	0.73%	2.52

Sampling – Private Schools

The NTPS 2023-24 private school sample will be roughly the same as for the NTPS 2020-21, which included 3,000 schools and school principals, with the goal of at least 1,750 interviews for each. This was roughly the same as the number of private school sampled from the Schools and Staffing Survey (SASS) 2011-12.

The sampling plan oversamples as follows, and any changes to oversampling, should they occur, will be provided in the main study package in winter 2022-23:

Elementary schools are sampled at a rate proportional to the measure of size;
Secondary schools are sampled at a rate proportional to 3 times the measure of size;
Combined schools are sampled at a rate proportional to 1.2 times the measure of size;
The oversampling rates for Nonsectarian schools are increased by an additional factor of 1.25 (e.g. the oversampling rate for Nonsectarian secondary schools is 3 ×1.25 = 3.75); and
The oversampling rates for Baptist schools are increased by an additional factor of 1.1 (e.g. the oversampling rate for Baptist combined schools is 1.2 ×1.1 = 1.32).

Under this design, not only all the precision goals are achieved but also all of the CVs for the teacher domains are less than 25% except for the vocational/technical teacher domain.

For teachers, the expected number of completed interviews is estimated to be proportional to the product of the final school sampling factor and the number of full time equivalent (FTE) teachers over schools in the domain. The overall target number of completed interviews is 4,500. Assuming the attrition rate for the 2023-24 NTPS will be similar to the rate for 2020-21 NTPS, the sample size needs to be 6,300 teachers in order to yield the expected number of completed teacher interviews. The teacher sample size for a sampled school should be proportional to the product of the final teacher multiplier (based on the expected attrition adjustment factors), final school oversampling factor, and measure of size for the school (square root of the number of full-time teachers).

Tables 6 and 7 show expected sample sizes, standard errors, and CVs for population percentages of 20% by key domains of school type, grade level, and region. Table 6 presents a portion of the analysis for private schools by affiliation, grade level, and region.

Table 6. NTPS 2023-24 private school domain expected interviews, standard errors, and design effects, based on NTPS 2020-21

School domain	Frame schools	Expected completed school interviews	For 20% population percentage				Min pop % for CV <30%
School domain	Frame schools	Expected completed school interviews	Expected standard error	95% CI half-width	Design effect	CV	Min pop % for CV <30%
All	24,984	1,750	1.24%	2.44%	1.70	6.22%	1.06%
Catholic	6,428	537	2.13%	4.18%	1.53	10.66%	3.06%
Other religious	12,006	664	2.00%	3.92%	1.66	10.01%	2.71%
Nonsectarian	6,550	549	2.17%	4.25%	1.62	10.84%	3.16%
Elementary	14,030	630	1.86%	3.64%	1.36	9.29%	2.34%
Secondary	2,609	501	1.96%	3.85%	1.21	9.81%	2.61%
Combined	8,345	620	1.94%	3.80%	1.45	9.69%	2.54%
Northeast	6,018	485	2.57%	5.04%	2.00	12.86%	4.39%
Midwest	6,024	373	2.66%	5.21%	1.65	13.30%	4.68%
South	8,081	562	2.10%	4.11%	1.55	10.48%	2.96%
West	4,861	330	2.79%	5.47%	1.61	13.96%	5.13%

Table 7 provides the analogous precision analysis for private school teachers.

Table 7. NTPS 2023-24 major domain expected private school teacher interviews, based on NTPS 2020-21

School domain	Frame teachers (FTE)	Expected completed teacher interviews	For 20% population percentage				Min pop % for CV <30%
School domain	Frame teachers (FTE)	Expected completed teacher interviews	Expected standard error	95% CI half-width	Design effect	CV	Min pop % for CV <30%
All	449,441	4,500	1.02%	2.00%	2.93	5.10%	0.72%
Catholic	136,810	1,528	1.80%	3.53%	3.10	9.01%	2.21%
Other religious	183,015	1,546	1.68%	3.29%	2.72	8.39%	1.92%
Nonsectarian	129,616	1,426	1.82%	3.56%	2.94	9.09%	2.24%
Elementary	174,278	1,246	1.71%	3.36%	2.28	8.56%	1.99%
Secondary	67,254	1,479	1.72%	3.37%	2.73	8.59%	2.01%
Combined	207,910	1,774	1.58%	3.10%	2.78	7.91%	1.71%
Northeast	128,835	1,359	1.96%	3.85%	3.27	9.81%	2.61%
Midwest	88,259	887	2.25%	4.40%	2.80	11.23%	3.39%
South	154,954	1,459	1.75%	3.42%	2.78	8.74%	2.08%
West	77,393	794	2.38%	4.67%	2.82	11.91%	3.79%

The 2023-24 NTPS will have an implicit stratification based on the proposed systematic sampling sort order, which uses a hierarchy of the following domains:

Three-level affiliation (Catholic, non-Catholic religious, nonreligious);
Three-level school span (elementary, secondary, combined);
Four-level Census region (Northeast, South, Central, Midwest);
Four-level urbanicity (city, suburb, town, rural);
Eleven-level affiliation;
Five-level school size (enrollment <100, 100-199, 200-499, 500-749, 750+);
State;
Highest grade;
Twelve-level urbanicity (large city, medium-sized city, small city, etc.);
Zip code;
School enrollment;
PIN number.

Teachers in traditional public, public charter, and private schools will be sampled from roster information provided by each participating sampled school or from the vendor (when the school does not provide teacher information). The target teacher completed interview sample sizes are designed to be proportional to the square root of the number of full-time teachers for each school and assume an attrition rate due to nonresponse.

Sampling – Principals within All Schools

For each sampled traditional public, public charter, and private school, the principal will be included in the survey as a result of the school being selected.

Survey Weights

Schools, principals, and teachers will be weighted by the inverse of the probability of selection. The final weight will contain adjustments for nonresponse and any other sampling or field considerations that arise after the sample has been drawn.

Response Rates

We expect the NTPS 2023-24 response rates to fall somewhere between those of NTPS 2017-18 and NTPS 2020-21. Specifically, we expect them to fall lower than the NTPS 2017-18 response rates given the long-term trend in declining response rates for federal surveys but not quite as low as the 2020-21 NTPS, since the survey was administered during the COVID-19 pandemic, which was a nontraditional school year due to widespread school closures and virtual learning environments. Table 8 provides the base-weighted response rates for NTPS 2017-18, and Table 9 provides the base-weighted response rates for NTPS 2020-21.

Table 8. Base-weighted response rates for NTPS 2017-18 by respondent and school type

School Type	Unit of Observation
School Type	Teacher	Principal	School
Traditional Public	76.9	70.7	72.9
Public Charter	75.4	63.4	67.5
Private	75.9	62.6	64.5

Table 9. Base-weighted response rates for NTPS 2020-21 by respondent and school type

School Type	Unit of Observation
School Type	Teacher	Principal	School
Traditional Public	62.6	68.2	65.8
Public Charter	59.5	66.7	63.4
Private	60.6	61.8	61.4

B.2 Procedures for the Collection of Information

Section B.2.1 describes the operations for the preliminary field activities for NTPS 2023-24, with Section B.2.1.1 describing special districts operation and Section B.2.1.2 the school precontact notification. Section B.2.2 describes school-level data collection procedures for the NTPS Screener. Section B.2.3 describes data collection procedures for the Teacher Questionnaire.

B.2.1 Preliminary Field Activities

B.2.1.1 Special Contact District Operation

Special contact districts require that a research application be submitted to and reviewed by the district before they will allow schools under their jurisdiction to participate in a study. Districts are identified as “special contact districts” prior to data collection because they were flagged as such during previous cycles of SASS, NTPS, the School Survey on Crime and Safety (SSOCS), or by other NCES studies (e.g., the School Pulse Panel (SPP)). Special contact districts are also identified during data collection when districts indicate that they will not complete the survey until a research application is submitted, reviewed, and approved.

Once a district is identified as a special contact district, basic information about the district is obtained from the NCES Common Core of Data (CCD). The basic information includes the NCES LEA ID number, district name, city, and state. The next step is to search the district’s website for a point of contact and any information available about the district’s requirements for conducting external research. Some districts identified as being a special contact district from the previous cycle may be incorrect and staff will verify whether a given district has requirements for conducting external research before proceeding.

The following are examples of the type of information that will be gathered from each district’s website in order to prepare a research application for submission to this district:

Name and contact information for the district office or department that reviews applications to conduct external research, and the name and contact information of the person in charge of that office;
Information about review schedules and submission deadlines;
Whether application fees are required, and if so, how much;
Whether a district sponsor is required;
Whether an online application is required, and if so, the link to the application if possible;
Information about research topics and/or agenda on which the district is focusing;
The web link to the main research department or office website; and
Research guidelines, instructions, application forms, District Action Plans, Strategic Plan or Goals, if any.

Recruitment staff will contact districts by phone and email to obtain key information not listed on the district’s website, (e.g., requirements for the research application, research application submission deadlines, etc.).

NTPS staff developed a generic research application that covers the information typically requested in district research applications. Staff will customize the generic research application to each district’s specific requirements that need to be addressed or included in the research application (e.g., how the study addresses key district goals, or inclusion of a district study sponsor), or submit the generic application with minimal changes to districts that do not have specific application requirements.

Using the information obtained from the district website or phone or email exchanges, a district research request packet will be prepared. Each research application will include the following documents, where applicable:

District research application cover letter;
Research application (district-specific or generic, as required by the district);
Study summary;
FAQ document;
Special contact district approval form;
Participant informed consent form (if required by the district);
NTPS Project Director’s resume;
Copy of questionnaires; and
Application fee (if required by the district).

Where required or requested, applications will include the draft 2023-24 NTPS questionnaires. The NTPS content that most closely matches the 2023-24 NTPS is that fielded in 2017-18 (for questionnaires other than the Teacher Listing Forms, TLFs) and these are provided in Appendix B of this submission. The teacher and principal longitudinal components to the NTPS are referenced in the research applications, and as such, the 2021-22 Teacher and Principal Follow-up Survey questionnaires are also included in Appendix B of this submission and will be included in applications as needed. Additionally, the 2020-21 NTPS questionnaires will be provided to districts that request them. Other information about the study may be required by the district and will be included with the application or provided upon request.

Approximately one week after the application is submitted to the district (either electronically or in hard copy, as required by the district), NTPS district recruitment staff will contact the district’s research office to confirm receipt of the package and to ask when the district expects to review the research application and when a decision will be made. If additional information is requested by the district (e.g., the list of sampled schools), recruitment staff will follow up on such requests and will be available to answer any questions the district may have throughout the data collection period.

Some districts charge a fee (~$50-200) to process research application requests, which will be paid as necessary. Special district operations will begin by contacting up to 100 “certainty” special contact districts for which, due to their size, it is certain that at least one school from their jurisdiction will be randomly sampled. Other special contact districts will be contacted after the sample is drawn in the spring of 2023.

B.2.1.2 School Precontact notification

The school precontact notification includes mailing a two-sided, full-color postcard to sampled schools to introduce the survey, promote survey recognition, and verify school mailing address. In previous administrations of the NTPS, about 4% of all school addresses get corrected by the U.S. Post Office in response to the precontact notification (previously a letter), saving time and effort during the main data collection period.

B.2.1.3 Endorsements from key groups and affiliations

The level of interest and cooperation demonstrated by key groups can often greatly influence the degree of participation of survey respondents. Endorsements will be sought from national, state, and private K-12 organizations and agencies, and will be listed on recruitment materials sent to NTPS 2023-24 sample members (see section B.3.1 of this document for more detail).

B.2.2 School-level Data Collection Procedures

In July 2023, all schools will receive a letter addressed to the principal at the school address (see Appendix A) which will include instructions for completing a brief screener interview online using the NTPS Screener internet instrument (see Appendix C). The purpose of the Screener is to determine the school’s eligibility for NTPS and to establish a survey coordinator. The survey coordinator will be asked to facilitate the completion of NTPS questionnaires within their school, and materials will be mailed to him or her throughout data collection. A reminder email will be sent to non-responding school principals in August 2023. Principals who do not self-screen will be contacted by telephone in mid to late August 2023.

Following the Screener data collection, NTPS 2023-24 main study data collection will begin with the collection of school-level questionnaires in September 2023. The details of the main study school-level data collection procedures and materials will be provided in the NTPS 2023-24 Main Study clearance request in winter 2022-23.

B.2.3 Teacher Data Collection

Teachers will be sampled weekly from completed or verified TLFs throughout data collection. As teachers are sampled, they will begin teacher data collection. The final details of all teacher data collection procedures and materials will be provided in the NTPS 2023-24 Main Study clearance request in winter 2022-23.

B.3 Methods to Secure Cooperation, Maximize Response Rates, and Deal with Nonresponse

This section describes the methods that NCES will use to secure cooperation, maximize response rates, and deal with nonresponse for NTPS 2023-24. Section B.3.1 details how NTPS plans to secure cooperation by leveraging its status as the primary source of information on K-12 schools and staffing in the United States. Section B.3.2 describes the methods that will be used to minimize nonresponse for NTPS 2023-24. The design is based on the results from NTPS 2020-21, which employed a number of different contact strategies aimed at boosting response rates. The final methods selected for NTPS 2023-24 will be specified in the NTPS 2023-24 Main Study submission in winter 2022-23.

B.3.1 Methods to Secure Cooperation and Maximize Response Rates

The entire survey process, starting with securing research cooperation from key public school groups and individual sample members and continuing throughout the distribution and collection of individual questionnaires, is designed to increase survey response rates. In addition, the following elements of the data collection plan, in particular, will contribute to overall success of the survey and will enhance the survey response rates.

Endorsements from key public school groups. The level of interest and cooperation demonstrated by key groups can often greatly influence the degree of participation of survey respondents. Endorsements are viewed as a critical factor in soliciting cooperation from state and local education officials, and endorsing groups will be listed on recruitment materials sent to NTPS 2023-24 sample members. NCES will seek endorsements for NTPS 2023-24 from the following national organizations and agencies:

American Association of School Administrators

American Counseling Association

American Association of School Librarians

American Federation of Teachers

American Montessori Society

American School Counselors Association

Association for Middle Level Education (formerly National Middle School Association)

Association for Supervision and Curriculum Development

Association of American Educators

Council of Chief State School Officers

Council of the Great City Schools

National Association of Elementary School Principals

National Association of Secondary School Principals

National Education Association

National Parent Teacher Association

The School Superintendents Association

Endorsements from key state public school groups. In addition, for NTPS 2023-24, like its 2020-21 predecessor, NCES will seek endorsement from the following state organizations and agencies (the number of state organizations and agencies are capped at two per state for efficiency of solicitation operations):

Alabama

Birmingham Federation of Teachers

Alabama Education Association

Alaska

Alaska Public Employees Association

Alaska Education Association

Arizona

American Federation of Teachers - Arizona

Arizona Education Association

Arkansas

Arkansas Education Association

California

American Federation of Teachers - California

California Teachers Association

Colorado

American Federation of Teachers - Colorado

Colorado Education Association

Connecticut

American Federation of Teachers - Connecticut

Connecticut Education Association

Delaware

Delaware State Education Association

District of Columbia

Washington Teachers' Union

Florida

Florida Education Association

Georgia

American Federation of Teachers - Georgia

Georgia Association of Educators

Hawaii

Hawaii State Teachers Association

Idaho

Idaho Education Association

Illinois

American Federation of Teachers - Illinois

Illinois Education Association

Indiana

American Federation of Teachers - Indiana

Indiana State Teachers Association

Iowa

Iowa State Education Association

Kansas

American Federation of Teachers - Kansas

Kansas National Education Association

Kentucky

Kentucky Education Association

Louisiana

American Federation of Teachers - Louisiana

Louisiana Association of Educators

Maine

Maine State Employee Association

Maine Education Association

Maryland

American Federation of Teachers - Maryland

Maryland State Education Association

Massachusetts

American Federation of Teachers - Massachusetts

Massachusetts Teachers Association

Michigan

American Federation of Teachers - Michigan

Michigan Education Association

Minnesota

Education Minnesota

Mississippi

American Federation of Teachers - Mississippi

Mississippi Association of Educators

Missouri

American Federation of Teachers - Missouri

Missouri Education Association

Montana

Montana Federation of Public Employees

Nebraska

Nebraska State Education Association

Nevada

Nevada State Education Association

New Hampshire

American Federation of Teachers - New Hampshire

New Hampshire Education Association

New Jersey

American Federation of Teachers - New Jersey

New Jersey Education Association

New Mexico

American Federation of Teachers - New Mexico

New Mexico Education Association

New York

New York State United Teachers

North Carolina

North Carolina Association of Educators

North Dakota

North Dakota United

Ohio

Ohio Federation of Teachers

Ohio Education Association

Oklahoma

American Federation of Teachers - Oklahoma

Oklahoma Education Association

Oregon

American Federation of Teachers - Oregon

Oregon Education Association

Pennsylvania

American Federation of Teachers - Pennsylvania

Pennsylvania State Education Association

Rhode Island

Rhode Island Federation of Teacher and Healthcare Professionals

Rhode Island Education Association

South Carolina

South Carolina Education Association

South Dakota

South Dakota Education Association

Tennessee

Tennessee Education Association

Texas

American Federation of Teachers - Texas

Texas State Teachers Association

Utah

American Federation of Teachers - Utah

Utah Education Association

Vermont

American Federation of Teachers - Vermont

Vermont Education Association

Virginia

Fairfax County Federation of Teachers

Virginia Education Association

Washington

American Federation of Teachers - Washington

Washington Education Association

West Virginia

American Federation of Teachers - West Virginia

West Virginia Education Association

Wisconsin

American Federation of Teachers - Wisconsin

Wisconsin Education Association Council

Wyoming

Wyoming Education Association

Endorsements from key private school groups. In addition to the endorsements from key public school organizations and agencies, NCES will also seek endorsements for NTPS 2023-24 from the following private school organizations:

Agudath Israel of America/Lefkowitz Leadership Initiative

American Association of Christian Schools

American Association of School Librarians

American Counseling Association

American Federation of Teachers

American Montessori Society

American School Counselors Association

Association for Middle Level Education

Association for Supervision and Curriculum Development

Association Montessori International

Association of American Educators

Association of Christian Schools International

Association of Christian Teachers and Schools

Association of Classical Christian Schools

Association of Military Colleges and Schools

Association of Waldorf Schools of North America

Christian Schools International

Council for American Private Education

Council of Chief State School Officers

Council of Islamic Schools of North America

Council of the Great City Schools

Evangelical Lutheran Church in America

Friends Council on Education

General Conference of Seventh-Day Adventists

Islamic School League of America

Jesuit Schools Network

Jewish Education Services of North America

Lutheran Church-Missouri Synod

National Association of Elementary School Principals

National Association of Episcopal Schools

National Association of Independent Schools

National Association of Private Special Education Centers

National Association of Secondary School Principals

National Catholic Educational Association

National Christian School Association

National Coalition of Girls’ Schools

National Council for Private School Accreditation

National Education Association

National Independent Private School Association

National Parent Teacher Association

Office of Education, General Conference of Seventh Day Adventists

Oral Roberts University Educational Fellowship

Prizmah: Center for Jewish Day Schools

RAVSAK: Jewish Community Day School Network

Southern Baptist Association of Christian Schools

The Association of Boarding Schools

The Jewish Education Project

The School Superintendents Association

Torah Umesorah National Society for Hebrew Day Schools

United States Conference of Catholic Bishops

Wisconsin Evangelical Lutheran Synod

Stressing the importance of the survey and the respondents' participation. Official letters, with all relevant obtained endorsements listed, will be used to motivate respondents to return their survey. NTPS 2023-24 respondent letters will be sent by the U.S. Census Bureau and signed by the NCES Commissioner. Communications in the form of both letters and emails will be personalized for the principal and survey coordinators, which is expected to have positive effects on the survey response rates.
Visible support from top-level Federal, State, and local education officials. Support of high-level officials in the U.S. Department of Education, State Education Agencies, and the sampled local school districts, is vital to the success of surveys of principals and teachers and thus obtaining endorsements from these officials is a critical step in NTPS data collection procedures. In addition to all routine endorsement sought and NTPS recruitment letters being signed by the NCES Commissioner, as the need arises, to secure adequate survey participation, top-level Education Department officials will be asked to support the NTPS 2023-24 data collection by endorsing the survey in writing and sending advance letters and notices directed to individual sampled districts and survey participants (principals) who need additional encouragement to participate.

B.3.2 Methods to Minimize Nonresponse

A major challenge in any survey is obtaining high response rates, and this is even more important today when response rates have been falling among federal surveys in general, and in NTPS in particular.

The main problem associated with nonresponse is the potential for nonresponse bias in the estimates produced using data collected from respondents. Bias can occur when respondents are systematically different from nonrespondents. Two approaches that will be used to reduce the potential for bias are (a) designing the data collection procedures and methods so as to reduce nonresponse (e.g., establishing survey coordinators) and (b) using statistical methods of sampling and weighting to reduce the effect of nonresponse on the estimates. While the statistical approaches are important in controlling biases and costs, the data collection procedures and methods are at the heart of a successful study.

Methods selected to minimize nonresponse in NTPS 2023-24 for the Screener build upon those used in NTPS 2020-21 and previous administrations, including actions that were taken late in the data collection to boost principal and teacher response rates. Detail about strategies to minimize nonresponse for the School, Principal, and Teacher Questionnaires will be included in the specified in the NTPS 2023-24 Main Study submission in winter 2022-23.

Data Collection Strategies to Minimize Nonresponse

Minimize survey burden on schools. NTPS survey procedures are designed to minimize burden on schools and sampled individuals (principals), and the survey instruments, the NTPS Screener for this submission, have been designed to be completed as quickly and easily as possible.

When designing the NTPS Screener, questionnaire design techniques were employed to minimize item nonresponse. The Screener from previous rounds of SASS and NTPS has been carefully analyzed to determine which items had the highest levels of item nonresponse. This information guides NCES in reviewing the clarity of item wording, definitions, and instructions. Items that are not considered to be effective or useful will be removed from the survey so as to streamline the questionnaire and ease the response burden.

Recruit survey coordinators. One of the main purposes of the NTPS Screener is to establish a “survey coordinator” at the school. Successive administrations of SASS and NTPS have shown that establish a survey coordinator is an important procedure to help maximize response rates for the School, Principal, and Teacher Questionnaires, because the survey coordinator serves as a primary point of contact for NTPS staff throughout data collection. The use of a survey coordinator is expected to help keep response rates high, provide some minimal data quality checks, and simplify the follow-up process by having one point of contact.
Personalize principal contact materials. To maximize the chances that the mailed NTPS 2023-24 Screener materials intended for the school principal successfully make it to the principal, these principal contact materials will be personalized with the principal’s name. Principals’ names are obtained from vendor-purchased school staff lists. If a principal’s name is not available from the vendor, clerical staff will research this information using school and district websites prior to the start of NTPS Screener data collection.
Use of email to target principals and survey coordinators. The last several administrations of NTPS demonstrated that email was an effective tool to drive participation for all of the NTPS surveys, including the Screener. It proved that principal email addresses could be effectively collected from school websites and from vendor purchased school data and that survey coordinator email addresses could be effectively collected during the screener interview. Because personalized emails carry low cost and may help boost response, throughout 2023-24 NTPS Screener data collection, principals will be contacted via email. The emails will include login information to access the NTPS Screener online instrument, in addition to text inviting and subsequently reminding these respondents to complete their survey online. The NTPS 2023-24 Screener will continue to collect an email address for the survey coordinator, and emails will be sent to that contact throughout the main study data collection.
Screener Telephone follow-up operation. The NTPS 2023-24 will include a telephone operation aimed at screening schools whose principal did not self-screen by logging into the Screener online instrument. The goal is to increase the Screener response rate as much as possible in order to establish a survey coordinator for as many sampled schools as possible. Having a survey coordinator serve as a single point of contact with the school throughout main data collection is expected to help keep response rates high, provide some minimal data quality checks, and simplify the follow-up process.
Consider new methods of minimizing nonresponse. NCES is considering the continued use of a non-monetary incentive to help increase response rates for the NTPS Screener. Further information about the use of incentives during the preliminary data collection activities for NTPS 2023-24 is provided in section B.4.1 of this document.

B.4 Tests of Methods and Procedures

The SASS/NTPS series of studies has a long history of testing materials, methods, and procedures to improve the quality of its data. Appendix D describes the tests that have most influenced the NTPS design, beginning with the 2014-15 NTPS Pilot Test and continuing through NTPS 2020-21. Section B.4.1 describes experiments proposed for NTPS 2023-24.

B.4.1 Tests Included in the NTPS 2023-24 Design

NCES is currently considering options for tests of methods, materials, and procedures to be conducted as part of NTPS 2023-24. NTPS 2023-24 is still in the planning stages, and so the plans described below are tentative and only apply to the preliminary data collection activities within this package. The final description of all data collection operations and tests, including final planned analyses and minimum detectable differences, for NTPS 2023-24 main data collection will be provided in the NTPS 2023-24 Main Study data collection submission in winter 2022-23.

Testing non-monetary incentives at the school level. The NTPS 2023-24 will include an experiment designed to examine the effectiveness of offering a non-monetary incentive to boost NTPS response. Specifically, this randomized experiment will assess the impact of including a colorful data wheel that displays state-level NTPS data from the 2020-21 collection in the Screener Letters to public schools, at which time the principal is invited to complete the Screener online, on Screener response.

Schools assigned to the experimental treatment will receive a data wheel in their mailed Screener package. This package will be addressed to the principal at the school address and also includes a letter introducing the NTPS with instructions for completing their Screener interview online using the NTPS Screener instrument. Schools assigned to the control group will include only the Screener letter in their package.

The design of the data wheel is still being finalized (see Appendix A for current version), and the final version will be included in a change request to this package later this year.

Following data collection, the experimental treatment will be evaluated against the control group at the school level. The analyses may be examined but not limited to:

Screener response rate;
Number of contacts;
Days to respond to Screener;
Data collection costs.

The response rates will be calculated directly and compared using significance tests for differences. The average number of contacts and average days to respond across the experimental groups will be used as a proxy for timeliness of response. A reduction in the average number of contacts could be used to justify the use cost of including the data product. Additionally, if schools within the experimental treatment respond in a more timely manner, survey coordinators would be established sooner, and main data collection could also begin earlier.

Using data collection costs associated with the mailout, the data product itself, and estimates for interviewer costs, an average cost-per-case can be determined within each experimental group. A reduction in cost-per-case could justify the use of non-monetary incentives to reduce the overall survey cost. Generally, using incentives of any form leads to more initial costs, however, if cases respond in a fewer number of contacts – specifically more costly contacts such as telephone follow-up by interviewers – this could lead to a reduction in overall cost.

B.4.2 Qualitative Testing in Preparation for NTPS 2023-24

Analyses were conducted on paradata collected during the 2020-21 NTPS Screener, and results of these analyses indicated that minor adjustments were needed for the Screener instrument in preparation for NTPS 2023-24. Additionally, further expert review is being done to ensure the Screener instrument adheres to instrument best practices. Appendix C includes example screenshots of the Screener instrument, which are based on the final 2020-21 NTPS Screener. Updated screenshots will be presented in a change request to this package later this year.

B.5 Individuals Responsible for Study Design and Performance

The following individuals are responsible for the NTPS 2023-24 study design, data collection, and analysis: Maura Spiegelman, Julia Merlin, and Andy Zukerberg at NCES; Shawna Cox, Walter Holmes, and Aaron Gilary at the U.S. Census Bureau; and Rebecca Medway and Jana Kemp at the American Institutes for Research (AIR).

1 If the 2021-22 CCD is not available as of early January in 2023, the most recently available CCD as of that date will be used instead.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2022-10-24

Part B NTPS 2023-24 Preliminary Activities