Change Sheet Memo Update 3

U NITED STATES DEPARTMENT OF EDUCATION

National Center for Education Statistics

August 29, 2019

MEMORANDUM

To: Robert Sivinski, OMB

From: Linda Hamilton, NCES

Through: Kashka Kubzdela, NCES

Re: National Assessment of Educational Progress (NAEP) 2019 and 2020 Long-Term Trend (LTT) Update 3 Changes to Materials (OMB# 1850-0928 v.17)

The National Assessment of Educational Progress (NAEP), conducted by the National Center for Education Statistics (NCES), is a federally authorized survey of student achievement at grades 4, 8, and 12 in various subject areas, such as mathematics, reading, writing, science, U.S. history, civics, geography, economics, technology and engineering literacy (TEL), and the arts. The National Assessment of Educational Progress Authorization Act (Public Law 107-279 Title III, section 303) requires the assessment to collect data on specified student groups and characteristics, including information organized by race/ethnicity, gender, socio-economic status, disability, and limited English proficiency. It requires fair and accurate presentation of achievement data and permits the collection of background, noncognitive, or descriptive information that is related to academic achievement and aids in fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and subpopulations of students and to monitor progress over time. The nature of NAEP is that burden alternates from a relatively low burden in national-level administration years to a substantial burden increase in state-level administration years when the sample has to allow for estimates for individual states and some of the large urban districts.

The request to conduct NAEP 2019 and 2020 was approved in September 2018 with the latest update to the NAEP 2020 plan, consisting of the Long Term Trend (LTT) assessment to be conducted during the 2019-20 school year, approved in August 2019 (OMB# 1850-0928 v.10-16). The LTT assessments are based on nationally representative samples of 9-, 13-, and 17-year old students, and have been used by NAEP since the early 1970s to provide measures of students’ educational progress over long time periods to allow for analyses of national trends in students’ performance in mathematics and reading. NAEP 2019 data collection has been concluded.

This request updates Part A, Part B, Appendix D3, and Appendix F to remove references to current or future submissions. The final version of the Long-Term Trend (LTT) Content of the MyNAEP System (Appendix J5) now includes revised screenshots to match previously approved text.

The table below details the changes for review in this submission that were made to each of the documents as compared to those approved in August 2019 (OMB# 1850-0928 v.16). These revisions do not alter the approved respondent burden nor the actual cost to the federal government for conducting this study.

Document	Changes
Part A	Removed all references to “August and October Submission” on pages 5 and 6 Updated Part A to refer to v. 17
Part B	Removed all references to “October Submission” on page 7
Appendix D3- LTT 2019-2020 Communications and Recruitment materials Update	Removed all references to “October Submission” on page 2
Appendix F-Item Library for NAEP 2019 & 2020 Survey Questionnaires	Removed all references to “October Submission” on page 2
Appendix H2- LTT 2020 Student Information Instructions Update	Updated screenshots to show a more complete picture of the screen display on pp. 13, 82, and 149 Updated screenshots with corrected grammar pertaining to the use of word “Data” on pp. 15, 25, 37, 47, 60, 72, 85, 95, 107, 117, 130, 140, 153, 163, 174, 184, 196, and 206
Appendix J5- LTT 2020 Content of the MyNAEP System	Removed all references to “August Submission” on page 2 Revised language to better refine communication materials and website instructions (see below) Updated all screenshots in J5 to better reflect the sampling frame for the LTT (by age of participant rather than grade level) Updated screenshots to reflect updated wording for the LTT on pp. 6, 7, 8, 13 (screenshot #2), 16, 32, 45, 50, 53, 55, 57, 60, 63, and 65

Part A (pp. 5-6)

The request to conduct NAEP in 2019 and 2020, including operational assessments, pilot tests, and special studies was approved in September 2018, with the latest change requests approved in June August 2019 (OMB# 1850-0928 v.10-1516).¹ This request (OMB#1850-0928 v.16), posted for a 30-day public comment period, is 17) updates Part A, Part B, Appendix D3, and Appendix F to update remove references to current or future submissions and provides revised screenshots of the Long-Term Trend (LTT) 2020 communication materials and data collection instruments to be used to conduct LTT during content of the 2019-2020 school year MyNAEP System (Appendix J5) to match previously approved text.

(…)

The library of possible items to be used in the NAEP 2019 and 2020 questionnaires is provided in Appendix F of this submission. The final versions of the 2019 questionnaires are provided in Appendices K1 (student questionnaires), K2 (teacher questionnaires), K3 (school questionnaires), and K-S (Spanish translations of applicable questionnaires), while the LTT 2020 questionnaires are in Appendix K4 (student questionnaires) and K4-S (Spanish translations of applicable LTT questionnaires: mathematics for ages 9 and 13). The items provided in these appendices are a subset of the item library provided in Appendix F. Some items have undergone non-substantive changes since their inclusion in Appendix F and a summary of changes is included before each final questionnaire (as applicable) in Appendices K1-K4. The final communication and recruitment materials to be used in NAEP 2019 are provided in Appendices D1 and D2 along with their Spanish translations as applicable, and final communication and recruitment materials to be used in LTT 2020 are provided in Appendix D3 along with their Spanish translations as applicable. Should any additional revisions be necessary for LTT 2020, they will be submitted by October 2019.​ The content of the 2019 MyNAEP system used by school coordinators to provide requested administration information is provided in Appendix J1 and for LTT 2020 in Appendix J5. Appendix J5 reflects the final text for the 2020 LTT MyNAEP system, the screenshots provided reflect the 2019 MyNAEP system. The final screenshots for LTT 2020 MyNAEP will be provided as a non-substantive change request by August 2019 (Appendix J5 and H2). The Spanish version of the 2019 MyNAEP is provided in Appendix J2, there will not be a need for the Spanish MyNAEP for LTT 2020. HSTS and MSTS MyNAEP websites are provided in Appendices J3 and J4. Some of the assessment, questionnaire, and recruitment materials are translated into Spanish. Specifically, Spanish versions of the student assessments and questionnaires are used for qualified English language learner (ELL) students when a bilingual accommodation is offered and for all students in Puerto Rico. Typically, this is done for all operational grade 4 and 8 assessments or ages 9 and 13 for LTT (note that a TEL bilingual accommodation has not been offered to date and no LTT bilingual accommodation is offered for reading). In years in which a Puerto Rico NAEP assessment is given, such as in 2019, Spanish versions of communication materials for parents, teachers, and school coordinators as well as teacher and school questionnaires are created by translating their English equivalents into Spanish. In addition, every year, Spanish versions of parent communication materials are used nationwide for Spanish-speaking parents. Should any additional revisions be necessary, they will be submitted by October 2019.

Part B (p. 7)

Note: Appendices D1 and D2 provide the full finalized communication and recruitment materials to be used in NAEP 2019. Some communication materials will be also used in a Spanish-language version and the translated versions are included in appendices D1 and D2 as applicable. Appendix D3 provides the LTT communication and recruitment materials for 2020. Should any additional revisions be necessary, they will be submitted by October 2019.

Appendix D3 (p. 2)

The following Appendices have been revised based on those in Appendix D1 and D2 of this submission (used in NAEP 2019 and marked below as “old”) and added to Appendix D3 (this document) for use in the 2019-2020 LTT (marked below as “new”):

• Facts for Teachers, Public Schools and Private Schools, Appendix D1-7 (old) - Appendix D3-15 (new),

• Templates for State Specific SD Inclusion Policy, Appendix D1-8 (old) - Appendix D3-16 (new),

• Templates for State Specific ELL Inclusion Policy, Appendix D1-9 (old) - Appendix D3-17 (new),

• Best Practices for Students age 17, Appendix D1-10 (old) - Appendix D3-18 (new),

• References for Best Practices, Appendix D1-11 (old) - Appendix D3-19 (new),

• Parent/Guardian Letter in Spanish, translation of Appendix D3-7 and D3-8 - Appendix D3-20 and D3-21 (new), respectively, and

• 2020 NCES Letter to Chiefs, Appendix D2-15 (old) - Appendix D3-22 (new).

Should any additional revisions be necessary, they will be submitted by October 2019.

Appendix F (p. 2)

This Appendix F provides the library of possible items to be used in the NAEP 2019 and 2020 questionnaires. Non-substantive wording changes to the questionnaire items (based on ongoing and future cognitive and pilot testing results), if any will be were made., along with tThe final selection of which items will be used in the final versions of the NAEP 2019 and 2020 questionnaires, will be are provided as outlined below. The number of items selected for each type of respondent will be appropriate to the allocated time (as indicated in the Supporting Statement Part A section A.12).

The final versions of all NAEP 2019 questionnaires in English language and Spanish-translated questionnaires are provided in Appendices K1, K2, K3, and K-S.

The 2020 data collection instruments are provided in Appendices K-4 and K4-S. Should any additional revisions be necessary, they will be submitted by October 2019.

The confidentiality citation has been updated in September 2018. The final student and teacher/school administrator log-in screens will reflect the new confidentiality citation, as reflected in Part A.

Appendix H2 (pages 15, 25, 37, 47, 60, 72, 85, 95, 107, 117, 130, 140, 153, 163, 174, 184, 196, and 206)

The following data you submitted has have been checked again against the 2017-2018 Common Core of Data (CCD) for your school.

The following data you submitted has have also been checked against historical data (if available) for your school.

If the data from your file is are correct as is, continue with the data checks process

Appendix J5

Table of Contents (p. 2)

The text provided in this document reflects the final text for the 2020 LTT MyNAEP system, but the screenshots provided reflect the 2019 MyNAEP system. The final screenshots for LTT 2020 MyNAEP will be provided as a non-substantive change request by August 2019 (except those on pages 15-34, which have been updated and reflect the final 2020 LTT MyNAEP system screenshots).

School Characteristics Information (p. 7)

• Student enrollment

• Charter school status

• Date school adjourns for winter break in December (Note: for age 17, this will read: “Date school adjourns for spring break”)

• Date school returns from winter break in January (Note: for age 17, this will read: “Date school returns from spring break” )

• School calendar type (traditional or year round)

• School start and end times

Prepare for Assessment (p.8)

• Review and verify the list of students selected for NAEP

• Complete information about how students with disabilities and English language learners will participate in NAEP

• Notify parents/guardians of sampled students

• Manage school and/or teacher questionnaires Note: there are no school or teacher questionnaires for the long-term trend assessments (LTT), so the Manage Questionnaires will not be listed on the left-hand menu.

• Plan assessment day logistics

• Encourage participation and motivate students to do their best

Additional Resources

• Instructions for the School Coordinator (see Additional Resources section of this document)

• Digitally-based Assessments Note: there are no digitally-based assessments for the LTT, so this will not appear.

Review and Verify List of Students Selected for NAEP (p. 9)

Additional Resources

• Information Needed to Review and Verify List of Students Selected for NAEP School Coordinator

(see Additional Resources section of this document)

• Dept. of Agriculture Memo Authorizing Release of NSLP to NAEP School Coordinator

(see Additional Resources section of this document)

Print Reports (p. 13)

Print Reports – REV-REP-1

After entering student data in the system

Select Print Summary Report to print a record of the student information entered in the system. Keep this report in the NAEP Storage Envelope and refer to it during the preassessment review call.

Select Submit to indicate you are finished with this section.

Complete SD/ELL Student Information-(Replaced red text on pages 17, 19, 20, 21, 25, 30 and 31)

Please note: Due to an artifact of the development process, the placeholder “Training J5” is being used in the screenshot. The state names will show on the actual site.

Please note: Due to an artifact of the development process, the placeholder phrase “Training J5” is being used in the screenshot. The applicable state name will show on the actual site in place of each instance of the “Training J5” text.

Record Assessment Details (p. 50)

Text for schools with Paper-only sessions:

Students will take NAEP on paper.

Review the information below, then record locations and start times for each session. Sessions can be tested together at the same in the same location.

Testing Locations

Select locations that…

• have flat desks or tables

Provide NAEP Team Instructions (p. 52)

Provide NAEP Team Instructions

Before the Assessment-PLN-TEAM-1

1. Where should the team park?

2. The team will bring their NAEP ID badges and check in at the front office. Do they need to follow any other school protocols?

3. Where is the best place for the team to unload the equipment for testing that is closest to the testing location?

Provide NAEP Team Instructions (p. 54)

During the Assessment-PLN-TEAM-2

1. The law prohibits students from photographing or sharing NAEP questions. Please assist us with one of these security measures:

Notify students that cell phones are banned from the testing location

Notify students that cell phones must be turned off and placed under with their seats belongings

No assistance can be provided

2. In case of an emergency, who should the NAEP team contact for assistance?

3. Is there anything scheduled, such as a fire drill or assembly, that might interrupt the assessment?

Encourage Participation (p. 57)

Additional Resources

• Measure Up News for the School Community (see Appendix D3-19)

• Facts for Teachers (see Appendix D3-15)

• Digitally-based Assessments Note: there are no digitally-based assessments for the LTT, so this will not appear.

Submit Current List (p. 65)

Upload File-USL-SUB-2

1. Does your student data file contain column headers?

2. Date list is accurate as of:

Click to show calendar (mm/dd/yyyy)

3. Select Choose File or Browse to locate and select the file on your computer. This must be an Excel file containing all currently enrolled X-year-old students at your school.

4. Select the Upload button.

Dept. of Agriculture Memo Authorizing Release of NSLP to NAEP (p. 80)

This memo replaces the United States Department of Agriculture (USDA) Letter

^Date: School Year 2019–20

^To: School Nutrition Program Personnel

^From: National Assessment of Educational Progress (NAEP)

^Subject: Policy on Limited Disclosure of Children's Eligibility Information to NAEP

The above information is available on the FNS website at https://www.fns.usda.gov/2017- edition-eligibility-manual-school-meals. https://www.fns.usda.gov/eligibility-manual-school-meals and then select, “Eligibility Manual.”

1 In late 2018, Congress requested for NCES to administer LTT, which is legislatively mandated, earlier than its originally planned next administration scheduled for the 2023-24 school year, and passed a bill for FY 2019 (P.L. 115-245, signed on 9/28/2018) that increased NAEP FY19 funding to allow for administration of the LTT NAEP assessment during the 2019-20 school year (https://www.nagb.gov/content/nagb/assets/documents/what-we-do/quarterly-board-meeting-materials/2018-11/04-executive-committee.pdf). In order to be able to comply with the Congressional request to conduct LTT during the 2019-20 school year and to meet the LTT 2019-20 study’s timeline while maintaining compliance with the Paperwork Reduction Act (PRA), due to this unanticipated event, in February 2019, the U.S. Department of Education (ED) and NCES requested under 44 U.S.C. 3507(j)(1) (“emergency clearance”) to begin participant recruitment and study materials printing for LTT 2020 by May 2019. Thus, at that time, NCES submitted an Information Collection Request (ICR) to OMB utilizing emergency review procedures in accordance with the PRA (P.L. 104-13, 44 U.S.C. Chapter 35) and 5 C.F.R. §1320.13 to announce revisions to the NAEP 2020 study plans and to provide for review the LTT 2020 plans, procedures, and materials, which were approved by OMB in April 2019 (OMB# 1850-0928 v.14). In February 2019, NCES also initiated a regular clearance process for that request (including a 60-day followed by a 30-day public comment periods), which was approved by OMB in June 2019 (OMB# 1850-0928 v.15). Updates to the LTT 2020 communication materials and data collection instruments were approved in August 2019 (OMB# 1850-0928 v.16).

550 12^th Street, S.W., Washington, DC 20202

Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.