Change Sheet Memo

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: September 30, 2019

TO: Robert Sivinski, OMB

THROUGH: Kashka Kubzdela, OMB Liaison, NCES

FROM: Sheila Thompson, National Research Coordinator for PIRLS, NCES

SUBJECT: Progress in International Reading Literacy Study (PIRLS 2021) Field Test Recruitment Change Request (OMB# 1850-0645 v.12)

The Progress in International Reading Literacy Study (PIRLS) is an international assessment of fourth-grade students’ achievement in reading. PIRLS reports on four benchmarks in reading achievement at grade 4 and on a variety of issues related to the education context for the students in the sample, including instructional practices, school resources, curriculum implementation, and learning supports outside of school. Since its inception in 2001, PIRLS has continued to assess students every 5 years (2001, 2006, 2011, and 2016), with the next PIRLS assessment, PIRLS 2021, being the fifth iteration of the study. Participation in this study by the United States at regular intervals provides data on student achievement and on current and past education policies and a comparison of U.S. education policies and student performance with those of the U.S. international counterparts. In PIRLS 2016, 58 education systems participated. The United States will participate in PIRLS 2021 to continue to monitor the progress of its students compared to that of other nations and to provide data on factors that may influence student achievement. PIRLS is coordinated by the International Association for the Evaluation of Educational Achievement (IEA), an international collective of research organizations and government agencies that create the assessment framework, the assessment instrument, and background questionnaires. The IEA decides and agrees upon a common set of standards and procedures for collecting and reporting PIRLS data, and defines the studies’ timeline, all of which must be followed by all participating countries. As a result, PIRLS is able to provide a reliable and comparable measure of student skills in participating countries. In the U.S., the National Center for Education Statistics (NCES) conducts this study.

In preparation for the PIRLS 2021 main study, all countries are asked to implement a field test in 2020 in order to evaluate new assessment items and background questions, to ensure practices that promote low exclusion rates, and to ensure that classroom and student sampling procedures proposed for the main study are successful. Data collection for the field test in the U.S. will occur from March through April 2020 and for the main study from March through June 2021. The submission describing the overarching plan for all phases of the data collection, including the 2021 main study and requesting approval for all activities, materials, and response burden related to the field test recruitment, scheduled to begin in May 2.019, was approved in April 2019 (OMB# 1850-0645 v.11). Since that time, NCES has secured a contractor for PIRLS and finalized the PIRLS 2021 field test recruitment materials and procedures. This request is to provide for the PIRLS 2021 Field Test the finalized sample sizes, associated burden estimates, and recruitment materials.

Changes of note in this submission include: (1) a small increase in burden associated with an increase in sample size; (2) an updated cost estimate, now that a contractor has been secured for the project; (3) an insertion claiming FERPA authorization to collect student records; (4) a dropped portion ($50) of an incentive for school coordinators (as an optional task was removed from their role); (5) a reorganization of data collection processes that replaces two forms from the school enrollment process with a single data collection process (for further detail please see the changes to section A.3 as reflected on p. 3 of this document); (6) in Appendix A, the addition of a Multi-Language Translation Notice, of Spanish translations of two parent contact materials (letter & FAQ), and of School Coordinator and School Incentive Templates; and (7) in Appendix A, revision, throughout, of the content of the “PIRLS Registration and Provide School Information (PSI) Guide” (pp. 36-38) and of MyPIRLS Website (pp.41-62).

A separate submission, with a 30-day public comment period, for all aspects of the Field Test, including data collection, plus main study recruitment will be submitted immediately upon approval of this change request, by early October 2019.

Changes to the estimated burden and costs to the federal government are detailed in this document. Revisions made to Part A, Part B, and Appendix A (Communication Materials) of the approved PIRLS 2021 field test recruitment request (OMB# 1850-0645 v.11) are detailed below.

The following edits were made Part A:

• REVISED - Updated sample numbers of students and assessment data collection dates for the field test with minor edits to clarify language and fix grammar (PREFACE, paragraph 5, page 2, see text in red font below)

Data collection for the field test in the U.S. will occur from March 1 through April 15, 2020. The U.S. plans to recruit sampled and will involve a sample of 45 public and private schools and assess about 1,000650 students. The student samples will be obtained by (selecting two classes from each school). The U.S. PIRLS 2021 main study will be conducted from March through June 2021 and will involve a nationally-representative sample of 285 schools and approximately 6,5900 students in the target population from 200 schools.

• REVISED – Updated information relevant to this submission (PREFACE, paragraph 2, page 3, see text in red font below)

Because PIRLS is a collaborative effort among many parties, the U.S. must adhere to the international schedule set forth by the IEA, including the availability of draft and final questionnaires. In order to meet the international data collection schedule for the spring 2020 field test, recruitment activities are scheduled to begin in May October 2019. Recruitment for the main study will begin in March of 2020 to align with recruitment for other NCES studies [e.g., the National Assessment of Education Progress (NAEP)], and for schools to put the assessment on their calendars. We expect the main study materials and procedures to be very similar to those used in the field test. However, as NCES does not yet have a national contractor and the international contractor has not yet issued guidance on PIRLS 2021 sampling or draft instruments, this submission requests approval only for the PIRLS 2020 field test recruitment. A second submission will follow in July October 2019, with a 30-day public comment period, to will request approval for PIRLS 2020 field test data collection and PIRLS 2021 main study recruitment. Additionally, should there be any revisions to recruitment materials or changes from the past field test sampling requirements, we will submit to OMB a change request with the final materials and recruitment details in early 2019.

• REVISED – Updated language about student questionnaire (section A.2, page 4, see text in red font below)

Student Questionnaire. Student information will be collected about home resources, motivation, self-concept, self-efficacy, and student characteristics such as gender and race/ethnicity. It should be administered to all students who have received parental permission to participate in PIRLS. Students will only receive a paper-and-pencil questionnaire Student questionnaire for digitalPIRLS will be administered via the student assessment player after the achievement assessment.

• REVISED – Updated language about data server (section A.3, page 4, see text in red font below)

Each participating nation is expected to adhere to the internationally prescribed design. In the U.S., the school and teacher questionnaires will be made available to school administrators and teachers online as the main mode of administration, with a paper-and-pencil backup to facilitate user preference for participation. The online questionnaires will be provided on the secure NCES Westat data server so that NCES will be able to control access to the data to ensure confidentiality and minimize disclosure risk.

• REVISED – Streamlined approach to information collection by implementing an e-file system that allows the researchers to collect two fewer forms while still collecting the same information (section A.3, page 5, see text in red font below)

The process of collecting teacher and student-teacher linkage information has been streamlined to improve user flexibility and efficiency. Teacher information is now collected through the MyPIRLS website on the “Submit Class List” page during the e-filing process. On that page, school coordinators are asked to enter a complete and current list of all of their school’s fourth-grade classes, including the student roster for each listed class as well as the name and email address of the associated reading teacher. Excel templates of the student list are posted on the “Submit Student List” page for the school coordinators to provide student information and link teachers to the students of each class listed on the “Submit Class List” page. This data collection procedure eliminates the need for the previously used Student-Teacher Linkage Form (STLF). Furthermore, information about the associated teachers is securely stored in Westat’s e-file system, which is connected to the database for the MyPIRLS website and is updated with teacher questionnaire participation status on a regular basis, making it easy and efficient for test administrators to track questionnaire status, and eliminating the need for the Teacher Tracking Form (TTF). This approach centralizes the information to be shared with the school coordinators in one secured online location through the MyPIRLS website, rather than having multiple forms the school coordinators have to verify and confirm at different stages of the data collection phase.

• REVISED – Updated main study data collection date (section A.6, page 5, see text in red font below)

The field test data collection is scheduled for March 1 through April 15, 2020, and the main study data collection for February March through May, June 2021. This schedule is prescribed by the international collective for PIRLS, and adherence to this schedule is necessary to establish consistency in survey operations among participating countries as well as to maintain trend lines.

• REVISED – Updated language to incentives and rationale with minor edits to clarify language and fix grammar (section A. 9, page 6, see text in red font below)

As in PIRLS 2016, in PIRLS 2021 schools will be offered $200 for participation. and, to address challenges that may be encountered with securing school cooperation, we will utilize a second-tier incentive during the main study that will allow us to offer up to $800 to schools that are historically very difficult to recruit. (…)

A similar second-tier recruitment strategy has been used in other international studies conducted by NCES. Most recently, in ICILS 2018 TIMSS 2019, we began offering offered the second-tier incentive very late, in to the middle of data collection, and attempted to turn around schools that had already refused, as well as newly activated substitutes and schools that were not doing any work to prepare for the assessment. aforementioned groups. We were able to successfully recruit about 20 over 85% of these eligible schools even with this late start, which brought us much closer to our was the target recruitment percentage. We expect the extra incentive strategy to be more successful in the PIRLS if we begin it during the fall, before data collection begins main study as well.

The school staff serving as School Coordinators will receive $100 for their time and effort in coordinating PIRLS assessment, plus $50 for running the PIRLS system check, and assisting with computer setup (these components may be delegated to a school IT coordinator if necessary).the PIRLS assessment. The School Coordinator serves a critical role in data collection, functioning as the central school contact, and facilitating arrangements for the assessments. They are asked to file class and student listing forms; arrange the date, time, and space for the assessment; and disseminate information to parents and students.

(…) Some schools also offer recognition parties with pizza or other treats for students who participate; however, these are not reimbursed by NCES or the contractor Westat.

Teachers will be offered a $20 Amazon gift card for completing the PIRLS teacher questionnaire. Historically, participation is high among school administrators without offering incentives; therefore, no incentive will be offered for completion of the school administrator questionnaire.

• REVISED – Refined the legal language on contact materials (section A.10, p.7, see text in red font below)

Letters and supporting materials will read:

The National Center for Education Statistics (NCES) is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). All of the information [you provide/ provided by school staff and students] may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). In the United States, PIRLS is conducted by NCES, part of the U.S. Department of Education, and the data are being collected by Westat. The U.S. Office of Management and Budget has approved the data collection under OMB # 1850-0645.

The following statement will appear on the login page for PIRLS and the front cover of the printed questionnaires (the phrase “search existing data resources, gather the data needed” will not be included on the student questionnaire):

The National Center for Education Statistics (NCES) is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

• REVISED – Updated sample numbers of schools and students and total burden time cost estimate, with minor edits to clarify language and fix grammar (A.12, page 9, see text in red font below)

This package shows the estimated burden to respondents for all PIRLS 2021 activities, and requests approval for the burden to respondents for field test recruitment (demarcated in black font in Table A.1). The For the field test, in order to assess the minimum required 1,400 students, we will sample size for the field study is 25 45 schools and 800 students and for the main study 150 schools and 4,000 1,650 students. The burden table assumes exceeding the minimum requirements and is based on a target yield of approximately 1,000 students in the field test, 5,000 students in (sampled from two randomly selected classes per school). For the main study, and 1,500 in the in order to assess the minimum required 5,000 students from 150 schools for the digitalPIRLS plus 1,500 students from 50 schools for the bridge study paperPIRLS, we will sample 285 schools and 6,900 students. The time required for students to respond to the assessment (cognitive items) portion of the study and associated directions are shown in gray italicized font and are not included in the totals because they are not subject to the PRA. The burden estimates also include burden for: (1) contacting states, districts, schools, and parents in order to recruit for the PIRLS field test and main study, including: (a) sending recruitment letters to districts and schools selected for each study, (b) contacting and seeking research approvals from special handling districts, where applicable, and (c) notifying parents of sampled students about their participation in either study; and (2) collecting data. and reviewing lists submitted by school coordinators. Burden estimates for the field test data collection and all aspects of the main study data are not requested at this time and are provided in Table A.1 for information purposes.

(…) Based on the estimated hourly rates for principals/administrators, school coordinators, teachers, and parents of $46.85, $29.25, $29.25, and $24.34, respectively¹, and the federal minimum wage of $7.25 as the hourly rate for the students, and based on the estimated total of 295 338 burden hours for PIRLS field test recruitment, the associated estimated respondent burden time cost is $8,554 9,645.

• REVISED – Updated burden estimates for Field Test Recruitment and Field Test Data Collection based on the revised sample numbers of schools and students for the field test (Table A.1, page 10, see text in red font below)

Activity	Sample size	Expected response rate	Number of respondents	Number of responses	Per respondent (minutes)	Total burden (hours)
Field Test Recruitment
Contacting Districts	4045	1	4045	4045	10	78
Contacting Schools	4045	1	4045	4045	20	1415
District IRB Staff Study Approval	5	1	5	5	120	10
District IRB Panel Study Approval	30	1	30	30	60	30
Parental notification	1,4001,650	1	1,4001,650	1,4001,650	10	234275
Total Field Test Recruitment			2,6921,775	2,6921,775		295338
Field Test Data Collection
Student
Assessment digitalPIRLS	1,1001,650	0.93	1,0231,403	1,0231,403	160	2,7283,742
Directions	1,1001,650	0.93	1,0231,403	1,0231,403	10	171234
Student Questionnaire	1,1001,650	0.93	1,0231,403	1,0231,403	50	8531,169
School Staff
School Administrator	4045	0.95	3843	3843	40	2629
Teacher (2 per school)	8090	0.95	7686	7686	40	5157
School Coordinator	4045	1	4045	4045	240	160180
Total Field Test Data Collection			1,1771,577	3,2234,383		3,9895,411
Total Field Test			2,6923,352	4,7386,158		4,2845,749
Total requested in this submission			1,5151,775	1,5151,775		295338

• REVISED – Updated annualized cost to the federal government with Table A-2 (section A.14, pages 10-11, see text in red font below)

The total cost to the federal government Federal Government for conducting the PIRLS 2021 full scale has not been determined as the national contract for this study has not yet been awarded. However, based on past administrations, the cost for the field test is estimated to be about $1,873,000. 1,832,621 over a 2-year period. All direct and indirect costs for this study are valued at an estimated $4,000,000 6,765,955 over five years, from March 2019 to March 2024. The final cost details and totals will be provided in this section once available.

Table A-2. Estimated costs for PIRLS 2021 field test and main study

Components with breakdown	Estimated costs
FIELD TEST (2020)
NCES salaries and expenses	101,400
Recruitment	410,146
Preparations (e.g., adapting instruments, sampling)	244,480
Data collection, scoring, and coding	1,076,595
Field Test Total (current package components)	$1,832,621
MAIN STUDY (2021)
NCES salaries and expenses	202,800
Recruitment	469,443
Preparations (e.g., adapting instruments, sampling)	293,974
Data collection, scoring, and coding	3,206,494
Reporting and dissemination	760,623
Main Study Total	$4,933,334
Grand total	$6,765,955

• REVISED – Updated tentative schedule (section A.16, page 11, see text in red font below)

Based on the data collected in the main study, the TIMSS & PIRLS International Study Center will prepare a report to be released in December 2021 2022. As has been customary, NCES plans to also release a report at the same time as the international report is released, interpreting the results for the U.S. audience. NCES reports on initial data releases are generally limited to simple bivariate statistics. There are currently no plans to conduct complex statistical analyses of PIRLS 2021 dataset. An example of the past PIRLS report can be found at https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2018017. In the spring of late 2022, the International Study Center will also prepare a technical report, describing the design and development of the assessment as well as the scaling procedures, weighting procedures, missing value imputation, and analyses. After the release of the international data, NCES plans to release the national data and an accompanying User’s Guide for the study.

Dates	Activity
April-December August 2019-January 2020	Prepare data collection manuals, forms, assessment materials, questionnaires
May October 2019—February 2020	Contact and gain cooperation of states, districts, and schools for field test
February 2020—March 2020	Select Sample classes and collect student samples lists
March 1, 2020—April 15, 2020	Collect field test data
May 15, 2020	Deliver raw data to international sponsoring organization
July 2020—August 2020	Review field test results
March 2020—February 2021	Prepare for the main study/recruit schools
February March 2021—June 2021	Collect main study data
June 2021-September 2021	Receive final Deliver perturbed data files from to international sponsors sponsoring organization
February 2023	PIRLS 2021 International Database distributed by international sponsoring organization
June December 2021—December 2021 2023	Produce report

The following edits were made in Part B:

• REVISED – Updated sample numbers of states, schools, and students for the field test with minor edits to clarify language and fix grammar (section B.1, see text in red font below)

The respondent universe for the PIRLS field test is all students enrolled in grade 4 that have a mean age of at least 9.5 years of age, during the 2019-2020 school year. The universe for the selection of schools is all types of schools in 715 populous states. A sample of 4045 schools will be selected for the field test, with the goal of obtaining participation from a minimum of 2540 schools. Within sampled schools, approximately 1,000 1,650 students will be selected for participation by drawing a random sample of two classes. Only students in intact classrooms of grade 4 students will be assessed at each grade. School administrators and teachers of the selected classrooms will also be asked to complete questionnaires.

• REVISED - Updated sample numbers of states, schools, and students for the field test with minor edits to clarify language and fix grammar/typos (section B.2, Field Test Sampling Plan and Sample, see text in red font below)

As required by the PIRLS International Study Center, the field test sample is to consist of at least 2545 schools with a minimum of 800 students assessed. The approximately 1,650 students samples will to be obtained by selecting selected for participation from a random sample of two classes from each school., which is estimated to yield a minimum of 1,400 students to be assessed. A probability sample of schools is not required for the field test, because the field test is designed only to test items, questions, and procedures. However, the sample must include a broad range of schools covering such characteristics as public (including charter schools), private, large, small, urban, and rural schools, and schools from a variety of different states.

The field test sample will be drawn after before the main study sample to avoid overlap, and schools will be selected for the field test from the set of schools not that may be included in the main study sample., though the chances of a school being selected for both samples are minimal. We will draw the field test sample from 7 or more15 states (typically. (Typically California, Illinois, New York, North Carolina Virginia, and Texas Georgia are among the selected states because of their large the variation in size and diverse demographics.) This approach will allow for achieving achieve the desired distribution of schools by region, poverty level, and ethnicity, and will inform the recruitment and data collection process for the main study.

Schools in the selected states that are not selected into the main study sample will comprise For the field test sampling frame and, schools in the selected states will be stratified by state, and high/low poverty, and public/private status, resulting in 1530 different strata. Serpentine sorting will be used to sort schools by locale (city, suburb, town, and rural), race/ethnicity status (“15 percent or above” or “below 15 percent” Black, Hispanic, Asian and Pacific Islander, and American Indian and Alaskan Native students), and fourth grade enrollment within each stratum. A purposive sample of 2545 schools will be selected for the field test that allocates equally to the separate states, although purposive selection of schools within the states may be conducted to ensure that to the extent possible, the proportion of schools in the field test closely aligns with the proportion of schools in the main study school sampling frame on the margins of the stratification and sort characteristics described previously. In addition, we will select the PIRLS field test sample so as to minimize overlap with the NAEP sample. Two replacement schools will be selected for each of the 4045 sampled schools from the same strata that will have the same sort characteristics as the corresponding sampled schools. Once the field test sample has been selected, a summary of the distribution of the characteristics of the selected schools will be prepared, showing the comparison with the national population of schools.

The student sampling procedures for the field test will correspond as closely as feasible to what is planned for the main study, so as to try out the operational procedures for student sample selection. The sample will be selected by selecting one or two classes per school, depending on the number of classes available at in grade 4. Each participating school will be asked to submit an exhaustive list of classes (that is, a list that accounts for each student in the grade exactly once). Smaller classes will be combined to form “pseudoclasses pseudo-classes” for the purposes of sampling. Once the list of classes is submitted, we will use a sampling algorithm in the sampling software provided by the IEA to select two classes (or pseudoclasses pseudo-classes) with equal probability. The student sample will then consist of all students in the selected classes.

• REVISED - Updated sample numbers of states, schools, and students for the field test with minor edits to clarify language and fix grammar/typos (section B.2, Main Study Sampling Plan and Sample, see text in red font below)

The school sample design for the main study must be more rigorous than that for the field test. It must be a probability sample of schools that fully represents the entire fourth grade population in the United States. At the same time, to ensure maximum participation, it must be designed so as to minimize overlap with other NCES studies involving student assessment that will be conducted around the same time. The main study will take place in the spring of 2021, about two months after the NAEP 2021 reading assessment. NAEP 2021 will assess several hundred thousand schools nationally, at grades 4, 8, and 128. To be fully representative, the PIRLS sample may include some schools that will have participated in the Main NAEP 2021 at the same grade 4. However, this number will be kept to a minimum.

The sample size for the PIRLS main study will be 200 schools. In order to assess the minimum required 5,000 students from 150 schools for the digitalPIRLS plus 1,500 students from 50 schools for the bridge study paperPIRLS, we will sample 285 schools and 6,900 students. (…)

Schools will be selected with probability proportional to size (PPS) sample, where the measure of size is based on the number of estimated classes students at grade 4, with schools expected to have either one or two classes being given the same selection probability. The use of this probability sample. A PPS design ensures that all students have an approximately equal chance of selection, given that two classes because the same sample size will be selected from each school, when possible, regardless of the size of the school. Note that we will modify this equal probability design in the following way: so as to increase It also improves cost-efficiency by increasing the available sample size of students in high poverty schools, we will double the probability of selection of each school with at least 50 percent number of students eligible for free or reduced-price lunch under NSLP, relative to other schools of the same size per school.

Student sampling will be accomplished by selecting one up to two classes per school.

• REVISED – Deleted additional incentive for School Coordinators (section B.3, approach to school recruitment, bullet 10, see text in red font below)

Offer a $100 incentive to the individual at the school identified to serve as the school coordinator, plus $50 for running the PIRLS system check, and assisting with computer setup on the day of the test administration (these components may be delegated to a school IT coordinator if necessary);

• REVISED – Updated staff names and contact number for Statistics Canada (section B.5, see text in red font below)

Staff [to be assigned] Sylvie LaRoche and Ahmed Almaskut from Statistics Canada (613-951-0007 863-9480).

• REVISED – Updated U.S. national contractor (section B.5, see text in red font below)

the U.S. national contractor (to be determined);, Westat; and

The following edits were made in Appendix A:

• REVISED – Inserted a field for the date in all letters (global – multiple pages)

• REVISED – Refined the legal language on all contact materials (global – multiple pages)

The National Center for Education Statistics (NCES) is authorized to conduct this study under the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543), and to collect students’ education records from educational agencies or institutions for the purpose of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). All of the information [you provide/ provided by school staff and students] may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). In the United States, PIRLS is conducted by NCES, part of the U.S. Department of Education, and the data are being collected by Westat. The U.S. Office of Management and Budget has approved the data collection under OMB # 1850-0645.

• REVISED – Updated the name of the contractors that will be administering PIRLS 2021 (global, multiple pages, see red font below for two typical examples of how this change appears)

The student assessment will be administered by a team of researchers from [XXX], Westat, on behalf of the National Center for Education Statistics (NCES).

The entire assessment is administered by trained staff from [National contractor to be determined] Westat, a research organization under contract to the U.S. Department of Education’s National Center for Education Statistics.

• REVISED – Clarified the technology used for the administration of PIRLS (global – multiple pages, see red font below)

For the first time, PIRLS will be completely electronic, and administered on a tablet or laptop. with a keyboard.

• REVISED – Changed the name of the study’s restricted-use website (global – multiple pages, see red font below)

PIRLS MyPIRLS is a restricted-use website that contains information on the Progress in International Reading Literacy Study (PIRLS).

• REVISED – Updated approval date on all materials that contain it (global – multiple pages, see red font below)

OMB No. 1850-0645, Approval Expires xx/xx/xxxx 04/30/2022.

• REVISED – Adjusted the incentive for School Coordinators (global – multiple pages, see red font below)

Participating schools will receive $200, and each school’s PIRLS school coordinator (the school staff person designated to work with PIRLS staff) will receive $150100 as a thank you for his or her time and effort.

• REVISED – Updated contact information for the new contractor and study support (global – multiple pages, see red font below)

If you have any questions, please do not hesitate to call [XXX] the PIRLS Help Desk at (XXX) XXX-XXXX 1-855-951-1215 or send an email to [XXX@xxxx). PIRLS@westat.com. You may also get more information about this study by visiting the PIRLS website at http://nces.ed.gov/surveys/PIRLS. If you have questions about your rights as a study participant, you may call [to be determined]. contact Laura Egan at 301-294-3840 or email LauraEgan@westat.com.

• REVISED – Removed details about the Student-Teacher Linkage Form (STLF) and Teacher Tracking Form (TTF) as these are no longer used because Westat has developed more efficient data collection procedures to obtain the student-teacher linkage and teacher information through the e-filing and data monitoring systems. The process of obtaining the information is still the same as described (i.e., collecting information from schools and requesting the school coordinators to link students to the teachers of the sampled classes and providing study-required information for those teachers). (pp. 58)

• REVISED – Refined details about the field test assessment and its administration (p. 8)

The main assessment will take approximately 2.5 hours and will include a paper and pencil an online questionnaire that asks students about themselves and their educational experience.

• ADDED – Added the following new materials:

  • Spanish language translations of the English versions of: (1) PIRLS Field Test Sample Parent Notification Letter (p. 9) and PIRLS Facts for Parents about the Field Test (p. 12)

  • Multi-Language Translation Notice (p. 14) – School coordinator may provide this translation notice to parents/guardians to indicate whether the parental/guardian consent materials need to be translated, if necessary.

• REVISED – Dropped the task of running the systems check of the digital systems from the school coordinator’s role and made a small change to the timeline for school coordinators (pp. 22-23)

Table 1. Activities Timeline

Provide information on 4th grade classes using the Class Listing Form.	Within 3 weeks one week of receipt of request returning from winter break
Complete Student Listing Form for selected class(es)	Within one week of selection of classes
Coordinate session logistics (dates, times, locations, consent type)	Within 3 weeks of receipt of request
Notify teachers, selected students, and parents of the study and benefit of participating	At least 3 weeks prior to scheduled session
Distribute parental consent forms to all selected students	At least 3 weeks prior to scheduled session
Monitor return of consent forms; distribute reminder letters as needed. Collect returned, signed forms and submit to TA.	During the 3 weeks from distribution to data collection day
Run a systems check of the digital system	At least 2 weeks prior to scheduled session
Notify/remind teachers and students about the data collection sessions	One week prior and one day prior to session
Assist the TA with getting the students to the sessions and setting up the room (if necessary)	On the day of the session
Coordinate completion of the administrator and teachers’ questionnaires.	Two weeks prior to the student session

Your TA will be in contact with you to track the return of consent forms, as well as exclusion statuses for students. Reminder forms will be sent home as needed. The TA will check the permission forms on the day of the first session to make sure we do not include anyone whose parents have not granted permission. Please keep the returned parent permission forms in a locked or secure location. Your TA will fax mail any returned forms from your school at the end of the data collection to a Westat for secure fax machine at [XXX] maintenance of records. Please let him/her know if you need to retain copies.

Assist on the Day of Student Sessions

(…)

We would also like for you or a teacher to remain in the room during the session administration to help maintain order and assist as needed.

Again, the TA will need to fax any returned permission forms at the end of the session. Please assist him or her in faxing these forms.

• REVISED – Updated the PIRLS Field Test FAQ (pp. 31-32)

Since 2001, PIRLS has been administered every 5 years. PIRLS 2021, the fifth study in the series, will involve students from more than 40 50 countries, including the United States.

Is it possible that my school will be selected for the main study as well as the field test?

No, It is highly unlikely that schools selected for the field test will not be selected for the 2021 main study.

Will all our fourth-grade students be asked to participate?

Probably not, unless your school only has two grade 4 classroom classrooms. Students with disabilities and English-Language learners may also be excused.

When will the field test be conducted?

The field test will be conducted between March 1 and April 30 15, 2020. [national contractor to be determined] Westat will work with schools to identify an assessment date convenient for the school in that time period.

How long does the field test take?

The field test assessment will take approximately 2.5 hours, including time for directions.

What are the benefits?

The nation as a whole benefits from PIRLS by having a greater understanding of how the reading knowledge and skills of U.S. 4^th graders compare with 4^th graders from other countries. The country benefits from the field test by ensuring that the questions used in the main study are fair and appropriate for U.S. students. To thank participating schools and individuals for their time and effort we offer tokens of appreciation: schools that participate in PIRLS will receive $200 and the school coordinator (staff person designated to assist with the study) will receive $150 100. Teachers who complete a survey will receive $20, and each student who participates will receive a small gift.

• REVISED – the Summary of School Activities: PIRLS Field Test (Spring 2020) as follows:

  Summary of School Activities: PIRLS Field Test
  	November-December 2019	January-March 2020 Prior to assessment day	March-April 2020 Assessment day	Benefits
  Principal	Designate a school coordinator	Complete a brief School Questionnaire on the characteristics of the school, its enrollment, resources, policies, and learning environment (available online or hardcopy) Share the importance of participation in the study with school staff	Support assessment day activities Encourage students to participate and do their best on the assessment	Represent other similar U.S. schools Receive a $200 check for the school
  School coordinator	SelectConfirm an assessment date convenient for your school	Arrange the day and location for the PIRLS sessions Confirm dates and location with PIRLS assessment staff Provide class listslist, student listslist, and contact information for grade 4 studentsteachers Notify teachers, selected students, and students’ parents of the study and benefit of participating Work with assessment staff to identify students with special needs Coordinate the principal’s completion of the School Questionnaire (online or hardcopy) Coordinate the teacher’s completion of the Teacher Questionnaire (online or hardcopy) Collect parental consent forms where required and submit to PIRLS staff Run a systems check of the digital -system	Confirm spaces for assessment are problem-free Collect complete School and Teacher Questionnaires (if not completed online) and give to assessment staff Ensure all sampled students attend the assessment sessions Meet with assessment staff and provide feedback about the assessment process	Receive a $100 check Receive an additional $50 for running the system check, and assisting with computer setup. These components may be delegated to a school IT coordinator if necessary.
  Teachers of eligible courses		Complete Teacher Questionnaire and return to school coordinator prior to assessment day (if not completed online)		Receive a $20 checkAmazon gift card Represent the United States in preparations for the international study
  Selected Students			Attend the assessment session, complete the assessment and Student Questionnaire	Receive a small thank-you gift Represent the United States in preparations for the international study
  [National contractor’s]Westat’s assessment staff	Work with the school to set assessment dates Help school coordinator with assessment details and logistics Protect school and student confidentiality	Call the school coordinator to discuss assessment day location(s) and student participation Select student sample and notify school of selected students Provide School and Teacher Questionnaires to the school coordinator for distribution	Conduct assessments from start to finish Furnish all the assessment materials, pencils, and test booklets Conduct a brief debriefing interview with the school coordinator at the end of the assessment Maintain security of all materials	Ensure quality and uniformity of data collected across the United States

• REVISED – Revised, throughout, the content of the “PIRLS Registration and Provide School Information (PSI) Guide” (pp. 36-38) and of MyPIRLS Website (pp.41-62), including: (a) replacing the placeholder TIMSS screenshots that modeled what the MyPIRLS website would look like with screenshots of the MyPIRLS website, (b) replacing examples of TIMSS certificates with PIRLS certificates, and (c) deleting: (i) Student-Teacher Linkage Form (STLF) Email; (ii) Student-Teacher Linkage Form (STLF) Template (Grade 4); and (iii) “Welcome to the U.S. ePIRLS Team!” card

• ADDED –School Coordinator Incentive Template (p. 71); and School Incentive Template (pp. 72)

1 The average hourly earnings of principals/education administrators in the May 2017 National Occupational and Employment Wage Estimates sponsored by the Bureau of Labor Statistics (BLS) is $46.85, for school coordinators and teachers is $29.25, and of parents is $24.34. Where mean hourly wage was not provided, it was computed from the mean annual wage assuming 2,080 hours per year. The student wage is based on the federal minimum wage. Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ data type: Occupation codes: Education Administrators, Elementary and Secondary Schools (11-9032), Education School Teachers, Except Special Education (25-2021), and all employees (00-0000); accessed on December 7, 2018.