Supporting Statement for
OMB Clearance Request
Part B
National Implementation Study of Student Support and Academic Enrichment Grants (Title IV, Part A)
Revision to reflect Congress’s creation of the Bipartisan Safer Communities Act (BSCA)/ Stronger Connections (SC) Grant Program
Revised September 2023
Original approved:
March 30, 2022
Section B: Data Collection Procedures and Statistical Methods 1
B.1. Respondent Universe and Sampling Methods 1
B.2. Procedures for the Collection of Information 1
B.2.1. Statistical methodology for stratification and sample selection. 1
B.2.3. Degree of accuracy needed for the purpose described in the justification 1
B.2.4. Unusual problems requiring specialized sampling procedures 2
B.2.5. Any use of periodic (less frequent than annual) data collection cycles to reduce burden. 2
B.3. Methods to Maximize Response Rates and Deal with Nonresponse 2
B.4. Test of Procedures and Methods to be Undertaken 3
B.5. Individuals Consulted on Statistical Aspects or Collecting or Analyzing Data 3
The Institute of Education Sciences (IES), U.S. Department of Education (the Department) requests clearance for a revision to a data collection to that OMB approved on March 30, 2022, for the National Implementation Study of Student Support and Academic Enrichment Grants (Title IV, Part A).1 The goal of the study is to develop a national picture of how states and districts are implementing the Title IV-A program. Survey data were collected from states and districts in the spring and summer of 2022, as intended (see Part B in the original clearance package). However, since OMB cleared this data collection, the Title IV-A program was significantly impacted by the Bipartisan Safer Communities Act (BSCA), which provides an additional $1 billion in Stronger Connections grants (SC) via Title IV-A to enhance states’ and districts’ ability to address acute school safety and student mental health needs in the aftermath of the coronavirus pandemic. To better align the study’s data collection plans and timeline to this recent development—Congress passed BSCA in June 2022, with funds only starting to flow to states in the fall—IES is requesting approval to conduct a follow-up state survey in lieu of the previously-approved optional district survey. No other changes are requested.
Below is a description of the statistical methods for analysis of data from the proposed 2023-24 Follow-up Survey of State Title IV-A Coordinators. Analyses of the 2022 surveys have followed the previously approved statistical methods described in the original Supporting Statement Part B. Additional background on the study and how it relates to the new BSCA-SC program is provided in Supporting Statement Part A that accompanies this revision request to the original clearance package.
The study will administer a follow-up survey to understand early BSCA-SC implementation to 50 of the 56 state Title IV‑A coordinators, including representatives from the 48 contiguous states, the District of Columbia, and Alaska. These 50 coordinators are from those states that were required, under the BSCA legislation, to award competitive BSCA-SC subgrants to high-need districts. The six other states and territories were not subject to these same BSCA requirements: Hawaii, Puerto Rico, and the Outlying Areas (American Samoa, Commonwealth of the Northern Mariana Islands, Guam, and the Virgin Islands). In these six states and territories, there are no districts to which the state can award BSCA-SC subgrants; rather, the SEA itself directs the operation of K-12 schools. As a result, the questions in the follow-up survey do not apply to these SEAs.2
Based on the 96 percent response rate to the 2022 state Title IV-A coordinator survey, IES anticipates a response rate of at least 85 percent for this new BSCA-SC survey. The anticipated fielding period for this survey is December 2023-February 2024.
As noted above, the follow-up state survey will include a census of 50 Title IV-A state coordinators and will not involve any statistical sampling procedures or stratification.
No estimation procedures are needed for the 2023-24 Follow-up Survey of State Title IV-A Coordinators because the survey will collect data from a census of state Title IV-A coordinators in all states that were required to award BSCA-SC subgrants competitively to high-need districts.
The study achieved a 96 percent response rate on the initial survey of state Title IV-A coordinators in 2022. IES therefore anticipates a similar response rate of at least 85 percent for the proposed 2023-24 Follow-up Survey of State Title IV-A Coordinators.
There are no unusual problems requiring specialized sampling procedures.
To minimize respondent burden, IES has carefully considered the minimum amount of data needed to answer the original research questions about Title IV-A implementation as well as the new research questions about BSCA-SC implementation (see accompanying Supporting Statement Part A for more details on the research questions) and how to structure the data collection. IES plans to administer the 2023-24 Follow-up Survey of State Title IV-A Coordinators beginning in winter of 2023-24. Because analyses of the 2022 survey of district Title IV-A coordinators yielded comprehensive information to address questions about regular Title IV-A implementation and because it would be too soon for a follow-up survey of district Title IV-A coordinators about BSCA-SC grants, IES has determined that the approved 2024 optional follow-up survey of district Title IV-A coordinators is no longer needed.
To communicate the importance of the data collection, motivate participation, and follow up with initial non-respondents to achieve our target response rate of at least 85 percent, the follow-up survey of state Title IV-A coordinators will follow similar procedures described in the original and approved information clearance request. The survey will include language citing that recipients of Title IV, Part A funds are expected to cooperate in Department evaluations (Education Department General Administrative Regulations, EDGAR, 34 C.F.R. § 76.591). Additionally, the web-based approach will allow the study team to identify non-respondents easily for follow-up to encourage participation and maximize response rates. To encourage participation, the study will use communication materials similar to those used to achieve a response rates of 96 percent on the 2022 survey of state Title IV-A coordinators:
2023 pre-survey notification of State Superintendents. The Department will send letters to state superintendents re-introducing the Abt Associates (Abt) study team and informing them about the goals of the 2023-24 Follow-up Survey of State Title IV-A Coordinators in light of the new BSCA-SC grant program. The letters will also include information on the data collection, analysis, and reporting timeline, provisions for maintaining anonymity of participants, and the benefits to be derived from the study. In addition, we will include the name of the Department contact and the Abt study director so that state superintendents or survey participants may ask questions about the study. The letters from the Department will support high survey response rates and add credibility to the email solicitation that will come from Abt a week or so later to encourage participation.
2023 pre-survey notification from Abt to state Title IV-A coordinators. Abt will send an email to all state coordinators describing the new follow-up survey about states’ BSCA-SC implementation. This letter will explain the purpose of the survey and request their participation.
2023 survey invitation from Abt to state Title IV-A coordinators. At the designated opening date of the survey, the study team will send an email invitation to state Title IV-A coordinators with a unique survey link, detailed instructions, the survey closing date, and project staff contact information. The survey will include detailed on-screen instructions. Throughout the data collection cycle, the Abt study team will monitor an email account for inquiries from potential respondents to ensure that participants receive prompt answers to questions or concerns.
Copies of informational letters to the state superintendents and state Title IV-A coordinators are included in Appendix A.
Survey Pilot Testing. Following similar procedures used to test the 2022 surveys, the study team, Abt refined the 2023-24 follow-up survey of state Title IV-A coordinators about BSCA-SC grants in response to a pilot test conducted with five state Title IV-A coordinators. (No substantive public comments were received during the initial 60-day public comment period.) To avoid losing states from the census of state coordinators, Abt will enter data from those who completed a pilot survey into the online form of the approved survey and ask them to review all existing responses for accuracy and complete any new or revised items. This procedure will allow Abt to make any necessary updates while minimizing burden on participants in the pilot test.
Pilot respondents from five states provided feedback on the clarity and content of the questions and the usability of the survey via cognitive interviews conducted while each respondent completed the survey. To estimate the time to complete the survey, a member of the study team used a stopwatch to record the time elapsed as each respondent read and answered survey items. Timing was temporarily paused when the respondent asked a clarifying question, or when the study team asked the respondent to provide feedback about the language, comprehensibility, relevance, and comprehensiveness of a set of survey items. As a result of pilot testing, Abt revised survey questions, response options, or supporting instructions that respondents found confusing and replaced ambiguous or vague terms with accurate and clear text. For example, the Abt study team:
Clarified that “awarding” subgrants to districts refers to notifying districts that they had received a BSCA-SC subgrant, which is not necessarily the same as actually issuing those subgrant funds to districts;
Edited questions about whether states sought “public input” to ensure that states answered accurately if they sought input from “the public or other stakeholders”;
Edited questions about the state’s procedures for awarding BSCA-SC subgrants to clarify the types of districts to which the questions referred;
Added a few new response options to ensure that states could accurately answer questions about the state’s definition of “high-need” and questions about the state’s priorities for districts’ use of BSCA-SC subgrant funds;
Moved the definitions of response options from introductory text at the start of one survey section to the individual question within that section where these definitions were most relevant.
Pilot respondents did not express concern about the survey structure. All pilot tests were completed in a shorter time period than originally estimated. On average, pilot respondents needed 20 minutes, rather than our initial, pre-pilot estimate of 30 minutes, to complete the survey, including the time needed to gather information and documentation needed for the survey.
The following individuals were consulted and will be responsible for the data collection and analysis:
Name |
Company |
Project Role |
|
Ellen Bobronnikov |
Abt |
Project Director |
|
Amanda Parsad |
Abt |
Senior Advisor |
|
Cristofer Price |
Abt |
Director of Analysis |
|
Carter Epstein |
Abt |
Data Collection Lead |
1 OMB control number 1850-0968 (https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=202107-1850-005)
2 IES anticipates coordinating data collection from these six states and territories with the Department’s Office of Elementary and Secondary Education or collecting data under a separate data collection effort in the future. IES will seek OMB clearance for that data collection separately, as needed. These six states and territories are not required to award BSCA-SC subgrants competitively to high-need districts, so the questions in the current follow-up survey do not apply to them.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Porsha Cropper |
| File Modified | 0000-00-00 |
| File Created | 2023-09-30 |