SUPPORTING STATEMENT - PART B for
OMB Control Number 0584-NEW:
SFA Survey on School Food Supply Chain Disruption
Holly Figueroa
Social Science Research Analyst
Office of Policy Support
Food and Nutrition Service
United States Department of Agriculture
1320 Braddock Place
Alexandria, Virginia 22314
Phone: 703-305-2105
Email: Holly.Figueroa@usda.gov
TABLE OF CONTENTS
B1. Respondent Universe and Sampling Methods 1
B2. Procedures for the Collection of Information 3
B3. Methods to Maximize the Response Rates and to Deal with Nonresponse 5
B4. Test of Procedures or Methods to be Undertaken 7
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 10
TABLES
Table B1. Summary of Respondent Universe and Expected Response Rates 2
Table B2. Individuals consulted on statistical aspects of study design 10
Appendix A. Section 28 of the Richard B. Russell National School Lunch Act and Section 305 of the Healthy, Hunger-Free Kids Act of 2010
Appendix B. SFA Survey on Supply Chain Disruption
Appendix C. SFA Contact Information Request
Appendix D. Survey Support Email
Appendix E. Reminder Email
Appendix F. Thank You Email
Appendix G. Pretest
Appendix H. Burden Table
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The respondent universe for this collection includes the 56 State Agency (SA) Child Nutrition (CN) directors that administer the CN Programs in the 50 States, District of Columbia, Guam, Puerto Rico, and U.S. Virgin Islands as well as the 19,050 school food authorities (SFAs), public and private, that administer the CN Programs at the local level. No sampling or weighting is required as the collection will be a census of both SAs and SFAs. Each SA will be asked to provide contact information for the SFAs under its jurisdiction, which each SA is uniquely positioned to provide; thus, a sample of SAs is not sufficient for the purposes of this collection. Additionally, given the anticipated wide variation in supply chain disruption that SFAs are experiencing and the expected need for information at the State and regional levels, there is not an efficient sample design that could closely match the level of detail that a census of SFAs will yield to support a full scale assessment of supply chain issues. This information is critical to helping FNS better understand, and provide tools and resources to address, the localized concerns and challenges school districts nationwide are facing in their efforts to feed children healthy and nutritious meals this school year.
Based on previous experience with State-level collections, the timeliness of the survey, and the minimal amount of burden involved on the part of State agencies for this collection, a 100 percent response rate is expected for the state-level SFA contact information request (Appendix C). Additionally, because supply chain disruption is an urgent issue affecting school districts nationwide and this short survey presents an opportunity for SFAs to communicate their challenges and frustrations directly to FNS, FNS expects all 19,050 SFAs to respond.
Table B1. Summary of Respondent Universe and Expected Response Rates
Respondent |
Universe |
Target completed cases |
Expected Response Rates |
|
|
|
|
State CN Agencies |
67 |
67 |
100% |
SFAs |
19,050 |
19,050 |
100% |
Total |
19,117 |
19,117 |
100% |
The approach to achieving high response rates builds on prior FNS survey experience as well as FNS’s relationships with its partners at the State and local levels. Throughout the pandemic, FNS, including the regional offices, has worked hand in hand with SAs to address new and emergent issues affecting the provision of meals to children. Based on what we are hearing from the regional offices and States’ expressed needs for support, we believe that State and local program operators will welcome this opportunity to share local supply chain challenges and experiences with FNS. The respondents that participated in the pretest of the instrument indicated that SFAs will welcome the opportunity to share their unique experiences this school year.
FNS will build support for this survey through engaging with regional and State partners to promote awareness and the importance of responding. Regional and State offices have consistently been in contact with the SFAs this school year to discuss specific supply chain concerns and while this survey will be the first time to systematically collect information from all SFAs, the respondents are already familiar with the communication process and topics. Because we know that SFAs consistently look to their SAs for guidance related to the CN Programs, FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix D) to let SFAs know that the survey is coming and to express their support for the collection. FNS intends to collect data over a 4-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix E) each week. FNS expects each SFA to receive 2 reminder emails, on average.
To further promote high response rates, the web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later. FNS has also created a dedicated email address through which SFAs may request assistance or ask questions. These tools will help SFAs to complete the survey whenever it is convenient for them to do so and will help FNS to track and respond to SFA requests for assistance in a timely manner. Additionally, States and SFAs will be informed that this a mandatory collection under the provisions of the National School Lunch Act.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;
Unusual problems requiring specialized sampling procedures; and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Because this is a census and a 100 percent response rate is expected, there is no need for sampling, weighting or nonresponse adjustments.
To facilitate the data collection, FNS will first obtain SFA contact information from the 56 State CN agencies that administer the CN Programs by sending SAs the SFA Contact Information Request email (Appendix C) with an attached spreadsheet to complete and return to FNS. SAs may also submit the information in other forms depending on the specific systems they use and their ability to export existing information. With the SFA Contact Information Request email, FNS will attach a Survey Support Email (Appendix D) and ask all SAs to distribute it to the SFAs in their respective States to let SFAs know that the survey is coming and to express their support for the collection. Shortly thereafter, FNS will email the SFA Survey on Supply Chain Disruption (Appendix B) to each SFA via the Qualtrics Survey Software platform.
FNS intends to collect data over a 4-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix E) from experienced survey support personnel at FNS each week. FNS expects each SFA to receive 2 reminder emails, on average. Upon completion of the survey, FNS will send each SFA a thank you email (Appendix F). Respondents may also email the survey team at FNS to request help in completing their survey or with technical issues.
Statistical Methodology for Stratification and Sample Selection. Because the collection will be administered via a census of State CN agencies and SFAs, there will be no sample stratification or selection.
Estimation Procedure. Because data will be collected from a census of State CN agencies and SFAs, no estimation procedures are necessary for this collection.
Degree of Accuracy Needed for the Purpose Described in the Justification: Because data will be collected from a census of State CN agencies and SFAs, this question is not relevant to this submission.
Unusual Problems Requiring Specialized Sampling Procedures: No specialized sampling procedures are planned.
Use of Periodic (Less Frequent than Annual) Data Collection Cycles to Reduce Burden. The data collection procedures will be conducted once. Concern regarding the periodicity of data collection cycles is not applicable.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The study is expected to achieve a 100 percent response rate for both the state-level collection of SFA contact information and the SFA Survey on Supply Chain Disruption. This means that the data collected will represent the entire universe of State CN Directors and SFAs, and rather than providing estimates to answer the research questions, we will be able to provide actual population totals. Thus, these data will provide reliable information on the extent and impacts of school food supply chain disruption that represent the full population.
Achieving the specified response rate involves contacting the States, securing their support for the study, and then contacting SFAs and offering support and completion reminders. The study team will use the following methods to maximize participation and reduce nonresponse:
FNS will inform States and SFAs that their participation in this collection is mandatory per the provisions of the National School Lunch Act.
FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix D) to let SFAs know that the survey is coming and to express their support for the collection.
The web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later, allowing respondents to complete the survey at their convenience.
SFAs that have not yet completed their surveys will receive a reminder email (Appendix E) each week. FNS expects each SFA to receive 2 reminder emails, on average.
Recruiting materials were carefully developed to emphasize the following points, which may resonate with respondents:
State agency and SFA participation in the collection are vital to providing FNS with information we need to support school districts during this difficult time.
This is an opportunity to communicate the specific issues each SFA is experiencing directly to FNS.
This survey will assist FNS and its partners to enhance the toolbox for school nutrition professionals working hard to make sure students have reliable access to healthy meals.
The survey was crafted with input from SFAs and will only take 20 minutes to complete, on average.
FNS created a dedicated email address through which SFAs may request assistance or ask questions to enable FNS to track and respond to SFA requests for assistance in a timely manner. Staff will be readily available to clarify survey questions and work with participants to resolve technical issues, such as difficulty logging on or advancing past pages. Personalized assistance bolsters the perceived legitimacy of the collection and will encourage respondents to persist in completing the survey.
If the response rate is lower than expected after reminder emails have been sent, FNS will compare the characteristics of the non-responding SFAs to those of all SFAs to determine if there are patterns in non-response across key characteristics such as SFA size, geographic location (State or region), or urbanicity (urban vs. rural). If there are identifiable patterns in nonresponse, FNS will consider providing more targeted outreach to the non-responding SFAs to encourage their participation. If necessary, FNS will also prepare suitable nonresponse adjustment weights to be used for estimates and will prepare suitable estimates of standard errors on critical outcomes prior to presenting results of the survey.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The study team pretested the SFA Survey on Supply Chain Disruption with nine SFAs in October 2021. Through the regional offices, FNS asked State agencies to recommend SFAs for the pretest with consideration for a set of identified criteria (described below). SFA respondents were selected based on their availability to participate within the pre-test timeframe and to ensure variation in participation across characteristics such as SFA size in terms of student enrollment, geographic location, urbanicity, percentage of students certified for free or reduced price meals, and use of a food service management company, in order to gather different perspectives on the survey.
FNS scheduled a 30-minute debriefing conversation with each of the 9 participants and sent each a link to complete the web survey via the Qualtrics Survey Software platform. In the invitation email, FNS requested feedback from the pretest SFAs on question wording, response options, and burden. Prior to each debriefing phone call, FNS sent the respondent a copy of his or her completed survey responses to facilitate the discussion. The pretested version of the survey and the debriefing protocol are available in Appendix G. Pretest participant names and titles are available in Question 8 of Supporting Statement Part A.
As described in Part A of the supporting statement, pretest participants reported spending from 5 to 30 minutes on the survey, with all but one finishing within the 20 minute estimate. All participants provided positive feedback on the overall content and structure of the survey, and suggested that the survey would be well-received by SFAs given its timeliness and ease of completion. While open-ended responses and the question about the extent of their current financial deficit (if applicable) took longer than other questions, most participants indicated that the 20 minute burden estimate was accurate and appreciated the opportunity to provide additional context via the open-ended responses when necessary. Participants largely thought that the response options offered were well-worded, understandable, and covered their situations well; however, a few made suggestions for items to add or alternative scales to use. Participants also suggested providing a “not applicable” option on a few questions.
The most prevalent issues reported during the pretest were with questions 1 and 2, which asked participants for their SFA ID numbers and to select the CN Programs that they have operated this school year. Most participants did not know their SFA ID numbers off the tops of their heads and many did not know where to find it. Additionally, they reported that it was unclear what time period they should report on for the CN Programs and whether they should select the National School Lunch and Breakfast Programs (NSLP/SBP) if they normally operate those programs but are using the nationwide waiver to provide breakfast and lunch under the NSLP Seamless Summer Option (SSO) this school year.
In response to participant feedback, FNS made the following changes to the survey:
Supplemented the language in the introduction
Removed Question 1 requesting SFA Name and ID – FNS will now get this information directly from State agencies, who have this information more readily available, as part of the SFA Contact Information Request (Appendix C)
Clarified the time period and added instructional details to Question 2
Switched the order of Questions 3 and 4 and removed an unnecessary descriptor from Question 3
Slightly adjusted the wording of the question, response options and examples for Question 5
Added a follow-up to Question 5 to assess changes in severity of the challenges reported over time
Made small adjustments to wording/examples in Question 7
Changed the response options for Question 8 regarding expected duration of challenges
Added a N/A option to Questions 9 and 10 regarding use of USDA Foods and USDA DoD Fresh
Added several response options to Questions 11 and 12 to better capture impacts challenges are having on SFAs and the methods participants reported using to address them
Clarified example calculation in Question 17
Added a N/A option to Question 19
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Table B5.1 lists staff consulted on statistical aspects of the design. The same staff will be responsible for collecting and analyzing the study data.
Table B2. Individuals consulted on statistical aspects of study design
Staff |
Title |
Phone |
|
Holly Figueroa, FNS |
Social Science Research Analyst |
703-305-2105 |
|
Maggie Applebaum, FNS |
Branch Chief, Special Nutrition Analysis Branch |
703-305-2578 |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Mathematica Standard Report Template |
| Author | MATHEMATICA |
| File Modified | 0000-00-00 |
| File Created | 2021-10-22 |