Supporting Statement Part B 0584-0607_Aug 5 2024

SUPPORTING STATEMENT - PART B for
OMB Control Number 0584-0607:
School Meals Operations (SMO) Study

August 5, 2024

Darcy Güngör
Social Science Research Analyst
Evidence, Analysis, and Regulatory Affairs Office
USDA, Food and Nutrition Service
1320 Braddock Place, Alexandria, VA 22314

Table of Contents

B1. Respondent Universe and Sampling Methods 1

B2. Procedures for the Collection of Information 2

B3. Methods to Maximize the Response Rates and to Deal with Nonresponse 3

B4. Test of Procedures or Methods to be Undertaken 5

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6

Appendices

1. National School Lunch Act

2. Web survey about school year 2023-24

3. Administrative data requests for fiscal year 2024

   1. FNS-10 administrative data request

   2. FNS-418 administrative data request

   3. FNS-44 administrative data request

4. Participant outreach:

   1. Study support email from FNS Regional Office to State Agency

   2. Advance email

   3. Study brochure

   4. Survey email

   5. Survey reminder email

   6. Survey reminder phone script

   7. Survey last chance post card

   8. Administrative data phone script

5. Web survey pre-test memo

6. FNS Privacy Act System of Records Notice

7. Confidentiality pledge

8. Review by National Agricultural Statistics Service

9. Burden table

1. Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

This is a revision of an approved information collection (OMB Control Number 0584-0607). The revision is to collect one additional year of study data for the School Meals Operations (SMO) Study.

The respondent universe for the additional SMO Study year will be 68 State Agency Child Nutrition (CN) Directors. This is a census of the 68 State Agencies that administer the National School Lunch Program (NSLP), School Breakfast Program (SBP), NSLP Seamless Summer Option (SSO), Summer Food Service Program (SFSP), and Child and Adult Care Food Program (CACFP) in all States and Territories. Given the total universe of only 68 eligible State Agencies, and their relatively distinct characteristics, there is not an efficient sample design that will closely match the comprehensive data on State policies and student meal service that a census will yield to better understand variation and localized concerns.

We expect a 100% response rate because:

• The first four years of the SMO Study have had a 100% response rate from State Agency respondents and, in addition to using the same successful data collection approach as previous study years, the respondents are now accustomed to the data collection.

• The Child Nutrition Program Operations Study II (OMB Control Number 0584-0607, expiration date 08/31/2022), which was the predecessor to the SMO Study, had a 100% response rate from State Agency respondents.

2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection;

• Estimation procedure;

• Degree of accuracy needed for the purpose described in the justification;

• Unusual problems requiring specialized sampling procedures; and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The SMO Study has used an annual data collection cycle to collect four years of data from State Agency respondents. Each study year has collected data with a web survey that corresponds to a school year (school years 2019-20 through 2022-23) and a request for administrative data that corresponds to a fiscal year (fiscal years 2020-2023). This information collection request is to collect a web survey about school year 2023-24 (Appendix B) and administrative data about fiscal year 2024 (Appendices C1-3).

We will email the respondents a link to their web survey (Appendix D4) and a request for a phone call to discuss the administrative data request (Appendix D8). Respondents will have 10 weeks (beginning in October 2024) to complete the survey and 10 weeks (beginning in March 2025) to provide the administrative data, which allows time to plan their approach for completion. Respondents will receive reminder emails and calls from trained survey support personnel and administrative data liaisons. Respondents may also call and/or email professional survey support specialists to request help completing their survey or with technical issues.

The information collection involves a census of State Agencies, and we expect a 100% response rate, so there is no need for sampling, weighting, or nonresponse adjustments. Rather, the state-level data will provide reliable answers to the study’s research questions that represent the full population. Likewise, no estimation procedures are necessary. We do not expect any unusual problems requiring specialized sampling procedures.

3. Methods to Maximize the Response Rates and to Deal with Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

We expect a 100% response rate. The data will represent the entire universe of State CN Directors and will result in actual population totals instead of estimates.

Achieving a 100% response rate involves contacting the State Agency respondents, securing their participation in the study, and offering support and completion reminders. The following methods have worked very well for the first four years of the SMO Study and will be employed again for this additional year of data collection:

• FNS headquarters will notify the FNS Regional Offices about the information collection and ask the FNS Regional Offices to email their regional State Agencies.

• The FNS Regional Offices will send a study support email to their regional State Agencies (Appendix D1) to request cooperation with the study.

• We will send the State CN Directors an advance email (Appendix D2) and Brochure (Appendix D3) prior to fielding the survey. These explain the purpose of the study and describe the data collection activities and timeline.

• We will hold telephone meetings with State Agencies to discuss the administrative data requests (Appendices C1-3) using the administrative data phone script (Appendix D8). We expect that State Agencies will be familiar with the request, which is the same as the prior four years of the SMO Study.

• We will send the State CN Directors the survey email (Appendix D4), which provides a direct link to their web survey, a timeframe for completion, brief information about the survey, and information about how to seek help if needed.

• As necessary, we will follow up with reminder emails (Appendix D5) at 2 or 3 weeks and at 5 or 6 weeks.

• As necessary, we will follow up by telephone beginning at 7 weeks to urge respondents to complete the survey (Appendix D6). We will use call scheduling procedures that are designed to call numbers at different times of the day (between 8am and 6pm) and days of the week (Monday through Friday) to improve the chances of finding a respondent at work.

• Toward the end of the field period for each survey, State Agencies that haven’t responded will be mailed the last chance post card (Appendix D7).

The following measures will facilitate successful data collection by reducing barriers for State Agency respondents:

• The study team will copy the FNS Regional Offices on all State Agency emails to promote participation and response. The Regional Offices are strong study liaisons who know their State Agency contacts very well. They will be kept closely informed about the project so that they will be able to answer questions from CN Directors and encourage participation. FNS Headquarters works closely with the FNS Regional Offices to help answer questions and to offer additional support.

• All State Agency emails will also provide a toll-free number and study email address so respondents have easy access to help.

• The outreach materials are succinct, clearly state their intent, and are written in plain language. They underscore the importance of the State Agency’s input on critical topic areas so FNS can make informed decisions that improve the lives of children.

• As described in Supporting Statement A, Response A3, we chose web- and email-based information collection methods to simplify the submission process and reduce burden in several ways:

  • A web-based survey reduces geographic and time constraints because repondents can access the survey from any location and at any time of the day or week. It provides flexibility to repondents, who can access the survey from a smartphone, tablet, laptop, or desktop computer. It also provides flexibility because the platform saves responses so repondents can leave and return to the survey as time allows. It is accessible to repondents who require assistive technology. It is programmed with skip logic so repondents answer only the questions that are relevant to them based on earlier responses. Study staff will be readily available to clarify survey questions and work with participants to resolve technical issues, such as difficulty logging on or advancing past pages. Personalized assistance bolsters the perceived legitimacy of the study and will encourage respondents to persist in completing the survey.

  • An electronic submission of administrative data provides flexibility. There is not a consistent platform across State Agencies to store, manage, and access administrative data; therefore, respondents may submit the requested data “in whatever format is easiest” (e.g., single file, multiple files, any standard file format) and may send the data by email or, if additional security is desired by the respondent, using the State Agency’s secure transfer platform or the study team’s secure transfer site hosted by Box.

4. Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

In February 2024, we pre-tested the new survey we developed for use in fall 2024. The pretest participants were the State CN Directors from four states that varied across relevant characteristics. We did not pretest the administrative data request because it has not changed from the first four years of the study.

We emailed a pdf version of the survey to the pretest participants and instructed them to track how long it took them to complete each section of the survey, including the time needed to gather relevant data or information, and to note any instructions or questions that were unclear or difficult to answer. We conducted 30-minute debriefing interviews with each pretest participant. The interviews focused on asking respondents to identify and share concerns about unclear questions or response options, questions that took too long to answer, burden, and the flow of the survey. We used cognitive methods to gauge respondents’ understanding of the intent of questions and response options, focusing on newly crafted or significantly revised survey items, or those that asked about topics that are complex or difficult to measure. Appendix E is a web survey pret-test memorandum that summarizes the pretest methods and findings.

As described in Supporting Statement A, Response A8, the pre-test respondents said the instructions were clear, and the survey was easy to understand generally. Respondents provided suggestions for improvements that we addressed in the final survey (Appendix B) and study support email from the FNS Regional Office (Appendix D1). In the final survey, we eliminated some questions to reduce the burden, re-ordered some questions to enable respondents to delegate sections to the appropriate staff members, and rephrased some questions to enhance clarity. In the study support email, we added that four states pre-tested the survey and recommended gathering the data ahead of time, and provided the survey questions in an attachment.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The table below lists staff consulted on statistical aspects of the design. The same staff will be responsible for collecting and analyzing the study data.

Mathematica staff	Title	Phone	Email
Kevin Conway	Project Director	609-750-4083	KConway@mathematica-mpr.com
Barbara Carlson	Senior Statistician	617-674-8372	BCarlson@mathematica-mpr.com
Andrew Gothro	Researcher	202-250-3569	AGothro@mathematica-mpr.com
Eric Grau	Senior Statistician	609-945-3330	EGrau@mathematica-mpr.com
Josh Leftin	Researcher	202-250-3531	JLeftin@mathematica-mpr.com
Sarah Forrestal	Senior Survey Researcher	609-945-6616	SForrestal@mathematica-mpr.com
Veronica Severn	Survey Researcher	617-715-6931	VSevern@mathematica-mpr.com
Liana Washburn	Researcher	202-250-3551	LWashburn@mathematica-mpr.com
Eric Zeidman	Senior Survey Researcher	609-936-2784	EZeidman@mathematica-mpr.com
USDA staff	Title	Phone	Email
Darcy Güngör, FNS	Social Science Research Analyst, Child Nutrition Evaluation Branch, Office of Policy Support		Darcy.Gungor@usda.gov
Conor McGovern, FNS	Branch Chief, Child Nutrition Evaluation Branch, Office of Policy Support		Conor.McGovern@usda.gov
Susannah Barr, FNS	Social Science Research Analyst, Child Nutrition Evaluation Branch, Office of Policy Support		Susannah.Barr@usda.gov
Maggie Applebaum, FNS	Deputy Associate Administrator, Child Nutrition Programs		Margaret.Applebaum@usda.gov
Janis Johnston, FNS	Director, Program Integrity & Innovation Division, Child Nutrition Programs		Janis.Johnston@usda.gov
Doug Kilburg, NASS	Statistician, National Agricultural Statistics Service		Douglas.Kilburg@usda.gov