Procurements Part A OMB Package Revised Final 11292018

Supporting Justification for OMB Clearance for the Study of School Food Authority (SFA) Procurement Practices

OMB Control Number 0584-NEW

Part A

November 29, 2018

Office of Policy Support
Food and Nutrition Service
United States Department of Agriculture
3101 Park Center Drive
Alexandria, VA 22302

Project Officer: Ashley Chaifetz
Telephone: 703-457-7741
Email: Ashley.Chaifetz@fns.usda.gov

Table of Contents—Part A: Justification

A.1 Circumstances That Make the Collection of Information Necessary 1

A.2 Purpose and Use of the Information 4

A.3 Use of Information Technology and Burden Reduction 8

A.4 Efforts to Identify Duplication and Use of Similar Information 9

A.5 Impact on Small Businesses or Other Small Entities. 9

A.6 Consequence of Collection the Infromation Less Frequently 9

A.7 Special Circumstances Relating to the Guideline of 5 CFR 1320 10

A.8 Comments in Response to Federal Register Notice and Efforts to Consult Outside Agency. 12

A.9 Explanation of Any Payment or Gift to Respondents. 17

A.10 Justification for Sensitive Questions. 17

A.11 Estimates of Annualized Burden Hours and Costs. 19

A.12 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 19

A.13 Annualized Cost to the Federal Government 23

A.14 Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information. 24

A.15 Explanation for Program Changes or Adjustments 24

A.16 Plans for Tabulation and Publication and Project Time Schedule. 25

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 30

A.18 Exceptions to Certifiation for Paper Reduction Act Submission 30

List of Tables

Table A.1 Overview of Data Collection Activities 7

Table A.2 Estimates of Respondent Burden 22

Table A.3 Annualized Cost to Respondents 23

Table A.4 Data Collection Schedule 25

List of Appendices

Appendix A1 Email Notification from FNS to Regional Offices

Appendix A2 Email Notification from Regional Offices to State Child Nutrition Directors

Appendix A3 Email Notification to State Child Nutrition Directors

Appendix A4 Email Notification to School Food Authority Directors

Appendix B1 Study of School Food Authority (SFA) Procurement Practices: Frequently Asked Questions – Web Survey and In-Depth Interview

Appendix B2 Study of School Food Authority (SFA) Procurement Practices: Frequently Asked Questions – In-Depth Interview

Appendix C1.a Pre-Survey Notification Letter (Web Survey)

Appendix C1.b Survey Notification Email (Web Survey with Link)

Appendix C1.c Survey Reminder Email 1 (Web Survey with Link)

Appendix C1.d Survey Reminder Email 2 (Web Survey with Link)

Appendix C1.e SFA Director Telephone Reminder Script (Web Survey)

Appendix C2.a Post-Survey Response Clarification Email (Web Survey)

Appendix C2.b Post-Survey Response Clarification Phone Call Script (Web Survey)

Appendix C3 Post-Survey Thank You Email (Web Survey)

Appendix C4.a Pre-Interview Notification Letter (In-Depth Interview)

Appendix C4.b Pre-Interview Scheduling Phone Call Script (In-Depth Interview)

Appendix C4.c Pre-Interview Reminder Email (In-Depth Interview)

Appendix C4.d Participant Confirmation Email (In-Depth Interview)

Appendix C5.a Post-Interview Response Clarification Email (In-Depth Interview)

Appendix C5.b Post-Interview Response Clarification Phone Call Script (In-Depth Interview)

Appendix D1 SFA Procurement Practices Web Survey

Appendix D1.a SFA Procurement Practices Web Survey (Web Version)

Appendix D2 SFA Procurement Practices In-Depth Interview Guide

Appendix E Thank You Email for Participation in Study

Appendix F1.a Federal Register 60-Day Notice Public Comment 1

Appendix F1.b Federal Register 60-Day Notice Public Comment 2

Appendix F1.c Federal Register 60-Day Notice Public Comment 3

Appendix F1.d Federal Register 60-Day Notice Public Comment 4

Appendix F1.e Federal Register 60-Day Notice Public Comment 5

Appendix F1.f Federal Register 60-Day Notice Public Comment 6

Appendix F1.g Federal Register 60-Day Notice Public Comment 7

Appendix F1.h Federal Register 60-Day Notice Public Comment 8

Appendix F1.i Federal Register 60-Day Notice Public Comment 9

Appendix F1.j Federal Register 60-Day Notice Public Comment 10

Appendix F1.k Federal Register 60-Day Notice Public Comment 11

Appendix F2.a Federal Register 60-Day Notice FNS' Response to Public Comment 1

Appendix F2.b Federal Register 60-Day Notice FNS' Response to Public Comment 2

Appendix F2.c Federal Register 60-Day Notice FNS' Response to Public Comment 3

Appendix F2.d Federal Register 60-Day Notice FNS' Response to Public Comment 4

Appendix F2.e Federal Register 60-Day Notice FNS' Response to Public Comment 5

Appendix F2.f Federal Register 60-Day Notice FNS' Response to Public Comment 6

Appendix F2.g Federal Register 60-Day Notice FNS' Response to Public Comment 7

Appendix F2.h Federal Register 60-Day Notice FNS' Response to Public Comment 8

Appendix F2.i Federal Register 60-Day Notice FNS' Response to Public Comment 9

Appendix F2.j Federal Register 60-Day Notice FNS' Response to Public Comment 10

Appendix F2.k Federal Register 60-Day Notice FNS' Response to Public Comment 11

Appendix G1 Estimates of Respondent Burden

Appendix G2 National Agricultural Statistics Service (NASS) Comments

Appendix G3 Response to National Agricultural Statistics Service (NASS) Comments

Appendix H1 Healthy Hunger-Free Kids Act Section 305

Appendix H2 Child Nutrition Act of 1966

Appendix H3 Richard B. Russell National School Lunch Act Amended Through February 2014 Section 28

Appendix H4      Richard B. Russell NSLP Act Amended through February 2014 Section 12

Appendix H5      7 CFR 210.21

Appendix H6      7 CFR 220.16

Appendix H7      7 CFR 225.17

Appendix H8      7 CFR 226.22

Appendix H9      2 CFR 200

Appendix I Procurement Practices – Final Memorandum on Pretest Results

Appendix J Data Collector Confidentiality Agreement

Appendix K Process of Data Collection

Part A: Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

Introduction. The purpose of the Study of School Food Authority (SFA) Procurement Practices is to describe and assess the practices of SFAs related to procuring goods and services for five Child Nutrition (CN) programs: National School Lunch Program (NSLP); School Breakfast Program (SBP); Fresh Fruit and Vegetable Program (FFVP); Summer Food Service Program (SFSP); and Child and Adult Care Food Program (CACFP), and to better understand how SFAs make decisions that lead to these procurement practices. Study findings will describe similarities and differences in how SFAs procure goods and services and elucidate key issues and experiences of SFAs. The study will go beyond previous studies that concentrated on a single food service or CN program (e.g., NSLP, SBP, or SFSP) or studies that focused on single procurement practices (e.g., use of Food Service Management Companies [FSMCs] or Cooperative Purchasing Agreements [CPAs]) at the SFA level. Using a nationally representative sample of SFAs, this study will be one of the first U.S. Department of Agriculture (USDA) Food and Nutrition Service (FNS) studies of SFA procurement practices for CN meal programs that comprehensively examines the use of FSMCs and CPAs, recordkeeping, local purchases, and food purchase specifications. Specifically, this study will help FNS

• identify and describe the various means through which self-operating SFAs develop and publish solicitations, evaluate and award contracts, and monitor procurement contracts for all school food purchases (i.e., goods and services paid from the nonprofit food service account, including, but not limited to, USDA Foods and commercial foods procurement, processing, and distribution);

• identify and describe the rationale, procedures, and recordkeeping practices used by SFAs with respect to their contracts with FSMCs;

• identify and describe the forms of group purchasing entities (e.g., cooperatives, agents, and third-party services) SFAs use to purchase food products and services; and

• assess the strengths and weaknesses of SFAs with respect to procurement-related expertise to develop solicitation and contract documents, evaluate bids/responses, negotiate terms and conditions, conduct contract oversight, and assess the availability of State Agency (SA)-provided technical assistance and training resources.

This study utilizes data from three sources: (1) Year 2 of the USDA/FNS Child Nutrition Program Operations Study II (CN-OPS-II) (OMB Control Number 0584-0607, expiration date 07/31/2020); (2) the SFA Procurement Practices Web Survey (web survey) of SFAs; and (3) the SFA Procurement Practices In-Depth Interview Guides (IDIs) with SFAs. The web survey and IDIs involve a new information collection request.

Program background. Recent legislation recognizes the importance of issues related to nutrition and costs in school meal programs. Furthermore, Section 305 of the Healthy, Hunger-Free Kids Act (HHFKA) of 2010 (HHFKA, Public Law 111-296, December 13, 2010; Appendix H1) strongly encourages participants in programs authorized under HHFKA and the Child Nutrition Act of 1966 (42 U.S.C. 1771 et seq.; Appendix H2) to cooperate with program research and evaluation being conducted on behalf of the USDA Secretary under those Acts.

Identifying current SFA procurement practices by examining SFA practices in school year (SY) 2017–18 will be used by FNS as it seeks to meet key program objectives that include enhancing nutrition assistance program efficiency and supporting American agriculture. The guiding principles of recent CN policies and regulations that relate to the research objectives and questions of this study are ensuring that SFA procurement practices are effective and efficient, and that SFAs procure domestic foods to the greatest extent possible.

Relevant legislation includes The Richard B. Russell National School Lunch Act (NSLA), as amended, which set procurement requirements for NSLP and SBP in 2014 (Appendix H3).¹ One such requirement for NSLP is the “Buy American” clause in Section 12 of the NSLA (Appendix H4), which requires the purchase of domestic agricultural commodities and domestic food products (produced and processed in the United States). Implementation of the Buy American provision should be carried out by including Buy American requirements in procurement procedures; SA prototype documents; all procurement solicitations and contracts, including domestic requirements in bid specifications; contract monitoring; and verifying cost and availability of domestic and non-domestic foods.²

Procurement for CN meal programs can be affected by federal regulations, policy memos, program guidance, and grant funding, in addition to State and local policies which sometimes have stricter guidelines. The procurement of all school meal goods and services must comply with procurement standards in CN programs; government-wide regulations in 7 CFR 210.21 (Appendix H5), 220.16 (Appendix H6), 225.17 (Appendix H7), 226.22 (Appendix H8); and government-wide regulations in 2 CFR 200 (Appendix H9). Procurement regulations are intended to ensure that SFA procurement procedures, and procedures of those purchasing on behalf of SFAs (such as FSMCs or group purchasing entities, including the use of cooperatives, agents, and third-party services), adhere to CN program and government-wide regulations and guidance, and State and local policies related to procurement.³

Understanding the impacts of these policies and regulations on SFAs provides important context for the study objectives, research questions, and study findings, especially since many of the federal rules and regulations related to HHFKA (Appendix H1) and NSLA (Appendices H2 and H3), as amended, continue to be updated and finalized. For example, in 2015, professional standards requirements for school nutrition personnel were finalized.⁴ Additionally, in 2016, FNS developed the Local Agency Procurement Review Tool, intended to assist SAs with their procurement reviews of SFAs. FNS continues to provide tools and training to SAs and SFAs to keep both groups informed and further assist them in how to interpret new regulations.

2. Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

How the information will be used. The information collected through this study will help FNS better understand how SFAs make decisions that lead to their procurement practices for CN programs. This information will assist FNS as it continues to work with SFAs to improve CN program operations. In particular, by identifying and describing the (1) various means through which self-operating SFAs develop and publish solicitations, evaluate and award contracts, and monitor procurement contracts for all school food purchases; (2) rationale, procedures, and recordkeeping practices used by SFAs with respect to their contracts with FSMCs; and (3) forms of group purchasing efforts SFAs use to purchase food products and services, FNS will have better information to help SFAs interpret FNS regulations and comply with program requirements. Similarly, by assessing the strengths and weaknesses of SFAs with respect to their procurement-related expertise and assessing the availability of SA-provided technical assistance and training resources, FNS will be able to provide more effective policy guidance, as well as supplemental training. In short, the information from the study will further enable FNS to pursue its mission to increase food security and reduce hunger by providing children access to food, a healthful diet, and nutrition education in a way that supports American agriculture and inspires public confidence.

From whom the information will be collected. Participants in this study include a subsample of SFA directors who participated in Year 2 of the USDA/FNS Child Nutrition Program Operations Study II (CN-OPS-II) (OMB Control Number 0584-0607, expiration date 07/31/2020) in school year 2017-18. The selected SFA directors will be asked to participate in two new data collection components for this study: (1) the web survey, and (2) the IDIs. Each instrument is estimated to average 90 minutes in length to complete.

First, a stratified random sample of 700 SFA directors will be selected from among the CN-OPS II Year 2 respondents to participate in the web survey (Appendix D1; Appendix D1.a). The sample of web survey respondents will be drawn based on CN-OPS-II survey responses such that a range of procurement practices (e.g., use of an FSMC, use of a CPA) are represented in the web survey sample along with a range of SFA characteristics (e.g., SFA size, urbanicity). The web survey data will be weighted in analyses to account for the stratified random sampling design and to adjust for nonresponse.

Next, a purposive subsample of 125 SFA directors who completed the web survey will be selected to participate in the IDIs (Appendix D2). Selection of SFAs for IDIs will be based on answers to web survey questions focused on the following topics: contracting, management, suppliers, decision-makers, and State monitoring. Overall, this process will ensure IDIs are conducted with SFAs that have a range of procurement practices of interest to FNS. IDI selection will also take into account SFA demographic and geographic characteristics to ensure representativeness. For example, the Study Team will ensure that SFAs from urban, suburban, and rural areas are included for the IDIs.

While these study requests are voluntary, the SFAs will be strongly encouraged to cooperate with them, as per Section 305 of HHFKA.⁵

How the information will be collected. Table A.1 summarizes the data collection plan. Both the web survey (Appendix D1) and the IDI (Appendix D2) will be administered with a subset of respondents (SFA directors) from the CN-OPS-II Year 2 sample. The research questions will be addressed through both the web survey and the IDIs using an integrative mixed-methods design, which informs findings through a synthesis of quantitative and qualitative data.

Table A.1. Overview of Data Collection Activities

Instrument	Respondent Category	Mode	Length	Number of Respondents6	Frequency	Purpose
SFA Procurement Practices Web Survey (Appendix D1)	SFA directors (subsample of CN-OPS-II Year 2 respondents)	Web	90 minutes	560	Once	Collect data on SFA contracting, management, suppliers, decision-makers, and State monitoring policies from a nationally representative sample of SFAs
SFA Procurement Practices Study In-Depth Interview Guide (Appendix D2)	SFA directors (subsample of web survey respondents)	Telephone	90 minutes	100	Once	Obtain detailed information about web survey responses through additional questions and probes

Information about the process of data collection can be found in Appendix K.

Frequency of information collected. This is a one-time data collection anticipated to take place in SY 2018–19. Data collection will span 31 weeks.

Information shared with any other organizations inside or outside USDA or the government. Results will be presented in aggregated form in the final study report, which will be made available in the research section of the USDA-FNS website: http://www.fns.usda.gov/ops/research-and-analysis. A dataset with deidentified responses to the web survey will also be made available to the public.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

FNS is committed to complying with the E-Government Act of 2002, to promote the use of technology. The web survey (Appendix D1) will be conducted electronically (560 respondents). The web survey includes a “Save and Continue” option, allowing respondents the ability to save their responses and continue the web survey later. Toward the end of the data collection period, SFA directors who have not completed the web survey will be contacted by telephone using the SFA Director Telephone Reminder (Web Survey) (Appendix C1.e) and will be offered the opportunity to complete their web survey via the telephone. Thus, all 560 responses to the web survey are expected to be electronic.

FNS estimates that out of the 5,813 total responses for this collection (3,779 responses and 2,034 nonresponses), approximately 9.6 percent (n = 560) of responses will be collected electronically. These are burden estimates presented in Appendix G1 and Table A.2.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.

Every effort has been made to avoid duplication. FNS has sole responsibility for administering the USDA school meal programs and has reviewed USDA reporting requirements, State administrative agency reporting requirements, and special studies by other government and private agencies. To our knowledge, there is no similar information available or being collected.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

Information being requested has been held to the minimum required for the intended use. Although smaller SFAs are involved in this data collection effort, they deliver the same program benefits and perform the same function as any other SFA. Thus, they maintain the same kinds of information on file. FNS estimates that 1 percent of web survey respondents (approximately six SFAs) will be small entities. Collecting the requested information will not have a significant economic impact on these small entities.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The proposed data collection plan involves one-time administration of the web survey and IDIs during SY 2018–19. If clarification of responses is necessary, the Study Team may follow up with a respondent via either email (Appendices C2.a and C5.a) or telephone (Appendices C2.b and C5.b) within one month of the end of the data collection period. There will be up to two attempts to reach the respondent to clarify their responses for both the web survey and the IDI. Gathering such data is essential to track the characteristics of the CN programs, such as the NSLP and SBP, which enable FNS to pursue its mission to increase food security and reduce hunger by providing children access to food, a healthful diet and nutrition education in a way that supports American agriculture and inspires public confidence in the CN programs. The purpose of this information collection is to help FNS better understand how SFAs make decisions that lead to their procurement practices for CN programs. This information will assist FNS as it continues to work with SFAs to improve CN program operations. Additionally, by assessing the strengths and weaknesses of SFAs with respect to their procurement-related expertise and assessing the availability of SA-provided technical assistance and training resources, FNS will be able to provide more effective policy guidance, as well as training to supplement the guidance. Without this data collection, FNS will not have the information necessary to understand SFA procurement practices, which would obstruct the agency’s ability to keep abreast of issues in administration and operation of these programs, thereby delaying the discussion, formulation, and implementation of suitable policies and affecting its ability to pursue its program mission.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no special circumstances. This collection of information is conducted in a manner consistent with the guidelines in the Code of Federal Regulations, 5 CFR 1320.5.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years even if the collection of information activity is the same as in prior years. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

Federal Register notice and comments

Notice of this study was published in the Federal Register on Tuesday, May 30, 2017 (Vol. 82, No. 102, pp 24667–24670). Seven responses were received (Appendices F1.a–F1.g): an email from Jean Public (Appendix F1.a); an email from Mark Rifkin (Appendix F1.b); an email from Tara Ray (Appendix F1.c); a letter from the Academy of Nutrition and Dietetics (Appendix F1.d); a letter from the Florida Department of Agriculture and Consumer Services (Appendix F1.e); a letter from the School Nutrition Association (SNA) (Appendix F1.f); and a letter from the National Farm to School Network (Appendix F1.g). FNS responded to each of these comments (Appendices F2.a–F2.g) as described below.

The email from Jean Public (Appendix F1.a) expressed concern with taxpayers paying for this study. FNS responded with an email (Appendix F2.a) indicating that Jean Public’s comments will be evaluated as required by 5 CFR 1320 and that comments related to the information collection will be addressed in this Information Collection Request (ICR) submission. FNS believes that this comment is not germane to this information collection as it does not address the necessity or practical utility of the collection nor does it provide comments concerning the cost, the time to complete the collection, burden, the frequency of the collection, or the method of the collection. Therefore, FNS has not addressed this comment beyond a generic response.

The email from Mark Rifkin of the Academy of Nutrition and Dietetics (Appendix F1.b) requested plans for this information collection. In response, FNS sent an email (Appendix F2.b) requesting clarification on what was meant by “plans” and that information can be found in the 60-day Federal Register Notice, as well as noting that additional information will be part of the OMB package. Mark Rifkin responded that he was interested in the data collection instruments and will look for them in the published OMB package.

The email from Tara Ray (Appendix F1.c) expressed concerns about the timing of the data collection and indicated a lack of clarity in the purpose for the information. She suggested that a survey of SA staff would be preferable so as to reduce workload on food service directors. FNS responded (via email) (Appendix F2.c), clarifying that this data collection will both inform and support FNS decision-making regarding procurement-related training, technical assistance, and policies, and that the data collection is not a procurement review.

The letter from the Academy of Nutrition and Dietetics (Appendix F1.d) offered support for the data collection and suggested that FNS work with States to provide training either before, or in conjunction with, the data collection. In response, FNS sent a letter (Appendix F2.d) expressing appreciation of their support, noting that there are several training opportunities on procurement practices for State CN directors at the federal level and that FNS expects that the State CN directors will participate in these trainings. FNS also clarified that this data collection is not an audit of SFAs, but instead, it is an exploration of procurement-related practices and decision-making. The response from FNS underscored that this study was designed to place as little burden on SFAs as possible, while still collecting important and relevant information.

The letter from the Florida Department of Agriculture and Consumer Services (Appendix F1.e) detailed a concern that the same SFAs appear to be repeatedly selected for surveys, and that it does not appear that other SFA responsibilities are taken into account when requests are made to complete surveys. The letter also provided recommendations for changes to the data collection, suggesting that (1) the SAs may be able to provide data on behalf of the selected SFAs, reducing the burden on SFAs; (2) the SAs be surveyed before the SFAs; and (3) if the SAs are unable to provide the specific requested information about the SFAs, then survey the SFAs. FNS sent a letter in response expressing appreciation of their support (Appendix F2.e) and also clarified that this data collection is not a procurement or administrative review, but instead, it is an exploration of procurement-related practices and decision-making. The response from FNS underscored that this study was designed to place as little burden on SFAs as possible, while still collecting important and relevant information.

The letter from SNA (Appendix F1.f) noted that this study is one of the first comprehensive reviews of SFA procurement practices and that the study will provide guidance and support for school meal programs. The letter also shared that SNA has already begun this process and suggested that it would be helpful if the recommendations were applicable to all districts, independent of whether they are self-operated or managed via an FSMC. It was also suggested that the SFAs be stratified by size, and that any new tools developed as a result of this study eliminate redundancies in the administrative review process. Concern was also expressed about the burden placed on SAs and SFAs selected to participate in this and other data collection projects. In a letter (Appendix F2.f), FNS expressed appreciation for SNA’s support for the study and noted that the SNA report, Solving the Procurement Puzzle, was provided to the Study Team to help inform this study’s development. The response from FNS also noted that the study has been designed to minimize burden to the extent possible.

In their letter, the National Farm to School Network (Appendix F1.g) indicated that they hope that the study will help improve USDA’s understanding of the challenges of procuring from local sources and to learn how procurement data collection can be streamlined to reduce administrative burden. The National Farm to School Network also noted that USDA should conduct the survey, and that the survey is necessary. They suggested that the survey be simple, electronic, and not overlap with other reporting requirements in an effort to minimize burden. In response (Appendix F2.g), FNS sent a letter expressing appreciation for the National Farm to School Network’s support for the study and noted that the SNA report, Solving the Procurement Puzzle, was provided to the Study Team to help inform this study’s development, and that the study has been designed to minimize burden to the extent possible.

Due to circumstances which delayed the submission of the full information collection request package within a year of the publication of the original notice, FNS republished the notice for public comment in the Federal Register (Volume 83, No. 161, pages 42101-42104) on August 20, 2018. The public comment period ended on October 20, 2018. FNS received four comments in response to this second notice.

One comment was received from Tracie Hartman (Appendix F1.h), who described frustration with her State in terms of paperwork requirements and another comment from the Colorado School Nutrition Association (Appendix F1.i) described inconsistent interpretation of State regulations regarding procurement. FNS responded that the comments will be shared with the appropriate program officials within FNS (Appendix F2.h; Appendix F2.i) but that the comments did not result in changes to the information collection plans. The School Nutrition Association (SNA) (Appendix F1.j) explained that they had no substantive changes from their previous comments but expressed concern that the number of respondents had increased and suggested incorporation into another FNS study. In response, FNS (Appendix F2.j) clarified that the number of respondents had not changed, though the burden was more accurately calculated, and that the study suggested for combination by SNA was the study used to sample this collection (CN-OPS-II). As with the initial comment, the email comment from Jean Public (Appendix F1.k) warranted only a generic response from FNS (Appendix F2.k) because the comments were not germane to the collection.

None of the comments resulted in changes to the study protocol or any of the data collection instruments.

Consultations outside the agency

In addition to soliciting comments from the public, FNS consulted with Prakash Adhikari from National Agricultural Statistics Service (NASS) for expert consultation about the availability of data, the design, level of burden, and clarity of instructions for this collection. NASS comments and the response to NASS comments are provided in Appendices G2 and G3, respectively.

Individuals were consulted about the burden estimate and other characteristics of the data collection (i.e., clarity of instructions, clarity of questions). Three SFA directors who consulted on the study during pretesting gave permission to FNS to publish their names and contact details: Amy Chappell, Owen County Schools (KY), amy.chappell@owen.kyschools.us, 502-484-3934; Jud Crane, Santa Rosa County School Board (FL), cranej@santarosa.k12.fl.us, 850-983-5000; and Bobbie Dawson, Washington County School Board (FL), bobbie.dawson@wcsdschools.com, 850-638-6222. Feedback from the pretest participants was then used to refine and finalize the data collection instruments, as summarized in the Procurement Practices – Final Memorandum on Pretest Results in Appendix I.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Study respondents will not receive any payments or gifts.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

FNS complies with the Privacy Act of 1974 (5 USC §552a). No confidential or personally identifiable information will be requested, reported, or maintained as a result of the data collection activities. None of the forms included in this collection require a Privacy Act Statement. The information gathered in this study will be kept private to the full extent allowed by law. Access to records is limited to those persons who process the records for the specific uses stated in the Privacy Act of 1974. The information will be kept private and will not be disclosed to anyone but the individuals conducting research in this investigation, except as otherwise required by law.

All data collected from the study will be reported in aggregate form so that it cannot be linked back to any individual responses. No names, phone numbers, or any other unique identifiers will be linked to the data or included in any public use data sets or reports. To ensure that SFA identifying information remains private, the contract executed between FNS and 2M Research (2M) requires that 2M create and keep data on secure networks and utilize data collectors that sign confidentiality agreements (Appendix J) binding them to protect private information. Each responding SFA will be assigned a unique ID number and the data will be provided to FNS by this ID number. Additionally, data are password-protected. A separate file associating the ID number with the responding SFA (identifiable by SFA, not individuals) will also be shared with FNS. FNS will keep this file private. Once the contract is over, the Study Team will destroy all files with SFA identifying information.

FNS published a system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on April 25, 1991, volume 56, pages 19078–19080, that discusses the terms of protections that will be provided to respondents.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

This information collection does not contain questions of a sensitive nature.

12. Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

Appendix G1 and Table A.2 show the estimates of the respondent burden for the proposed data collection, including the number of respondents, frequency of response, average time to respond, and annual hour burden. With this submission, there are 760 respondents (620 respondents and 140 nonrespondents); 5,813 responses (3,779 from respondents and 2,034 from nonrespondents); and 1,268.47 burden hours (1,207.01 for all respondents and 61.46 for all nonrespondents).

These estimates reflect guidance from program officials, the agency’s prior experience in collecting data, and the pretesting of instruments and protocols. Burden has been minimized to the extent possible. The web survey will be programmed to auto-edit questions based on responses and skip patterns. The IDIs will be based on the procurement practices used by SFAs, as indicated in the web survey. As a result, SFAs with fewer different procurement practices (i.e., no FSMCs or group purchasing agreements) in place will receive fewer questions. For example, the Special Nutrition Program Operations Study (SN-OPS), School Year 2013–14 Report⁷ indicated that only 21 percent of SFAs used an FSMC in SY 2013–14. As a result, it is anticipated that the majority of SFAs will not complete the FSMC components of the web survey and IDI.

The survey and interview instruments have been designed to be completed solely by the SFA directors, the only local government respondents included in this collection. However, they are allowed to reach out to their colleagues for assistance.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

The estimates of annualized costs to State and local governments are based on the burden estimates and utilize the U.S. Department of Labor, Bureau of Labor Statistics, May 2017 National Occupational and Wage Statistics for Occupational Groups 999200: State Government (https://www.bls.gov/oes/current/naics4_999200.htm) and 611000: Educational Services (http://www.bls.gov/oes/current/naics3_611000.htm). Annualized costs are based on the mean hourly wage for each job category. The estimated annualized cost for State government, which includes State CN directors (Occupation Code 11-9030), is $228.94 ($44.89/hr. x 5.10 hours). The estimated cost of this data collection for local government, which includes SFA directors (Occupation Code 11-9039), is $58,670.90 ($46.44/hr. x 1,263.37 hours). The estimated annualized total cost to respondents is $58,899.70.⁸

Table A.2. Estimates of Respondent Burden

Table A.3. Annualized Cost to Respondents

13. Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The total cost to the Federal Government is $1,060,996.94 over a period of 3.67 years (September 2016–May 2020), resulting in an annualized cost to the Government of $289,099.98. The largest cost to the Federal Government is to pay the Study Team $1,037,290.14 over a 44-month period to conduct the study and deliver data files and reports. This represents an average annualized cost of $282,640.37 for Study Team labor, other direct costs, and indirect costs. This information collection also assumes a total of 520 hours of federal employee time annually for a GS-12, Step 6 at $45.59 per hour, for a total cost of $23,706.80 annually. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2018 for the Washington, DC locality.⁹

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.

This submission is a new information collection request as a result of program changes and will add approximately 1,268.47 burden hours and 5,813 responses to OMB’s information collection inventory.

16. For collections of information whose results are planned to be published, outline plans for tabulation and publication.

Data collection will begin following OMB approval and run for approximately 31 weeks. Data file preparations will follow immediately so that data analysis can begin soon after data collection is complete. The draft final report will be submitted to FNS approximately 53 weeks after OMB approval, and presentation materials will be submitted 62 weeks after OMB approval. There will be a final report containing the results from both the web survey and the IDIs.

Table A.4. Data Collection Schedule

Activity	Due Date
Pretest of web survey and IDI guide	June–August 2017
Email Notification sent to State CN directors	One week post-OMB approval
Pre-Survey Notification Letter sent to SFA directors	Two weeks post-OMB approval
Survey Notification Email sent to SFA directors	Three weeks post-OMB approval
Web survey data collection	Starting three weeks post-OMB approval and lasting 31 weeks
Survey Reminder Emails sent to SFA directors	Starting four weeks post-OMB approval
Post-Survey Response Clarification Email (web survey)	Starting five weeks post-OMB approval
Post-Survey Response Clarification Phone Call (web survey)	Starting five weeks post-OMB approval
Post-Survey Thank You Email	Starting four weeks post-OMB approval
Pre-Interview Notification Letter (IDI)	Starting seven weeks post-OMB approval
Pre-Interview Scheduling Phone Call (IDI)	Starting eight weeks post-OMB approval
Pre-Interview Reminder Email (IDI)	Starting 10 weeks post-OMB approval
Participant Confirmation Email (IDI)	Starting nine weeks post OMB approval
Thank-You Email for Participation in Study	Starting 10 weeks post-OMB approval
Data file preparation	39 weeks post-OMB approval and lasting 43 weeks
Data analysis and reporting	39 weeks post-OMB approval and lasting 43 weeks
Publication of findings	May 2020

a. Analysis of the web survey data

Web survey data will be used to identify which SFAs have characteristics for which further information about their procurement practices is desired. Such characteristics will include the extent to which the SFA uses certain procurement methods, and the motivating factors and circumstances that influenced their choice of methods. Once these characteristics are identified, the Study Team will select a sample of SFAs for IDIs. Selection of SFAs for IDIs will be purposive in nature, based on answers to specific web survey questions. Overall, this process will ensure IDIs are conducted on SFAs with a range of procurement practices of interest to FNS. IDI selection will also take into account SFA demographic and geographic characteristics to ensure representativeness. For example, the Study Team will ensure that SFAs from urban, suburban, and rural areas are included for the IDIs.

The analysis of quantitative data will be cross-sectional in nature and provide a “snapshot” of SFA procurement practices. The Study Team will assess SFA-level nonresponse (a.k.a. unit nonresponse) and conduct nonresponse bias analysis if needed. If the unit nonresponse rate is greater than 20 percent (OMB requires an 80 percent or greater response rate),¹⁰ the Study Team will conduct a nonresponse bias analysis to assess if the missing pattern is random by comparing the key variables in the collected data against the most recent SFA demographics. If significant differences are found, the Study Team will make sample adjustments (e.g., reweighting and sample raking) to compensate for the potential bias resulting from the unit nonresponse. The web survey data will be weighted to account for the stratified random sampling design and sample release, as well as to adjust for nonresponse. The data will also be calibrated to population control totals in order to produce nationally representative estimates. Under this approach, the responding cases will be weighted by the inverse of the predicted probability of response, using a weighting class methodology that divides the propensity scores into classes and assigns the average score within the class to each case.

Descriptive statistics, including frequency distributions and cross tabulations, will be generated for each research question. The analysis will include descriptive statistics for the web survey responses, including comparisons (e.g., statistical tests of differences) between SFAs, by procurement approach (i.e., procurement model), and other demographic variables of interest such as geographic region or size of the SFA. Given the complex nature of the sample design and the estimates, the Study Team proposes to utilize a jackknife variance replication method¹¹ to simplify the computation of the statistical significance of the descriptive statistics. This method will also account for the nonsampling error associated with the nonresponse adjustment and any subsequent post-stratification or calibration.

b. Analysis of the IDI data

The objective of the qualitative analysis is to enhance the information obtained through the web survey data and expand upon the descriptive summaries discussed above. The qualitative analysis will result in narrative summaries and illustrative quotes that further describe the similarities and differences in procurement practices between SFAs in the various procurement models.

After coding all the interviews in a qualitative analysis software (NVivo), analysts from the Study Team will review the coded data and identify themes, a process that focuses on discovering similarities, differences, and patterns within the data.¹² Distilling the data into themes will help provide answers to the research questions and bring depth to FNS’s understanding of the different procurement models that the Study Team has established over the course of this project. By identifying similarities and differences, the data can be clustered into themes for each research question and across the procurement models.

Narrative summaries and illustrative quotes will answer the study research questions by describing, for example, the practices at each SFA type, operational and technical hurdles, decision-making strategies, and other factors that influence their procurement practices. For example, FNS is interested in the various CPAs and how they differ, how the competitive procurement process is conducted for cooperative purchasing, and improvements to program efficiency. To address this interest, the Study Team will identify various structures for the agreements and similarities and differences in how the competitive procurement process is conducted within and across these SFAs by examining these emerging themes. They might also analyze SFAs that reported not participating in cooperative purchasing as a contrast; similar to analysis of SFAs that use cooperative purchasing, they will look for any commonalities among this group or contextual factors related to SFA decisions.

To confirm the findings from the qualitative analysis, the Study Team will triangulate, where possible, IDI results with those from the web survey. This process will likely help the Study Team identify instances where they should review qualitative findings, confirming that they are fully supported by the content of the coded interviews and that the analysis is comprehensive and well-developed. The Study Team’s approach to triangulation and mixed-methods interpretation is described below. Text segments will be exported from NVivo to Excel for ease of viewing, for comparison with the quantitative data for similarities and differences, and to facilitate analysis.

An important aspect of the analysis will be to relate findings to the different procurement models to be developed from responses to the CN-OPS-II Year 2 survey using cluster analyses as described in Part B. For example, it is possible that SFAs’ use of cooperative purchasing and competitive procurement themselves could be a major or minor characteristic of one or more procurement models. Relationships between the research findings and procurement models—whether single or multiple models—will be discussed throughout the final report.

c. Integration of quantitative and qualitative data

A key part of the analysis will be to synthesize qualitative and quantitative results to answer research questions that are addressed through both the web survey and the IDIs. This integration is the culmination of the integrative mixed-methods design as noted in section A2 (above) and described in more detail in Part B, in which survey and interview data are brought together to establish a set of interdependent findings to address the study’s common research questions. The mixed-method analysis will be based on a convergent mixed-methods design, which entails survey and interview data collection and analysis during a similar timeframe, followed by an integrated analysis. The integration will produce “a whole through integration that is greater than the sum of the individual qualitative and quantitative parts.”¹³

d. Methods of dissemination

The Study Team will produce a final report that will be made available on the FNS website.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency will display the expiration date for OMB approval of the information collection on all instruments and recruitment materials.

18. Explain each exception to the certification statement identified in Item 19 "Certification for Paperwork Reduction Act."

There are no exceptions to the certification statement. The agency is able to certify compliance with all provisions under Item 19 of OMB Form 83-I.

1 Richard B. Russell National School Lunch Act (2014), 42 U.S.C. 1751, as amended through P.L. 113­‑179. Retrieved from: http://www.fns.usda.gov/sites/default/files/NSLA.pdf

2 U.S. Department of Agriculture. (2017, June 30). Compliance with and enforcement of the Buy American Provision in the National School Lunch Program. (Memorandum SP 38-2017). Retrieved from: https://fns-prod.azureedge.net/sites/default/files/cn/SP38-2017os.pdf

3 Generally, an SFA conducts the procurement of food; however, there are a few SAs that procure goods and services on behalf of SFAs. In some States, the State Distributing Agency conducts procurement for processors for USDA Foods processed end-products on behalf of SFAs.

4 Professional Standards for State and Local School Nutrition Programs Personnel as Required by the Healthy, Hunger-Free Kids Act of 2010, 7 C.F.R. § 210 and 235 (2015).

5 HHFKA of 2010 states: “States, State educational agencies, local educational agencies, schools, institutions, facilities, and contractors participating in programs authorized under this Act and the Child Nutrition Act of 1966 (42 U.S.C 1771 et seq.) shall cooperate with officials and contractors acting on behalf of the Secretary, in the conduct of evaluations and studies under those Acts.”

6 The estimated number of respondents assumes an 80 percent response rate for each instrument. Thus, the total sample size for the web survey is 700 SFA Directors (700 x .80 = 560), and the total sample size for the IDIs is 125 (125 x .80 = 100).

7 Murdoch, J., Campbell, A., Condon, E., Fox, M. K., Harrison, R., Miller, M., . . . Shen, Y. (2014). Special Nutrition Program Operations Study (SN-OPS) SY 2013–14 report. U.S. Department of Agriculture. Retrieved from https://www.fns.usda.gov/sites/default/files/ops/SNOPSYr3.pdf

8 The sum of the State and local government cost estimates listed here is $58,899.84 ($228.94 + $58,670.90) and differs slightly from the estimate shown in the burden table ($58,889.70; Appendix G1) due to rounding. To minimize confusion, the estimate of the total annualized cost to respondents has been adjusted here to match the estimate shown in the burden table.

9 Office of Personnel Management. (2018). Salary table 2018-GS. Retrieved from https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/DCB_h.pdf

10 See, Guideline 1.3.4 in Office of Management and Budget. (2006). Office of Management and Budget standards and guidelines for statistical surveys. Washington, DC: Author. Retrieved from https://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf

11 Shao, J., & Wu, C. J. (1989). A general theory for jackknife variance estimation. Annals of Statistics, 17(3), 1176–1197.

12 Crabtree, B., & Miller, W. (1999). Using codes and code manuals: A template organizing style of interpretation. In B. F. Crabtree & W. L. Miller (Eds.), Doing qualitative research (2nd ed.) (pp. 163–177). Thousand Oaks, CA: Sage Publications.

13 Fetters, M. D., & Freshwater, D. (2015). The 1 + 1 = 3 integration challenge. Journal of Mixed Methods Research, 9(2), 115–117. doi: 10.1177/1558689815581222