Appendix G2 National Agricultural Statistics Service (NASS) Comments

April 9, 2018

Appendix G2. National Agricultural Statistics Service (NASS) Comments

Review of OMB Part A and B: Supporting Justification for OMB Clearance for the

Food and Nutrition Service’s Study of

School Food Authority (SFA) Procurement Practices

Conclusion:

The paper addresses and explains very clearly the purpose, expectation, process, cost and statistical methods used for the study of SFA procurement practices. All topics are well defined and supported by tables, examples and the pertinent information such as OMB control number, expiration date etc. The statistical methods used for the study are well detailed and - for the most part - justified. However, The docket uses an extreme amount of acronyms to the point that it is somewhat difficult to read. Please ensure that each one is clearly defined and acronyms are only introduced when necessary.

The purpose of the study is clearly described with detailed introduction and background. The data collection process is thoroughly presented with anticipated dates, cost and timeline. There are various methods proposed to increase the response rate such as following up and reminding the respondents (SFAs) throughout the survey. The sample size is also high enough in order to ensure desired sample size in each of the four major and four minor procurement models.

Overall, the study uses sound statistical techniques in sampling, data collection, stratification, estimation and degree of accuracy. Each of the statistical methods, except a few that need further clarification (comments below), are justified and seem appropriate.

Summary:

Part A: JUSTFICATION

A1 Circumstances, Legal and Administrative Requirements that necessitate the information collection

1. Introduction:

Good- Thoroughly provides and outlines the information to follow in the paper

2. Program Background:

Good- Well defined with examples of previous works and proper citation

A2 Usefulness of Information

1. How the Information will be used:

Good - the study will further enable FNS to pursue its mission to increase food security and reduce hunger

2. From whom the information will be collected:

Good - subsample of SFA directors who participated in Year 2 of the USDA/FNS Child Nutrition Program Operations Study

3. How the information will be collected:

Needs Clarification - Tables included to summarize the data collection plan for both web survey and the in depth interview (IDI). How was it determined that 10% of respondents would need clarification.

4. Frequency of information collected:

Good - One-time data collection, spanning 31 weeks

A3 Information shared with any other organizations inside or outside USDA or the government.

Needs Clarification-. On page 11, it is somewhat confusing when the number of responses were referenced (5,813). It seemed like the sample size had suddenly changed. Perhaps add more clarification to final paragraph on page 11.

A4 Efforts to identify duplication

Good- FNS claims to have every effort made to avoid duplication of previous study

A5 Methods used to minimize burden on small businesses or entities

Good- Only about 1 percent of the web survey respondents will be small entities and will not have a significant economic impact on them

A6 The consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequent

Good- Defines why the impact if the study were not to be conducted.

A7 Any special circumstances that would cause an information collection to be conducted in a manner

Not Applicable- There are no special circumstances

A8 Provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice

1. Federal Register notice and comments

Good- Published in the Federal Register on Tuesday, May 30, 2017

2. Consultations outside the agency

Good- FNS consulted with NASS for expert consultation about the availability of data, the design, level of burden, and clarity of instructions for this collection

A9 Explain any decision to provide any payment or gift to respondents

Not Applicable- Study respondents will not receive any payments or gifts

A10 Assurance of confidentiality provided to respondents

Good- No confidential information will be requested, reported, or maintained as a result of the data collection activities

A11 justification for any questions of a sensitive nature

Not Applicable- This information collection does not contain questions of a sensitive nature

A12 Estimates of the hour burden of the collection of information

1. The number of respondents, frequency of response, annual hour burden

Good- Provided in a table - 760 respondents, 5,813 responses and 1264.11 hours

2. Estimates of annualized cost to respondents for the hour burdens

Good- Provides the broken down cost of data collections and wage of state government – totals cost to respondents is $49,752.96

A13 Estimates of the total annual cost burden to respondents resulting from the collection of information

Not Applicable- There are no capital/start-up or ongoing operation/maintenance costs

associated with this information collection

A14 Estimates of annualized cost to the Federal government

Good- The total cost to the Federal Government is $1,122,633.79 over a period of 3.67 years

A15 Reasons for any program changes or adjustments reported

Good- addition of burden hours and responses to OMB’s information collection inventory

A16 Plans for tabulation and publication

Good- Well planned schedule and table provided for the following tasks:

1. Analysis of the web survey data

2. Analysis of the IDI data

3. Integration of quantitative and qualitative data

4. Methods of dissemination

A17 The reasons that display the expiration date for OMB approval would be inappropriate

Good- The agency will display the expiration date for OMB approval of the information collection on all instruments and recruitment materials

A18 Exception to the certification statement

Good- There are no exceptions to the certification statement

Part B: COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1 Describe the potential respondent universe and any sampling or other respondent selection method to be used

Needs Clarification- Well described with accompanying tables. Please include clarification on presence of nonresponse bias in first study that is being used to sample from for this survey. They claim they have shown a propensity to participate in their studies but are there any other fundamental differences between respondents and non-respondents.

B2 Describe the procedures for the collection of information

1. General data collection procedures

Good- Provided in a table

2. Statistical methodology for stratification and sample selection

Good- stratified sampling design with equal probability sampling of the SFAs within each stratum

3. Estimation procedure

Good- Use of sampling weights and nonresponse adjustments

4. Degree of accuracy needed for the purpose described in the justification

Good- Large sample size selected to achieve the desired precision

5. Unusual problems requiring specialized sampling procedures

Not Applicable – FNS does not anticipate any unusual problems

6. Any use of periodic (less frequent than annual) data collection cycles to reduce burden

Not Applicable – This is a one-time study

B3 Describe methods to maximize response rates and to deal with issues of nonresponse

Good- Maximize the response rate using follow up and reminders

B4 Describe any tests of procedures or methods to be undertaken

Good- Uses various sampling techniques to ensure proper representation of SFAs

B5 Provide the name and telephone number of individuals consulted on statistical aspects of the design

Good- Provided as a table

Reviewer:

Prakash Adhikari, Mathematical Statistician

Summary, Estimation, and Disclosure Methodology Branch

National Agricultural Statistics Service

United States Department of Agriculture

T: (202) 720 5467

Email: Prakash.Adhikari@nass.usda.gov