Supporting Justification for OMB Clearance for the National School Lunch Program and School Breakfast Program Access, Participation, Eligibility, and Certification Study II (APEC-II Study)
Part A
Final
April 23, 2012
Project Officer: Reneé Arroyo-Lee Sing
CONTENTS
PART A. JUSTIFICATION 1
1. Circumstances Making the Collection of Information Necessary 1
2. Purpose and Use of the Information 2
3. Use of Information Technology and Burden Reduction 7
4. Efforts to Identify Duplication and Use of Similar Information 8
5. Impacts Small Businesses or Other Small Entities 9
6. Consequences of Collecting the Information Less Frequently 9
7. Special Circumstances Relating to the Guideline of 1320.5(D)(2) 10
8. Comments in Response to the Federal Register Notice and Efforts to Consult with Persons Outside the Agency 10
9. Explanation of Any Payments or Gifts to Respondents 11
10. Assurances of Confidentiality Provided to Respondents 12
11. Justification for Sensitive Questions 14
12. Estimates of Hour Burden Including Annualized Hourly Costs 15
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers 18
14. Annualized Cost to the Federal Government 18
15. Explanation for Program Changes or Adjustments 18
16. Plans for Tabulations and Publication and Project Time Schedule 19
17. Display of Expiration Date for OMB Approval 22
18. Exception to the Certification Statement Identified in Item 19.0 of Form OMB 83-1 22
REFERENCES 23
TABLES
Table A12.1. Reporting Estimates of Hour Burden and Annualized Cost 16
ATTACHMENT A: Tables and Figures
ATTACHMENT B: School District and SFA Contact Documents
ATTACHMENT C: School food authority (SFA) Survey (sfa district director questionnaire)
ATTACHMENT D: SFA Reimbursement Claims Data Forms
ATTACHMENT E: Application Data Abstraction Form
ATTACHMENT F: Certified and Denied Applicant Sampling Forms
ATTACHMENT G: CEO Student Sampling
ATTACHMENT H: School Meal Count Verification Forms
ATTACHMENT I: Meal Transaction Observation Form
ATTACHMENT J: Changes in Student Certification and Enrollment Form
ATTACHMENT K: Participation Data Protocol
ATTACHMENT L: Household Contact Documents
ATTACHMENT M: Household Survey
ATTACHMENT N: Mathematica Confidentiality Pledge
ATTACHMENT O: State Contact Documents
ATTACHMENT P: PUBLIC COMMENTS
ATTACHMENT Q: RESPONSE TO PUBLIC COMMENTS
PART A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
This is a reinstatement with change of previously approved collection for which approval has expired. The school meal programs administered by the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA) are a cornerstone of the nation’s nutrition safety net for low-income children. FNS has long been committed to ensuring that the meals provided in schools are healthful and contribute to children’s dietary requirements.
To preserve the integrity of the programs, FNS has conducted research and developed policies and initiatives to ensure that meal benefits reach eligible students and districts receive their correct reimbursements. The first Access, Participation, Eligibility, and Certification (APEC) study (OMB Number 0584-0530; Expiration Date August 31, 2008), conducted by Mathematica Policy Research for FNS and released in November 2007 (http://www.fns.usda.gov/ora/menu/published/CNP/FILES/apecvol1.pdf) to comply with the Improper Payments Information Act (IPIA) of 2002, provided the first reliable national estimates of erroneous payments made to school districts for the national school lunch program (NSLP) and the school breakfast program (SBP), covering school year (SY) 2005–06, as a result of both certification and noncertification errors. APEC found that erroneous payments in the school meal programs were significant as defined by the Office of Management and Budget (OMB). APEC also developed models that FNS could use to project erroneous payments on an annual basis.
In response to APEC’s findings, FNS initiated measures to improve Federal and State oversight and technical assistance to identify, recover, and reduce erroneous payments in the school meal programs. Other important changes in the school meal programs have occurred through FNS’s work with Congress. The 2004 Child Nutrition Reauthorization Act and the Healthy, Hunger-Free Kids Act of 2010 (1) added several regulations and changed some existing ones in an effort to improve the accuracy of the certification process; and (2) required other actions to help FNS reduce improper payments.
Given the increased attention to improper payments, as well as concerns that the econometric model developed under APEC to project erroneous payments may no longer be useful as a result of changing policies and procedures, FNS is seeking to renew OMB Clearance 0584-0530 granted in August 2005 to conduct the National School Lunch Program and School Breakfast Program Access, Participation, Eligibility, and Certification Study-II (APEC-II Study), a replication of the first APEC study, for SY 2012–13. The study includes several objectives (detailed in Table B2.1 in Attachment A) addressed in APEC, but expands its scope via (1) an examination of alternatives for producing State-level models; and (2) the inclusion of a recently introduced provision, the Community Eligibility Option (CEO).
2. Purpose and Use of the Information
Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.
To comply with the Improper Payments Information Act (IPIA) of 2002 (P.L. 107-300) and Executive Order 13520, USDA must report on the prevalence of erroneous payments in the NSLP and SBP on an annual basis, and if erroneous payments are significant, take actions to reduce improper payments and report on the efficacy of those actions. APEC-II will produce national estimates of erroneous payments for SY 2012–13 based on new collection of primary data. In addition, as it is cost prohibitive and infeasible to conduct a large nationally representative study on a yearly basis, we will use these data to update and improve upon the models developed in APEC and used annually by FNS to generate estimates of erroneous payments in years it does not collect national data. Therefore, as in the first APEC study, we propose to address APEC-II study objectives using a multistage–clustered sample design, which will include representative samples of school districts, schools (public and private), applicants for free and reduced-price meals, and directly certified students participating in the NSLP and SBP in the contiguous 48 States and the District of Columbia. We will collect data for each of these samples from several sources, including household surveys, abstraction from applications and direct certification records, observation of cashier transactions and collection of administrative records on school and school food authority (SFA) meal counts and claiming data, and FNS administrative data. We will collect similar data from SFAs and schools participating in the CEO. Table B2.1 (Attachment A) summarizes the overall research design, explicitly linking the proposed research plans to the objectives. Figure B2.1 (Attachment A) summarizes the sample design.
On behalf of FNS, the information for the APEC-II study will be collected and analyzed by Mathematica and its subcontractors Westat, Inc., and Decision Information Resources (DIR). Table B2.2 (Attachment A) summarizes our data collection plan. The table shows, for each data collection, the mode, respondent, target number of completed interviews and response rates, and key data elements to be collected.
School Food Authority Survey (SFA District Director Questionnaire) (Attachment C). A self-administered questionnaire will be collected from a representative sample of 175 SFAs selected from all SFAs in public and private school districts that participate in the NSLP and SBP. The questionnaire will collect information on the characteristics of the sampled SFA and on selected characteristics of the schools sampled (usually three schools per SFA). For the respondent, we will target the person who knows the most about the district’s administrative practices regarding the school meal programs—typically the district’s food service director. In January and February 2013, we will send the self-administered questionnaire to the SFA director. The questionnaire will be completed on paper and returned to Westat with a self addressed stamped envelope or by fax. The questionnaire contains quantitative questions that will require the SFA director to look at reports or other sources to respond. If necessary, we may conduct a short telephone interview after we receive and review the questionnaires. There is no additional survey instrument associated with the telephone follow-up, as it will focus on missing or ambiguous information provided in each SFA’s self-administered questionnaire. The telephone follow-up will be approximately 10 minutes and will collect clarifying information.
Household Survey (Attachment M). Mathematica field staff will administer in-person interviews to parents or guardians of children in our scientifically selected, representative samples of certified free/reduced-price students (3,835) and denied applicants (585 students) from 130 districts and 390 schools during SY 2012–13. The household survey will collect information on (1) household composition (who lives with the sampled student); (2) income from jobs for each household member and income from sources other than employment; (3) the household’s participation in SNAP, FDPIR, and TANF; (4) sampled students’ participation in the NSLP and SBP; (5) parents’ and children’s perceptions of meal program quality; and (6) demographic information about the student and household.
Mathematica staff will obtain lists of students by certification status from SFAs in order to draw the samples of students and households. Based on program regulations (Section 305 of the Healthy, Hunger-Free Kids Act), we will not need to obtain consent to obtain students’ meal program applications and direct certification documents, their certification status, and changes in certification during the course of the school year. In addition, to the extent a school district requires further legal authority for disclosure of information and records without parental consent, we will provide support for disclosure as being in compliance with the new FERPA regulation, published on December 2, 2011, at 76 FR 75604-1 amending the Family Education Educational Rights and Privacy Act, 34 C.F.R. 99, (New FERPA Amendment). During the district recruiting calls, the evaluation team will provide information to the school districts regarding legal authority and support for their disclosure of the requested information and records without parental consent and determine whether the district agrees disclosure is appropriate and, if so, negotiate appropriate non-disclosure agreements with each district. If the district does not agree, we will then identify the type of consent needed for the release of information recorded on the application forms, changes in meal certification status, changes in enrollment status, and program participation at schools that keep participation information on individual students during the school year.
We will send advance letters (Attachment L) to sampled students’ parents and contact them to both conduct the household survey and, if required, obtain permission (by passive or active method as discussed with the relevant district) to abstract student information recorded on the application forms, changes in meal certification status, changes in enrollment status, and program participation during the school year. The advance letters also will address the protection of respondents’ privacy and note that participation will not affect certification for free or reduced-price meals or participation in any other program. Further, if parental consent is required by the district, the parental consent form (Attachment L), which the household respondent will receive a copy after signing, also addresses these issues.
The purpose of the in-person interview is to obtain accurate, documented income amounts for each student’s family in order to measure the household’s true eligibility status. This information will be used to estimate overall certification error as well as certification error due to household misreporting and to administrative error. For households that applied for the school meal programs—certified and denied applicant households—these data will be compared with information on the household’s school meal application and the SFA’s certification decision to assess the prevalence of certification error and the amounts of erroneous payments and their sources.
Abstraction of Meal Program Applications, Program Participation Data, and Other Data on Students. Field interviewers will collect data appearing on the meal program benefit applications for the free and reduced-price certified student and denied applicant samples. The forms used for abstracting data from meal program applications are included in Attachment E. We will also collect data on meal program participation (Attachment K) for sampled students in the free and reduced-price meal samples, as well as for the denied applicants in districts and schools that compile and keep such individual data. Wherever possible, we will get participation information covering the entire school year. We will collect the data in two waves: one for the first semester and one for the second semester. We anticipate that at least half of the districts will track participation at the individual level and most will do so electronically. In those cases, we will request copies of relevant data files. Some schools may keep paper records or may not be able to provide electronic data, and will instead provide paper printouts. We will convert the data on these printouts into electronic data files and merge them into our analytic files. Finally, Mathematica central office staff will contact districts just prior to the end of the school year and request information regarding any change in certification status or enrollment for students in the research sample (Attachment J).
Meal-Counting and -Claiming Data Collection. Field interviewers will collect data to measure error that occurs after eligibility is determined, which consists of two main sources: (1) cashier error and (2) counting and reimbursement claiming error. Through observation of cashier transactions, field interviewers will collect information on the degree of accuracy with which cashiers classify meals as reimbursable. Field interviewers also will collect information on each school’s breakfast and lunch counts and claims made to SFAs for meals served and in turn how SFAs consolidate and report the schools meal counts and claims they receive to State agencies for reimbursement. Forms used to record cashier transactions are included in Attachment I. Forms used to record counting and claiming are included in Attachment D for data collected from the SFA and Attachment H for data collected from schools. The respondent is asked to record the counting and claiming data by target week and target month (Attachment D1-D2 and H2-H3). Optional reporting forms are available to report the meal count data in a daily format (H1) and the reimbursement claim data as a consolidation of all available data (D3) based upon the school or SFA’s available data format. Additionally, for schools that use supplemental reporting for school meal count verification, the supplemental reporting data is entered on the Meal Count Form – Supplemental Form (H4). These data are necessary to estimate erroneous payments due to noncertification error, and will be combined with the estimates of erroneous payments due to certification error, appropriately adjusted, to derive measures of total erroneous payments for SY2012–13.
Collecting Administrative Data for Modeling Erroneous Payments in Future Years. To support the enhancement of APEC’s models for estimating erroneous payments in future years, we will collect data from several administrative sources, including district-level administrative data from the SFA Verification Summary Reports (Form FNS-742; OMB number 0584-0026; Expiration Date March 31, 2013), other district-level administrative data from State child education/nutrition agencies, public school district-level data from the Common Core of Data (CCD) and decennial census, and private school-level data from the Private School Survey (PSS). The protocols used to obtain these data from State child education/nutrition agencies are included in Attachment O.
3. Use of Information Technology and Burden Reduction
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.
FNS is committed to complying with the E-government Act, 2002 to promote the use of technology. The information to be collected for this study will come from existing records and data, surveys, and interviewer observations of meal transactions and meal count and reconciliation activities during school visits. Wherever possible, improved technology has been incorporated into the data collection to reduce respondent burden. Information that is available to the contractor from a centralized source has not been included in the data collection instruments. Electronic mail will be used, when possible, to send reminders and other communications to district and school staff. (Advance letters and emails are in are in Attachment B.)
All in-person interviews with respondents will use computer-assisted personal interviewing (CAPI) technologies (see table below). Use of CAPI will make possible accurate skip patterns, customized wording for State-specific TANF and SNAP names and income reference periods, response code validity checks, and consistent checking and editing, all of which improve the pace and flow of the interviews and thus reduce respondent burden. We will also be prepared to handle the provision of school and district records electronically, in place of the production of hard-copy documents or completion of specific forms.
In total, 80% of the responses will be collected electronically (6414 responses out of 7974). The following instruments will be collected electronically: C - SFA District Director Questionnaire; Telephone follow-up Interview for Survey Data Clarification; D - SFA Reimbursement Claim Verification Forms; F - Certified and Denied Applicant Sampling; G - CEO Student Sampling File Documentation Form; H - School Meal Count Verification Form; J - Changes in Student Certification and Enrollment Form; K -Participation Data Protocol; M - Household Survey; and O.3-Administrative Data Items Being Requested from State Agencies.
4. Efforts to Identify Duplication and Use of Similar Information
Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.
Every effort has been made to avoid duplication of data collection efforts. These efforts include a review of USDA reporting requirements, State administrative agency reporting requirements, and special studies by government and private agencies.
FNS has the responsibility for administering the USDA school meal programs. It funds State agencies which, in turn, fund local SFAs. Within this structure, LEAs are responsible for certification and verification activities and SFAs are responsible for food service delivery. SFAs report on their activities to the State agency, which reports to FNS by way of seven regional offices. Other than extant, district-level administrative data from the SFA Verification Summary Reports (Form FNS-742), public school district-level data from the CCD and decennial census, and private school-level data from the Private School Survey, the information required for this study is not currently reported to FNS on a regular basis in a standardized form nor available from any other previous, contemporary study.
5. Impacts Small Businesses or Other Small Entities
If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The sample of SFAs and schools will include private schools proportional to their population among all participating schools in the NLSP and SBP. Because these schools will be part of the national estimates of error rates and erroneous payments, it is important that a consistent methodology be employed across the full sample, including private schools. Therefore, no methods to minimize burden are specifically being used with these entities. Private schools on average are smaller than public schools and most typically are smaller SFAs (often constituted of a single school). By virtue of their being small schools and SFAs, the burden associated with completing the SFA survey and reimbursement claims data may be relatively less, due to simpler record look ups and data compilation, and they are likely to have less students sampled compared to an SFA with multiple schools in the study.
6. Consequences of Collecting the Information Less Frequently
Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
If this data collection were not performed, USDA would be prevented from meeting its federal reporting requirements under IPIA to annually measure erroneous payments in the NSLP and SBP and identify the sources of erroneous payments.
Most data being collected for the study involve a one-time data collection with no repetition. SFAs will be contacted additional times during data collection in order for a sample of newly certified students to be selected later in the school year that would not have been eligible for selection at the initial point of sampling at the beginning of the school year, to learn of changes in students’ certification and enrollment during the school year, and to provide NLSP and SBP participation data that spans the entire school year for students participating in the study.
7. Special Circumstances Relating to the Guideline of 1320.5(D)(2)
Explain any special circumstances that would cause an information collection to be conducted in a manner:
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.
The proposed data collection is consistent with the guidelines set forth in Section 1320.5(D)(2). As discussed in Part B of this OMB supporting statement, the selection of SFAs to be included in the study is designed to provide a nationally representative sample of public and private SFAs, including a representative sample of SFAs participating in the CEO. Similarly, the selection of schools and students within these SFAs is designed to provide nationally representative samples.
8. Comments in Response to the Federal Register Notice and Efforts to Consult with Persons Outside the Agency
If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.
Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.
An announcement of FNS’s intent to seek approval to collect this information provided an opportunity for public comment on this study. This announcement, published in the Federal Register, Volume 77, Issue 27, February 9, 2012, pages 6776-6778, specified a 60-day period for comment ending April 9, 2012. Public comments and responses are included in Attachments P and Q, respectively.
Consultations about the research design, sample design, data sources and needs, and study reports occurred during the study’s design phase and will continue to take place throughout the study. The purpose of these consultations is to ensure the technical soundness of the study and the relevance of its findings and to verify the importance, relevance, and accessibility of the information sought in the study.
The OMB Supporting Statement was reviewed by:
Jennifer Maiwurm, Statistical Methods Branch, Statistics Division, USDA/NASS, 202-720-5467
Other individuals outside the agency who will review and comment upon key documents produced by the study are:
David Betson, Ph.D., Associate Professor of Public Policy and Economics, University of Notre Dame, 574-631-5068
Jay Breidt, Ph.D., Professor and Associate Chair of the Department of Statistics, Colorado State University, 970-491-5269
Marianne Bitler, Ph.D., Associate Professor, University of California, Irvine, 949-824-5606
9. Explanation of Any Payments or Gifts to Respondents
Explain any decision to provide any payment or gift to respondents, other than re-enumeration of contractors or grantees.
Permission is requested to offer a financial token of appreciation to promote cooperation and full participation in the household survey for the planned study. Sample members will be offered a $25 gift card upon completion of the in-person survey, including providing requested documentation. The token of appreciation will be cited in the advance letter sent to sampled households (Attachment L).
Research summarized by Singer and Kulka (2000) indicates that financial incentives can be effective. They conclude that they significantly reduce survey nonresponse and are cost-effective, lowering the overall cost and burden for most surveys. A similar payment offered to households completing a survey in the first APEC study yielded an average response rate of 83 percent. It is important for the quality and comparability of APEC-II for a similar response rate to be achieved.
10. Assurances of Confidentiality Provided to Respondents
Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
The information provided in this study, including through the collection NSLP and SBP applications (and direct certification documents), will be kept private to the extent allowed by law. Applications will contain private information, such as names, addresses, dates of birth, Social Security numbers, and program participation information.
More specifically, the application for school meal benefits requires the current income, the names of all household members and the social security number of the adult household number who signs the application or indication that such adult does not possess a social security number. Section 9(b) of the National School Lunch Act (Public Law 103-448) restricts the use or disclosure of any eligibility information to persons directly connected with the administration or enforcement of the program. It also authorizes States and local school food authorities to conduct verification of the eligibility for free and reduced price meals. The social security numbers may be used to identify household members in carrying out efforts to verify the correctness of information stated on the application.
Individuals participating in this study will be assured that the information they provide will not be published in a form that identifies them. No identifying information will be attached to any reports. Identifying information will not be included in any public use dataset. Names and phone numbers, or any other unique identifier, will not be linked to the data. Interview and observation respondents will be assigned a unique ID number and analysis will only be conducted on data sets that include these unique ID numbers. Records are kept in physically secured rooms and/or cabinets. Paper records are segregated and physically secured in locked cabinets. Electronic data will be maintained on secured, password-protected servers. Discarded materials containing private information will be shredded. Identifying information will be accessible only by approved contractor staff. Such accessibility is limited to those who have direct responsibility for providing and maintaining sample information.
APEC-II data will be processed and stored on the contractor’s password-protected local area network (LAN). The contractor, Mathematica, protects its LAN with several security mechanisms available through the network operating system. Access to private information stored on LAN directories is restricted to authorized project staff by means of identification and passwords. In addition, network servers containing private information are kept in a locked area. Access to names and phone numbers will be destroyed within 12 months of the end of the contract. Additionally, all permanent and temporary employees of Mathematica and subcontractors are required to sign a confidentiality agreement (Attachment N) emphasizing the importance of confidentiality and describing their obligation. Mathematica holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP). The FWA covers all federally supported or conducted research involving human subjects. Mathematica intends to submit all study instruments and procedures to an Institutional Review Board for the purpose of safeguarding research participants’ rights and welfare. IRB approval is anticipated prior to the distribution of study materials to potential respondents.
FNS does not have any connection to the personal data collected and will not handle any data containing identifying information. FNS’s aggregated report contains no personal information and is publicly posted. Data will be presented in aggregate statistical form only.
Participants in this study will be subject to assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. It also provides for the confidential treatment of records maintained by a Federal agency according to either the individual’s name or some other identifier. This requires that before personal identifying information (such as taxpayer identifying numbers) may be shared with other entities, a Privacy Act notice must first be published. Therefore, the Food and Nutrition Service published such a Privacy Act notice (system of records) to specify the uses to be made of the information in this collection. Access to records is limited to those persons who process the records for the specific uses stated in this Privacy Act notice. The information will be kept private and will not be disclosed to anyone but the individuals conducting research in this investigation, except as otherwise required by law. A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on March 31, 2000, Volume 65, Number 63, and is located on pages 17251-17252 discusses the terms of protections that will be provided to respondents.
11. Justification for Sensitive Questions
Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
With the exception of questions in the household survey about demographics and household composition, receipt of income by source for individual household members, and receipt of income to the household as a whole, the household surveys and interviews with SFAs do not involve questions of a sensitive nature. All respondents will be informed that they can decline to answer any question they do not wish to answer and that there are no negative consequences for not participating. Respondents will also be assured of privacy at the outset of the interview, and, if required by the district, receive a copy of their parental consent form which addresses the issue. All survey responses will be held secure; respondents’ answers will not be reported to school officials or any other program or agency, but will be combined with the responses of others so that individuals cannot be identified. FNS and the contractor will comply with the requirements of the Privacy Act. All the questions have been used extensively in previous surveys with no evidence of harm.
The following household survey questions may be considered sensitive items: Questions on income sources (Section H) and amounts (Section I); and questions on receipt of public assistance (Section J).
Questions about income and the receipt of public assistance are necessary to establish the family’s actual eligibility for free and reduced-price meal benefits. Without them, the study will not be able to compare students’ certification status with estimated eligibility status to estimate certification error and derive estimates of erroneous payments in the NSLP and SBP for SY 2012–13. Questions similar to those concerning income receipt by persons in the household and public assistance receipt by the household and questions requesting documentation of income reported have been used successfully in the first APEC study and multiple other FNS studies.
12. Estimates of Hour Burden Including Annualized Hourly Costs
Provide estimates of the hour burden of the collection of information. The statement should:
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.
Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.
A total of 5,363 burden hours and a total annualized cost to respondents of $97,953.13 are estimated for this study. Table A12.1 shows sample sizes, estimated burden, and annualized costs for each data collection component. All data collection is taking place during a single year, SY 2012–13. The estimates are based on prior experience with comparable instruments on APEC and a pretest of the household survey held in March 2012 (see Section B.4 for information on the pretest).
Table A12.1. Reporting Estimates of Hour Burden and Annualized Cost
Affected Public |
Respondent |
Estimated Number of Respondents |
Frequency of Responses |
Total Annual Responses |
Average Burden Hours per Response |
Total Annual Burden Estimate (hours) |
Total Annualized Cost of Respondent Burden |
State and local agencies |
School Food Service Directors |
|
|
|
|
|
|
SFA Survey |
|||||||
Completed |
160 |
1.00 |
160 |
1.500 |
240.000 |
$8757.60a |
|
Attempted |
9 |
1.00 |
9 |
0.083 |
0.747 |
$27.26a |
|
Telephone Interview for Survey Data Clarification |
|||||||
Completed |
160 |
1.00 |
160 |
0.167 |
26.720 |
$975.01a |
|
SFA Reimbursement Claims Data |
|||||||
Completed |
139 |
1.00 |
139 |
1.000 |
139.000 |
$5072.11a |
|
Certified and Denied Applicant Sampling |
|||||||
Completed |
124 |
2.00 |
248 |
0.250 |
62.000 |
$2262.38a |
|
CEO Student Sampling |
|||||||
Completed |
45 |
1.00 |
45 |
2.000 |
90.000 |
$3284.10a |
|
School Liaisons |
|||||||
School Meal Count Verification Form |
|||||||
Completed |
429 |
1.00 |
429 |
1.000 |
429.000 |
$15654.21a |
|
Changes in Student Certification and Enrollment Form |
|||||||
Completed |
384 |
1.00 |
384 |
0.500 |
192.000 |
$7006.08a |
|
Participation Data |
|||||||
Completed |
384 |
2.00 |
768 |
0.500 |
384.000 |
$14012.16a |
|
States |
|||||||
Program Data |
|||||||
Completed |
49 |
1.00 |
49 |
8.000 |
392.000 |
$15037.12c |
|
SA SUBTOTALS |
1,883.00 |
--- |
2,391.00 |
--- |
1,955.48 |
$72,088.03 |
|
Business- for/not-for Profit (Private Schools) |
School Food Service Directors |
|
|
|
|
|
|
SFA Survey |
|||||||
Completed |
6 |
1.00 |
6 |
1.500 |
9.000 |
$372.96d |
|
Attempted |
--- |
1.00 |
--- |
0.083 |
--- |
--- |
|
Telephone Interview for Survey Data Clarification |
|||||||
Completed |
6 |
1.00 |
6 |
0.167 |
1.002 |
$41.52d |
|
SFA Reimbursement Claims Data |
|||||||
Completed |
6 |
1.00 |
6 |
1.000 |
6.000 |
$248.64d |
|
Certified and Denied Applicant Sampling |
|||||||
Completed |
6 |
2.00 |
12 |
0.250 |
3.000 |
$124.32d |
|
School Liaisons |
|||||||
School Meal Count Verification Form |
|||||||
Completed |
6 |
1.00 |
6 |
1.000 |
6.000 |
$248.64d |
|
Changes in Student Certification and Enrollment Form |
|||||||
Completed |
6 |
1.00 |
6 |
0.500 |
3.000 |
$124.32d |
|
Participation Data |
|||||||
Completed |
6 |
2.00 |
12 |
0.500 |
6.000 |
$248.64d |
|
BUS SUBTOTALS |
42.00 |
--- |
54.00 |
--- |
34.00 |
$1,409.54 |
|
Households |
Households |
||||||
Household Survey |
|||||||
Pretest |
4 |
1.00 |
4 |
0.750 |
3.000 |
$21.75b |
|
Completed |
4,420.00 |
1.00 |
4,420 |
0.750 |
3315.000 |
$24033.75b |
|
Attempted |
1,105.00 |
1.00 |
1,105 |
0.050 |
55.250 |
$400.56b |
|
HH SUBTOTALS |
5,529.00 |
--- |
5,529.00 |
--- |
3,373.25 |
$24456.06 |
|
GRAND TOTAL |
7,454.00 |
--- |
7,974.00 |
--- |
5,362.72 |
$97,953.13 |
|
a North American Industry Classification System (NAICS) 999300: Local Government (SOC Code 11-0000) = $36.49/hour
b Federal minimum wage = $7.25/hour
c North American Industry Classification System (NAICS) 999200: State Government (SOC Code 11-0000) = $38.36/hour
d North American Industry Classification System (NAICS) 611100: Elementary and Secondary Schools, Privately Owned (SOC Code 11-0000) = $41.44/hour
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.
There are no capital or startup or ongoing operation or maintenance costs associated with this data collection.
14. Annualized Cost to the Federal Government
Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.
The total estimated cost of the study to the federal government is $8,324,235 over a period of four years (December 2011 through October 2015), resulting in an annualized cost of $2,081,059. This represents the contractor’s costs for labor, other direct costs, and indirect costs and includes the salary of the assigned FNS project officer. The cost of the FNS employee, Social Science Research Analyst/ Project Officer, involved in project oversight with the study is estimated at GS-13, step 1 at $44.23 per hour based off 2080 hours per year, totaling $36,799.36 annually from an estimated 832 hours per year. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2011 for the Washington, DC locality).
15. Explanation for Program Changes or Adjustments
Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.
This collection is a reinstatement of a previously approved information collection resulting in a program change of 5,363 burden hours. APEC-II includes several objectives addressed in APEC-I, but expands on that study via (1) State-level modeling (2) the inclusion of a recently introduced provision, the CEO, in the SY 2012–13 national estimate of erroneous payments.
16. Plans for Tabulations and Publication and Project Time Schedule
For collections of information whose results are planned to be published, outline plans for tabulation and publication.
Study Schedule. The planned schedule for the activities in the APEC-II study is as follows:
Project Activity |
Dates |
Select SFA and School Samples |
February 2012–August 2012 |
Train Field Interviewers and Coordinate Data Collection Schedule |
August 2012 |
Conduct Data Collection |
September 2012–July 2013 |
Prepare Data Files |
January 2013–September 2013 |
Analyze Data, Develop and Estimate Models, Prepare Final Reports and Journal Articles |
August 2013–October 2015 |
Analysis Plan. The APEC-II study will provide national estimates of erroneous payments due to certification and noncertification error made under NSLP and SBP based on primary data collection in SY 2012–13. The study’s national estimates of erroneous payments are comprehensive, fully accounting for the most utilized certification processes (applications and direct certification) as well as special provisions, for example, schools using Provisions 2 or 3 and districts and schools using the CEO. The study will provide improved, updated estimation models for FNS staff to use when annually updating erroneous payment estimates for NSLP and SBP using available extant data and will also produce State-level estimation models. FNS will use all estimates from APEC-II to meet its reporting requirements under the IPIA. Separate analyses will be conducted to meet the following study objectives.
Generate National Estimates of Erroneous Payments Due to Certification Error. We will produce, separately, national estimates of erroneous payments (overpayment, underpayment, and overall total) made under the NSLP and SBP in SY 2012–13 as a result of the misclassification of meal eligibility status of certified students and denied applicants who participate in these programs. We will calculate erroneous payments over a sample month based on number of meals consumed and a comparison of students’ certification and eligibility status and then compute a weighted sum of students’ monthly erroneous payments to generate a national estimate of erroneous payments over the full school year. We will address certification errors and erroneous payments separately for CEO schools because individual students are not certified in CEO schools.
Determining Sources of Erroneous Payments Due to Certification Error. After estimating total erroneous payments, we will estimate the proportion of erroneous payments due to two alternative sources: (1) administrative error by the LEA in processing applications and (2) household misreporting of income or other family circumstances on the application. We will use application and other SFA-compiled student information and certification status to determine whether or not any erroneous payments made for meals consumed by the student were due to administrative error by the SFA. To estimate the proportion of erroneous payments due to household misreporting of eligibility on the application, we will compare the household’s reported income, household size, and SNAP, TANF, and FDPIR status obtained from the household survey versus the information on their application.
Estimating Erroneous Payments Due to Certification Error for Districts and Schools Using Special Provisions. Most students are certified annually based on applications and direct certification processes. Exceptions are schools participating in Provision 2 and 3 and the CEO. In Provision 2 and 3 schools not in their base year, erroneous payments due to certification are the result of certification errors that occur during the application and direct certification process in the base year. As was done in APEC-I, we will impute the rates of erroneous payments in Provision 2 and 3 non–base year schools based on rates of erroneous payments in similar Provision 2 and 3 base year schools for which we collect certification and eligibility data using a simple regression modeling approach. The sum of erroneous payments from Provision 2 and 3 non–base year schools will be added to the national estimate of non-Provision 2 and 3 and Provision 2 and 3 base year schools, properly weighted to account for their representation in the meal programs.
For districts and schools participating in the CEO, the reimbursement for a CEO group is a weighted sum of payments for free meals and for paid meals calculated using the CEO group’s free claiming percentage (FCP), paid claiming percentage (PCP), and total reimbursable NSLP and SBP meals, counted separately. We will assess erroneous payments for CEO groups by taking the difference between the reimbursements claimed using their FCP and PCP and reimbursements suggested by our calculations of what the true rates should be. Summing total overpayments and underpayments will yield the estimate of overall erroneous payments for CEO groups.
Estimates of Erroneous Payments Due to Meal-Counting and -Claiming Errors. We will estimate meal-counting and -claiming errors—both amounts and sources— on a national level based on a sample of 130 SFAs and 390 schools. We will estimate errors at key functional points in the administrative process, including errors that cashiers make at the point-of-sale and three types of aggregation errors (errors in transcribing data, errors in totaling data from individual cash registers, and errors in districts’ claims to State agencies for reimbursement). Our basic approach to estimating these types of noncertification error will involve collecting data on each type of error separately for the NSLP and SBP at each school and district in our sample. We will derive estimates of error rates for each meal type (free, reduced-price, and paid) within each sampled school and an estimate of total dollar error for the school. Our national estimate is the sum of the dollar amount of error across all schools divided by the sum of the schools’ reimbursements for all meals, weighted to a nationally representative sample of the population of reimbursable meals.
Refine Existing Estimation Models. FNS staff have used models developed as part of APEC to generate annual estimates of overpayments, underpayments, and overall erroneous payments in the NSLP and SBP to meet reporting requirements to OMB in years in which primary data are not collected. APEC-II will update models of certification error, develop models to predict noncertification error, modify the models so that they are appropriate for State-level estimates, and test the models for accuracy.
Our methodological approach includes the following steps:
Estimate a district-level econometric model of error rates.
Apply the model to predict error rates for each district nationally.
Combine the error rates with administrative data to calculate district-level erroneous payments.
Aggregate the payments to the national or State level separately for the NSLP and SBP.
Specifying the model at the district level allows us to incorporate data available only at the district level that are likely to be highly predictive of erroneous payments.
17. Display of Expiration Date for OMB Approval
If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
The agency plans to display the expiration date for OMB approval of the information collection on all instruments.
18. Exception to the Certification Statement Identified in Item 19.0 of Form OMB 83-1
Explain each exception to the certification statement identified in Item 19 "Certification for Paperwork Reduction Act."
There are no exceptions to the certification statement.
REFERENCES
U.S. Department of Agriculture, Food and Nutrition Service, Office of Research, Nutrition and Analysis, NSLP/SBP Access, Participation, Eligibility, and Certification Study – Erroneous Payments in the NSLP and SBP, Vol. I: Study Findings, by Michael Ponza, et al. Project Officer: John R. Endahl. Alexandria, VA: 2007. http://www.fns.usda.gov/ora/menu/published/CNP/FILES/apecvol1.pdf
Singer, E., and R.A. Kulka. “Paying Respondents for Survey Participation.” Survey Methodology Program, No. 092. The University of Michigan, Institute for Social Research, Survey Research Center, May 2000. http://www.isr.umich.edu/src/smp/ Electronic%20Copies/92.pdf (accessed January 13, 2005).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Justification for OMB Clearance for the National School Lunch Program and School Breakfast Program Access, Participat |
| Author | Computer & Network Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |