EPICCS OMB Part A (Revised Final 7-28-16)

Supporting Statement – Part A for

OMB Control Number 0584-XXXX

Erroneous Payments in Child Care Centers Study (EPICCS)

July 2016

Chanchalat Chanhatasilpa, Ph.D.

Social Science Research Analyst

Office of Policy Support

Food and Nutrition Service

United States Department of Agriculture

3101 Park Center Drive

Alexandria, Virginia 22302

Phone: 703-457-6791

Email: Chanchalat.Chanhatasilpa@fns.usda.gov

This page is intentionally blank

TABLE OF CONTENTS

PART A. JUSTIFICATION 1

1. Circumstances That Make the Collection of Information Necessary 1

2. Purpose and Use of the Information 2

3. Use of Information Technology and Burden Reduction 8

4. Efforts to Identify Duplication 9

5. Impacts on Small Businesses or Other Small Entities 10

6. Consequences of Collecting the Information Less Frequently 10

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 11

8. Comments to the Federal Register Notice and Efforts for Consultation 11

9. Explanation of Any Decisions to Provide Any Payment or Gift to Respondents 12

10. Assurances of Confidentiality Provided to Respondents 14

11. Justification for Any Questions of a Sensitive Nature 15

12. Estimates of the Hour Burden of the Collection of Information 16

13. Estimates of Other Total Annual Cost Burden 18

14. Estimates of Annualized Cost to the Federal Government 22

15. Explanation of Program Changes or Adjustments 22

16. Plans for Tabulations, and Publication and Project Time Schedule 23

17. Displaying the OMB Approval Expiration Date 25

18. Exceptions to the Certification Statement Identified in Item 19 25

TABLES AND FIGURES

Table A2-1 Summary of Data Collection Instruments

and Forms 5

Table A12-1 Estimates of Respondent Burden Including

Annualized Hourly Cost 19

Table A16-1 Proposed Study Schedule 23

LIST OF APPENDICES TO EPICCS OMB SUPPORTING STATEMENT

APPENDIX A: APPLICABLE STATUTES, REGULATIONS AND REFERENCE DOCUMENTS

A1. RICHARD B. RUSSELL NATIONAL SCHOOL LUNCH ACT (AS AMENDED THROUGH P.L. 113–79, ENACTED FEBRUARY 07, 2014)

A2. IMPROPER PAYMENTS INFORMATION ACT OF 2002 (P.L. 107-300)

A3. EXECUTIVE ORDER 13520—REDUCING IMPROPER PAYMENTS

A4. IMPROPER PAYMENTS ELIMINATION AND RECOVERY ACT (IPERA) OF 2010 (P.L. 111-204)

A5. IMPROPER PAYMENTS ELIMINATION AND RECOVERY IMPROVEMENT ACT (IPERIA) OF 2012 (P.L. 112-248)

A6. OFFICE OF INSPECTOR GENERAL (OIG) USDA’S F Y 2014 COMPLIANCE WITH IMPROPER PAYMENT REQUIREMENTS

A7. FNS FY 2014 RESEARCH AND EVALUATION PLAN

A8. HEALTHY HUNGER-FREE KIDS ACT 2010 (P.L. 111-296)

A9. M-15-02 – APPENDIX C TO CIRCULAR NO. A-123, REQUIREMENTS FOR EFFECTIVE ESTIMATION AND REMEDIATION OF IMPROPER PAYMENTS

APPENDIX B: STUDY MATERIALS

STATE STUDY NOTIFICATION AND ADMINISTRATIVE REQUEST

B1. STATE NOTIFICATION LETTER & ADMINISTRATIVE DATA REQUEST

B2. STATE INSTRUCTIONS FOR SUBMISSION OF ADMINISTRATIVE DATA

B3. STATE USER ACCOUNT SHEET

B4. GUIDELINES FOR THE EPICCS ADMINISTRATIVE DATA REQUEST

B5. CONFIRMATION EMAIL (CONFIRMING DATA FILE IS COMPLETE)

B6. STATE SAMPLE SELECTION NOTIFICATION LETTER (NOTIFYING STATE OF SAMPLED SPONSORS/CENTERS)

B7. TEMPLATE LETTER FROM STATE TO SPONSORS (NOTIFICATION OF SELECTION FOR STUDY)

SPONSOR/CENTER STUDY NOTIFICATION

B8. LETTER OF ENDORSEMENT

B9. SPONSOR NOTIFICATION LETTER

B10. INDEPENDENT CHILD CARE CENTER NOTIFICATION LETTER

B11. TEMPLATE LETTER FROM SPONSOR TO SAMPLED CENTER (NOTIFICATION OF SELECTION FOR STUDY)

B12. CENTER (SCCC & HSC) NOTIFICATION LETTER

B13. FACT SHEET (FOR SPONSORS AND CHILD CARE CENTERS)

B14. CALL GUIDE AND WORKSHEET FOR SPONSORS

B15. CALL GUIDE AND WORKSHEET FOR CENTERS

HOUSEHOLD RECRUITMENT-NAMES

B16. NAMES INTRODUCTORY LETTER

B17. NAMES INTRODUCTORY LETTER—SPANISH

B18. NAMES BROCHURE

B19. NAMES BROCHURE—SPANISH

B20. NAMES RECRUITMENT CALL SCRIPT

B21. NAMES RECRUITMENT CALL SCRIPT—SPANISH

APPENDIX C: DATA COLLECTION MATERIALS

C1. DATA COLLECTION SUMMARY

SPONSOR/CENTER DATA COLLECTION

C2. SPONSOR & CHILD CARE CENTER PRE-VISIT INTERVIEW

C3. CENTER ENROLLMENT FORM

C4. INCOME ELIGIBILITY APPLICATION ABSTRACTION FORM

C4a. ROUND 2 & 3 APPLICATION ABSTRACTION SCHEDULING CALL GUIDE

C5. MEAL OBSERVATION FORM – RESTAURANT/CAFETERIA STYLE

C6. MEAL OBSERVATION FORM – FAMILY STYLE

C7. EXAMPLE OF MEAL TRANSACTION SAMPLING

C8. CENTER ATTENDANCE FORM FOR TARGET MONTH

C9. CENTER MEAL COUNTS FOR TARGET MONTH

C10. (SPONSORED) CENTER MEAL COUNT CLAIM TO SPONSOR FOR TARGET MONTH

C11. SPONSOR MEAL CLAIM TO STATE FOR TARGET MONTH

C12. (INDEPENDENT) CENTER MEAL CLAIM TO STATE FOR TARGET MONTH

C13. SPONSOR SURVEY COVER LETTER

C14. CHILD CARE CENTER SPONSOR SURVEY

C15. HEAD START SPONSOR SURVEY

C16. EXTENDED ATTENDANCE DATA REQUEST

C17. MEAL COUNTS FOR OBSERVATION MONTH

C18. STATE MEAL CLAIM REQUEST

NAMES DATA COLLECTION

C19. NAMES SURVEY CONSENT FORM

C20. NAMES SURVEY CONSENT FORM—SPANISH

C21. NAMES APPOINTMENT REMINDER LETTER

C22. NAMES APPOINTMENT REMINDER LETTER—SPANISH

C23. NAMES SURVEY INCOME WORKSHEET

C24. NAMES SURVEY INCOME WORKSHEET—SPANISH

C25. NAMES SURVEY

C26. NAMES SURVEY—SPANISH

C27. NAMES SURVEY INCOME SOURCE SHOW CARD

C28. NAMES SURVEY INCOME SOURCE SHOW CARD—SPANISH

C29. NAMES SURVEY INCENTIVE RECEIVED FORM

C30. NAMES SURVEY INCENTIVE RECEIVED FORM—SPANISH

APPENDIX D: SUMMARY OF PUBLIC COMMENTS

APPENDIX E: RESPONSE TO PUBLIC COMMENTS

APPENDIX F: WESTAT CONFIDENTIALITY PLEDGE

APPENDIX G: WESTAT FEDERAL-WIDE ASSURANCE

APPENDIX H: WESTAT EPICCS IRB APPROVAL LETTER

APPENDIX I: WESTAT INFORMATION TECHNOLOGY AND SYSTEMS SECURITY POLICY AND BEST PRACTICES

APPENDIX J: EPICCS STATE SAMPLE SELECTION MEMO

APPENDIX K: EPICCS CENTER SAMPLE SELECTION MEMO #1

APPENDIX L: EPICCS CENTER SAMPLE SELECTION MEMO #2

APPENDIX M: EPICCS BURDEN TABLE

This page is intentionally blank

PART A. JUSTIFICATION

1. Circumstances That Make the Collection of Information Necessary

Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The Child and Adult Care Food Program (CACFP), administered by the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA), is authorized at section 17 of the National School Lunch Act (42 U.S.C. 1766) (Appendix A1). The CACFP supports day care centers through reimbursements of costs for serving nutritious meals and snacks to eligible children and adults. During Fiscal Year 2015 CACFP served nearly of 3.3 million children daily through about 21,000 sponsoring organizations enrolled at over 64,000 child care centers. ^{1, 2}

The Improper Payments Act (IPIA) of 2002 (P.L.107-300) (Appendix A2) set annual requirements for Federal programs, such as CACFP, to report estimates of improper payments in an effort to improve program integrity. Further guidance was provided in a 2009 Executive Order (Appendix A3) and by the Improper Payments Elimination and Recovery Act (IPERA) of 2010 (P.L.111-204) (Appendix A4), and the Improper Payments Elimination and Recovery Improvement Act (IPERIA) of 2012 (P.L. 112-248) (Appendix A5), which amended and expanded IPIA requirements. Despite administrative changes and investigations to address directives, FNS has struggled to fully address these requirements as found by the USDA Inspector General³. The Fiscal Year 2014 Compliance with Improper Payment Requirements Audit Report (50024-0008-11) (Appendix A6) reported that CACFP has remained non-compliant with reporting requirements for four consecutive years in not reporting estimates for total payment errors in the CACFP. Additionally, the report recommended that FNS take immediate, additional actions to bring its programs into compliance.

The Erroneous Payments in Child Care Centers Study (EPICCS) will focus on CACFP operations in participating child care centers and their sponsoring organizations⁴. EPICCS will be critical for FNS’s annual compliance with IPERA in the child care center component of CACFP. Collection of these data were part of FNS’s Research and Evaluation Plan for FY 2014 (Appendix A7) in order to provide a comprehensive measure on the level of erroneous payments for the child care center component.⁵ In summary, data collected for EPICCS will be used by FNS to inform its policy-making and regulatory processes for maintenance and improvements to program integrity. Appendix A includes appropriate statutes, regulations, and reference documents pertaining to the EPICCS.

2. Purpose and Use of the Information

Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate how the agency has actually used the information received from the current collection.

IPERA requires filing of risk assessments and measurements of improper payments for each fiscal year. EPICSS data will produce national estimates of improper payments (hereafter referred to as erroneous payments) in the child care center component of CACFP. Additionally, due to prohibitive costs for conducting large, nationally representative studies each year, these data will be also used to develop estimation models to use in estimating annual improper payments. The study’s three objectives are:

1. Obtain national estimates for three types of erroneous payments: certification (incorrect eligibility classification), aggregation (errors in compiling meal counts and claims), and meal claiming errors (improperly claiming a meal as reimbursable);

2. Develop models for calculating annual, national estimates for all three types of errors; and

3. Describe methodologies for generating State-level erroneous payment estimates in a White Paper.

While participation in this study is part of the Healthy, Hunger-Free Kids Act (HHFKA) (Appendix A8) requirements, sponsor, center, and household participation is voluntary and will not impact receipt of any benefits. The information collected from this study may be shared with other departments within the USDA and the government (as determined by FNS).

Pre-data collection activities included building a national support base for the study, obtaining letters of endorsement from National CACFP organizations for use as part of study notification to sponsors and centers, and working with the State agency to build the sample frame. As part of their participation in CACFP⁶, sponsors and centers are notified about the study and their selection into the sample, asked to confirm their participation, and asked to provide administrative data. Study materials include letters to introduce the study, brochures and fact sheets to describe it, and scripts for follow-up phone calls. Senior staff (including former CACFP professionals) will engage sponsors and centers for participation via mail, email, and telephone. These pre-data collection activities will include verification, and updating as needed, of sponsor and center information received from the State agency. Recruitment activities include recruiting households to participate in the Study. Household recruitment will include a recruitment packet sent via mail, followed by a follow up recruitment call from a data collector. Recruitment materials (including pre-recruitment materials) are found in Appendices B1 – B21.

Pre-data collection activities also included cognitive testing of the survey and methods for the household survey component, and consultations with sponsor and center directors for their expertise and feedback and the overall data collection approach. These activities were conducted with fewer than nine respondents.

Data collection for EPICCS will address the three types of erroneous payments: certification errors (administrative and household reporting), aggregation errors, and meal claiming errors. The data collected for certification errors will take place in three rounds (October 2016, February 2017, and June 2017) to ensure more complete coverage of households who submitted income eligibility applications over the course of the year. Table A2-1 presents a summary of the data collection instruments and forms, and type of error each will address. All of the described information collection will be conducted once for the EPICCS. The contractor will deliver summary reports as described in the response to question 16 of this document along with the corresponding data sets and codebook (with de-identified data). Appendix C1 includes a summary of the data collection activities and how the information will be used, and Appendices C2-C30 include the data collection instruments and forms.

Table A2-1. Summary of Data Collection Instruments and Forms

Instrument / Form (Appendix Reference)	Mode of Data Collection	Key Data Elements	Type of Error
Sponsor and Center Data Collection
Sponsor & Child Care Center Pre-visit Interview (Appendix C2)	Phone / Email	Sponsor and school information needed to plan for data collection visit
Center Enrollment Form (Appendix C3)	Data abstraction	Record of students enrolled with certification status	Certification
Income Eligibility Application Abstraction Form (Appendix C4)	Data abstraction	Household size and income Participation in SNAP, TANF, etc. Eligibility determination	Certification
Meal Observation Form – Restaurant / Cafeteria Style (Appendix C5)	Observation and data abstraction	Food production records Observation of meals served, including serving size	Meal Claiming
Meal Observation Form – Family Style (Appendix C6)	Observation and data abstraction	Food production records Observation of meals served, including serving size	Meal Claiming
Example of Meal Transaction Sampling (Appendix C7)	Data Abstraction	(If necessary) Number of serving locations and serving periods	Meal Claiming
Center Attendance Form for Target Month (Appendix C8)	Data Abstraction	Attendance for enrolled students	Aggregation
Center Meal Counts for Target Month (Appendix C9)	Data abstraction	Meal count for each meal (classroom/unit level) for target month Daily and monthly meal count for each meal (center level) for target month	Aggregation
(Sponsored) Center Meal Count Claim to Sponsor for Target Month (Appendix C10)	Data abstraction	Meal count and claims submitted to sponsor by center for target month	Aggregation
Sponsor Meal Claim to State for Target Month (Appendix C11)	Data abstraction	Meal claims reported to State by sponsor on behalf of center during the target month	Aggregation
(Independent) Center Meal Claim to State for Target Month (Appendix C12)	Data abstraction	Meal count and claims submitted to State for target month	Aggregation
Sponsor Survey Cover Letter (Appendix C13)	Mailed Letter	Summary of sponsor survey and instructions
Child Care Center Sponsor Survey (Appendix C14)	Self-Administered Mailed questionnaire	Sponsoring organization characteristics Training and site monitoring practices Staffing levels Oversight and management approaches
Head Start Sponsor Survey (Appendix C15)	Self-Administered Mailed questionnaire	Sponsoring organization characteristics Training and site monitoring practices Staffing levels Oversight and management approaches
Extended Attendance Data Request (Appendix C16)	Electronic Data Request	Extended attendance records for students with completed household surveys	Certification
Meal Counts for Observation Month (Appendix C17)	Electronic data request	Center Meal Counts for month of meal observations	Aggregation
State Meal Claim Request (Appendix C18)	Electronic data request	Meal claims submitted to USDA	Aggregation
NAMES Survey Data Collection
NAMES Survey Consent Form (Appendix C19 and C20)	In Person	Purpose of study and information to be collected Risk and privacy Compensation Voluntary Participation	Certification
NAMES Appointment Reminder Letter (Appendix C21 and C22)	Mailing	Survey appointment date and time	Certification
NAMES Income Worksheet (Appendix C23 and C24)	Hardcopy worksheet sent via mail	Sources of income Acceptable forms of documentation	Certification
NAMES Survey (Appendix C25 and C26)	In Person Interview Computer Assisted Computer Interview (CAPI)	Household size and income Participation in SNAP, TANF, etc. Meal participation and satisfaction	Certification
NAMES Survey Income Source Show Card (Appendix C27 and C28)	In Person	List of income sources	Certification
NAMES Incentive Received Form (Appendix C29 and C30)	In Person	Document of incentive received

3. Use of Information Technology and Burden Reduction

Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

FNS is committed to complying with the E-government Act, 2002 to promote the use of technology. Most information to be collected for this study will come from existing records and interviewer observations, rather than depending on center or sponsor staff effort to provide the needed data. Wherever possible, improved technology has been incorporated into the data collection to reduce respondent burden. For example, electronic data collection forms will be pre-populated with data obtained during the pre-recruitment, recruitment, and pre-visit interviews, as well as data derived from child care center enrollment rosters.

Data collectors will use portable scanners to scan records that will subsequently be entered directly into an electronic form on a laptop computer or tablet. This will minimize the length of time at the center/sponsor, and burden on that staff. Data collectors will securely transmit via broadband internet connections on a daily basis. Laptops will be configured with security settings that feature numerous technical controls to protect study data and maintain the laptop’s integrity during the communication sessions. Access to the private data networks is controlled by many security features including a firewall for port filtering and an SSL/VPN gateway that requires user authentication. The described approach not only reduces the burden of responding for child care centers but also for processing the collected data.

In addition, administration of the in-person NAMES survey (Appendix C25) will use computer-assisted personal interviewing (CAPI). Thus, 100% of the 5,400 household survey responses will be collected electronically. Use of CAPI automates skip patterns, customizes wording, completes response code validity checks, and applies consistent editing checks. These features will improve the pace and flow of the interviews and thus reduce respondent burden. The research team will also be prepared to receive and process electronic records (Appendices C16, C17, & C18) from child care centers and State agencies, in place of the production of hard-copy documents or completion of specific forms. We estimate that 90% (approximately 833) of these records request will be collected electronically⁷. FNS estimates that out of the 64,916 total responses for this collection, 9.6% (6,233 responses) will be submitted electronically.

4. Efforts to Identify Duplication

Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Question 2.

Every effort has been made to avoid duplication of data collection efforts. These efforts include a review of USDA reporting requirements, State administrative agency reporting requirements, and special studies by government and private agencies. These efforts have found that reliable and nationally representative research on the existence of improper payments in CACFP’s child care center setting does not exist. USDA’s Office of the Inspector General (OIG) indicated agreement to this assessment in the most recently released consolidated financial statement for the department. ⁸

While little research has been conducted on improper payments in child care centers, more information is available on the sources of improper payments in the Family Day Care Homes (FDCH) component of CACFP, the National School Lunch Program (NSLP), and the School Breakfast Program (SBP). However, the unique characteristics and processes of child care centers indicate that findings from other studies on erroneous payments may not be fully transferable. Thus, a separate study on erroneous payments in child care centers is warranted to meet reporting requirements under IPERA (Appendix A4).

5. Impacts on Small Businesses or Other Small Entities

If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

A portion of the sampled child care centers will be small entities, and thus the data collection is likely to create some burden. Study procedures will minimize this burden to the degree possible by relying on the data collectors for primary data collection. Further, the data collection effort does not require centers to collect new data. The requests for records will include data that are routinely obtained or compiled as part of their CACFP participation. While some child care center staff time will be needed to provide access to child care center records, the data collector will independently abstract the needed information.

There are four respondent categories for this data collection: State agencies, sponsors, child care centers, and households representing children served in child care centers. The total estimated number of respondents (including non-respondents) for this data collection is 8,921. This includes: 25 State CN agency administrators, 25 State CN data managers, 474 sponsor staff, 450 sponsor data managers, 450 center staff and 6,750 households. Approximately 5.58 percent (498) are expected to be considered small entities. This includes 142 centers (30% of 474 centers) and 356 sponsors (75% of 474 sponsors).

6. Consequences of Collecting the Information Less Frequently

Describe the consequence to Federal program or policy activities if the collection is not conducted, or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If this data collection were not performed, FNS would be unable to meet its reporting obligations under IPERA to annually measure and report erroneous payments in the CACFP and identify the sources of erroneous payments as outlined in Appendix A4 to Circular No. A-123, Requirements for Effective Estimation and Remediation of Improper Payments.⁹ (Appendix A9). This study will be conducted once as a one-time study.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

Explain any special circumstances that would cause an information collection to be conducted in a manner:

• Requiring respondents to report information to the agency more often than quarterly;

• Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• Requiring respondents to submit more than an original and two copies of any document;

• Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

The collection of information is conducted in a manner consistent with the guidelines in 5 CFR 1320.5. None of the special circumstances listed are applicable to this data collection effort.

8. Comments to the Federal Register Notice and Efforts for Consultation

If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8 (d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least every 3 years even if the collection of information activity is the same as in prior years. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

A notice was published in the Federal Register on November 18, 2015 (Volume 80, pages 72041 - 72043). The public comment period ended on January 19, 2016. One public comment was received (Appendix D). The comment was unrelated to the information collection¹⁰. The response to the public comment is included in Appendix E.”

Consultations about the research design, sample design, data sources and needs, and study reports occurred during the study’s instrument design phase and will continue to take place throughout the study with senior team members. The purpose of these consultations is to ensure the technical soundness of the study and the relevance of its findings and to verify the importance, relevance, and accessibility of the information sought in the study. Other individuals outside the agency who will review and comment upon key documents produced by the study are:

Lori Harper, National Agricultural Statistics Service (NASS), Methods Division, 202-690-0694.

9. Explanation of Any Decisions to Provide Any Payment or Gift to Respondents

Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Permission is requested to offer financial incentives of up to $50 based upon the respondent’s level of participation. All respondents completing the household survey would be provided $30. Respondents who obtain and provide documents to verify income will receive an additional $20. The incentives will be provided in the form of a gift card that can be used anywhere Visa is accepted. This approach, along with providing a household income worksheet prior to the household survey, is proposed to address a limitation in prior APEC studies (noted by FNS in a recent solicitation¹¹) in which only a small percentage of households provided documentation to verify income, which is needed to more accurately measure household reporting error. In July 2015, Westat conducted a pilot test of these procedures designed to improve the response rate of income documentation with the household survey. In summary, all nine respondents provided some form of acceptable income documentation, either in hardcopy or for review from their smartphone. Thus, we propose incorporating these procedures with some enhancements based on the pilot study findings in EPICSS, which will improve the rates of income documentation being provided during the interview.

Singer and Ye¹² completed a systematic review of articles on the use of incentives to enhance response rates published after 2002. They found that incentives increase response rates for all modes of administration. They also found that increasing incentive amounts continue to improve response rates. Another conclusion of this review was that monetary incentives were more effective than promised gifts. It is likely that incentives influence response rates either through facilitating contact with the potential respondent or by stimulating cooperation. In another meta-analysis of monetary incentives, Mercer and colleagues at Westat looked at dose-response analysis effects of increasing monetary incentive amounts on response rates. ¹³ They found a strong, nonlinear effect for increased response rates through increasing monetary incentive amounts, which was strongest for prepaid incentives for mailed surveys.

APEC II, conducted during School Year (SY) 2012-2013, was approved by OMB to provide an incentive of $25 for completion of the household survey. For APEC III, we propose a slightly higher base incentive of $30 because the APEC III survey has a few additional questions, and will be conducted five years later in SY 2017-2018 ($5 more). The additional $20 incentive is added to reflect the additional task of reviewing and completing the worksheet in advance, and compile the necessary documentation. Also, the sensitive nature of the information requested (income documentation) was a factor. In summary, the incentives serve as a ‘token of appreciation for the respondents’ time and participation, which has been shown to enhance participation rates.

10. Assurances of Confidentiality Provided to Respondents

Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Study participants will be subject to assurances as provided by the Privacy Act of 1974 (5 USC §552a), which requires the safeguarding of individuals against invasion of privacy; these assurances will be documented in an informed consent form (Appendices C19 and C20). In addition, all Westat project staff and subcontractors have signed a Westat Confidentiality Pledge (Appendix F). All procedures planned for the study will ensure the privacy and security of electronic data during the data collection and processing period following the system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports published in the Federal Register on April 25, 1991, volume 56, pages 19078 – 19080, which discusses the terms of protections that will be provided to respondents. Names and phone numbers will not be linked to participants’ responses, survey respondents will have a unique ID number, and analysis will be conducted on data sets that include only respondent ID numbers. Finally, to protect the privacy of the children, their parents, and the centers, we will perform a nondisclosure analysis on each of the files slated to become Public Use files and take the appropriate remedial actions. In addition, we will conduct an Initial Risk Analysis by evaluating all the potential personal, geographic, and other identifiers in the files that could be disseminated using a rigorous step-by-step process. Finally, we will summarize the outcomes and implications and write a disclosure mitigation plan. After review and approval by the FNS COTR, we will apply the approach.

All collected data will be securely transmitted to Westat using closed and secure data transmission. Any hardcopy materials will be stored in locked file cabinets. Electronic data can only be accessed using a log-on routine with approved user identification and a strong password. Thus, access to records is limited to those persons who process the records for the specific uses stated in the Privacy Act notice. FNS does not have any connection to the personal data collected and will not handle any data containing identifying information. The compiled report for FNS will contain no personal information and is publicly posted. Data will be presented in aggregate statistical form only. Names and phone numbers will be destroyed two years after the completion of analysis and delivery of the contractual reports.

Westat’s Institutional Review Board (IRB) serves as the organization’s administrative body and all research involving interactions or interventions with human subjects is within its purview. Westat holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP) (Appendix G). The FWA covers all federally supported or conducted research involving human subjects. The project has submitted the study instruments and protocol to the Westat Institutional Review Board (IRB) for review and approval. The Westat IRB approved the Study for data collection. (Appendix H)

11. Justification for Any Questions of a Sensitive Nature

Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The NAMES survey includes sensitive questions, including questions on demographics, household composition, income, and receipt of public assistance from federal, State and/or county agencies. The consent form will inform all respondents of their right to decline to participate or answer any question they do not wish to answer without negative consequences. Similar questions were asked in the APEC studies, with success and no evidence of harm. Being part of the study will not affect any USDA benefits received by programs or families participating in this data collection.

Questions on income and the receipt of public assistance are necessary to establish the family’s actual eligibility for free and reduced-price CACFP meal benefits and will be used to estimate certification error and derive estimates of erroneous payments.

As previously mentioned, all procedures planned for the study will follow the system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports. Further, this study will adhere to Westat’s Information Technology and Systems Security Policy and Best Practices (Appendix I).

Our security plans are very detailed and specify multiple levels of controls for protecting data. These protections include management (e.g., certification, accreditation, and security assessments, planning, risk assessment), operational (e.g., awareness and training, configuration management, contingency planning), and technical (e.g., access control, audit and accountability, identification and authentication) controls that we implement to secure study data. All work will fully comply with all Government-wide guidance and regulations as well as USDA Office of the Chief Information Officer (OCIO) directives, guidelines and requirements.

12. Estimates of the Hour Burden of the Collection of Information

Provide estimates of the hour burden of the collection of information. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. The statement should:

1. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

The estimated burden for this information collection including the number of respondents, frequency of response, average time to respond, and annual hour burden are provided in Table A12-1 Estimates of Respondent Burden Including Annualized Hourly Cost, and in Appendix M. A summary of the burden appears in the table below.

B)Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

The estimate of respondent cost is based on the burden estimates and uses the U.S. Department of Labor, Bureau of Labor Statistics, May 2014 National Occupational and Wage Statistics, Occupational Group (25-0000) Job category 11-9031 – Educational Administrators, Preschool and Child Center or Program was used to estimate annualized costs for managers or directors at the State agencies, sponsor, and child care centers (http://www.bls.gov/oes/current/oes119031.htm). Annualized costs were based on the mean hourly wage for each job category. The hourly wage rate used for the State child nutrition agency director is $43.61 (Occupation code 11-9031 "Education Administrators, Preschool and Childcare Center/Program;" industry "State Government"). The hourly wage rate used for State child nutrition data manager and sponsor level data manager is $39.56 (Occupation Code15-1141 Database Administrator). The hourly wage rate used for the sponsor director is $25.09 (Occupation Code 11-9031). The hourly wage rate used for center director/managers is $22.57 (Occupation Code 11-9031, Child and Day Care Services Industry). The estimated annualized cost for the household survey respondent uses the Federal minimum wage of $7.25¹⁴. The total annualized hour burden to the public is 13,906 hours and estimated annualized cost is $196,237.34.

13. Estimates of Other Total Annual Cost Burden

Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.

There are no capital/startup or ongoing operation/ maintenance costs associated with this information collection.

Table A12-1 Estimates of Respondent Burden Including Annualized Hourly Cost (See Appendix M) (page 1 of 3)

Table A12-1 Estimates of Respondent Burden Including Annualized Hourly Cost (continued, page 2 of 3)

Table A12-1 Estimates of Respondent Burden Including Annualized Hourly Cost (continued, page 3 of 3)

14. Estimates of Annualized Cost to the Federal Government

Provide estimates of annualized cost to the Federal government. Provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The total estimated cost of the study to the Federal government is $5,671,976.30 including contractor and Federal government employee costs. The average annual cost to the Federal government including contractor and Federal government employee cost is $1,134,395.12.

The total estimated cost to the contractor is $5,581,871 over a period of five years (September 2014 through September 2019), representing an average annualized cost of $1,116,374. This represents the contractor’s costs for labor, other direct costs, and indirect costs.

The total estimated cost to the Federal government for the FNS employee, Social Science Research Analyst/ Project Officer, involved in project oversight with the study is estimated at $90,105.60 over a period of five years. This represents an estimated annual cost of $18,021.12 (GS-12, step 6 at $43.32 per hour¹⁵, 416 hours per year). Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2016 for the Washington, DC locality).

15. Explanation of Program Changes or Adjustments

Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.

This submission is a new information collection request as a result of program changes and will add 13,906 hours of burden and 64,916 responses to OMB’s inventory.

16. Plans for Tabulations, and Publication and Project Time Schedule

For collections of information whose results are planned to be published, outline plans for tabulation and publication.

Table A16-1 below provides the schedule for EPICCS activities.

Table A16-1 Proposed Study Schedule

Project Activity	Schedule
Sampling of Sponsors and Centers	October – December 2015
Preparation for Data Collection	February – August 2016
Train Data Collectors	September 2016
Conduct Data Collection	September 2016 – July 2017*
Prepare Data Files & Analyze Data	August 2017 – March 2018
Draft, Revised and Final Reports and Estimate Models	April 2018-May 2019
Draft, Revised, and Final Briefing Materials	May 2019 – June 2019
Additional Reports and Analyses	May 2019- September 2019

* Data collection will not begin prior to September 2016. The data collection and analysis schedule will be adjusted as needed if OMB approval is not received by September 2016.

Prepare Study Findings Report (Objective 1). The weighted statistical tables will be used to derive the national estimates of erroneous payments due to certification, aggregation, and meal claiming errors. The values needed for the EPICCS report includes the following items.

• Calculation of Certification Rate Errors – Once the applicant’s true certification status is determined; it will be compared with the eligibility level of the current certification status from the abstracted records. Following the APEC methodology, the “broad certification error rate” will also be computed. The broad certification error rate represents the percentage of children who are either certified for some level of benefits when they are not eligible for either free or reduced-price benefits or who are not certified when they are eligible for at least reduced-price benefits.

We will use application-level sampling weights to estimate the annual rates of the administrative errors and household reporting errors. These sampling weights are computed by using the center and application level selection probabilities. Since we are selecting applications three times throughout the year to capture all applications, the resulting error weights will be unbiased “base” weight. When applied to the sampled applications, the resulting error estimates are representative of the universe of applications submitted during the entire year. Furthermore, we will compare applicants sampled from each of the three data collection periods using data on the application to determine whether there is any statistically significant differences among applicants who apply at different times during the year.

• Calculation of Aggregation Error Rates. Other erroneous payments could be due to errors unrelated to a child’s eligibility for CACFP benefits, such as errors in aggregating meal counts at the child care center or during transfers from the center to the sponsor or State child nutrition agency. Three values will be calculated: incidence rate, total dollar amount, and dollar-based error rate.

• Calculation of Meal Claiming Errors. These errors occur when a meal is incorrectly classified as reimbursable, when it is not. That is, the meal served does not meet the specific meal patterns required for CACFP. Three values will be calculated for each of these error types: incidence rate, total dollar amount, and a dollar-based error rate.

A study findings report will be prepared to summarize the estimated error rates as determined from EPICCS data.

Estimation Models for National Certification and Non-Certification Errors (Objective 2). Reliable estimation models will also be developed, based on this nationally representative data, for FNS to use for updating erroneous payment estimates each year. The methodology will be documented in a methodology report and delivered to FNS for public or limited dissemination. This effort will employ Bayesian estimation through Markov Chain Monte Carlo parameters as described in Part B of this Supporting Statement.

State Level Methodologies for Estimation (Objective 3). A white paper on methodologies will be prepared that will describe the use of EPICCS analyzed data and small area estimation (SAE) to compute State level estimates of erroneous payments. The SAE methodology takes advantage of existing relationships between the characteristics of interest and the auxiliary variables, and through the modeling “borrow strength” across areas to improve the small area estimates. The methodology for computing the estimates will enhance findings from EPICCS with available auxiliary variables for the entire population such as the census or the American Community Survey (ACS).

17. Displaying the OMB Approval Expiration Date

If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency plans to display the expiration date for OMB approval of the information collection on all instruments.

18. Exceptions to the Certification Statement Identified in Item 19

Explain each exception to the certification statement identified in Item 19 of the OMB 83-I "Certification for Paperwork Reduction Act."

There are no exceptions to the certification statement.

1 Child care providers participate in the CACFP under the umbrella of a sponsoring organization that assumes fiscal responsibility and provides training and monitoring to ensure that its providers comply with all of the CACFP regulations. However, independent child care centers may act as their own sponsor (i.e., self-sponsor) for the CACFP. As such they are included in counts of both sponsors and providers.

2 All program data are taken from the FNS National Data Bank Data (NDB), last accessed on March 17, 2016 at: http://www.fns.usda.gov/sites/default/files/datastatistics/keydata-december-2015.pdf.pdf

3 Office of the Inspector General, USDA’s Fiscal Year 2013 Compliance with Improper Payment Requirements, Audit Report 50024-0008-11, May 2015. Accessed July 9, 2015 at: http://www.usda.gov/oig/webdocs/50024-0008-11.pdf

4 The household survey component of the study is referred to as National Assessment of Meal Eligibility and Services (NAMES).

5 Food and Nutrition Service, Research and Evaluation Plan – Fiscal Year 2014, February 19, 2014 http://www.fns.usda.gov/sites/default/files/2014RandE.pdf

6 Sponsor and center participation complies with the Health, Hunger Free Act, and requirements as a participant in CACFP.

7 The total number of electronic requests from child care centers and State agencies are 900 and 25 respectively, for a total of 925 electronic requests.

8 USDA OIG, Department of Agriculture’s Consolidated Financial Statements for Fiscal Years 2014 and 2013 (December 2014), accessed January 20, 2015 at: http://www.usda.gov/oig/50401-0007-11.pdf.

9 Accessed January 9, 2015 at: http://www.whitehouse.gov/sites/default/files/omb/memoranda/2015/m-15-02.pdf

10 The comment, included in Appendix D, was related to the application process for new CACFP agencies.

11 Draft Access, Participation, Eligibility and Certification (APEC) III Performance Work Statement: National Estimates of NSLP and SBP Erroneous Payments (May 21, 2015). Accessed July 10, 2015 at: https://www.fbo.gov/index?s=opportunity&mode=form&id=fc2768a8411c3ff0f25b42a153009b28&tab=core&_cview=0 .

12 Singer, E and Ye, C; The Use and Effects of Incentives in Surveys. The ANNALS of the American Academy of Political and Social Science, January 2013 vol. 645 no. 1 112-141, accessed January 9, 2015 at: http://ann.sagepub.com/content/645/1/112.full.pdf+html.

13 Mercer, A, Caporaso, A., Cantor, D. and Townsend R; How Much Gets You How Much? Monetary Incentives and Response Rates in Household Surveys. Public Opinion Quarterly, Spring 2015 vol. 79, no. 1 102-129.

14 http://www.dol.gov/general/topic/wages/minimumwage

15 Office of Personal Management, General Schedule, accessed March 11, 2016 at: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2016/DCB_h.pdf