DCM-FRP Generic OMB Memo

Memorandum

Date: October 5, 2016

To: Stephanie Tatham, OMB Desk Officer

Through: Ruth Brown, United States Department of Agriculture, Office of the Chief Information Officer

From: Lynnette Thomas

Food and Nutrition Service, Branch Chief, Planning & Regulatory Affairs

Re: Generic OMB Clearance No. 0584-0606 – Pretest for Evaluation of Direct Certification with Medicaid Demonstrations

The Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA) is requesting approval for pretesting the data collection procedures and instruments for the Evaluation of Direct Certification with Medicaid Demonstrations under Approved Generic OMB Clearance No. 0584-0606.

This request is to acquire clearance to conduct research with State Child Nutrition and Medicaid agency officials, School Food Authority (SFA) directors, and school district food service staff to develop, test, and improve evaluation data collection instruments and methodologies. The following information is provided for your review:

1. Title of the Project: Evaluation of Direct Certification with Medicaid Demonstrations

2. Generic Approval Control Number: 0584-0606, Expires 03/31/2019

3. Public Affected by this Project:

State, Local, Tribal Governments:

• State Child Nutrition Agency Staff, State Medicaid Agency Staff, SFA Staff

4. Number of Respondents & Research Activities:

Respondents	Research Activity	Number of Participants
State Child Nutrition Agency Staff	Site Visit Scheduling Call	7
	On-Site Interviews	21
	On-Site Observations	14
	Site Visit Follow-Up Telephone Interview	7
	Administrative Records Request	7
	State Cost Data Collection Tracking Logs	7
	State Cost Data Collection Clarification Call	7
	Pretest Debriefing Interview	7
State Medicaid Agency Staff	Site Visit Scheduling Call	7
	On-Site Interviews	21
	On-Site Observations	14
	Site Visit Follow-Up Telephone Interview	7
	State Cost Data Collection Tracking Logs	7
	State Cost Data Collection Clarification Call	7
	Pretest Debriefing Interview	7
SFA Staff	Site Visit Scheduling Call	18
	On-Site Interviews	54
	On-Site Observations	36
	Site Visit Follow-Up Telephone Interview	18
	Pretest Debriefing Interview	9
Total Unique Participantsa		96

^aState Child Nutrition Agency, State Medicaid Agency, and SFA staff are all expected to participate in multiple data collection activities, including the site visits, follow-up interviews, debriefings, and the completion of cost tracking logs and administrative records requests, where applicable. Each individual is counted only once in the total.

5. Time Needed Per Response:

Recruitment is not needed because State Child Nutrition Agency and State Medicaid Agency staff applied to participate in the demonstration. The States will contact the selected districts and invite them to participate.

Table 5. Time Needed for Research Activities by Audience

Respondents	Research Activity	Time (hours per respondent)
State Child Nutrition Agency Staff	Site Visit Scheduling Call	0.5
	On-Site Interview	1
	On-Site Observations	0.5
	Site Visit Follow-Up Telephone Interview	1
	Administrative Records Request	4
	State Cost Data Collection Tracking Logs	3
	State Cost Data Collection Clarification Call	0.3
	Pretest Debriefing Interview	2
State Medicaid Agency Staff	Site Visit Scheduling Call	0.5
	On-Site Interview	1
	On-Site Observations	0.5
	Site Visit Follow-Up Telephone Interview	1
	State Cost Data Collection Tracking Logs	3
	State Cost Data Collection Clarification Call	0.3
	Pretest Debriefing Interview	1.5
SFA Staff	Site Visit Scheduling Call	0.5
	On-Site Interview	1
	On-Site Observations	0.5
	Site Visit Follow-Up Telephone Interview	1
	Pretest Debriefing Interview	1
Total		24.1

6. Total Burden Hours on Public:

Affected Public		(a) Instrument	(b) Instrument/ Appendix ID	(c) Respond-ents	(d) Frequency of Response	(e) Est. Total Annual Responses (c x d)	(f) Hours per Response	(g) Total Burden Hours (e x f)
State, Local/ Tribal Employees	State Child Nutrition Agency Staff	Site Visit Scheduling Call	A-1	7	1	7	0.5	3.5
		On-Site Interview	A-2	21	1	21	1	21
		On-Site Observations	A-3	14	1	14	0.5	7
		Site Visit Follow-Up Telephone Interview	A-4	7	1	7	1	7
		Administrative Records Request	B	7	2	14	4	56
		State Cost Data Collection Tracking Logs	C-1	7	3	21	3	63
		State Cost Data Collection Clarification Call	C-2	7	3	21	0.3	6.3
		Pretest Debriefing Interview	D	7	1	7	2	14
	State Medicaid Agency Staff	Site Visit Scheduling Call	A-1	7	1	7	0.5	3.5
		On-Site Interview	A-2	21	1	21	1	21
		On-Site Observations	A-3	14	1	14	0.5	7
		Site Visit Follow-Up Telephone Interview	A-4	7	1	7	1	7
		State Cost Data Collection Tracking Logs	C-1	7	3	21	3	63
		State Cost Data Collection Clarification Call	C-2	7	3	21	0.3	6.3
		Pretest Debriefing Interview	D	7	1	7	1.5	10.5
	SFA Staff	Site Visit Scheduling Call	A-1	18	1	18	0.5	9
		On-Site Interview	A-2	54	1	54	1	54
		On-Site Observations	A-3	36	1	36	0.5	18
		Site Visit Follow-Up Telephone Interview	A-4	18	1	18	1	18
		Pretest Debriefing Interview	D	9	1	9	1	9
Total				96a		345		404.1

^aState Child Nutrition Agency, State Medicaid Agency, and SFA staff are all expected to participate in multiple data collection activities, including the site visits, follow-up interviews, debriefings, and the completion of cost tracking logs and administrative records requests, where applicable. Each individual is counted only once in the total.

Total burden hours on public: 404.1 hours

• Total burden hours on State Child Nutrition Agency staff: 177.8

• Total burden hours on State Medicaid Agency staff: 118.3

• Total burden hours on SFA staff: 108

7. Project Purpose, Methodology, and Formative Research Design:

Background

The National School Lunch Program (NSLP) is the largest child nutrition program in the United States and, along with the School Breakfast Program (SBP), it is a cornerstone of the government’s efforts to provide nutritious meals to schoolchildren. Students from low-income families are eligible to receive school meals for free or at a reduced price. The evaluation of demonstrations of Direct Certification with Medicaid for Free or Reduced Price Meals (DCM-F/RP) will investigate the processes, challenges, and outcomes of using Medicaid data to directly certify students to receive free or reduced price (F/RP) meals. This request covers the first year of data collection for the DCM-F/RP demonstration, which will serve as the pretest to support the OMB clearance process for data collection in future years of the demonstration (in which the demonstration will be expanded to include additional States).

All students enrolled in schools participating in the NSLP or SBP are eligible to receive subsidized school meals, but the subsidies that the USDA provides are much larger for meals served to students who are certified to receive F/RP meals. Students in families with incomes at or below 130 percent of the Federal poverty level (FPL) are eligible for free meals, as are students who participate in one of several public-assistance programs (discussed below). Reduced-price meals are provided to students whose families have incomes between 130 and 185 percent of the FPL. Students who have not been certified for F/RP meals pay full price for their school meals. Students can become certified through two main methods:¹

1. Certification through application. Historically, most students who receive F/RP school meals have become certified based on household information reported in an application submitted to the school district. To become certified in this way, households must either (1) provide detailed information on household size and income or (2) demonstrate that they are “categorically eligible,” because they participate in one of several public assistance programs, such as the Supplemental Nutrition Assistance Program (SNAP), Temporary Assistance for Needy Families (TANF), or the Food Distribution Program on Indian Reservations (FDPIR). The district assesses the application information to determine whether the household meets the eligibility requirements.

2. Direct certification. In recent years, increasing numbers of students have been automatically determined eligible for free meals through direct certification rather than an application. Direct certification typically involves matching administrative records from programs that confer categorical eligibility with student enrollment records to identify and automatically certify eligible children for free school meals. All districts participating in the NSLP that certify students, including private schools, are required to directly certify students in SNAP households.² FNS also encourages direct certification of students in TANF and FDPIR households. Students documented as foster children, homeless, migrant, runaway or participating in Head Start can also be directly certified for free school meals. In some States, the districts conduct direct certification; in other States, a State agency conducts direct certification and provides the results to the districts.

Recently, FNS has sought ways to further expand direct certification, including experimenting with allowing direct certification based on data from means-tested programs that do not confer categorical eligibility. In particular, FNS solicited applications from States to participate in a demonstration to directly certify students for free school meals based on income eligibility identified through Medicaid data. Five States began conducting direct certification using Medicaid (DCM) data in school year (SY) 2012–2013, and the demonstration expanded to three additional States over the following two years. Because students receiving Medicaid are not categorically eligible for free meals, the demonstration authorized selected States and districts to use income information from Medicaid enrollment or eligibility files to determine eligibility and directly certify students found to be eligible for free meals.³ Students were eligible if they were (1) enrolled in Medicaid and (2) from a household with Medicaid gross income not exceeding 133 percent of the FPL.⁴ Other students in a household with a child who meets these criteria were also eligible for direct certification for free meals under DCM. The legislation specified the use of gross income “before the application of any expense, block, or other income disregard,” rather than net income for determining eligibility under DCM. However, the determination of eligibility through DCM relies on the definition of household used by the Medicaid Agency, which may differ from that used on NSLP/SBP applications.

The new demonstration, DCM-F/RP, differs from the previous demonstration in important ways. First, guidelines for assessing eligibility were revised to reflect changes in Medicaid income and household definitions under the Patient Protection and Affordable Care Act of 2010. Students can now be certified for free meals under DCM-F/RP if their household income is at or below 130 percent of the FPL for the family size used by Medicaid. Second, in addition to certifying students for free meals, the selected States will use data from the Medicaid files to identify those in households with gross income between 130 and 185 percent of the FPL and directly certify them to receive reduced-price meals. FNS has authorized six States to begin implementing DCM-F/RP in SY 2016–2017: Florida, Massachusetts, Nebraska, Utah, Virginia, and West Virginia. California will be implementing a variation of DCM-F/RP that (1) uses 133 percent of the FPL as the income threshold for free meal eligibility and (2) includes undocumented immigrant students who are enrolled in Medicaid coverage for emergency conditions only, but would be eligible for full Medicaid if not for their undocumented status.

Purpose

This purpose of the pretest is to develop, test, and improve evaluation data collection instruments and methodologies for use in studying the DCM-F/RP demonstration. The pretest data collection conducted during the first year of the demonstration will support the later OMB clearance process for subsequent rounds of data collection in future years of the demonstration.

More broadly, the study of the DCM-F/RP demonstration will investigate the processes, challenges, and outcomes of using Medicaid data to directly certify students to receive F/RP meals. Direct certification has improved access to free school meals while easing the burden for families and district staff by reducing the use of household applications as well as the number of students subject to verification of school meal benefit eligibility. The DCM-F/RP demonstrations represent an expansion of direct certification. The study will help FNS understand how States are implementing the demonstration, identify challenges encountered, and examine the demonstration’s effectiveness in improving certification and participation outcomes and the implications for Federal reimbursement totals and State administrative costs. This information will help FNS identify best practices that will shape future replication and improvement under wider rollout of the policies.

Methodology/Research Design

• On-Site Interviews: To document the direct-certification processes pertaining to the policies under study, we will use in-depth case studies that trace the relevant direct-certification workflow step by step. The core of these studies will be in-person site visits in each participating State, during which we will interview program and technical staff at the State and district levels who are involved in implementing DCM-F/RP. Interviews will provide detailed descriptions of DCM-F/RP procedures at the State and district levels and specific changes needed to initiate DCM-F/RP. We will ask open-ended questions and follow up with adaptive probes based on the information provided by the respondent.

• On-Site Observations: During the site visits, we will also conduct observations of key steps in the DCM-F/RP process. We will ask technical staff to show us how they complete steps such as accessing and reviewing data, conducting data matching, validating matches, certifying matched students in point-of-sale systems, and obtaining the data needed to complete the FNS-742 form.

• Site Visit Follow-up Telephone Interviews: Later in the school year, we will conduct telephone interviews with respondents at the same State agencies and districts visited during site visits. These interviews will provide updated information on how the DCM-F/RP demonstration has changed in each State since the site visits. They will also provide an opportunity to learn whether and how challenges identified during the site visit have been resolved, and explore any additional challenges that emerged later.

• Administrative Records Request: We will collect from State Child Nutrition agencies district-level administration data on certification and participation outcomes. In particular, we will collect data on certification status, method, and basis. Where available, we will also gather information on DCM-F/RP match results and the certification status, method, and basis for matched students before that match. We will also collect statewide administrative data from each State on each district’s total number of reimbursable lunches and breakfasts served, by reimbursement category. This information will be collected for both the year before implementing DCM-F/RP and the first year of implementing DCM-F/RP.

• State Cost Data Collection Tracking Logs: To analyze administrative costs related to implementing DCM-F/RP, we will collect data three times during the school year on resource expenditures from relevant State Child Nutrition Agency and Medicaid Agency staff. These data will cover labor, infrastructure, software, and other costs. They will reflect additional costs States incurred to implement the new demonstration, beyond those associated with existing State work on direct certification through other programs such as SNAP. For States involved in the first DCM demonstration, we will identify costs associated with implementing new DCM-F/RP components. We will use an Excel tool to collect the data and we may follow up with staff who provide the cost information on any data that require clarification.

• Debriefing Interviews: We will conduct a set of debriefing interviews with respondents to obtain their perspectives on how the data collection processes and instruments worked and suggestions for improving them. In addition, we will ask staff responsible for providing administrative data extracts and completing cost logs to estimate the amount of time it took to prepare the requested data files and logs. We will conduct these debriefing interviews by telephone. The length of the debriefing interview will vary by respondent type, depending on the number of pretest activities in which they participated.

Design/Sampling Procedures

We will collect data from State Child Nutrition and Medicaid Agency staff in all seven States participating in the first year of the DCM-F/RP demonstration and from district staff in 18 districts selected from those States. Including State-level Child Nutrition and Medicaid Agency staff in all demonstration States will yield a thorough understanding of how each State implemented the demonstration and how State-level policies and procedures vary across the study States. Including 18 districts will provide information on districts with varied characteristics, including public and private schools, and ranging in size, urbanicity, and levels of F/RP meal eligibility. We propose selecting two districts in each of the five States with central matching systems and four districts in each of the two States that do not conduct central matching. We will ask the State Child Nutrition Agency staff to recommend districts to include in this convenience sample and will use information from FNS-742 and NCES Common Core Data to assess diversity along key characteristics.

Recruitment and Consent

• State Child Nutrition Agency Staff: Recruitment is not necessary for the State Child Nutrition agencies, because they applied to participate in the demonstration. Participation in the research is a condition of the demonstration.

• State Medicaid Agency Staff: Recruitment is not necessary for the State Medicaid agencies, as they applied to participate in the demonstration. Participation in the research is a condition of the demonstration.

• SFA Staff: As detailed above, with the help of State Child Nutrition Agency staff, we will identify school districts in the demonstration States with varied characteristics. We will ask the State agencies to reach out to the districts to explain the study requirements and encourage them to participate. We will then telephone the districts to plan the site visits.

In advance of the interviews, all individual respondents will be informed about the voluntary nature of their participation and the intended use of the findings.

Incentives

No incentives will be provided.

Data Analysis

Qualitative analysis. To begin the analysis of qualitative data, we will draft an internal analytic memo after each site visit that will summarize findings from the in-depth interviews and observations. We will construct an analysis framework based on the relevant research questions and emergent themes from the analytic memos. We will use this framework to conduct a thorough analysis of all interview and observation notes and State documents, using either qualitative analysis software (such as NVivo) or detailed theme tables in Microsoft Excel. The analysis will identify patterns across different States and districts, focusing on matching procedures, challenges, and successes in implementing DCM-F/RP.

Quantitative analysis. To assess the effects of DCM-F/RP on certification, participation, and Federal reimbursement outcomes, we will use different comparative methods for States implementing the demonstration statewide and for California, which is implementing the demonstration in a subset of districts.

• Statewide DCM-F/RP States. We will use administrative records data to compute for each district measures of key certification, participation, and Federal reimbursement outcomes in the year prior to DCM-F/RP implementation and in the first year of DCM-F/RP. We will use quantitative pre/post district fixed effects analysis to estimate the changes in certification that accompany DCM-F/RP implementation, controlling for measurable time-varying district characteristics (such as enrollment and local economic conditions) and all district characteristics that are fixed over time.

• California. We will use a double-difference approach to estimate effects of the demonstration on certification, participation, and Federal reimbursement costs in California, which will be implementing a variation of DCM-F/RP in 14 districts. Our model will compare changes in key outcomes in demonstration districts with the changes in comparison districts selected to maximize their similarity to demonstration districts. We will use propensity score matching to select a comparison group from among districts that are not implementing DCM-F/RP. The baseline year for the change is SY 2013–2014, the year before DCM was implemented.

For other outcomes, we will use descriptive analyses. For example, we will analyze data on State administrative costs to estimate the total State-level administrative costs of the new demonstration, as well as breakdowns of these costs by agency, timing, and category (such as labor).

8. Confidentiality:

Participants in this study will be subject to assurances and safeguards as provided by the Privacy Act of 1974 (5 USC 552(a)), which requires the safeguarding of individuals against invasion of privacy. A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on April 25, 1991, Volume 56, page 19078, discusses the terms of protections that will be provided to respondents.

All information collected for the evaluation will be used for research purposes only. Data will be collected only in aggregate form so that individual students will not be identified. All employees at Mathematica sign a confidentiality pledge emphasizing its importance and describing their obligation. Access to identifying information on interview respondents will be limited to those who have direct responsibility for collecting or analyzing the data and for providing and maintaining sample information. At the conclusion of the research, these data will be destroyed.

9. Federal Costs:

The original estimated cost to the government for conducting this data collection will be approximately $1,506,727. This estimate includes all contract direct and indirect costs associated with the pretest.

10. Research Tools/Instruments:

• Appendix A: Qualitative Data Collection Materials

  • A-1: Site Visit Scheduling Call Protocol

  • A-2: Semi-structured Site Visit Interview Protocol Questions

  • A-3: Site Visit Observation Protocol

  • A-4: Semi-structured Follow-up Interview Protocol Questions

• Appendix B: Administrative Records Request

• Appendix C: State Cost Data Collection Materials

  • C-1: Instructions for State Cost Data Tracking Logs

  • C-21a-b: State Cost Data Tracking Logs

  • C-23: Semi-structured State Cost Log Clarification Protocol Questions

• Appendix D: Pretest Debriefing Interview Protocol Questions

1 Some school districts use alternative procedures that do not involve certifying individual students each year. Districts participating in Provision 2 or Provision 3 serve all meals for free, conduct certification in a base year, and are reimbursed in later years based on claims from that base year. Under the Community Eligibility Provision (CEP), schools in high-poverty areas may choose to serve free breakfasts and lunches to all students, without requiring applications. Eligible schools or districts that choose to participate in CEP receive the Federal free reimbursement rate for up to 100 percent of meals served, depending on the school's or district's percentage of "identified students" certified for free meals through means other than applications.

2 Schools participating in Provision 2 or 3 in a non-base year or in the CEP do not certify students.

3 Students could not be directly certified for reduced-price meals through DCM under this first demonstration.

4 The Healthy Hunger-Free Kids Act of 2010 (HHFKA) allowed a slightly higher income threshold (133 percent of the FPL) for DCM than is otherwise allowed (130 percent of the FPL).