Feasibility of Computer Matching in the National School Lunch Program

APPENDIX B
IN-DEPTH, SEMI-STRUCTURED INTERVIEW PROTOCOLS

1.

State Child Nutrition Staff

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate the time you and your staff took to complete the survey we sent you about
direct certification processes in your state, and your participation in today’s in-depth interview.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at
a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to
ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?
Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your State, the role your agency plays in direct certification, and explore the possibility of future
changes in those methods.
1.

Please briefly describe the role of your agency in the direct certification process.

GO TO SECTION II

2

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview and
probe any inconsistencies.]
A. DIRECT CERTIFICATION PROCESS
1.

How is direct certification currently done in your State/district? Please describe the process.
PROBE: Is direct certification conducted primarily at the state level or at the district level?
[If at the district level, ask:] What guidelines does the state provide districts concerning
data matching for NSLP direct certification?
PROBE: Does the state require or recommend the use of specific identifiers and/or matching
rules for SFAs to use? What are they?
PROBE: Do you directly certify participants in any public assistance programs other than
SNAP? Which ones? (TANF, Medicaid, etc.). Why/Why not?

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct
certification?
What types of staff at your agency are involved in the process? What tasks does each
type perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

Is there much variation at the local level in the way functional responsibilities for direct
certification are carried out? If yes, please describe.
Does the direct certification process differ by district characteristics such as size? If so,
how? Why?
Does the direct certification process differ for private schools? If so, how? Why?

3

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

4.

Does the direct certification process use computer matching? [If so, ask:]
Is computer matching done at the state or local level?
[If at local/regional level, ask:]
How widely is computer matching for NSLP direct certification used among the
districts in your state?
Does the use of computer matching vary by district characteristics such as size or
public/private? If so, how?

5.

Were any technology or other infrastructure changes necessary for you to participate in the
direct certification process? Please describe.

6.

How is information related to direct certification provided and to whom (State agency,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification / matching?
PROBE: [District-level matching sites:] How is the information used for direct certification
provided to school districts?
PROBE: [State-level matching sites:] How are the lists of directly certified students
provided to school districts?
PROBE: Is there a system in place that allows schools or SFAs to look up individual
students?
PROBE: Who is responsible for notifying households of direct certification?
PROBE: Are households required to “opt in,” or are they given the option to “opt out” of
direct certification?

7.

[For district-level matching sites:] Who is responsible for compiling data from SNAP
[TANF, Medicaid, etc.] and preparing it for district use?
Are districts sent files containing all SNAP [TANF, Medicaid, etc.] cases in the state, or
only cases in their area? If the latter, what data elements are used to determine which
cases to send to each district?
[If state uses TANF or Medicaid data in direct certification, ask:] Do districts receive
separate data files for each program (SNAP, TANF, Medicaid, etc.) or a single file
containing all relevant programs? If the latter, does the state child nutrition agency
compile the data from different programs, or does the data from the SNAP agency also
contain the other programs’ data?

4

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

8.

What is the timeline for direct certification activities?
PROBE: When is the first file of SNAP[/TANF/Medicaid/other] data for direct certification
provided?
PROBE: How often is SNAP[/TANF/Medicaid/other] data for direct certification provided
during the school year?
PROBE: What is the reference point for the first file of enrollment data used for direct
certification? Is it based on prior year enrollment lists, or does it include new enrollees
and/or transfers?
PROBE: How often are the enrollment data files used for direct certification updated? Do
updated enrollment data include all students or only new students/transfers?
PROBE: [If more than once,]
[Ask States:] Do you know how many SFAs use direct certification data after the first files
of the school year?
[Ask SFAs:] Do you use direct certification data after the first files of the school year?
PROBE: When are direct certification data available to SFAs?
[State-level matching sites:] When are districts given the list of their directly certified
students?
PROBE: When are households notified that they have been directly certified?

9.

[If state-level matching state, ask:] In addition to the matching conducted at the state-level,
are individual SFAs allowed or encouraged to perform matches or identify additional
students for direct certification at the local level?
If so, why?
How does the district-level direct certification process work? What data files do they
use? How often do they conduct the process?

10. Why were the current direct certification methods and procedures chosen?
PROBE: Why does your State conduct direct certification at the [State/district] level?
PROBE: Why does/doesn’t your State/district use computer matching for direct
certification?

5

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
1.

Please describe the computer matching process:

2.

Which organization conducts the computer match between data from different sources?
Is the matching done by agency staff or a contractor?

3.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Do you use data from any programs other than SNAP? If not, why not?
PROBE: Is a Statewide Student Information System (SSIS) used for NSLP direct
certification?
If not, what is the source of the information on students enrolled in schools in the
state/district? [Interviewer: Refer to Field Guide for list of possible data sources.]

4.

What enrollment information is provided for NSLP direct certification? [Interviewer: Refer
to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

5.

[For each program providing data (SNAP, TANF, Medicaid, etc.), ask:]
What information does the [SNAP/TANF/Medicaid/etc.] agency provide for NSLP direct
certification?
[Interviewer: Refer to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

6.

Does the student enrollment file include information from the household’s NSLP
application?

7.

What is the time period covered by the student enrollment and other data used for the
match? How often is each data source updated?

6

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

8.

How are the data transferred from SNAP [/TANF/Medicaid/etc.] to the child nutrition
agency?

9.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

10. How often is each type of data transferred? What is the timeline for the data transfer
process?
11. What data use agreements cover the data transfers?
12. What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups, etc.)?
13. What data quality checks are performed on the data received?
14. How often are the data from different sources matched? What is the timeline for the
matching? Why is matching done at those times?
PROBE: After the initial data match, are later rounds of matching conducted when updated
data is received (from which agencies), when new students enroll in the SFA, or for some
other reason?
15. [If more than once, ask:]After the initial data match, are later rounds of matching conducted
in batches, or are students looked up individually?
Are the later matches conducted by the same staff who conduct the initial match?
Do the processes used for subsequent matches differ from the initial match in any other
ways?
How much variation is there in the specific identifiers and matching rules used at the
local level? What causes this variation?
[If computer matching done at the district level, ask:] Did you consult with other
districts in deciding how to set up your data matching system?
16. What specific identifiers and matching rules does your State use for NSLP direct
certification data matching?
Do you require exact matches for each of these identifiers, or do you use probabilistic
matching for some/all?

7

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

17. Can you provide information on the number of children in the SNAP [TANF, Medicaid,
etc.] file and the number matched to the student enrollment data? (If more than one match,
refer to the largest match this school year.)
Do you keep files with indicators of which students were matched and which were not?
Is the file continually updated during the school year to indicate matches?
[If district-level matching state, ask:]Do districts inform the state agency of the
outcomes of their matching process? Do districts send the state a file indicating whether
each case in the SNAP [/TANF/Medicaid/etc.] file was successfully matched or not?
18. What do you do about unmatched SNAP [/TANF/Medicaid/etc.] children? Do you have a
process for identifying and dealing with “possible” matches?
PROBE: Are there secondary match rules for these cases? What are they?
19. What do you do about duplicate matches?
PROBE: Are there secondary match rules for these cases? What are they?
20. Does your matching process include any specific procedures that respond to FNS’s new
policy on Extending Categorical Eligibility to Additional Children in a Household? Please
describe.
21. Why were the current identifiers and matching rules chosen?
What are the advantages and disadvantages of the identifiers and matching rules that you
use?
Did you consider alternatives to these identifiers and rules? Why did you reject those
alternatives?
22. Did your agency incur any startup costs in implementing a computer matching system?
Did SFAs have to implement any technology upgrades to use direct certification?
Who developed the software and data systems used for direct certification?
Did you consult with other states/districts in developing your system? If so, why were
these states/districts chosen for consultation?
What are the costs of maintaining the computer matching system now that it has been set
up?
GO TO SECTION III

8

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

III. GRANTS
1.

2.

Has your State/district applied for any grant funding to support direct certification?
[If no:]
Do you have plans to apply for grant funding to support direct certification in the future?
If so, what would you use grant funding for?
[Interviewer: Skip to Section IV]

3.

[If yes:]
Was this a direct certification grant from FNS, or some other type of grant?
Was the grant funded under the American Recovery and Reinvestment Act?

4.

Was your application approved and funded?

5.

[If no:]
What did you propose to use the grant funding for?
[Interviewer: Skip to Section IV]

6.

[If yes:]
When was the grant awarded?

7.

What was the amount of the grant?

8.

What activities were planned under the grant?
PROBE: Expansion of direct certification to include programs like TANF, Medicaid, and
SCHIP in addition to SNAP? Investment in technology (to enhance software, develop online
query systems, or integrate existing programs’ data systems)? To overcome specific
challenges? To develop/improve direct verification process?
Are grant funds being used for a pilot? If so, when will full implementation take place?

9.

Which activities have been implemented?

10. How has your implementation of grant activities deviated from your original plans? Why
did you alter your plans?
11. Who has been involved in activities under the grant? Describe their roles.
12. What challenges have you encountered in implementing grant activities? How have you
addressed these challenges?

9

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

13. Do you feel like the grant has accomplished its goals?
What results have you seen from the grant activities? Do these results meet your
expectations?
How do you track the success of the grant activities?
[If no results yet, ask:] When do you expect to see results?
14. Will the State/district continue these activities after the grant funding ends?
GO TO SECTION IV

10

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
1.

When (in what school year) did your State/district begin directly certifying students for the
NSLP?

2.

How long have you used your current methods for direct certification?

3.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP (besides any discussed in Section III).
[Probe for any changes indicated in survey.]
PROBE: Has direct certification always been conducted at the [State/district] level in your
State?
PROBE: Have the roles played by each agency changed at all? How?
PROBE: Have the data sources or specific data elements or algorithms used for matching
changed? How?
PROBE: Has the frequency with which matches are conducted changed? How?

4.

When were these changes made?

5.

Why were these changes made?
PROBE: Have you made any changes to your direct certification procedures or guidance for
SFAs in response to FNS’s new policy on Extending Categorical Eligibility to Additional
Children in a Household that became effective in SY2009-2010?
PROBE: Most states have experienced large increases in the SNAP caseload recently. Have
such changes affected your direct certification procedures at the State or local level?

6.

Do you consider these changes to be an improvement or not?

7.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

11

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe
PROBE: [If district-level matching used] Have you considered conducting the matching at
the state level?
PROBE: If a Statewide Student Information System (SSIS) is not currently used for NSLP
direct certification: does your State plan to develop a SSIS or use an existing SSIS for this
purpose? What changed to make this possible?
PROBE: [If TANF/Medicaid are not used to directly certify students] Have you considered
directly certifying students based on programs other than SNAP, such as TANF or
Medicaid? Why/ why not?
PROBE: Do you plan any changes to the specific data elements or algorithms used for
matching? If so, how?
PROBE: Do you plan any changes to frequency of matching during the school year?
PROBE: Are you planning to make any changes to direct certification as a result of FNS’s
policy to extend categorical eligibility to all children in a household?
PROBE: Most states have experienced large increases in the SNAP caseload recently. Are
you planning any changes in direct certification procedures in response to such changes?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

4.

Is there anything you would like to change about your State's direct certification process?
Why haven’t those changes been made? What obstacles stand in the way of such changes?

12

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
1.

Would it be possible to expand the current direct certification process to use information
from other means-tested programs, such as TANF, Medicaid or SCHIP, child care
assistance, or energy assistance? How feasible is this option?
If so: What data would be required? What steps would be necessary? What, in your view,
would be the greatest challenges?
If not: Why not?
PROBE: Do the eligibility criteria differ between NSLP and other programs?
PROBE: Are the necessary data not available?
PROBE: Would confidentiality concerns prevent sharing the data between programs?

2.

[For sites where a Statewide Student Information System (SSIS) is not used for direct
certification:] Why is the SSIS not currently used for NSLP direct certification?

[For sites where TANF/Medicaid data systems are not used for direct certification:]
Next, we’d like to talk about data sharing between schools and programs like TANF and
Medicaid.
3.

Do any school districts exchange data with TANF, Medicaid/SCHIP, or other means-tested
programs?
[If so:]
What is the purpose of these data exchanges? (For example, do you/SFAs provide data
on NSLP certifications for referrals to the Medicaid or SCHIP program?)
How does the process work?
PROBE: What are the sources of the data (district or statewide student information systems,
or SFA food service information systems (POS etc.)?
How are the data exchanged?
PROBE: Who conducts the match (the school district or the program)?
What data elements are used for matching?
PROBE: Does the process vary by SFA?
PROBE: Does the process use computer matching for all school districts? If not, what are
the barriers to wider use of computer matching?
PROBE: If manual methods are used (e.g., sharing copies of application forms), what would
make an electronic process feasible for some or all SFAs in your State?
13

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

PROBE: What are the main challenges in the matching process as it currently operates?
(technology, staffing, cost, school district cooperation, privacy or other legal restrictions?)
4.

[If no data sharing with TANF/Medicaid, ask:] Have State or local agencies in your State
considered this option? What do you see as the pros and cons of such data sharing? What
would make it feasible for some or all SFAs in your State?

GO TO SECTION V

14

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your State?
PROBE: How effectively does the current system identify children receiving SNAP
[/TANF/Medicaid/etc.]?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

2.

Does the effectiveness of direct certification vary by any district characteristics, such as size
or public vs. private schools?

3.

How often do families receiving SNAP [/TANF/Medicaid/etc.] submit NSLP applications?
Why does the direct certification process miss these families?

4.

How do you measure the effectiveness of your direct certification process?
[If district-level matching, ask:] Do you assess effectiveness at the district level?
Are any written reports available on direct certification outcomes within the state? If so,
could we have a copy?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Have some types of districts had greater success than others in implementing direct
certification? What types of districts have been most successful?

5.

Are there any unique state or local factors in your State that contribute to the success you’ve
had in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State?

15

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

2.

Which aspects of the direct certification process do you think are most challenging?

3.

[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.
Have any legal issues or consent requirements limited the ability to use computer matching
for direct certification in your State/district?

4.

[If computer matching is not used, ask:]What are the barriers to use of computer matching?

5.

[If computer matching is used for some, but not all, SFAs in the State, ask:] What are the
barriers to wider use of computer matching among SFAs in your State?

6.

What kinds of problems do school districts encounter in receiving and using direct
certification data?
What assistance does the state provide?
How do these problems or other factors affect whether school districts use the information
from computer matching?

7.

Have you encountered any barriers or challenges to implementing the policy to extend
categorical eligibility to all children in a household?

8.

Do key barriers to direct certification vary by district characteristics such as size or
public/private? If so, how?

9.

How have you attempted to address the challenges you’ve encountered? Have the
challenges been resolved?

10. In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?
11. Are there any unique state or local factors that contribute to the challenges you’ve
encountered in implementing direct certification?

16

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

1.

What advice would you give to another state that was struggling with implementing direct
certification effectively?

2.

[If the state/district has experience with more than one method—either because of a change,
or district-level differences within the state, or if the direct certification process uses both
the letter method and computer matching—ask:] What are the challenges and advantages of
each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match SNAP
data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
SNAP but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?

4.

Are there direct certification methods or procedures you would advise other States/districts
to avoid? Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

GO TO SECTION VI

17

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH STATE CHILD NUTRITION STAFF

VI. CONCLUDING QUESTIONS
1.

Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your State/district?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

END OF INTERVIEW

18

2.

State Education Staff

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate your participation in today’s in-depth interview about direct certification
processes.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at
a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to
ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?
Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your State, the role your agency plays in direct certification, and explore the possibility of future
changes in those methods.
1.

Is your agency involved with NSLP direct certification?

2.

Please briefly describe the role of your agency in the direct certification process.

GO TO SECTION II

2

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview and
probe any inconsistencies.]
A. DIRECT CERTIFICATION PROCESS
1.

How is direct certification currently done in your State? Please describe the process.

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct
certification?
Why did your agency take on the role(s) it plays?
What types of staff at your agency are involved in the process? What tasks does each
type perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

How is information related to direct certification provided and to whom (State agencies,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification/matching? [district-level matching sites:] How is the information
used for direct certification provided to school districts?
PROBE: Who is responsible for notifying households of direct certification? Are
households required to “opt in,” or are they given the option to “opt out” of direct
certification?

4.

What is the timeline for direct certification activities?
PROBE: When is the first file of SNAP[/TANF/Medicaid/other] data for direct certification
provided?
PROBE: What is the reference point for the first file of enrollment data used for direct
certification? Is it based on prior year enrollment lists, or does it include new enrollees
and/or transfers?

3

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

PROBE: How often are the enrollment data files used for direct certification updated? Do
updated enrollment data include all students or only new students/transfers?
PROBE: When are households notified that they have been directly certified?
5.

Why were the current direct certification methods and procedures chosen?

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
1.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Do you use data from any programs other than SNAP? If not, why not?
PROBE: Is a Statewide Student Information System (SSIS) used for NSLP direct
certification? If not, what is the source of the information on students enrolled in schools in
the State/district? [Interviewer: refer to Field Guide for list of possible data sources.]

4.

What enrollment information is provided for NSLP direct certification? [Interviewer: refer
to Field Guide for list of possible data elements.] Is there a data dictionary that lists the file
specifications, data elements, their definitions, formats, etc? Could we have a copy of that?

5.

Does the student enrollment file include information from the household’s NSLP
application?

6.

What is the time period covered by the student enrollment data used for the match? How
often is the data source updated?

7.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

8.

How often is each type of data transferred? What is the timeline for the data transfer
process?

9.

What data use agreements cover the data transfers?

10. What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups, etc)?

4

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

11. Did your agency have input on how the matching process works?
PROBE: Did you have input regarding the choice of identifiers, matching rules, file format,
and/or who has access to your agency’s data?
12. [Interviewer Instruction: only ask if the agency is involved in a lead role, including
conducting a data match.] Why were the current identifiers and matching rules chosen?
13. Did your agency incur any startup costs in implementing a computer matching system?
GO TO SECTION IV

5

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
1.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP.
[Probe for any changes indicated in survey.]

2.

PROBE: Have the roles played by each agency changed at all? How?
When were these changes made?

3.

Why were these changes made?

4.

Do you consider these changes to be an improvement or not?

5.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe.
PROBE: [If a Statewide Student Information System (SSIS) is not currently used for NSLP
direct certification:] Does your State plan to develop a SSIS or use an existing SSIS for this
purpose? What changed to make this possible?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
[Interviewer instruction: details on data sources currently used for direct certification are
discussed in earlier sections. In this section, we ask for information on data sources that are
not currently used for direct certification, to explore the possibility of their use in the future.]
1.

[For sites where a Statewide Student Information System (SSIS) is not used for direct
certification:] Next, we’d like to ask about your student enrollment data systems. Does the
SEA collect individual student data? Are these data maintained in a Statewide Student
Information System (SSIS)?

6

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

[If so, ask:]
How often are these data collected from districts?
What are the functions of the SSIS? Are there legal or regulatory restrictions on the
information that can be maintained in the SSIS?
What data elements are collected?
PROBE: Does the SSIS contain date of birth and Social Security Number of students? What
information does the system contain for parents or guardians?
PROBE: Does the SSIS collect information on the free/reduced-price NSLP status of
individual students? If not, why not? Does the SSIS include any information from the
household’s NSLP application?
PROBE: Does the SSIS include all LEAs in the state? If not, why not? Are there certain
types of districts that are typically included/excluded? Does the SSIS include private
schools? What would have to happen to reach statewide implementation?
PROBE: Do school districts have direct access to the SSIS? Do they access the system to
view student information, update student information, or both?
PROBE: What is the process for updating information in the SSIS? How often is it updated?
PROBE: Why is the SSIS not currently used for NSLP direct certification?
2.

What identifying and demographic data are the hardest to keep accurate in your databases,
and why?
PROBE: Is there a problem with duplicate records? If so, how large is the problem and what
causes it?
PROBE: Is there a problem with timeliness of data, either overall or with certain types of
districts or data? If so, how large is the problem and what causes it?

3.

[For sites where TANF/Medicaid data systems are not used for direct certification:] Next,
we’d like to talk about data sharing between schools and programs like TANF and
Medicaid. Do your school districts exchange data with TANF, Medicaid/SCHIP, or other
means-tested programs?

7

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

[If so, ask:]
What is the purpose of these data exchanges? (For example, is the SFA involved with
computer matching for Medicaid administrative claiming (MAC)? Do you provide data on
NSLP certifications for referrals to the Medicaid or SCHIP program? Does Medicaid
participate in a data matching program of individual records to determine Medicaid
reimbursements to school districts?)
How does the process work?
PROBE: What are the sources of the data (district or statewide student information systems,
or SFA food service information systems (POS etc.)?
PROBE: How are the data exchanged?
PROBE: Who conducts the match (the school districts or the program)?
PROBE: What data elements are used for matching?
PROBE: Does the process use computer matching in any school districts? If not, what are
the barriers to computer matching?
PROBE: If manual methods are used (e.g., sharing copies of application forms), what would
make an electronic process feasible for your SFAs?
PROBE: What are the main challenges in the matching process as it currently operates?
(technology, staffing, cost, school district cooperation, privacy or other legal restrictions?)

8

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your State?
PROBE: How effectively does the current system identify children receiving SNAP
[/TANF/Medicaid/etc.]?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

2.

How do you measure the effectiveness of your direct certification process?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Are there any unique state or local factors in your State that contribute to the success you’ve
had in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State?

2.

Which aspects of the direct certification process do you think are most challenging?

3.

[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.
PROBE: Have any legal issues or consent requirements limited the ability to use computer
matching for direct certification in your State?

4.

[If computer matching is not used, ask:] What are the barriers to use of computer matching?

5.

What kinds of problems do school districts encounter in receiving and using direct
certification data?

9

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

What assistance does the State provide?
How do these problems or other factors affect whether school districts use the information
from computer matching?
6.

How have you attempted to address the challenges you’ve encountered? Have the
challenges been resolved?

7.

In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?

8.

Are there any unique state or local factors that contribute to the challenges you’ve
encountered in implementing direct certification?

D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

1.

What advice would you give to another State that was struggling with implementing direct
certification effectively?

2.

[If the State has experience with more than one method—either because of a change, or
district-level differences within the State, or if the direct certification process uses both the
letter method and computer matching—ask:] What are the challenges and advantages of
each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match SNAP
data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
SNAP but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?

10

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

4.

Are there direct certification methods or procedures you would advise other States to avoid?
Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

11

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH EDUCATION AGENCY STAFF

VI. CONCLUDING QUESTIONS
1.

Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your State?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

12

3.

State SNAP Staff

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate your participation in today’s in-depth interview about direct certification
processes.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at
a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to
ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?

Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your State, the role your agency plays in direct certification, and explore the possibility of future
changes in those methods.
1.

Is your agency involved with NSLP direct certification?

2.

Please briefly describe the role of your agency in the direct certification process.

GO TO SECTION II

2

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview
and probe any inconsistencies.]
A. DIRECT CERTIFICATION PROCESS
1.

How is direct certification currently done in your State/district? Please describe the process.

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct certification?
Why did your agency take on the role(s) it plays?
What types of staff at your agency are involved in the process? What tasks does each type
perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

How is information related to direct certification provided and to whom (State agency,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification / matching?
PROBE: [District-level matching sites] How is the information used for direct certification
provided to school districts?

4.

What is the timeline for direct certification activities?
PROBE: When is the first file of SNAP data for direct certification provided?
PROBE: How often is SNAP data for direct certification provided during the school year?

5.

Why were the current direct certification methods and procedures chosen?

3

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
1.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Does your state use data from any programs other than SNAP? If not, why not?

2.

What information does the SNAP agency provide for NSLP direct certification?
[Interviewer: refer to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

3.

What is the time period covered by the student enrollment and other data used for the
match? How often is each data source updated?

4.

How are the data transferred from SNAP to the child nutrition agency?

5.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

6.

How often is each type of data transferred? What is the timeline for the data transfer
process?

7.

What data use agreements cover the data transfers?

8.

What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups, etc.)?

9.

Did your agency have input on how the matching process works?
PROBE: Did you have input regarding the choice of identifiers, matching rules, file format,
and/or who has access to your agency’s data?

10. [Interviewer Instruction: Only ask if the agency is involved in a lead role, including
conducting a data match.] Why were the current identifiers and matching rules chosen?
11. Did your agency incur any startup costs in implementing a computer matching system?
4

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

GO TO SECTION IV

5

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
1.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP.
[Probe for any changes indicated in survey.]
PROBE: Have the roles played by each agency changed at all? How?

2.

When were these changes made?

3.

Why were these changes made?
PROBE: Most states have experienced large increases in the SNAP caseload recently. Have
such changes affected your direct certification procedures at the State or local level?

4.

Do you consider these changes to be an improvement or not?

5.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe
PROBE: Most states have experienced large increases in the SNAP caseload recently. Are
you planning any changes in direct certification procedures in response to such changes?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

6

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
[For sites where TANF/Medicaid data systems are not used for direct certification:] Next, we’d
like to talk about your data systems and about integration and data sharing between SNAP,
TANF, and Medicaid.
1.

What is the organizational relationship of the State SNAP, Medicaid, and TANF agencies?
Is there a joint application process for Medicaid and SNAP?

2.

Is the eligibility management system for programs like SNAP, TANF, and Medicaid
integrated in your State?

3.

Are other agencies integrated with this system? Which agencies?

4.

Does your State maintain a client registry or client index of social service clients? If so:
Which agency maintains this information system?
What other agencies are included?
How frequently does your agency send information to this system?
What types of information does your agency send to this system?

5.

What identifying and demographic data are the hardest to keep accurate in your databases,
and why?
PROBE: Is there a problem with duplicate records? If so, how large is the problem and what
causes it?
PROBE: Is there a problem with timeliness of data, either overall or with certain types of
districts or data? If so, how large is the problem and what causes it?

GO TO SECTION V

7

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your State?
PROBE: How effectively does the current system identify children receiving SNAP?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Are there any unique state or local factors in your State that contribute to the success you’ve
had in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State?

2.

Which aspects of the direct certification process do you think are most challenging?

3.

[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.
PROBE: Have any legal issues or consent requirements limited the ability to use computer
matching for direct certification in your State/district?

4.

[If computer matching is not used, ask:]What are the barriers to use of computer matching?

5.

How have you attempted to address the challenges you’ve encountered? Have the
challenges been resolved?

6.

In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?

8

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

7.

Are there any unique state or local factors that contribute to the challenges you’ve
encountered in implementing direct certification?

D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

1.

What advice would you give to another state that was struggling with implementing direct
certification effectively?

2.

[If the state has experience with more than one method—either because of a change, or
district-level differences within the state, or if the direct certification process uses both the
letter method and computer matching—ask:] What are the challenges and advantages of
each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match SNAP
data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
SNAP but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?

4.

Are there direct certification methods or procedures you would advise other States/districts
to avoid? Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

GO TO SECTION VI

9

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH SNAP STAFF

VI. CONCLUDING QUESTIONS
1.

Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your State/district?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

END OF INTERVIEW

10

4.

State Medicaid/TANF/Other Staff

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate your participation in today’s in-depth interview about direct certification
and related processes.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at
a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to
ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?

Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your State, the role your agency plays in direct certification, and explore the possibility of future
changes in those methods.
1.

Is your agency involved with NSLP direct certification?
[If not:]
Has your agency considered becoming involved in direct certification? Are you familiar
with the NSLP direct certification process?
[Skip to Section IV.3]

2.

[Interviewer Instruction: only ask if the agency is currently involved in direct certification.]
Please briefly describe the role of your agency in the direct certification process.

GO TO SECTION II

2

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
A. DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview
and probe any inconsistencies.]

1.

[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
How is direct certification currently done in your State? Please describe the process.
Do you directly certify participants in any public assistance programs other than SNAP?
Which ones? (TANF, Medicaid, etc.)
Why/why not?

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct certification?
Why did your agency take on the role(s) it plays?
What types of staff at your agency are involved in the process? What tasks does each type
perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

How is information related to direct certification provided and to whom (State agency,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification / matching?
PROBE: [District-level matching sites] How is the information used for direct certification
provided to school districts?

4.

What is the timeline for direct certification activities?

3

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

PROBE: When is the first file of TANF/Medicaid/other data for direct certification
provided?
PROBE: How often is TANF/Medicaid/other data for direct certification provided during
the school year?
5.

Why were the current direct certification methods and procedures chosen?

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching, and only if the agency is currently involved in direct certification.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
1.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Does your state use data from any programs other than SNAP? If not, why not?

2.

[For each program providing data (TANF, Medicaid, etc.), ask:]
What information does the TANF/Medicaid/etc. agency provide for NSLP direct
certification?
[Interviewer: refer to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

3.

What is the time period covered by the student enrollment and other data used for the
match? How often is each data source updated?

4.

How are the data transferred from TANF/Medicaid/etc. to the child nutrition agency?

5.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

6.

How often is each type of data transferred? What is the timeline for the data transfer
process?

7.

What data use agreements cover the data transfers?

8.

What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups, etc.)?
4

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

9.

Did your agency have input on how the matching process works?
PROBE: Did you have input regarding the choice of identifiers, matching rules, file format,
and/or who has access to your agency’s data?

10. Why were the current identifiers and matching rules chosen?
11. Did your agency incur any startup costs in implementing a computer matching system?
GO TO SECTION IV

5

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
1.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP.
[Probe for any changes indicated in survey.]
PROBE: Have the roles played by your agency changed at all? How?

2.

When were these changes made?

3.

Why were these changes made?

4.

Do you consider these changes to be an improvement or not?

5.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe
PROBE: [If TANF/Medicaid are not used to directly certify students]: Have you considered
directly certifying students based on programs such as TANF or Medicaid? Why/why not?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
1.

Would it be possible to expand the current direct certification process to use information
from other means-tested programs, such as TANF, Medicaid, or SCHIP, child care
assistance, or energy assistance? How feasible is this option?
If so:

6

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

What data would be required? What steps would be necessary? What, in your view, would
be the greatest challenges?
If not: Why not?
PROBE: Do the eligibility criteria differ between NSLP and other programs? Are the
necessary data not available? Would confidentiality concerns prevent sharing the data
between programs?
[For sites where TANF/Medicaid data systems are not used for direct certification:] Next, we’d
like to talk about your data systems and about integration and data sharing between SNAP,
TANF, and Medicaid.
2.

What is the organizational relationship of the State SNAP, Medicaid, and TANF agencies?
Is there a joint application process for Medicaid and SNAP?

3.

Is the eligibility management system for programs like SNAP, TANF, and Medicaid
integrated in your State?

4.

Are other agencies integrated with this system? Which agencies?

5.

Does your State maintain a client registry or client index of social service clients?

6.

What identifying and demographic data are the hardest to keep accurate in your databases,
and why?
PROBE: Is there a problem with duplicate records? If so, how large is the problem and what
causes it?
PROBE: Is there a problem with timeliness of data, either overall or with certain types of
districts or data? If so, how large is the problem and what causes it?

7.

[If data systems SNAP, TANF, and Medicaid not integrated, ask:] Does Medicaid share
eligibility and/or income information with other means-tested programs for purposes of
identifying Medicaid enrollees who may be eligible for other programs or establishing or
verifying eligibility for applicants to other programs? If so:
Which means-tested programs receive eligibility or income information from the Medicaid
agency?
How often is the information furnished?
Which data elements are shared?
How are the data matched?
What agreements govern this information sharing?
7

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

8.

Does the Medicaid application contain informed consent or disclosure of data sharing
activities? Do federal or State regulations prohibit Medicaid from sharing data with other
means-tested programs?

[For sites where TANF/Medicaid data systems are not used for direct certification:]
Next, we’d like to talk about data sharing between schools and programs like TANF and
Medicaid.
9.

Do any school districts exchange data with TANF, Medicaid/SCHIP, or other means-tested
programs?
[If so:]
What is the purpose of these data exchanges? (For example, is the SEA or SFAs involved
with computer matching for Medicaid administrative claiming (MAC)? Do SFAs provide
data on NSLP certifications for referrals to the Medicaid or SCHIP program? Does
Medicaid participate in a data matching program of individual records to determine
Medicaid reimbursements to school districts?)
How does the process work?
PROBE: What are the sources of the data (district or statewide student information systems,
or SFA food service information systems (POS etc.)? How are the data exchanged?
PROBE: Who conducts the match (the school district or the program)? What data elements
are used for matching?
PROBE: Does the process vary by SFA?
PROBE: Does the process use computer matching for all school districts? If not, what are
the barriers to wider use of computer matching?
PROBE: If manual methods are used (e.g., sharing copies of application forms), what would
make an electronic process feasible for some or all SFAs in your State?
PROBE: What are the main challenges in the matching process as it currently operates?
(technology, staffing, cost, school district cooperation, privacy or other legal restrictions?)

10. [If no data sharing with TANF/Medicaid, ask:] Have State or local agencies in your State
considered this option? What do you see as the pros and cons of such data sharing? What
would make it feasible for some or all SFAs in your State?

GO TO SECTION V

8

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your State?
PROBE: How effectively does the current system identify children receiving
TANF/Medicaid/etc.?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Are there any unique state or local factors in your State that contribute to the success you’ve
had in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State?

2.

Which aspects of the direct certification process do you think are most challenging?

3.

[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.
Have any legal issues or consent requirements limited the ability to use computer
matching for direct certification in your State/district?

4.

[If computer matching is not used, ask:] What are the barriers to use of computer matching?

9

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

5.

How have you attempted to address the challenges you’ve encountered? Have the
challenges been resolved?

6.

In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?

7.

Are there any unique state or local factors that contribute to the challenges you’ve
encountered in implementing direct certification?

[For those not involved in direct certification, ask:]
8. What barriers stand in the way of your agency being involved in direct certification?
D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
1.

What advice would you give to another state that was struggling with implementing direct
certification effectively?

2.

[If the state has experience with more than one method—either because of a change, or
district-level differences within the state, or if the direct certification process uses both the
letter method and computer matching—ask:] What are the challenges and advantages of
each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match
TANF/Medicaid/other data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
TANF/Medicaid/other but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?

10

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

4.

Are there direct certification methods or procedures you would advise other States/districts
to avoid? Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

GO TO SECTION VI

11

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TANF/MEDICAID/OTHER
AGENCY STAFF

VI. CONCLUDING QUESTIONS

1.

[Interviewer Instruction: Only ask if the agency is currently involved in direct certification.]
Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your State/district?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

END OF INTERVIEW

12

5.

LEA/SFA Staff

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate your participation in today’s in-depth interview about the direct
certification processes used in your district.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at a
time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to ask
you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?

Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your district, the role your agency plays in direct certification, and explore the possibility of
future changes in those methods.
1.

Please briefly describe the role of your staff in the direct certification process.

GO TO SECTION II

2

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview and
probe any inconsistencies.]
A. DIRECT CERTIFICATION PROCESS
1.

How is direct certification currently done in your district? Please describe the process.
PROBE: Is direct certification conducted primarily at the state level or at the district level?
[If at the district level, ask:] What guidelines does the state provide districts concerning data
matching for NSLP direct certification?
PROBE: Does the state require or recommend the use of specific identifiers and/or matching
rules for SFAs to use? What are they?
Do you directly certify participants in any public assistance programs other than SNAP?
Which ones? (TANF, Medicaid, etc.). Why/Why not?

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct certification?
What types of staff at your agency are involved in the process? What tasks does each type
perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

Does the direct certification process use computer matching? [If so, ask:]
Is computer matching done at the state or local level?

4.

Were any technology or other infrastructure changes necessary for you to participate in the
direct certification process? Please describe.

5.

How is information related to direct certification provided and to whom (State agency,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification / matching?
3

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

PROBE: [District-level matching sites:] How is the information used for direct certification
provided to school districts?
PROBE: [State-level matching sites:] How are the lists of directly certified students
provided to school districts?
PROBE: Is there a system in place that allows schools or SFAs to look up individual
students?
PROBE: Who is responsible for notifying households of direct certification? Are households
required to “opt in,” or are they given the option to “opt out” of direct certification?
6.

[For district-level matching sites:] Who is responsible for compiling data from SNAP
[TANF, Medicaid, etc.] and preparing it for district use?
PROBE: Are districts sent files containing all SNAP [TANF, Medicaid, etc.] cases in the
state, or only cases in their area? If the latter, what data elements are used to determine
which cases to send to each district?
PROBE: [If state uses TANF or Medicaid data in direct certification, ask:] Do districts
receive separate data files for each program (SNAP, TANF, Medicaid, etc.) or a single file
containing all relevant programs? If the latter, does the state child nutrition agency compile
the data from different programs, or does the data from the SNAP agency also contain the
other programs’ data?
7. What is the timeline for direct certification activities?
[Interviewer Instruction: Only ask next four probes if the district is involved in a lead role,
including conducting a data match:]
PROBE: When is the first file of SNAP[/TANF/Medicaid/other] data for direct certification
provided?
PROBE: How often is SNAP[/TANF/Medicaid/other] data for direct certification provided
during the school year?
PROBE: What is the reference point for the first file of enrollment data used for direct
certification? Is it based on prior year enrollment lists, or does it include new enrollees
and/or transfers?
PROBE: How often are the enrollment data files used for direct certification updated? Do
updated enrollment data include all students or only new students/transfers?
[If more than once, ask:] Do you use direct certification data after the first files of the school
year?

4

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

PROBE: When are direct certification data available to SFAs?
PROBE: [State-level matching sites:] When are districts given the list of their directly
certified students?
PROBE: When are households notified that they have been directly certified?
8.

[If state-level matching site, ask:] In addition to the matching conducted at the state-level,
are individual SFAs allowed or encouraged to perform matches or identify additional
students for direct certification at the local level?
If so, why?
How does the district-level direct certification process work? What data files do you use?
How often do you conduct the process?

9.

[Interviewer Instruction: Only ask if the district is involved in a lead role, including
conducting a data match] Why were the current direct certification methods and procedures
chosen?
PROBE: Why does/doesn’t your State/district use computer matching for direct
certification?

5

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
[Interviewer Instruction: Ask Questions 1-4 only if the district is involved in a lead role,
including conducting a data match:]
1.

Which organization conducts the computer match between data from different sources? Is
the matching done by agency staff or a contractor?

2.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Do you use data from any programs other than SNAP? If not, why not?
PROBE: [Interviewer Instruction: Ask of all] Is a Statewide Student Information System
(SSIS) used for NSLP direct certification? If not, what is the source of the information on
students enrolled in schools in the state/district? [Interviewer: Refer to Field Guide for list of
possible data sources.]

3.

What enrollment information is provided for NSLP direct certification? [Interviewer: Refer
to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

4.

[For each program providing data (SNAP, TANF, Medicaid, etc.), ask:]
What information does the [SNAP/TANF/Medicaid/etc.] agency provide for NSLP direct
certification? [Interviewer: Refer to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

5.

Does the student enrollment file include information from the household’s NSLP
application?

6.

[Interviewer Instruction: Only ask if the district is involved in a lead role, including
conducting a data match:] What is the time period covered by the student enrollment and
other data used for the match? How often is each data source updated?

6

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

7.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

[Interviewer Instruction: Ask Questions 8-18 only if the district is involved in a lead role,
including conducting a data match:]
8. How often is each type of data transferred? What is the timeline for the data transfer
process?
9.

What data use agreements cover the data transfers?

10. What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups..etc)?
11. What data quality checks are performed on the data received?
12. How often are the data from different sources matched? What is the timeline for the
matching? Why is matching done at those times?
PROBE: After the initial data match, are later rounds of matching conducted when updated
data is received (from which agencies?), when new students enroll in the SFA, or for some
other reason?
13. [If more than once, ask:]After the initial data match, are later rounds of matching conducted
in batches, or are students looked up individually?
Are the later matches conducted by the same staff who conduct the initial match?
Do the processes used for subsequent matches differ from the initial match in any other
ways?
[If computer matching done at the district level, ask:] Did you consult with other districts in
deciding how to set up your data matching system?
14. What specific identifiers and matching rules does your district use for NSLP direct
certification data matching?
Do you require exact matches for each of these identifiers, or do you use probabilistic
matching for some/all?
15. Can you provide information on the number of children in the SNAP [TANF, Medicaid,
etc.] file and the number matched to the student enrollment data? (If more than one match,
refer to the largest match this school year.)
Do you keep files with indicators of which students were matched and which were not? Is
the file continually updated during the school year to indicate matches?
7

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

[If district-level matching state, ask:] Do districts inform the state agency of the outcomes of
their matching process? Do districts send the state a file indicating whether each case in the
SNAP [/TANF/Medicaid/etc.] file was successfully matched or not?
16. What do you do about unmatched SNAP [/TANF/Medicaid/etc.] children? Do you have a
process for identifying and dealing with “possible” matches?
PROBE: Are there secondary match rules for these cases? What are they?
17. What do you do about duplicate matches?
PROBE: Are there secondary match rules for these cases? What are they?
18. Does your matching process include any specific procedures that respond to FNS’s new
policy on Extending Categorical Eligibility to Additional Children in a Household? Please
describe.
19. Did your district have input on how the matching process works?
PROBE: did you have input regarding the choice of identifiers, matching rules, file format,
and/or who has access to your district’s data?
[Interviewer Instruction: Ask Questions 20 and 21 only if the district is involved in a lead
role, including conducting a data match:]
20. Why were the current identifiers and matching rules chosen?
What are the advantages and disadvantages of the identifiers and matching rules that you
use? Did you consider alternatives to these identifiers and rules? Why did you reject those
alternatives?
21. Did your district incur any startup costs in implementing a computer matching system?
Did you have to implement any technology upgrades to use direct certification?
Who developed the software and data systems used for direct certification?
Did you consult with other states/districts in developing your system? If so, why were these
states/districts chosen for consultation?
What are the costs of maintaining the computer matching system now that it has been set
up?
GO TO SECTION III

8

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

III. GRANTS
A. USES OF DIRECT CERTIFICATION AND ARRA PROGRAMS
1.

Has your district applied for any grant funding to support direct certification?

2.

[If no:]
Do you have plans to apply for grant funding to support direct certification in the future?
If so, what would you use grant funding for?
[Interviewer: Skip to Section IV]

3.

[If yes:]
Was this a direct certification grant from FNS, or some other type of grant?
Was the grant funded under the American Recovery and Reinvestment Act?

4.

Was your application approved and funded?

5.

[If no:]
What did you propose to use the grant funding for?

[Interviewer: Skip to Section IV]

6.

[If yes:]
When was the grant awarded?

7.

What was the amount of the grant?

8.

What activities were planned under the grant?
PROBE: Expansion of direct certification to include programs like TANF, Medicaid, and
SCHIP in addition to SNAP? Investment in technology (to enhance software, develop online
query systems, or integrate existing programs’ data systems)? To overcome specific
challenges? To develop/improve direct verification process?
Are grant funds being used for a pilot? If so, when will full implementation take place?

9.

Which activities have been implemented?

10. How has your implementation of grant activities deviated from your original plans? Why
did you alter your plans?
11. Who has been involved in activities under the grant? Describe their roles.
9

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

12. What challenges have you encountered in implementing grant activities? How have you
addressed these challenges?
13. Do you feel like the grant has accomplished its goals?
What results have you seen from the grant activities? Do these results you’re your
expectations?
How do you track the success of the grant activities? [If no results yet, ask:] When do you
expect to see results?
14. Will the district continue these activities after the grant funding ends?
GO TO SECTION IV

10

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
1.

When (in what school year) did your district begin directly certifying students for the
NSLP?

2.

How long have you used your current methods for direct certification?

3.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP (besides any discussed in Section III).
[Probe for any changes indicated in survey.]
PROBE: Has direct certification always been conducted at the [State/district] level in your
State?
PROBE: Have the roles played by each agency changed at all? How?
PROBE: Have the data sources or specific data elements or algorithms used for matching
changed? How?
PROBE: Has the frequency with which matches are conducted changed? How?

4.

When were these changes made?

5.

Why were these changes made?
PROBE: Have you made any changes to your direct certification procedures or guidance for
SFAs in response to FNS’s new policy on Extending Categorical Eligibility to Additional
Children in a Household that became effective in SY2009-2010?
PROBE: Most states have experienced large increases in the SNAP caseload recently. Have
such changes affected your direct certification procedures at the State or local level?

6.

Do you consider these changes to be an improvement or not?

7.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

11

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe.
PROBE: [Interviewer Instruction: Only ask if the district is involved in a lead role,
including conducting a data match:] Do you plan any changes to the specific data elements
or algorithms used for matching? If so, how?
PROBE: Do you plan any changes to frequency of matching during the school year?
PROBE: Are you planning to make any changes to direct certification as a result of FNS’s
policy to extend categorical eligibility to all children in a household?
PROBE: Most states have experienced large increases in the SNAP caseload recently. Are
you planning any changes in direct certification procedures in response to such changes?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
1.

[If computer matching not used for direct certification:] We’d like to ask about other ways
in which technology is used in the NSLP application and verification processes.
Are computers used to determine eligibility status in the NSLP application process? How
much data is entered from applications for this procedure?
Can households submit NSLP applications via the Internet? Does your SFA use the Internet
to communicate with the State Child Nutrition agency?
Is computerized sampling used to select samples for verification?
How is categorical eligibility verified?
PROBE: Are SNAP or TANF administrative records used for this process? Who does this?
What SFA records are used?

[Interviewer instruction: Details on data sources currently used for direct certification are
discussed in earlier sections. In this section, we ask for information on data sources that are not
currently used for direct certification, to explore the possibility of their use in the future.]
[For sites where a Statewide Student Information System (SSIS) is not used for direct
certification:] Next, we’d like to ask about your student enrollment data systems.
12

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

2.

Does the SEA collect individual student data? Are these data maintained in a Statewide
Student Information System (SSIS)?
[If so, ask:]
How often are these data collected from districts?
Why is the SSIS not currently used for NSLP direct certification?
[For sites where TANF/Medicaid data systems are not used for direct certification:] Next,
we’d like to talk about data sharing between schools and programs like TANF and
Medicaid.

3.

Does your school district exchange data with TANF, Medicaid/SCHIP, or other meanstested programs?
[If so, ask:]
What is the purpose of these data exchanges? (For example, is the SFA involved with
computer matching for Medicaid administrative claiming (MAC)? Do you provide data on
NSLP certifications for referrals to the Medicaid or SCHIP program? Does Medicaid
participate in a data matching program of individual records to determine Medicaid
reimbursements to school districts?)
How does the process work?
PROBE: What are the sources of the data (district or statewide student information systems,
or SFA food service information systems (POS etc.)?
PROBE: How are the data exchanged?
PROBE: Who conducts the match (the school district or the program)?
PROBE: What data elements are used for matching?
PROBE: Does the process use computer matching in your school district? If not, what are
the barriers to computer matching?
PROBE: If manual methods are used (e.g., sharing copies of application forms), what would
make an electronic process feasible for your SFA?
PROBE: What are the main challenges in the matching process as it currently operates?
(technology, staffing, cost, school district cooperation, privacy or other legal restrictions?)

4.

[If no data sharing with TANF/Medicaid, ask:] Have you considered this option? What do
you see as the pros and cons of such data sharing? What would make it feasible for your
SFA?

13

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your district?
PROBE: How effectively does the current system identify children receiving SNAP
[/TANF/Medicaid/etc.]?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

2.

How often do families receiving SNAP [/TANF/Medicaid/etc.] submit NSLP applications?
Why does the direct certification process miss these families?

3.

How do you measure the effectiveness of your direct certification process?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Have you identified any cost savings from using direct certification instead of processing
NSLP applications?

5.

Are there any unique local factors in your district that contribute to the success you’ve had
in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State/district?

2.

Which aspects of the direct certification process do you think are most challenging?
[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.

14

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

PROBE: Have any legal issues or consent requirements limited the ability to use computer
matching for direct certification in your State/district?
3.

[If computer matching is not used, ask:] What are the barriers to use of computer matching?

4.

What kinds of problems do school districts encounter in receiving and using direct
certification data?
What assistance does the State provide?
How do these problems or other factors affect whether school districts use the information
from computer matching?

5.

Have you encountered any barriers or challenges to implementing the policy to extend
categorical eligibility to all children in a household?

6.

How have you attempted to address the challenges you’ve encountered? Have the
challenges been resolved?

7.

In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?

8.

Are there any unique local factors that contribute to the challenges you’ve encountered in
implementing direct certification?

D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

1.

What advice would you give to another district that was struggling with implementing direct
certification effectively?

2.

[If the district has experience with more than one method—either because of a change, or if
the direct certification process uses both the letter method and computer matching—ask:]
What are the challenges and advantages of each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match SNAP
data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
SNAP but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
15

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?
4.

Are there direct certification methods or procedures you would advise other districts to
avoid? Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

16

OMB No.: 0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH LEA/SFA STAFF

VI. CONCLUDING QUESTIONS
1.

Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your district?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

17

6.

State and Local Technical (IS) Staff

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

NSLP DIRECT CERTIFICATION IMPROVEMENT STUDY
My name is [X], from Mathematica Policy Research.
As you may know, Mathematica is collecting information from state and local child nutrition,
education, and SNAP staff regarding direct certification for the National School Lunch Program
(NSLP). The study is being conducted for the U.S. Department of Agriculture, Food and
Nutrition Service (FNS). The purpose of the study is to provide FNS with a comprehensive
picture of the direct certification practices employed by States and districts and to explore the
relationship between these practices and overall direct certification performance measures.
Under the Richard B. Russell National School Lunch Act and the Child Nutrition Act of 1966,
FNS has authority to conduct this study under its responsibility for the development and
implementation of national policy for the National School Lunch Program. Participation is
voluntary and the information you provide will be kept private to the extent allowed by law.
We really appreciate your participation in today’s in-depth interview about direct certification
processes.
Our final report will describe the range of responses expressed by staff, and may list the names
of agencies and school districts who contributed information, but we will not quote you or
anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track
of everything you are saying. May I record our discussion so that I can listen to it later when I
write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at
a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to
ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be
discussing today?
Public Burden Statement: Public reporting burden for this collection of information is estimated to average 80
minutes per response, including the time for reviewing instructions, and preparing for and participating in the
interview. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for this project is
0584-0529. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services,
Office of Research and Analysis, Room 1014, Alexandria, VA 22302, ATTN: PRA (0584-0529).

1

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

I.

INTRODUCTION/OVERVIEW

Direct certification is used in the National School Lunch Program (NSLP) to certify students
who receive benefits from the Supplemental Nutrition Assistance Program (SNAP) and
sometimes other public assistance programs without requiring their households to complete an
NSLP application. We want to talk with you about the methods used for direct certification in
your state/district, the role your agency plays in direct certification, and explore the possibility of
future changes in those methods.
1.

Please briefly describe the role of your staff in the direct certification process.

GO TO SECTION II

2

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

II. CURRENT DIRECT CERTIFICATION PROCESS
[INSTRUCTIONS TO INTERVIEWER: Review information from survey prior to interview and
probe any inconsistencies.]
A. DIRECT CERTIFICATION PROCESS
1.

How is direct certification currently done in your State/district? Please describe the process.
PROBE: Is direct certification conducted primarily at the state level or at the district level?
[If at the district level, ask:] What guidelines does the state provide districts concerning
data matching for NSLP direct certification?
PROBE: Does the state require or recommend the use of specific identifiers and/or matching
rules for SFAs to use? What are they?
Do you directly certify participants in any public assistance programs other than SNAP?
Which ones? (TANF, Medicaid, etc.). Why/Why not?

2.

What agencies are involved in the direct certification process? (Child Nutrition, Education,
SNAP, TANF, Medicaid, etc.)
Which agency bears primary responsibility for direct certification?
What are the roles of other State agencies and local entities in direct certification? What
other agencies does your agency interact with in conducting its role in direct certification?
What types of staff at your agency are involved in the process? What tasks does each type
perform?
What agreements do you have with other agencies involved in NSLP direct certification?
How do these agreements address data formatting, recipient notification, security and
confidentiality, problem resolution, and data retention?

3.

Is there much variation at the local level in the way functional responsibilities for direct
certification are carried out? If yes, please describe.

4.

Does the direct certification process use computer matching? [If so, ask:]
Is computer matching done at the state or local level?

5.

Were any technology or other infrastructure changes necessary for you to participate in the
direct certification process? Please describe.

3

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

6.

How is information related to direct certification provided and to whom (State agency,
school districts, or parents)?
PROBE: How are the data used for direct certification transmitted to the entity that conducts
the direct certification / matching?
PROBE: [District-level matching sites] How is the information used for direct certification
provided to school districts?
PROBE: [State-level matching sites] How are the lists of directly certified students provided
to school districts?
PROBE: Is there a system in place that allows schools or SFAs to look up individual
students?

7.

What is the timeline for direct certification activities?
PROBE: When is the first file of SNAP[/TANF/Medicaid/other] data for direct certification
provided?
PROBE: How often is SNAP[/TANF/Medicaid/other] data for direct certification provided
during the school year?
PROBE: What is the reference point for the first file of enrollment data used for direct
certification? Is it based on prior year enrollment lists, or does it include new enrollees
and/or transfers?
PROBE: How often are the enrollment data files used for direct certification updated? Do
updated enrollment data include all students or only new students/transfers?

8.

[If state-level matching site, ask:] In addition to the matching conducted at the state-level,
are individual SFAs allowed or encouraged to perform matches or identify additional
students for direct certification at the local level?
How does the district-level direct certification process work? What data files do they use?
How often do they conduct the process?

9.

Why were the current direct certification methods and procedures chosen?

4

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

B. COMPUTER MATCHING PROCESS (INCLUDING INFORMATION SYSTEMS,
DATABASES, AND DATA ELEMENTS, FORMATS, VARIABLE DEFINITIONS,
DATA-MATCHING ALGORITHMS, AND COMPUTER CODE USED)
[INSTRUCTIONS TO INTERVIEWER: This section will be asked only for sites that use
computer matching.]
Now we’d like to talk in more detail about the computer matching methods used for direct
certification.
1.

Please describe the computer matching process.

2.

Which organization conducts the computer match between data from different sources? Is
the matching done by agency staff or a contractor?

3.

Which agencies provide data for the match? (education, SNAP, TANF, Medicaid, etc.)
PROBE: Do you use data from any programs other than SNAP? If not, why not?
PROBE: Is a Statewide Student Information System (SSIS) used for NSLP direct
certification? If not, what is the source of the information on students enrolled in schools in
the state/district? [Interviewer: Refer to Field Guide for list of possible data sources.]

4.

What enrollment information is provided for NSLP direct certification? [Interviewer: Refer
to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?

[For each program providing data (SNAP, TANF, Medicaid, etc.), ask:]
5. What information does the [SNAP/TANF/Medicaid/etc.] agency provide for NSLP direct
certification? [Interviewer: Refer to Field Guide for list of possible data elements.]
Is there a data dictionary that lists the file specifications, data elements, their definitions,
formats, etc? Could we have a copy of that?
6.

Does the student enrollment file include information from the household’s NSLP
application?

7.

What is the time period covered by the student enrollment and other data used for the
match? How often is each data source updated?

8.

How are the data transferred from SNAP [TANF/Medicaid/etc.] to the child nutrition
agency?

5

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

9.

[If district-level matching, ask:] How are the data transferred from the state agency to the
SFAs that conduct the matching?

10. How often is each type of data transferred? What is the timeline for the data transfer
process?
11. What data use agreements cover the data transfers?
12. What safeguards are in place for data security (such as access requirements, secure file
transfer protocols, specific log-ins for internet look-ups..etc)?
13. What data quality checks are performed on the data received?
14. How often are the data from different sources matched? What is the timeline for the
matching? Why is matching done at those times?
PROBE: After the initial data match, are later rounds of matching conducted when updated
data is received (from which agencies?), when new students enroll in the SFA, or for some
other reason?
[If more than once, ask:]
15. After the initial data match, are later rounds of matching conducted in batches, or are
students looked up individually?
Are the later matches conducted by the same staff who conduct the initial match?
Do the processes used for subsequent matches differ from the initial match in any other
ways?
How much variation is there in the specific identifiers and matching rules used at the local
level? What causes this variation?
16. What specific identifiers and matching rules does your State/district use for NSLP direct
certification data matching?
Do you require exact matches for each of these identifiers, or do you use probabilistic
matching for some/all?
17. Can you provide information on the number of children in the SNAP [TANF, Medicaid,
etc.] file and the number matched to the student enrollment data? (If more than one match,
refer to the largest match this school year.)
Do you keep files with indicators of which students were matched and which were not? Is
the file continually updated during the school year to indicate matches?

6

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

[If district-level matching state, ask:] Do districts inform the state agency of the outcomes of
their matching process? Do districts send the state a file indicating whether each case in the
SNAP [/TANF/Medicaid/etc.] file was successfully matched or not?
18. What do you do about unmatched SNAP [/TANF/Medicaid/etc.] children? Do you have a
process for identifying and dealing with “possible” matches?
PROBE: Are there secondary match rules for these cases? What are they?
19. What do you do about duplicate matches?
PROBE: Are there secondary match rules for these cases? What are they?
20. Does your matching process include any specific procedures that respond to FNS’s new
policy on Extending Categorical Eligibility to Additional Children in a Household? Please
describe.
21. Why were the current identifiers and matching rules chosen?
What are the advantages and disadvantages of the identifiers and matching rules that you
use? Did you consider alternatives to these identifiers and rules? Why did you reject those
alternatives?
22. Did your agency incur any startup costs in implementing a computer matching system?
Did SFAs have to implement any technology upgrades to use direct certification?
Who developed the software and data systems used for direct certification?
Did you consult with other states/districts in developing your system? If so, why were these
states/districts chosen for consultation?
What are the costs of maintaining the computer matching system now that it has been set
up?
GO TO SECTION IV

7

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

IV. CHANGES TO DIRECT CERTIFICATION
A. RECENT CHANGES IN DIRECT CERTIFICATION PROCEDURES
1.

Please describe any changes in the direct certification process since you first began directly
certifying students for the NSLP (besides any discussed in Section III).
[Probe for any changes indicated in survey.]
PROBE: Have the data sources or specific data elements or algorithms used for matching
changed? How?
PROBE: Has the frequency with which matches are conducted changed? How?

2.

When were these changes made?

3.

Why were these changes made?
PROBE: Have you made any changes to your direct certification procedures or guidance for
SFAs in response to FNS’s new policy on Extending Categorical Eligibility to Additional
Children in a Household that became effective in SY2009-2010?

4.

Do you consider these changes to be an improvement or not?

5.

What are the major advantages and disadvantages of the different procedures you’ve used?
What are the trade-offs?

B. CHANGES TO DIRECT CERTIFICATION PROCEDURES PLANNED FOR THE
FUTURE
1.

Are you planning or considering any changes in your NSLP direct certification procedures
in the future? If so, please describe.
PROBE: Do you plan any changes to the specific data elements or algorithms used for
matching? If so, how?
PROBE: Do you plan any changes to frequency of matching during the school year?

2.

What are the reasons for the planned changes?

3.

What is the expected timeline of these changes?

8

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

C. CAPACITY TO CHANGE DIRECT CERTIFICATION PROCEDURES
1.

[If respondent is at the district level and computer matching not used for direct
certification:] We’d like to ask about other ways in which technology is used in the NSLP
application and verification processes.
Are computers used to determine eligibility status in the NSLP application process? How
much data is entered from applications for this procedure?
Can households submit NSLP applications via the Internet? Does your SFA use the Internet
to communicate with the State Child Nutrition agency?
Is computerized sampling used to select samples for verification?
How is categorical eligibility verified?
PROBE: Are SNAP or TANF administrative records used for this process? Who does this?
What SFA records are used?
[Interviewer instruction: details on data sources currently used for direct certification are
discussed in earlier sections. In this section, we ask for information on data sources that are
not currently used for direct certification, to explore the possibility of their use in the
future.]
[For sites where respondent is at the district level and TANF/Medicaid data systems are not
used for direct certification:] Next, we’d like to talk about data sharing between schools and
programs like TANF and Medicaid.

2.

Does your school district exchange data with TANF, Medicaid/SCHIP, or other meanstested programs?
[If so, ask:]
What is the purpose of these data exchanges? (For example, is the SFA involved with
computer matching for Medicaid administrative claiming (MAC)? Do you provide data on
NSLP certifications for referrals to the Medicaid or SCHIP program? Does Medicaid
participate in a data matching program of individual records to determine Medicaid
reimbursements to school districts?)
How does the process work?
PROBE: What are the sources of the data (district or statewide student information systems,
or SFA food service information systems (POS etc.)?
PROBE: How are the data exchanged?
PROBE: Who conducts the match (the school district or the program)?

9

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

PROBE: What data elements are used for matching?
PROBE: Does the process use computer matching in your school district? If not, what are
the barriers to computer matching?
PROBE: If manual methods are used (e.g., sharing copies of application forms), what would
make an electronic process feasible for your SFA?
PROBE: What are the main challenges in the matching process as it currently operates?
(technology, staffing, cost, school district cooperation, privacy or other legal restrictions?)

10

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

V. SUCCESSES, CHALLENGES, AND LESSONS LEARNED
A. STATES’ ASSESSMENT OF EFFICIENCY/EFFECTIVENESS
1.

How well does direct certification work in your State/district?
PROBE: How effectively does the current system identify children receiving SNAP
[/TANF/Medicaid/etc.]?
[If computer matching used, ask:] What do you think about the accuracy of computer
matching?

2.

How do you measure the effectiveness of your direct certification process?
[If district-level matching, ask:] Do you assess effectiveness at the district level?

B. DIRECT CERTIFICATION SUCCESSES
1.

What are the greatest successes you’ve had in implementing direct certification?

2.

Which aspects of the process do you think work best?

3.

What have been the results of any changes you’ve made in the process?

4.

Are there any unique state or local factors in your State/district that contribute to the success
you’ve had in implementing direct certification?

C. BARRIERS TO DIRECT CERTIFICATION
1.

What are the main challenges or barriers you’ve encountered in implementing direct
certification in the NSLP in your State/district?

2.

Which aspects of the direct certification process do you think are most challenging?
[If computer matching is used, ask:] What barriers have you faced in the computer matching
process?
PROBE: Have you encountered barriers related to technology, data issues, SFA buy-in, cost,
accuracy, or parent response? Please describe.
PROBE: Have any legal issues or consent requirements limited the ability to use computer
matching for direct certification in your State/district?

3.

[If computer matching is not used, ask:] What are the barriers to use of computer matching?
11

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

4.

[If computer matching is used for some, but not all, SFAs in the State, ask:] What are the
barriers to wider use of computer matching among SFAs in your State?

5.

What kinds of problems do school districts encounter in receiving and using direct
certification data?
What assistance does the State provide?
How do these problems or other factors affect whether school districts use the information
from computer matching?

6.

How have you attempted to address the challenges you’ve encountered? Have the challenges
been resolved?

7.

In hindsight, do you think anything could have been done to avoid or mitigate these
challenges?

8.

Are there any unique state or local factors that contribute to the challenges you’ve
encountered in implementing direct certification?

D. POTENTIAL LESSONS
CERTIFICATION

LEARNED

FROM

IMPLEMENTING

DIRECT

1.

What advice would you give to another state/district that was struggling with implementing
direct certification effectively?

2.

[If the state/district has experience with more than one method—either because of a change,
or district-level differences within the state, or if the direct certification process uses both
the letter method and computer matching—ask:] What are the challenges and advantages of
each direct certification approach you’ve used?

3.

Are there particular procedures/methods you think are critical for efficient direct
certification?
PROBE: Many States are now using centralized data matching for direct certification, while
others use decentralized methods. What is your opinion about the best way to match SNAP
data with student enrollment data?
PROBE: What are the best strategies for directly certifying children who are enrolled in
SNAP but not matched with student enrollment data?
PROBE: What is the right frequency for data matching?
PROBE: What are some of the best ways to provide direct certification to private schools?
PROBE: What are the most successful ways of implementing FNS’s policy of Extending
Categorical Eligibility to Additional Children in a Household?
12

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

PROBE: Are there any aspects of direct certification that we have not yet talked about that
could be considered a best practice?
4.

Are there direct certification methods or procedures you would advise other States/districts
to avoid? Which ones?

5.

Is there anything else states should keep in mind when considering changes to their direct
certification processes?

6.

What do you think USDA can do to improve direct certification?

13

OMB No.:0584-0529
Expiration date: MM/DD/YYYY
PROTOCOL FOR SEMI-STRUCTURED INTERVIEWS WITH TECHNICAL STAFF

VI. CONCLUDING QUESTIONS
1.

Is there anything we haven’t covered that you’d like us to know about how NSLP direct
certification is implemented in your State/district?

2.

If I have any additional questions later, as I write up my notes from our conversation, may I
call you to follow up?
Thank you for your time.

14