R. Pre-test Methods and Summary of Findings

Appendix R. Pretest Methods and Summary of Findings

OMB No. 0584-[NEW]

Assessment of Mandatory E&T Programs

Month XX, 2018

Project Officer: Jordan Younes

Office of Policy Support

Food and Nutrition Service

U.S. Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22303

703-305-2935

Jordan.younes@fns.usda.gov

M emorandum

Date: April 12, 2018

To: Andrew Burns and Jordan Younes, FNS

From: Claire Wilson, Carole Trippe, Brian Estes, and Brittany McGill, Insight Policy Research
Gretchen Rowe and Kristen Joyce, Mathematica Policy Research

Subject: Assessment of Mandatory E&T Programs: Pretest Findings and Recommendations

Insight and Mathematica pretested the following data collection instruments for the “Assessment of Mandatory Employment and Training (E&T) Programs” study:

1. State SNAP director and E&T director interview protocol

2. Local SNAP office director interview protocol

3. Local SNAP office eligibility worker interview protocol

4. E&T provider supervisor and frontline staff interview protocol

5. Administrative data collection instructions and variable list

The primary objective of the pretest was to ensure the instruments will be clear and easy for respondents to understand. Specific pretest objectives follow:

• Identify problems related to communicating intent or meaning of questions.

• Determine whether respondents could provide the information requested.

• Identify problems with introductions, instructions, or explanations.

• Assess the time needed to complete the questionnaire and other respondent burden issues.

1. Recruitment and Data Collection Methods

On January 24, 2018, FNS sent a one-page overview of the study via email to Minnesota’s Economic Assistance and Employment Supports Division Director, requesting the State’s participation in the pretest. Insight’s qualitative task lead followed up with the director via email to provide additional information and followed up by telephone to request Minnesota’s participation.

After securing Minnesota’s participation, the task lead scheduled an interview to pretest the State SNAP director and E&T director interview protocol. Insight also discussed the need to pretest three additional interview protocols: one with a local office director, a second with an eligibility worker, and a third with an employment and training provider who serves mandatory SNAP E&T participants.

The State SNAP director provided contact information for SNAP office representatives from two counties. One of the counties operated a mandatory E&T program; the other operated a voluntary E&T program. Insight contacted the SNAP office representative from the mandatory county by email. The email briefly described the study, the purpose of the pretest, and the topics to be covered in the interview. Insight followed up with the local representative by phone to formally request her participation and scheduled the pretest.

The State SNAP director also provided contact information for two E&T providers. Insight forwarded the contact information for the E&T providers to Mathematica to coordinate and conduct the pretest of the E&T provider instruments. After learning these providers were in a voluntary E&T county, Insight requested contact information for two providers in mandatory E&T counties from the State SNAP director. Mathematica recruited one of these providers for the pretest.

Each interview pretest was scheduled for 90 minutes to facilitate testing a 60-minute instrument and follow-up questions. Each interview was recorded with the permission of respondents, and a research analyst took detailed notes during the interview.

For the pretest of the administrative data request and instructions, Insight held an initial call with a representative from the Economic Assistance and Employment Supports Division to describe the study and administrative data collection and to schedule a 60-minute conference call to obtain feedback on the materials. Next, Insight sent the administrative data collection instructions and requirements document (see appendix E) to the respondent and asked her to review the documentation before the scheduled call. Specifically, Insight asked the representative to provide feedback on (1) the instructions for preparing and submitting the data extract, including which records to include, the format for the file, and the time period for the data; and (2) the list and description of variables requested, including whether the variables were clearly defined, and the level of detail and quality of the data that could be provided. Finally, Insight held a 60-minute call on February 22 to solicit feedback on the materials.

2. Findings

In Minnesota, SNAP is administered at the county level, with some counties operating mandatory programs and others operating voluntary programs. For the purposes of the pretest, we conducted interviews with SNAP representatives and E&T providers who serve mandatory E&T participants.

As discussed below, we learned through the local office pretests that employment services and eligibility services are administered by different departments in Minnesota. As a result, the employment services representatives could not address questions about eligibility, and the eligibility representatives could not address some questions about participation tracking. This situation suggests the need to gather information about how services are organized and coordinated in a given State or location, and what entities need to be “at the table” to provide a comprehensive understanding of how clients move through the program and where they can drop off in the process. We plan to include these issues as discussion topics with the study States during the planning phase for the site visits to ensure smooth execution of site visits and comprehensive data collection.

1. State SNAP Director and E&T Director Interview Protocol

Insight interviewed the director of Minnesota’s Economic Assistance and Employment Supports Division and the director of the SNAP E&T program on February 27.

1. Duration

The interview lasted 75 minutes, suggesting the need to reduce the number of questions. After the pre-test was completed, approximately 20 percent of the interview questions were deleted to ensure that the interviews were reduced from 75 minutes to an average of 60 minutes.

2. General Findings

The interviewees felt the questions were clearly worded, but they seemed uncertain how to answer some questions. For example, they could not answer questions about how long the current policies had been in effect, how the State decided on exemption criteria (these policies are determined at the county level), and how compliance-related policies were formulated. The interviewees could address every topic but had limited details on the training provided to eligibility workers regarding exemption policies and the determination process. We plan to include a list of the interview topics in advance, which will enable respondents to better prepare for discussions.

3. Question-by-Question Findings and Recommendations

Table 1 provides our findings and recommendations for specific questions. See appendix A for the revised State SNAP director and E&T director interview protocol, including suggested changes mentioned in table 1.

Table 1. Item-Level Recommendations for the State SNAP and E&T Director Interview Protocol

Question Number From Draft Instrument	Findings/Observations	Recommendations
B.1. How long have the current mandatory E&T policies been in effect? [If recently changed from a voluntary program:] Can you tell me about why [State] made this change? Which entities (e.g., Governor, legislature, SNAP agency) drove the effort to make the change? Were there any challenges in making this change? Describe.	Some State SNAP representatives may consider it a challenging question because it is broadly worded and requires respondents to rely on memory.	We recommend dropping this question because it does not directly address any research questions and extends the interview. We recommend the interviewer begin this section by reviewing the exemption criteria and asking the respondent to offer any corrections or updates, thereby establishing the respondent as the expert.
B.3. What are the reasons the State makes E&T participation mandatory in [selected counties/for selected groups]? [If the State administers a mandatory and voluntary program:] How did you determine where you would offer a mandatory or voluntary program? Which factors were most important in driving that decision? Are there challenges with administering and operating both types of programs? Describe.	The respondents noted they were following the Federal regulations and seemed somewhat confused by this question. Question 3b was redundant with the challenges section.	Reword the question to make intent clearer: “As you may know, States have discretion in determining whether to make participation in E&T services mandatory or voluntary for SNAP participants. Can you describe the reasons for operating a mandatory program [in State/selected counties or for selected groups]?” Drop B.3.b.
C.1. Can you please describe the components that are available to mandatory E&T participants?	The question would be less burdensome to the respondents if the interviewer referenced the components identified in the State plan.	Revise the question: “Based on your State plan, I understand [State] offers [insert relevant components such as job search, workfare, apprenticeship services]. Are other services available to mandatory E&T participants, specifically, that I did not mention? Are any of the components I mentioned no longer available?”
D.1. First, how did [State] formulate its current policies regarding compliance? How did the State determine the length of the compliance period? How did the State establish parameters for satisfactory compliance?	Initially, respondents simply described the current policies regarding compliance, not how they were formulated. When the interviewer followed up with a question about how the State arrived at these policies, respondents were not sure how to answer but noted the policies were based on the Federal guidelines.	We recommend dropping this question as it is not directly related to the research questions. Instead of asking D.1.a, we suggest expanding questions on conciliation (D.5). For example, for States that have a conciliation period, we may want to ask: How did you decide to offer a conciliation period? How do you think the conciliation period affects sanction rates, if at all?
D.3. To the degree you know, what kinds of barriers lead to noncompliance upfront (i.e., before entering E&T)? What barriers do mandatory E&T clients face in sustaining participation?	Although the respondents could answer the first question, we can ask it only of the local office staff to make the State director interview shorter.	Drop this question from State SNAP Director interview protocol.
F.5. I understand States are expected to measure E&T outcomes to determine whether the program components are helping clients become employable. What outcome measures does [State] collect?	We can reduce the burden on the respondent and perhaps time on this question by reviewing the outcomes identified in the State plan and asking for updates.	Revise the question: “I understand States are expected to measure E&T outcomes to determine whether the program components are helping clients become employable. Your State plan identifies [list measures in State plan]. Are there any other measures you collect that I did not mention? Any you no longer collect?”

2. Local SNAP Office Director Interview Protocol

To pretest the local office director protocol, the Insight team spoke with the Work Services program director who oversees employment and training services in one of Minnesota’s mandatory counties. The county supervisor for employment counselors also participated in the interview.

Because different departments administer the Work Services program and the eligibility services program in this county, the interviewees noted questions pertaining to eligibility (e.g., training for eligibility workers and screening for exemptions) should be directed to our point of contact for eligibility services.

1. Duration

The interview lasted 65 minutes. No changes to the length of the protocol are needed.

2. General Findings

Several questions appeared to be too difficult to answer on the spot (e.g., how would you describe the local economy; what policy or programmatic changes within the past year have affected your E&T program?). In table 2, we provide some suggested revisions for making such questions easier to answer. Prior to the interview, the study team can send the interviewee a short list of topics to be covered to help the respondent prepare for the interview.

Some responses suggested a need to reframe the way the question is asked. For example, question D.4 asks, “How often would you say workers make incorrect referrals to mandatory E&T?” The employment counselor supervisor replied that individuals are not incorrectly referred; they simply may not have their documents with them. In that case, the eligibility worker refers them to E&T until they present those documents to show they meet one or more exemption criteria. We suggest rewording the question to steer the conversation away from a discussion of worker error and move it in the direction of gathering information about how an exempt participant might get referred to E&T, how commonly that it happens, and how workers receive feedback about such cases.

The respondents easily answered questions about local capacity to serve E&T clients, common reasons for noncompliance, and challenges associated with operating a mandatory E&T program. They may have found these questions easier to answer because the questions invited candid discussion of participant barriers and the limited availability of resources to address those barriers in a comprehensive way.

3. Question-by-Question Findings and Recommendations

Table 2 provides our findings and recommendations for specific questions. The revised local office director interview protocol, including suggested changes from table 2, appears in appendix B.

Table 2. Item-Level Recommendations for the Local Office Director Interview Protocol

Question Number From Draft Instrument	Findings/Observations	Recommendations
B.1. How would you describe the local economy? What jobs are most in demand in this area (low-, middle-, and/or high-skill jobs)? Does your SNAP E&T program offer mandatory clients any services specifically designed to prepare them for these jobs? Are there jobs or industries that have disappeared in recent years?	The respondents had some difficulty answering these questions. The overarching question about the local economy may be too broad. The services offered in this county were limited, so the respondents seemed uncertain how to answer the question about whether the program offers services specifically designed to prepare them for local jobs. They said they did not know how to answer the question about jobs or industries that have disappeared.	We ask a question similar to B.1.a (“What jobs are most in demand in this area? Does your SNAP E&T program offer mandatory clients any services specifically designed to prepare them for these jobs?”) later in the interview (“H.2. How well do you think each E&T component prepares clients for employment in a local job market?”). We suggest moving the first question in B.1.a to H.2 and deleting the second question under B.1.a. We also recommend deleting the following questions, which are not directly related to the research questions: “B.1. How would you describe the local economy? b. Are there jobs or industries that have disappeared in recent years?”
B.3. What policy or programmatic changes within the past year (if any) have affected your E&T program? Describe. Did these changes affect participation, sanctions, or outcomes? [If aware of changes, probe accordingly:] ABAWD time limit, State options, reporting requirements, sanction policies, changes to caps on participant reimbursements? Changes to allowable reimbursements? Expansion of available services? Anything else?	This question elicited a long pause, and the respondents seemed unsure how to answer. After some thought, they mentioned opening additional locations where mandatory participants could go for the orientation.	We suggest focusing this question on specific policies and practices: “E&T policies and practices can change from time to time. I understand your State [identify any changes reported during the State director interview]. What effect, if any, do you think those changes have had on E&T participation? Sanction rates? Outcomes?” If no changes are reported by the State director, ask, “How have your State’s E&T program or practices changed within the past year? What effect did these changes have on E&T participation? Sanction rates? Outcomes?”
C.1. What kinds of training do SNAP eligibility workers (or the individuals who screen for exemptions) receive on implementing mandatory E&T policies?	The respondents said this question should be directed to the eligibility services team.	No change. Though the pretest respondents could not answer this question, we may find local office directors have information to share during interviews. We will also discuss the various interview topics with the State during the planning stage of the site visit to ensure we identify the most appropriate interview respondents to capture as much data as possible.
D.1. As you know, clients must be screened for Federal, State, and any local exemptions. What is the process for determining whether a SNAP client is exempt from the mandatory E&T requirements? D.2. Does the eligibility worker typically make a determination at the time of screening, or is there a delay between the time when the worker gathers relevant information about the client’s circumstances and the actual determination? D.3. In your opinion, how complicated or straightforward is the process for determining whether a SNAP client is exempt from mandatory E&T?	The respondents said these questions should be directed to the eligibility services team.	No change. Though the pretest respondents could not answer this question, we may find local office directors have information to share during interviews. We will also discuss the various interview topics with the State during the planning stage of the site visit to ensure we identify the most appropriate interview respondents to capture as much data as possible.
D.4. How often would you say workers make incorrect referrals to mandatory E&T?	The employment counselor supervisor replied that participants are not incorrectly referred; they are just referred until they present documentation showing they should be exempt.	We suggest rewording to omit the phrase “incorrect referrals”: “How often does a participant who qualifies for an exemption get referred to E&T?”
E.1. If State has E&T providers: To begin, I want to make sure I am aware of all the E&T providers that work with SNAP mandatory E&T participants. Can you review that with me? Can you tell me about the services each provider offers? How did each of these providers connect with SNAP; in other words, what prompted your office to begin referring mandatory E&T participants to these partners? E.2. How do the SNAP office and E&T providers communicate to coordinate services? How well does this approach work?	These questions were not relevant to this county because the mandatory E&T program is operated by the county. Although there are a few providers serving the county that have contracted with the State to offer E&T services, those providers primarily serve voluntary participants. The respondents did describe how those providers communicate with employment services.	No change.
G.13. Are clients more or less likely to be sanctioned depending on the component they are assigned to? If so, how would you explain this variation? Do certain components require more time and/or effort to meet compliance requirements? Please explain.	This question was not applicable in this county because the E&T services offered are limited.	No change.
G.14. Do participant reporting requirements vary between components? Please explain.	This question was not applicable in this county because the E&T services offered are limited.	No change.
G.15. How does your office track and report sanctions to the State?	Respondents noted the eligibility worker can answer this question.	No change.

3. Local SNAP Office Eligibility Worker Interview Protocol

To pretest the eligibility worker protocol, we interviewed a human services supervisor from a local office in the previously mentioned mandatory county. She supervises the eligibility workers who screen participations for E&T exemptions.

1. Duration

The interview lasted 60 minutes. No changes to the length of the protocol are needed.

2. General Findings

The respondent had no difficulty understanding or answering the questions in the protocol, and the conversation flowed smoothly. She noted that question D.2 (“How do you make a ‘good cause’ determination?”) should be directed to the E&T services program because that office determines whether a participant had good cause for not attending orientation, meeting with their employment counselor, or meeting some other requirement.

3. Question-by-Question Findings and Recommendations

Table 3 provides our findings and recommendations for specific questions. The revised local office eligibility worker interview protocol, including suggested changes from table 3, appears in appendix C.

Table 3. Item-Level Recommendations for the Local Office Eligibility Worker Interview Protocol

Question Number From Draft Instrument	Findings/Observations	Recommendations
B.4. How do you learn about incorrect referrals? At what point do you typically learn about incorrect referrals? What happens when you learn of an incorrect referral? Is there a way to correct it? If so, please explain. How frequently would you say mandatory E&T participants are incorrectly referred to E&T?	The respondent acknowledged clients are sometimes incorrectly referred but did not have any data to estimate the frequency of incorrect referrals. For consistency across protocols, we recommend making the same change recommended for the local office director interview protocol.	Reword the question: “Sometimes a participant who qualifies for an exemption may get referred to E&T. How do you learn about these incorrect referrals?” Also reword B.4.c: “How often does a participant who qualifies for an exemption get referred to E&T?”
E.5. Approximately how many (what percentage) of your clients who receive a NOAA comply with the requirements in time to avoid being sanctioned?	Respondent noted someone from E&T services could answer this question, but she did not have this information.	No change.
E.8. Approximately how many (what percentage of) participants return to SNAP and E&T after being sanctioned?	Respondent noted someone from E&T services could answer this question, but she did not have this information.	No change.
F.1. Are you responsible for tracking and reporting participation data for mandatory E&T clients to the State? If so, what data do you track and report?	Respondent noted someone from E&T services could answer this question, but she did not have this information.	No change.

4. E&T Provider Supervisor and Frontline Staff Interview Protocol

E&T Provider Supervisor

On March 16, Mathematica interviewed the executive director of a SNAP E&T provider in a rural area of the State that serves mandatory SNAP E&T participants.

1. Duration

The interview was completed within 62 minutes. No changes to the length of the protocol are needed.

2. General Findings

The interviewee thought the questions were clearly worded and easy to understand. However, he deferred some questions about data reporting and tracking to the E&T worker. Otherwise, he could address all the topics covered in the protocol. He suggested sending data requests to respondents before the interview to obtain more detailed information on program statistics. He did not understand the term “certification” and asked the interviewer to clarify what it meant. The interviewers will ensure they specify that certification relates to obtaining a degree or training certifications (question F.4).

3. Question-by-Question Findings and Recommendations

Table 4 provides our findings and recommendations for specific questions. The revised provider interview protocol, including suggested changes from table 4, appears in appendix D.

E&T Provider Frontline Staff

On March 19, Mathematica staff interviewed a job counselor from a provider in rural Minnesota who serves mandatory SNAP E&T participants.

1. Duration

The interview lasted 58 minutes. No changes to the length of the protocol are needed.

2. General Findings

The interviewee thought the questions and the terminology were clear. She commented that some questions were open-ended to the extent she answered subsequent questions before they were asked and then felt the questions were repetitive. This feedback suggests the interviewer will need to pay close attention to which questions have already been answered while advancing through the protocol. The interviewee could address all topics covered in the protocol.

3. Question-by-Question Findings and Recommendations

Table 4 provides our findings and recommendations for specific questions. The revised provider interview protocol, including suggested changes from table 4, is in appendix D.

Table 4. Item-Level Recommendations for the SNAP E&T Provider and Worker Interview Protocol

Question Number From Draft Instrument	Findings/Observations	Recommendations
B.5. Do you know how many other employment and training providers there are in the area? Do the services they provide overlap with yours? Do you refer the clients you serve to any of these other organizations to provide additional E&T services you are not able to provide (co-enrollment)? Do you ever refer clients to these other organizations for their primary services because your organization is not a good fit for the SNAP E&T participants? How frequently?	The respondent answered there were no other employment and training providers in the area that provide SNAP E&T but later mentioned there are American Job Centers and local community colleges that also provide training.	With this question, we would like to identify other employment and training providers that offer comprehensive employment and training services, including assessment, case management, job search and training, and support services. We suggest the following revision to clarify the meaning of “employment and training providers” and explain we are not looking only for other SNAP E&T providers: “Do you know how many other employment and training providers in the area provide comprehensive services (e.g., assessment, job search and training, support services) to clients? [If necessary, clarify we are not interested only in providers that serve SNAP E&T participants.]”
C.4. Do you primarily receive new clients through referrals or walk-ins? Do you know if a client is referred from a certain program? How? Does the referral process/frequency differ by agency? Does SNAP E&T primarily use referrals? How do you receive SNAP E&T referrals? Describe the process. How frequently do you receive them? Provided through MIS or outside of systems?	The respondent answered the main question about SNAP E&T participants, but we wanted to collect this information on all the organization’s new clients first and then compare how it differed from the process for receiving new SNAP E&T clients.	Add the following language to clarify we are asking about all clients, not just SNAP E&T clients: "In general, do you primarily receive new clients to your agency/organization through referrals or walk-ins? “
D.12. How frequently do SNAP E&T participants complete the training and find employment? [Probe for percentage.] What types of employment are generally found? How does the type of employment found vary by type of service, if at all? Are these rates and types of employment similar for other clients you serve? Can these jobs help make participants self-sufficient, or do they still need SNAP and other types of assistance to supplement income? If not, why not?	The respondent at first said she did not feel she had the experience to make a guess about how often SNAP E&T participants complete training and find employment. However, when we asked to give us her sense of the situation, she was able to respond.	Add the following language to clarify the respondent’s best estimate is sufficient: “How frequently do SNAP E&T participants complete the training and find employment? [Probe for percentage. If respondent does not know, probe for his or her best estimate. Ask if he or she can provide these data after the interview.]” Request available statistics from the organization before or after the interview when we want precise information.
F. Data Tracking and Reporting	This section was marked to be answered by directors only, but the director thought the E&T worker would be better suited to answer those questions.	Specify which questions in section F are more high-level questions to be answered by directors (e.g., questions 2, 9) and more detailed questions to be answered by E&T workers (e.g., questions 4, 8).
F.4. What types of data do you track for each client (beyond what is required by SNAP)? For instance, do you track metrics on— Skill gains/certifications?	The director did not understand the term “certification” and asked for clarification.	Revise the probe to define certification as follows: “b. Skill gains/acquiring degrees or training certifications?”

3. Administrative Data Collection Findings

This section presents the pretest results for the administrative data collection materials. Administrative data will be used to examine the characteristics of mandatory SNAP E&T participants and to assess the E&T services, sanctions, case closures, and other outcomes associated with E&T participation. The pretest was held on February 22, with a respondent from the Economic Assistance and Employment Supports Division of the Minnesota Department of Human Services.

1. Results and Recommendations

Based on the feedback received, we recommend making the following revisions to the data collection materials. The revised document appears in appendix E.

1. Only include individuals aged 16 to 59 in the SNAP data files. We recommend excluding individuals who are under age 16 and over age 59 from the data request. Doing so will reduce the amount of data States will have to provide, enable them to apply the same programming specifications in all States, and ensure the universe of individuals includes those who could potentially be identified as work registrants, E&T participants, and ABAWDs. Because we will be collecting 29 months of data, we would ask for participants aged 13 to 62 (some 13-year-olds will be 16 by the end of the period, and some 59-year-olds will be 62 at the end of the period).

2. Only provide data for months in which participants are in the State caseload system. Our original instructions required States to include 29 rows (months) of data for all individuals and to create dummy rows of data for months in which the participant is not on SNAP and not in the caseload system. Requiring States to create dummy rows for individuals not in their system may result in errors or inconsistencies in the data. Therefore, we recommend asking States to submit data only for those individuals who are receiving SNAP and who are in their data system. Insight will add dummy rows as needed in the data cleaning and preparation stage to account for months in which participants are not in the data system.

3. Add a variable for individual-level earnings. Some States may have individual-level earnings data for SNAP participants (if the participant was employed during the eligibility determination). This individual-level data would facilitate identification of which household member is working and receiving earnings.

4. Add a variable for the start and end dates of the E&T sanction. Some States may provide information on the start and end dates of a sanction. The start and end dates would be useful in confirming or calculating the length of each sanction, particularly when there is not a separate variable for length of E&T sanction.

5. Make three minor edits to existing variables. We recommend the following three minor edits to existing variables: (1) change the variable “E&T sanction iteration” to “E&T sanction number” and update variable name and description accordingly, (2) change the variable “Certification/recertification date” to “Certification date” and define the certification date as “The last review date associated with household’s SNAP benefit month (i.e., disposition date for reference month),” and (3) change the variable “Type of E&T service assigned” to “Type of E&T component assigned” and update variable name and description accordingly. These revisions are expected to help clarify the meaning of these variables for State data managers.

6. Include both a data manager and SNAP policy expert on consultative calls. Because of the complexity of many of the variables requested, it is important to include both a policy expert and a technical data manager in the consultative discussions with each State. The data manager may need to consult a SNAP policy person, for example, to clarify certain variables or identify which codes should be used.

Based on feedback from Minnesota, we have additional general comments on the administrative data collection:

1. Recipients may be assigned to multiple E&T components and/or providers in a month, so we will be prepared to add additional variables as needed to collect information on multiple components or providers.

2. Most States probably will not have data on whether the participant “cured” noncompliance before being sanctioned. If a participant cured noncompliance, the sanction simply may not be recorded.

3. In Minnesota, E&T providers submit outcome data directly to the State, and the State links the data to the SNAP recipient. However, the data entered by job counselors are not always complete or consistent. Other States may have similar issues with the quality of E&T outcome data.

4. Minnesota agreed the quality of the E&T outcome data will be better starting in FY2017, when States were required to begin collecting reporting measures and outcome data to submit to FNS. The resulting request for 29 months of data maximizes the number of months between October 1, 2016, and the anticipated OMB clearance date of February 2019. We will adjust the total number of months of data once actual OMB clearance is received (to maximize the number of months up to 30 months).

Appendix A.
Revised State SNAP Director Interview Protocol

My name is [name], and I’m a researcher at [Insight Policy Research/Mathematica Policy Research, working with Insight]. As you may know, Insight is conducting the Assessment of Mandatory E&T Programs study for the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture. This study examines how mandatory State SNAP E&T programs are administered and how the programs help SNAP clients move toward economic self-sufficiency. We are interested in better understanding the rates of participation, sanction, and employment among mandatory clients. We also want to explore reasons why mandatory E&T clients are most often sanctioned and when sanctions happen, such as at initial referral, intake and assessment, or service referral. This study also seeks to understand what data are available on how well mandatory programs help SNAP clients gain skills, certificates and credentials, and stable employment.

My colleagues and I are currently visiting local SNAP offices here and in five other States to collect information from a wide range of stakeholders involved in operating a mandatory SNAP E&T program. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues will be very helpful to the study.

I want to let you know that your participation in this interview is voluntary, and your responses will be kept private to the extent provided by law. We will not share the information you provide with anyone outside the study team, including your supervisor and State-level staff. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalties if you refuse to participate in part or at all.

We will take notes over the course of the interview and would like to record the conversation so we can remember the information we collect. We will use this information in our reports to FNS, describing the range of responses expressed by staff. The reports might list the States that contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of States participating in the study, there is a possibility a response could be correctly attributed to you.

I expect our conversation will take about 60 minutes. First, do you have any questions for me about the project in general or what we will be discussing today?

Do I have your permission to record the conversation?

[Confirm permission before recording starts.]

1. Introduction

I’d like to start with a few questions about your roles and responsibilities.

1. What is your official job title or position?

2. What are your primary responsibilities?

3. How long have you worked at [agency]? How long in this position?

2. Background on State’s E&T Policies

Next I’d like to get your perspective on some of the factors that helped shape [State]’s current policies regarding SNAP E&T.

1. I understand some SNAP clients in your State are exempt from mandatory E&T, such as [identify exempt groups identified in the State plan]. Are there other exempt groups I did not mention?

1. How did [State] decide on these exemption criteria?

2. As you may know, States have discretion in determining whether to make participation in E&T services mandatory or voluntary for SNAP participants. Can you describe the reasons for operating a mandatory program [in State/selected counties or for selected groups]?

1. [If the State administers a mandatory and voluntary program:] How did you determine where you would offer a mandatory or voluntary program? Which factors were most important in driving that decision?

3. Have there been any policy or programmatic changes within the past year that have affected your E&T program? Describe.

1. [If aware of changes, probe accordingly:] ABAWD time limit, State options, reporting requirements, sanction policies, changes to caps on participant reimbursements? Changes to allowable reimbursements? Expansion of available services? Anything else?

2. How did these changes affect participation, sanctions, or outcomes?

1. When changes are made (e.g., a specific county becomes exempt, a particular population is subject to new requirements), what typically drives those changes?

4. What kind of training does the State offer SNAP staff related to implementing mandatory E&T policies and issuing sanctions for noncompliance?

1. [Probe for:] Who receives the training? Frequency (including frequency of retraining)? Materials used? Training format (e.g., in person, online, mandatory/optional)?

3. E&T Components and Support Services

Now I’d like to talk about the services offered to mandatory E&T clients.

1. Based on your State plan, I understand that [State] offers [insert relevant components such as job search, workfare, apprenticeship services]. Are other services available to mandatory E&T participants, specifically, that I did not mention? Are any of the components I mentioned no longer available?

1. How are decisions made about what components to offer?

2. How do the components offered to mandatory E&T participants vary across different areas of the State (if at all), and what accounts for those differences?

3. How much discretion do local areas have in offering different types of components?

2. Can mandatory E&T participants choose among various services, or do they get assigned to a particular component?

1. [If they can choose among various components:] Have you found that some components are more “in demand” or more heavily used than others? Are there some components that mandatory participants never or rarely choose?

2. What do you think accounts for such differences?

3. To what extent do you think E&T components are well aligned with the current job market?

1. If markets vary by local area, do the components in a given county or geographic area align with the local markets?

4. What supports are offered to mandatory E&T clients?

1. What are the limits on reimbursements?

2. Are these limits determined at the State or local level?

5. Are there any caps on time or restricting of “slots” for certain components? Describe.

1. If so, who determines these restrictions? Are decisions based on funding?

4. Compliance and Sanctions

We are particularly interested in gaining a better understanding of your sanction policies and potential “dropoff” points, such as where a mandatory participant is at risk of being found noncompliant with the E&T requirements.

1. Can you provide a high-level summary of what the State requires of a mandatory client to remain in compliance with E&T?

1. What are the most common reasons for noncompliance?

2. Do clients tend to become noncompliant initially (by never engaging in services), or do they tend to become noncompliant after engaging in services?

3. Does the State offer a conciliation period to noncompliant mandatory E&T participants?

1. If not, what are the reasons for not offering conciliation?

2. [if yes] How did you decide to offer a conciliation period? How do you think the conciliation period affects sanction rates, if at all?

4. When is good cause determined?

1. Is it the client’s responsibility to demonstrate good cause, or does the State try to determine whether the client has good cause?

5. If an E&T provider thinks a referred client has good cause—in other words, he or she believes the client should be exempt from mandatory E&T—is there a process in place that allows the provider to communicate that information back to the SNAP office? Please describe.

6. Does the State track of how many clients who receive a Notice of Adverse Action (NOAA) comply with program requirements before a sanction is imposed? If so, approximately what percentage of clients comply?

7. Does your office track return rates to SNAP after a sanction is imposed? If so, what is your sense of how many individuals return to SNAP after being sanctioned, and how quickly they return?

8. Is there a process for reviewing NOAAs to make sure they are understandable to the average SNAP client?

5. Compliance Monitoring

Now I’d like to ask a few questions about the process of monitoring participation in E&T services.

1. To what extent are local SNAP offices and E&T providers monitored at the State level?

1. Who handles that monitoring? For example, are the programs monitored by SNAP, the Department of Labor, or a different office?

2. Do you monitor E&T screening and intake procedures? How long it takes to notify clients of their E&T status and refer them to a provider? How is that accomplished?

2. What kind of quality assurance procedures are in place, if any, to monitor that sanctions are made correctly?

1. Do you ever conduct case reviews of sanctioned cases? How are these conducted?

6. E&T Data

Now I’d like to ask some questions about the E&T data your office collects and reports to FNS.

1. What E&T-related data does the State collect?

1. How do you obtain data on client status, E&T activities, and outcomes?

2. How do you obtain the data you use to produce reports for FNS?

3. Do you produce other reports for internal program management purposes?

9. What kind of information is in these reports? Do you share the internal reports with local offices?

2. Does the State track how many people participate in each component?

3. Does the State track how long nonexempt SNAP recipients usually participate in E&T?

4. Have E&T participation rates remained fairly consistent annually and across the State?

1. [If not:] How have participation rates changed? What do you think accounts for the change? Have there been any policy changes that may have led to a spike or drop in sanction rates?

5. I understand States are expected to measure E&T outcomes to determine whether the program components are helping clients become employable. Your State plan identifies [list measures in State plan]. Are there any other measures you collect that I did not mention? Any you no longer collect?

1. How did the State choose those measures?

2. How are measures reported? Who does the reporting? How often are data submitted?

3. How are these data used (e.g., are they used to inform programmatic changes or allocation of resources)?

7. Challenges and Lessons Learned

Finally, I would like to learn more about any challenges you have had in operating your mandatory E&T program and any lessons learned you can share with others.

1. What is most challenging about operating a mandatory SNAP E&T program?

1. Do these challenges vary across different areas of the State?

2. Can you think of ways to improve mandatory E&T?

3. Can you think of any changes in policies or administrative practices that might lead to greater participation in SNAP E&T?

2. Is there anything you would change about the way you serve SNAP E&T participants?

3. Do you feel the program is meeting the goals of your agency?

4. What lessons have you learned in serving this population?

1. What advice would you give to other agencies beginning to serve mandatory SNAP E&T participants?

8. Wrap-Up

Thank you for answering all our questions.

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

That completes our questions for you. Thank you very much for speaking with us.

Appendix B.
Revised Local SNAP Office Director Interview Protocol

My name is [name], and I’m a researcher at [Insight Policy Research/Mathematica Policy Research, working with Insight]. As you may know, Insight is conducting the Assessment of Mandatory E&T Programs study for the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture. This study examines how mandatory State SNAP E&T programs are administered and how the programs help SNAP clients move toward economic self-sufficiency. We are interested in better understanding the rates of participation, sanction, and employment among mandatory clients. We also want to explore reasons why mandatory E&T clients are most often sanctioned and when sanctions happen, such as at initial referral, intake and assessment, or service referral. This study also seeks to understand what data are available on how well mandatory programs help SNAP clients gain skills, certificates and credentials, and stable employment.

My colleagues and I are currently visiting local SNAP offices and E&T providers here and in five other States to collect information from a wide range of stakeholders involved in operating a mandatory SNAP E&T program. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues will be very helpful to the study.

I want to let you know that your participation in this interview is voluntary, and your responses will be kept private to the extent provided by law. We will not share the information you provide with anyone outside the study team, including your supervisor and State-level staff. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalties if you refuse to participate in part or at all.

We will take notes over the course of the interview and would like to record the conversation so we can remember the information we collect. We will use this information in our reports to FNS, describing the range of responses expressed by staff. The reports might list the States that contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of SNAP offices participating in the study, there is a possibility a response could be correctly attributed to you.

I expect our conversation will take about 60 minutes. First, do you have any questions for me about the project in general or what we will be discussing today?

Do I have your permission to record our conversation? [Confirm permission before recording starts.]

1. Background

I’d like to start with a few questions about your role and responsibilities.

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. Target Population and Local Area

Let’s talk about your SNAP E&T clients.

1. Who is targeted for mandatory SNAP E&T in this area?

1. I understand some SNAP clients in your county are exempt from mandatory E&T, such as [identify exempt groups listed in State plan]. Is that accurate? Are there other exempt groups I did not mention?

2. What are the reasons these groups are exempt?

3. Are there any local areas in this region that are not subject to mandatory SNAP E&T? If so, what are reasons for those exemptions?

2. E&T policies and practices can change from time to time. I understand your State [identify any changes reported during State Director interview]. What effect, if any, do you think those changes have had on E&T participation? Sanction rates? Outcomes?

1. a. [If no changes are reported by State Director, ask]: How have your State’s E&T program or practices changed within the past year? What effect did these changes have on E&T participation? Sanction rates? Outcomes?

3. Training

Next we are interested in knowing more about how you train your eligibility workers on mandatory E&T policies and procedures.

1. What kinds of training do SNAP eligibility workers (or the individuals who screen for exemptions) receive on implementing mandatory E&T policies?

1. Who participates in the training?

2. Is the training in person, online?

3. How often does the training occur?

4. What does the training cover (e.g., screening for exemptions? monitoring for compliance? issuing sanctions?)

2. How do workers stay up to date on current policies regarding E&T requirements, standards for compliance, and exemptions?

4. Intake in E&T Programs

Let’s discuss the procedures for screening, referral, and notification of mandatory clients in E&T services.

1. As you know, clients must be screened for Federal, State, and any local exemptions. What is the process for determining whether a SNAP client is exempt from the mandatory E&T requirements?

1. Do eligibility workers review a list of exemptions with the client, or do they rely on the client to report any barriers they may have?

2. Does screening occur face to face, or by telephone, or using some other mode of communication?

3. What sorts of guidance or aids do workers use during the screening process, if any? [Request copies.]

2. Does the eligibility worker typically make a determination at the time of screening, or is there a delay between the time when the worker gathers relevant information about the client’s circumstances and the actual determination?

1. [If the worker makes the determination at the time of screening:] What additional information do they provide to the client at that time, aside from letting them know they must participate in E&T services? Does the worker direct the client to a particular office, give a phone number to call, or a list of E&T providers? Do mandatory participants receive any hard copy or electronic instructions?

2. [If there is a delay between screening and determination:] How are clients notified about their E&T status? For example, do they get a notice in the mail, a phone call, an email? What additional information do mandatory participants receive upon learning that they are required to participate in E&T services? For example, are they instructed to schedule an orientation or contact an E&T provider? Do they receive hard copy or electronic instructions?

[Request copies of any notification documents.]

3. In your opinion, how complicated or straightforward is the process for determining whether a SNAP client is exempt from mandatory E&T?

3. Are some exemptions more difficult to screen for than others?

4. Can you think of any factors that may contribute to determination errors?

5. Do you think telephone interviews or task-based systems (i.e., shared case management) affect an eligibility worker’s ability to identify an exemption?

4. How often does a participant who qualifies for an exemption get referred to E&T?

5. How are incorrect referrals identified? Describe.

1. When a worker makes an incorrect referral, at what point does the worker typically learn he or she made an error?

2. How do these errors get corrected?

6. Are there any other steps in the intake process that your office is responsible for after making the referral? Please describe.

5. E&T Components and Support Services

We’d also like to better understand your relationship with E&T providers and the components and support services that are available to your clients. [Note: Interviewer will know what whether and which providers the office works with based on initial discussions with the State to coordinate the site visit.]

1. If State has E&T Providers: To begin, I want to make sure I am aware of all the E&T providers that work with SNAP mandatory E&T participants. Can you review that with me? Can you tell me about the services each provider offers?

1. How did each of these providers connect with SNAP—in other words, what prompted your office to begin referring mandatory E&T participants to these partners?

2. How do the SNAP office and E&T providers communicate to coordinate services? How well does this approach work?

3. Can mandatory E&T participants choose among various E&T services, or do they get placed or assigned to a particular service?

1. [If they can choose among various services:] Have you found that some services are more “in demand” or more heavily used than others? Are there some services that mandatory participants never or rarely choose? What do you think accounts for such differences?

4. What supports, if any, does your office provide to mandatory E&T clients?

1. What supports do they most commonly need?

2. How do clients learn about the supportive services that are available? Do they have to ask for them, or does the worker describe them to the client? When are participants informed about these supports?

3. What do your workers tell clients about participant reimbursements? Do you think all workers let participants know that they are exempt from E&T if costs exceed supports?

5. To what extent do you think the supports offered by your office and the providers cover the costs associated with participating in the program?

1. How does your office determine whether a participant should be exempt from participating in E&T because of the cost?

6. Local Capacity

1. Does your office currently have sufficient resources and capacity to serve mandatory E&T clients, including required participant reimbursements?

1. [If insufficient resources, ask:] What resources are you lacking? How does this affect the services you provide?

2. What would happen if participation reached 100 percent?

1. What additional resources would your office need?

2. Do you think the local E&T providers have the collective capacity to serve the entire mandatory SNAP E&T population?

1. [If providers are used, ask:] Are some providers limited in the number of clients they can serve?

2. [If participants can choose components, ask:] Have you seen demand for a particular component exceed the providers’ capacity?

7. Compliance and Sanctions

Next I’d like to talk about potential “dropoff” points in the process, where a mandatory participant is at risk of being found noncompliant with the E&T requirements.

1. What are the participation requirements for clients in order to remain compliant? [Probe for:] How many days or hours are they required to participate? What steps are they required to take?

2. How do you track how clients are meeting their E&T requirements?

3. How do E&T providers track whether clients are meeting their E&T requirements?

1. Do they track who is referred versus who shows up? How?

2. Do they track ongoing participation? How?

3. How well do you think providers are tracking clients? Please explain.

4. Do providers have difficulty tracking certain acts of noncompliance? Is participation more difficult to track for certain components?

4. What information do providers report to your office? How? [Probe for:] What component the client is enrolled in? If they complete the program? Receive a credential or certificate? Enroll in another component? Get a job?

5. How do providers report noncompliance to your office?

1. What other information do they report?

2. How frequently do they provide data?

6. Have you ever had difficulties with any providers? Describe.

1. How did you address the difficulties?

7. [If State offers conciliation process, ask:] Can you describe your conciliation process for noncompliant clients? [Probe for:] When does conciliation happen—before the Notice of Adverse Action (NOAA) goes out? As part of the NOAA?

1. How long is the conciliation period?

2. How does your office determine whether a client comes into compliance? What information do you use?

8. How and when do SNAP workers determine “good cause” for noncompliant clients?

1. Do they make the determination themselves or rely on the client to demonstrate good cause?

2. Do workers ever use information from providers when determining good cause? From employers? Please describe.

9. Let’s talk about NOAAs. What sorts of questions do clients ask about NOAAs?

1. Are there are any parts of NOAAs that clients seem to have difficulty understanding?

[Request copies of NOAAs if haven’t already.]

10. As far as you know, what are some of the reasons clients don’t comply with the requirements outlined in a NOAA before being sanctioned?

11. At what step in the process do most sanctions occur in your office (e.g., between screening and E&T orientation, between orientation and participation in activities)?

1. Can you think of anything that could be done to reduce the likelihood of noncompliance at that point?

12. What barriers do noncompliant mandatory E&T clients most commonly report for not being able to comply upfront (i.e., before entering E&T)?

1. What do you think are the challenges or obstacles mandatory SNAP E&T clients face in sustaining participation?

13. Are clients more or less likely to be sanctioned depending on the component they are assigned to? If so, how would you explain this variation?

1. Do certain components require more time and/or effort to meet compliance requirements? Please explain.

14. Do participant reporting requirements vary between components? Please explain.

15. How does your office track and report sanctions to the State?

16. How does your office monitor whether sanctions are made correctly, if at all?

Do you ever conduct case reviews of sanctioned cases? How are these reviews conducted?

17. How frequently would you say mandatory E&T clients leave the program because they…

1. Are sanctioned for noncompliance?

2. Do not meet the deadline to recertify?

3. Begin earning wages above the threshold to qualify for SNAP?

8. Monitoring Effectiveness of E&T Services

1. How does your office track what happens to a mandatory E&T participant once he or she is referred to E&T?

1. Do you or the providers monitor the employment trajectory or the types of jobs clients secure following their participation in E&T? Describe.

2. What jobs are most in demand in this area (low-, middle-, and/or high-skill jobs)? How well do you think each E&T component prepares clients for employment in the local job market?

9. Challenges and Lessons Learned

Finally, I would like to learn more about any challenges you have had in operating your mandatory E&T program and any lessons learned you can share with others.

1. What is most challenging about operating a mandatory SNAP E&T program?

1. Can you think of ways to improve mandatory E&T?

2. Has the State SNAP agency been helpful in resolving any of these issues?

1. What else could they do to support you?

2. Is there anything you would change about the way you serve SNAP E&T clients?

3. What lessons have you learned in serving this population?

1. What advice would you give to other local offices beginning to serve mandatory SNAP E&T clients?

10. Wrap-Up

Thank you for answering all our questions:

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

That completes our questions for you. Thank you very much for speaking with us.

Appendix C.
Revised Local SNAP Office Eligibility Worker
Interview Protocol

My name is [name], and I’m a researcher at [Insight Policy Research/Mathematica Policy Research, working with Insight]. As you may know, Insight is conducting a study of Mandatory E&T Programs for the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture. This study examines how mandatory State SNAP E&T programs are administered and how the programs help SNAP clients move toward economic self-sufficiency. We are interested in better understanding the rates of participation, sanction, and employment among mandatory E&T clients. We also want to explore reasons why mandatory E&T clients are most often sanctioned and when sanctions happen, such as at initial referral, intake and assessment, or service referral. This study also seeks to understand how well mandatory programs help SNAP clients gain skills, certificates and credentials, and stable employment.

My colleagues and I are currently visiting local SNAP offices and E&T providers here and in five other States to collect information from a wide range of stakeholders involved in operating a mandatory SNAP E&T program. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues will be very helpful to the study.

I want to let you know that your participation in this interview is voluntary, and your responses will be kept private to the extent provided by law. We will not share the information you provide with anyone outside the study team, including your supervisor and State-level staff. You may refuse to answer any question, and you may stop the interview at any time. There will no penalties if you refuse to participate in part or at all.

We will take notes over the course of the interview and would like to record the conversation so we can remember the information we collect. We will use this information in our reports to FNS, describing the range of responses expressed by staff. The reports might list the names of States that contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of SNAP offices participating in the study, there is a possibility a response could be correctly attributed to you.

I expect our conversation will take about 60 minutes. First, do you have any questions for me about the project in general or what we will be discussing today?

Do I have your permission to record our conversation? [Confirm permission before recording starts.]

 

1. Background

I’d like to start with a few questions about your roles and responsibilities.

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. Intake for E&T Programs

Let’s talk about screening and referral procedures for mandatory E&T clients.

1. How do you screen for Federal, State, or local exemptions from mandatory SNAP E&T? Please walk me through the determination process. [Probe for:]

1. Are clients screened in person? By phone? Some combination of methods?

2. Do you walk through the list of possible exemptions with the client, or do you ask clients to identify any barriers that might exempt them?

3. Do you make an assessment about a client’s qualification for an exemption, or is that done within the MIS as the client answers questions?

1. If in the MIS, do any of the answers to questions require your judgment (such as disability status)?

1. Do you use a guide or desk aid to conduct the screening interview? If so, how helpful do you find the guide? [Collect guide if available.]

2. Are there specific documents clients need to present to demonstrate they meet one or more exemption criteria?

1. In your opinion, how complicated or straightforward is the process for determining whether a SNAP client is exempt from mandatory E&T?

   1. Are some exemptions more difficult to screen for than others?

   2. Have you encountered circumstances where the client’s situation is complex or falls into a “gray area,” and you were not sure what determination to make? If so, what do you do in those circumstances? How do you reach a decision?

   3. Can you think of any factors that may contribute to determination errors?

   4. How often do you think errors happen?

2. What is the process for referring nonexempt participants to E&T?

   1. Is there a lag time between screening and notification? How long is the lag?

   2. Do you discuss the determination with the client or mail a notification letter? [Collect sample notification letter.] Does that discussion or letter include information about what clients can do if they believe they meet the exemption criteria, but they were determined to fall under the mandatory requirements?

   3. What can clients do if they believe they should be exempt, but they are referred for mandatory E&T?

   4. How frequently do clients try to appeal a determination?

3. Sometimes a participant who qualifies for an exemption may get referred to E&T. How do you learn about these incorrect referrals?

   1. At what point do you typically learn about an incorrect referral?

   2. What happens when you learn of an incorrect referral? Is there a way to correct it? If so, please explain.

   3. How often does a participant who qualifies for an exemption get referred to E&T?

4. Once clients learn they are required to participate in E&T, what happens next?

   1. Is there some type of orientation? Where does it happen (e.g., in this office, or are clients referred to a provider for orientation)?

   2. When does the orientation typically happen?

1. [If orientation is conducted by provider, ask:] Do participants contact the provider, or does the provider contact the participant about completing the orientation?

2. [If orientation is conducted by local office, ask:] How is the orientation scheduled, and how quickly?

3. Is orientation on a set schedule every week, or do you schedule based on flow of clients?

4. What information is provided?

6. How do you determine what E&T provider to refer a client to?

7. What can you tell me about the enrollment process; for example, does the client need to show up at an office, school, or other site to sign up for a particular E&T service? Does the provider make any attempts to reach the client?

3. E&T Tracking and Reporting

Now let’s talk about the relationship you have with SNAP E&T providers. [Note: Interviewer will know what whether and which providers the office works with based on initial discussions with the State to coordinate the site visit.]

1. How do you communicate with E&T providers about a client’s status with regard to participation in E&T and compliance with requirements?

1. How frequently do you communicate with providers? Does frequency vary depending on the provider?

2. How do you know if a participant reports to a provider? Engaged in an E&T component for the requisite hours?

3. How is noncompliance reported? What is the typical timeframe for reporting noncompliance?

4. What are some of the challenges you have seen with respect to communicating with providers about mandatory E&T participants’ compliance?

5. Have you ever had difficulties with any providers? Describe.

1. How did you address the difficulties?

4. Conciliation and Good Cause Determination

We are interested in understanding the steps you need to take after a provider reports noncompliance.

1. [If State offers conciliation process, ask:] Can you describe the conciliation process for noncompliant clients?

1. How long is the conciliation period?

2. How do you determine whether a participant comes into compliance? What information do you use?

1. How do you make a “good cause” determination?

   1. Do you assess good cause based on your own observations, or is it incumbent upon the client to show that he or she has good cause?

   2. Are there specific criteria or guidelines you use for determining good cause?

   3. Do you ever use information from providers when determining good cause? From employers? Please describe.

5. Compliance and Sanctions

We are particularly interested in gaining a better understanding of all the points where a mandatory client might receive a Notice of Adverse Action (NOAA) or be sanctioned for noncompliance.

1. Can you walk me through the process of issuing a NOAA? [Probe for:]

1. How are NOAAs issued? For example, are they sent by regular mail, by email, or are some clients notified by phone?

2. At what point are NOAAs issued?

1. What are clients’ options if they have questions about NOAAs?

2. Do you receive many questions about NOAAs from recipients?

   1. If so, what kinds of questions do you get?

   2. Are you typically able to answer the questions right away, or do you sometimes need to investigate further to find an answer?

   3. Is there anything you would change about NOAAs? Explain.

3. At what step in the process do sanctions most often occur (e.g., between screening and E&T orientation, between orientation and participation in activities, during participation in activities)? Why?

   1. Can you think of anything that could be done to reduce the likelihood of noncompliance at that point?

4. Approximately how many (what percentage) of your clients who receive a NOAA comply with the requirements in time to avoid being sanctioned?

5. Based on your experience, what percentage of mandatory E&T clients never take the first step toward compliance, such as completing the orientation?

   1. What are some of the reasons clients fail to take that step?

6. What about clients who start participating in some kind of E&T but then drop out? What are some of the reasons they report for no longer participating?

7. Approximately how many (what percentage) of participants return to SNAP and E&T after being sanctioned?

   1. What is your sense of how quickly individuals return to SNAP after being sanctioned?

8. Does this office periodically review a portion of sanctioned cases to ensure the sanctions were issued accurately?

9. Do you have ideas for how to improve the sanction process? Or to improve compliance?

6. Monitoring Effectiveness of E&T Services

We’re also interested in learning about the metrics you report to the State about mandatory E&T and any thoughts you have about the effectiveness of the mandatory E&T program.

1. Are you responsible for tracking and reporting participation data for mandatory E&T clients to the State? If so, what data do you track and report?

1. How do you report the data? [Probe for: What format of the data? Database? MIS export?]

2. How frequently do you report data?

1. In your opinion, do the services local E&T providers offer help E&T clients become more employable?

   1. What is your sense of the providers’ success rate in helping clients find employment?

   2. Do you have access to such data?

2. What do you hear from clients with respect to the E&T services offered? To what extent do they feel they are getting valuable training or gaining skills that can help them get a job?

3. What are the most common barriers your clients face in finding employment?

   1. What other services or supports do you think would help in addressing those barriers?

   2. Are there other exemptions you would suggest for your clients?

7. Training

Now I have a few questions about training you receive related to mandatory E&T policies and procedures.

1. What kind of training have you received on practices for implementing mandatory E&T policies?

1. How often does the training occur?

2. Is the training in person, online?

3. What does the training cover (e.g., screening for exemptions, monitoring for compliance, issuing sanctions)?

4. Is there any other information that would have been helpful in implementing mandatory E&T correctly?

1. How do you stay up to date on current policies regarding E&T requirements, standards for compliance, and exemptions?

8. Challenges and Lessons Learned

Finally, I would like to learn more about any challenges you have had with your mandatory E&T program and any lessons learned you can share with others.

1. What is most challenging about implementing a mandatory SNAP E&T program?

1. Can you think of ways to improve mandatory E&T?

2. What additional support or guidance might be helpful for eligibility workers like you?

1. What lessons have you learned in serving this population?

   1. What advice would you give to other eligibility workers serving mandatory SNAP E&T clients?

9. Wrap-Up

Thank you for answering all our questions:

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

That completes our questions for you. Thank you very much for speaking with us.

Appendix D.
Revised E&T Provider Supervisor and Frontline Staff Interview Protocol

My name is [name], and I’m a researcher at [Insight Policy Research/Mathematica Policy Research, working with Insight]. As you may know, Insight is conducting the Assessment of Mandatory E&T Programs study for the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture. This study examines how mandatory State SNAP E&T programs are administered and how the programs help SNAP participants move toward economic self-sufficiency. We are interested in better understanding the rates of participation, sanction, and employment among mandatory participants. We also want to explore reasons why mandatory E&T participants are most often sanctioned and when sanctions happen, such as at initial referral, intake and assessment, or service referral. This study also seeks to understand what data are available on how well mandatory programs help SNAP participants gain skills, certificates and credentials, and stable employment.

My colleagues and I are currently visiting providers here and in five other States to collect information from a wide range of stakeholders involved in providing SNAP E&T to mandatory participants. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues will be very helpful to the study.

I want to let you know that your participation in this interview is voluntary, and your responses will be kept private to the extent provided by law. We will not share the information you provide with anyone outside the study team, including your supervisor and State-level staff. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalities if you refuse to participate in part or at all.

We will take notes over the course of the interview and would like to record the conversation so we can remember the information we collect. We will use this information in our reports to FNS, describing the range of responses expressed by staff. The reports might list the names of organizations that contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility a response could be correctly attributed to you.

I expect our conversation will take about 60 minutes. First, do you have any questions for me about the project in general or what we will be discussing today?

Do I have your permission to record our conversation? [Confirm permission before recording starts.]

[Unless marked otherwise, the questions should be asked of both provider directors/supervisors and frontline staff. Questions for the directors/supervisors only are marked with an asterisk (“*”); questions for frontline staff only are marked with an inverted V (“^”).]

Unless we specify otherwise, all of the questions we ask are referring to your work with mandatory SNAP E&T participants. When you are working with a participant, do you know if they are a mandatory or voluntary participant? [If yes, ask how. If no, note this for how you interpret the answers to the questions.]

1. Background

I’d like to start with a few questions about your roles and responsibilities.

1. What is your official job title or position?

2. What are your primary responsibilities?

3. How long have you worked at [agency/organization]? How long in this position?

2. Target Population and Local Area*

Now I would like to talk about what types of clients your [agency/organization] serves and get some background on your local area.

Target Population

1. Are your services targeted to certain populations (e.g., homeless, youth, veterans, at a certain education level, interested in a certain industry)?

2. What geographic area does your program serve?

3. What human services program clients do you serve (e.g., referrals from TANF, SNAP E&T, WIOA)?

4. How long have you been serving SNAP E&T program participants?

Local Area

5. Do you know how many other employment and training providers in the area provide comprehensive services (e.g., assessment, job search and training, support services) to clients? [If necessary, clarify that we are not interested only in providers that serve E&T participants.]

1. Do the services they provide overlap with yours?

2. Do you refer the clients you serve to any of these other organization to provide additional E&T services you are not able to provide (co-enrollment)?

3. Do you ever refer clients to these other organizations for their primary services because your organization is not a good fit for the SNAP E&T participants? How frequently?

6. What other kinds of supports are there in the area (e.g., transportation, food banks, clothing closets, housing assistance)?

7. What are the services most in need by the people in this local area?

1. How would you describe the local economy? What jobs are most in demand in this area (low-, middle-, and/or high-skill jobs)? What industries are on the decline?

3. Organization, Staffing, and Communication*

Next I would like to talk about how [agency/organization] is organized and staffed and how you communicate with other organizations about SNAP participants.

1. Do you have a formal contract to serve SNAP E&T participants?

1. Is the contract directly with the SNAP agency, or do you contract with another organization to serve SNAP recipients? Describe.

2. Does your contract specify how many SNAP E&T participants you will serve per month/year? If so, describe.

3. What are the primary service requirements under your contract?

4. Have you had any challenges with this contract, working with SNAP, or serving SNAP E&T participants? What aspects could be improved?

5. Are there any lessons you have learned about establishing and maintaining collaborations with the SNAP agency? [If they are contracting with the SNAP agency]

2. How do you determine what services to offer at your organization (e.g., best practices, local labor data/demand, assessment of skills needed by participants, dictated by the funding organization)?

1. Are there contractual/regulatory constraints on the services you provide to SNAP E&T participants? Describe.

3. What types of staff work with SNAP E&T participants?

1. Do staff serve all clients or do you have dedicated staff for clients from each type of program?

2. Have your staff received any training about SNAP E&T procedures and requirements from other organizations/agencies? If so, describe who provided the training, the frequency, materials provided, format, and content of the training. Is it ongoing?

4. In general, do you primarily receive new clients to your agency/organization through referrals or walk-ins?

1. Do you know if a client is referred from a certain program? How? Does the referral process/frequency differ by agency?

2. Does SNAP E&T primarily use referrals?

3. How do you receive SNAP E&T referrals? Describe the process. How frequently do you receive them? Provided through MIS or outside of systems?

5. How much contact do you have with the SNAP agency? For what reasons and with whom?

1. Do you have a primary point of contact or do you interact with multiple individuals?

2. How frequently are you in contact with the SNAP agency about E&T in general or individual participants in particular?

3. Have there been any challenges with communication? If so, how could it be improved?

6. Does your organization currently have sufficient resources and capacity to serve all of the mandatory E&T clients that are referred?

1. Do you have capacity to serve more clients? How many?

2. What additional resources would your organization need to serve these clients?

4. Client Flow^

I would like you to describe the client flow through the program from referral to completing services, or dropping out of the program.

1. To start, how do people enter this program generally?

1. Is it different for SNAP E&T participants? [Explore referral questions from section C with frontline staff, as needed.]

2. What other information does the SNAP office provide about SNAP E&T participants?

Intake Process

2. Please describe the intake process. [Probe for the following.]

1. Generally, do SNAP E&T participants come directly from the SNAP office to you, or are they referred from another agency? Do you know how many steps there are prior to coming to your organization?

2. About how much time passes from the when the participant is enrolled in SNAP and when you receive a referral? [Or if no referral, when the participant shows up here?]

3. Is there an orientation? Is orientation in a group setting or one-on-one? Who conducts the orientation? What information is given about mandatory SNAP E&T?

4. Is there an intake interview? What information is collected? Who conducts this interview? Is a caseworker/career advisor assigned before or after this interview? How are they assigned?

5. Do you conduct an assessment of every client? Including SNAP E&T participants? Describe the assessment and the tests included (how long is the assessment, in what format is it conducted, what organization conducts it). How are results used?

6. During assessment, do you ever identify SNAP E&T participants that you feel are not ready to participate in your program because they have excessive barriers (like a disability) or do not meet your program requirements ( for example, have a certain level of education or can pass a drug test)? If so, what are the most common reasons for this? What do you do in these instances? How often would you say this happens?

3. Do your staff conduct any outreach to encourage mandatory SNAP E&T participants to participate/comply? If so, what type of outreach?

1. Does the SNAP agency provide any guidance or materials to use for conducting outreach?

Training/Employment Services

4. What training and employment services are offered at this organization (e.g., education, vocational training, work experience, job readiness training, follow-up services)?

1. What services are offered to SNAP E&T participants? [Probe for how these services are determined.]

2. Do all services offered to SNAP participants meet their mandatory participation requirement? If not, which do not?

5. How long is each service available to a client? How long on average does the service take to complete?

1. Is this different for SNAP E&T participants?

2. Overall, how long are participants in services, on average?

6. May clients receive multiple services?

1. Are they generally offered different services concurrently or sequentially? [Probe for whether there is a structure order of activities or are there several options for moving from one activity to another]

2. How common is it for mandatory E&T participants to receive multiple services? Which services?

Supportive Services

7. What supportive services are available at your organization? [Probe for types, amount, and duration.]

1. What is the process for approving supportive services? Is it the same for SNAP E&T participants; do you need to contact the SNAP agency for approval before providing supports?

2. If the SNAP agency does not approve a support or it is not covered by SNAP, what do you do?

3. Which supports are provided to SNAP E&T participants most frequently (about what percentage, if known)?

4. What supports are provided least frequently to SNAP E&T participants? Why?

5. What types of referrals do you provide for support services (to other organizations or programs)?

6. To your knowledge, do SNAP E&T participants receive other support services from the SNAP agency (such as transportation or childcare assistance)?

7. Do you find that the supports typically cover the client’s expenses? If not, what is needed?

8. Are SNAP participants ever unable to participate because they need a support service that is not covered by SNAP? How often would you say this happens?

1. What happens in that case? Do you notify the SNAP agency for further assistance?

Case Management

8. Please describe the case management provided to clients. [Probe for the following.]

1. Who provides it?

2. Typically, how much time is spent with clients in an average month? Is the time different for SNAP E&T? Is there a minimum level of contact required?

3. What is the typical format of meetings (in person, telephone, email, text)? Who generally initiates contact?

4. Do you use any case management tools or models as part of your services (e.g., team-based case management across programs, clinically certified staff assessments, case monitoring)?

5. How responsive are clients to case management (do they attend meetings/respond in a timely manner/seem to find contact useful)?

6. Do SNAP E&T participants need more/less/same level of contact as other clients?

9. How are clients assigned to a specific service? [Probe for the following.]

1. Is the assigned service based on the results of an assessment/test? At the discretion of the caseworker? At the discretion of the client? What criteria are used for each type of service?

2. Are clients aware of all the services available, or are they told only about those deemed most appropriate for them based on an assessment or their intake interview?

3. Generally, how much input does the client have in selecting services to participate in? Does this input vary for SNAP E&T?

Participation

10. In which E&T services do SNAP E&T participants most frequently participate? Why?

1. Which services do they participate in least frequently? What are the reasons for this?

2. Are some services in higher demand but have limited space?

3. [If a mismatch between what is offered and what is received]: Are there reasons SNAP E&T participants tend to not participate in [X, Y, Z] services? [Probe for: Is this about their preferences, their skills/abilities, or are limited slots available?] Are most participants encouraged to enter into these services, or do they tend to apply to a limited group of people?

Client Outcomes

11. Where are the most frequent dropoff points for SNAP E&T participants (e.g., never show for first meeting, do not return after assessment, do not show for assigned services)?

1. Is this pattern similar for other clients?

2. What are the reasons participants tend to drop off at these points?

12. How frequently do SNAP E&T participants complete the training and find employment? [Probe for percentage. If respondent does not know, probe for his or her best estimate. Ask if he or she can provide these data after the interview.]

1. What types of employment are generally found? How does the type of employment found vary by type of service, if at all?

2. Are these rates and types of employment similar for other clients you serve?

3. Can these jobs help make participants self-sufficient, or do they still need SNAP and other types of assistance to supplement income? If not, why not?

13. What is the process for when a SNAP E&T participant fails to comply with requirements or never shows up for services?

1. How much guidance do you receive from the SNAP agency about the process for identifying and reporting noncompliance?

2. How long do you give a participant before considering the individual noncompliant? Are a certain number of contacts required?

3. Do you reach out to the participant to try to reengage? Describe that process.

4. After assessments, how frequently do you determine that SNAP E&T participants should qualify for an exemption or good cause not to participate in services? What is the process in these circumstances?

5. Perceptions About SNAP E&T Participants

Now let’s talk about your thoughts on the types of challenges SNAP E&T participants have, how they are different from your other clients, and how well they understand what is required of them.

1. What differences, if any, do you notice between SNAP E&T participants and the other clients you serve?

2. What kinds of barriers do SNAP E&T participants face starting or participating in services?

1. Which of these barriers most interfere with participation (which are most significant)? Are the barriers different for those having difficulty complying initially versus those who have been in the program but struggle to complete it?

2. Are these factors different from those you see in other clients you serve? If so, describe the differences.

3. Can you address any of these barriers through your program? If so, how?

3. Do the barriers participants face differ by the type of activity? Do you think the services offered to SNAP E&T participants are appropriate for their needs, barriers/abilities, and interests? Why?

4. Do you believe the mandatory nature of the program affects participants’ engagement? Or the way you offer services?

5. In your experience with SNAP E&T participants, how well do they understand their E&T requirements?

1. Do they tend to know why they were told to come to your organization?

2. Do they understand the documents the SNAP office provides to them regarding their work requirements? Do they ever ask you to help explain letters they receive from SNAP? What do they have difficulty understanding?

3. Do they realize they could lose SNAP benefits if they do not comply?

6. Do SNAP E&T participants generally seem interested in participating?

7. For those SNAP E&T participants who complete the program/services, do you believe they are prepared to compete for in-demand jobs in your community?

1. How successful have they been in obtaining and retaining these types of jobs?

6. Data Tracking and Reporting

Next, I would like to discuss the kinds of data you track, the systems you use, and the reports you develop for managing your work.

1. What percentage of your SNAP E&T referrals ultimately participate in services?

1. Do you track the number of participants who appear initially versus those who do not appear after referral?

2. At what point do those who start services tend to stop participating? What do you think leads to dropoff at this point? [Probe for additional dropoff points.]

2. *How many mandatory SNAP E&T participants are you currently serving?

1. Is that number generally how many you serve in an average month?

2. Approximately how many unique participants do you serve each year?

3. Do you have sufficient resources to serve all the SNAP participants referred to you?

4. Are you currently able to support more E&T participants than specified in your contract? If so, how many more?

3. How long do SNAP E&T participants generally participate in services?

1. Is this more, less, about the same as the other clients you serve?

4. ^What types of data do you track for each client (beyond what is required by SNAP)?

For instance, do you track metrics on—

1. Services started/completed?

2. Skill gains/acquiring degrees or training certifications?

3. Job placement, retention, advancement? Over what period?

4. Earnings and types of employers?

5. Characteristics of clients?

5. What types of systems do you use to track data on your SNAP E&T participants? Are those data systems linked to the SNAP agency? How do you transfer/receive information to/from the SNAP agency?

6. What type of reporting are you providing to SNAP [or other contracting organization] for each participant? How? How frequently?

1. Did you need to create new systems/processes to track SNAP participants? Mandatory versus voluntary?

2. Does the SNAP agency conduct any monitoring/oversight of the program? Describe this process and frequency.

7. How do you notify the SNAP agency about noncompliance?

1. How frequently do you report on noncompliance?

8. ^Do you require clients to provide you with any documentation of their participation or work efforts? If so, describe what is tracked for different services.

1. Is any of this documentation provided to the SNAP agency?

9. *Do you use the data collected for any internal purposes?

1. Why kind of reports do you produce? How are they used?

2. Are there other types of data that would be helpful for your monitoring/program management? Describe. What are the limitations to obtaining these data?

7. Challenges and Lessons Learned

Finally, I would like to learn more about any challenges you have had in working with the SNAP E&T program and any lessons learned you can share with others.

1. What is most challenging about serving mandatory SNAP E&T participants?

1. Can you think of ways to improve this situation?

2. Has the SNAP agency been helpful in resolving any of these issues?

2. What if anything would you change about the way you serve SNAP E&T participants? Why?

3. What lessons have you learned in serving the mandatory SNAP E&T population? What advice would you give to other providers who may be considering serving this population?

8. Wrap-Up

Thank you for answering all our questions.

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

That completes our questions for you. Thank you very much for speaking with us.

Appendix E.
Administrative data collection instructions and variable list

What is the purpose of this administrative data request?

This document provides instructions and requirements for submitting SNAP administrative caseload data as part of the Assessment of Mandatory E&T Programs study being conducted by FNS. These data will be used to examine the characteristics of mandatory SNAP E&T participants and to assess the E&T services, sanctions, case closures, and other outcomes associated with E&T participation.

The following sections provide detailed instructions for preparing and submitting the data extract, including guidance on which records to include in the file, the time period for the data, a specific list of caseload data variables needed, the format for the file, how to handle missing data, how to ensure confidentiality, and the process for submitting data.

Insight will arrange a consultative discussion with State staff familiar with the State data systems and policies to discuss the administrative data to be provided and convey the level of detail and quality of the data that can be provided on E&T participants and their outcomes.

What SNAP records should be included in the file and for how many months?

The file should include monthly data on individuals aged 13 to 62 who received SNAP benefits at any point during the 29-month period from October 1, 2016, through February 28, 2019.

There should be up to 29 rows of data (1 row per month) for every individual included in the file. Months in which an individual is not in your State caseload system can be left blank or excluded from the file.

What variables should be included in the file?

Table A provides a list and description of the variables that should be included in the file for every individual who received SNAP at any point during the 29-month period from October 1, 2016, through February 28, 2019. Please provide the variables in the order shown in table A.

If codes are used to identify information (e.g., reasons for exemptions, reasons for case closures), please submit a crosswalk of these codes and their descriptions with your file.

What is the file format?

Preferred file formats are comma-separated values (.csv) or Excel (.xls), although other formats are acceptable, including SAS datasets. Each file should have 1 record per participant-month (from for October 2016 through February 2019), and each record should include all variables in the variable list (see table A).

How should missing/unknown information be handled?

Missing values should be indicated by a BLANK space. Please do NOT fill unknown values with zeroes. Zero should ONLY indicate an actual zero value, such as zero dollar income. For example, if the participant’s household stops receiving SNAP benefits, the household size and benefit fields should be BLANK, not zero.

For all indicator variables, a value of “1” should represent “yes,” while a value of “0” should represent “no.”

How will Insight ensure confidentiality of State data?

The data will be maintained on a secure server and available only to key project personnel needed to clean and tabulate the data. No personal identifiers will be disclosed in reports.

How should we submit the files?

To protect the data, please submit the file using Insight’s secure FTP system, a secure file transfer site that encrypts both commands and data, preventing passwords and sensitive information from being accessed during transmission. Instructions for using this system will be sent separately.

If possible, submit a test data file with 3 months of data to Insight. The test file should include data on SNAP participants identified as work registrants at any point during the period January–March 2018. This will enable Insight to review all data and clarify any remaining concerns before the first data submission.

Questions or concerns?

If you have any questions or concerns, contact the study’s principal investigator, Carole Trippe, at ctrippe@insightpolicyresearch.com or 703.504.9498.

Table A. Requested Variable List

Variable Requested	Variable Name	Variable Description	Code/Categories Examples
Individual-Level Variables
Reference month and year	REF_MONTH_YEAR	Benefit reference month and year for the SNAP participant’s SNAP household If no SNAP benefit was received that month, still include a row for that month with data for the CLIENT_ID and REF_MONTH_YEAR		MMYYYY
Client ID	CLIENT_ID	Participant’s individual SNAP ID provided by the State to identify individual participants within a SNAP household If no SNAP benefit was received that month, still include a row for that month with data for the CLIENT_ID and REF_MONTH_YEAR		Example: 2130768 (character variable)
Social Security Number	SSN	SNAP participant’s Social Security Number		Example: 510981319
Received SNAP benefits	SNAP_FLAG_PER	Whether the participant received SNAP benefits this month		0 = no 1 = yes
Date of birth	DOB	SNAP participant’s date of birth		MMDDYYYY
Gender	GENDER	SNAP participant’s gender		1 = male 2 = female
Race/ethnicity	RACE_ETHN	SNAP participant’s race and ethnicity		Provide definitions of codes in separate crosswalk
Marital status	MARITAL	SNAP participant’s marital status (e.g., married, single)		Provide definitions of codes in separate crosswalk
Relationship to case head	RELATION	SNAP participant’s relationship to the case head (e.g., head of household, spouse, child, adult)		Provide definitions of codes in separate crosswalk
Primary language spoken or language used on application	LANGUAGE	Codes used to identify primary language for participant or participant’s household		Provide definitions of codes in separate crosswalk
Whether participant has received a high school diploma or GED	EDUCATION	Codes used to identify level of school completed (or whether SNAP participant received a high school diploma or GED)		Provide definitions of codes in separate crosswalk
Participant’s earned income amount	P_EARNED_INC	Individual-level monthly earned income for the SNAP participant		Example: 200
Work registrant exemption status	WR_EXEMPT	Codes used to identify whether SNAP participant is exempt or not exempt from work registration		Provide definitions of codes in separate crosswalk
SNAP E&T exemption status	E&T_EXEMPT	Codes used to identify whether SNAP participant is exempt or not exempt from E&T		Provide definitions of codes in separate crosswalk
Voluntary or mandatory SNAP E&T participation	MAND_OR_VOL	Indicator of whether the SNAP participant is identified as voluntary or mandatory E&T participant		1 = voluntary 2 = mandatory
SNAP E&T referral status	ET_REFERRAL	Whether the SNAP participant was referred to SNAP E&T		0 = no 1 = yes
SNAP E&T participation status	ET_PART	Whether the referred SNAP E&T individual participated in E&T component		0 = no 1 = yes
E&T sanction status	ET_SANCT	Whether the individual is under E&T sanction this month		0 = no 1 = yes
Start and end dates of E&T sanction	ETSANCT_DATES	Start and end dates of the E&T sanction		MMDDYYYY - MMDDYYYY
Reason for E&T sanction	ETSANCT_REASON	Codes used to identify reason for E&T sanction		Provide definitions of codes in separate crosswalk
Length of E&T sanction	ETSANCT_LENGTH	Number of months of assigned sanction for sanctioned E&T participants		Example: 2
E&T sanction number	ETSANCT_NUM	E&T sanction number or iteration		1 = first sanction 2 = second sanction 3 = third sanction 4 = other
Good cause determination for noncompliant E&T participant	GOODCAUSE	Whether the noncompliant E&T participant was granted good cause determination		0 = no 1 = yes
Work registration sanction status	WR_SANCT	Whether the SNAP participant is under work registration sanction this month		0 = no 1 = yes
Reason for work registration sanction	WRSANCT_REASON	Codes used to identify reason for work registration sanction, if available		Provide definitions of codes in separate crosswalk
Length of WR sanction	WRSANCT_LENGTH	Number of months of assigned sanction for sanctioned work registrants		Example: 2
ABAWD exemption status	ABAWD_EXEMPT	Codes used to identify exemption status for ABAWD from ABAWD time limits		Provide definitions of codes in separate crosswalk
Receipt of NOAA	NOAA_RECEIPT	Whether the E&T participant received a NOAA related to mandatory E&T requirements		0 = no 1 = yes
Date of NOAA	NOAA_DATE	Date of NOAA related to participant’s mandatory E&T requirements		MMDDYYYY
Whether participant “cured” noncompliance before being sanctioned	CURED	Whether E&T participant complied with requirement cited in NOAA before sanction went into effect		0 = no 1 = yes
Household-Level Variables
SNAP case identification (ID)	CASE_ID	Household or case ID provided by the State to identify the SNAP household of the individual participant		Example: 02076085 (character variable)
Received SNAP benefits (household)	SNAP_FLAG_HHLD	Whether the participant’s SNAP household received SNAP benefits this month		0 = no 1 = yes
SNAP benefit amount	BEN_AMT	Total SNAP benefit received by the SNAP household this month (rounded to nearest dollar)		Example: 357
SNAP household size	HH_SIZE	Number of individuals in the participant’s SNAP household this month		Example: 3
Certification date	CERT_DATE	The last review date associated with household’s SNAP benefit month (i.e., disposition date for reference month)		MMDDYYY
County serving the case	COUNTY	County code or name of the county serving the participant’s SNAP household		Provide definitions of county codes in separate crosswalk
SNAP household’s gross income amount	GROSS_INC	Total monthly gross income for the participant’s SNAP household (i.e., gross income used to determine SNAP eligibility before deductions)		Example: 700
SNAP household’s net income amount	NET_INC	Total monthly net income for the participant’s SNAP household (i.e., net income after all deductions from gross income used to determine SNAP benefit amount)		Example: 600
SNAP household’s earned income amount	EARNED_INC	Total monthly earned income for the participant’s SNAP household		Example: 300
Closed case	CLOSED_CASE	Indicator that the SNAP case is closed		0 = no 1 = yes
Closure date	CLOSED_DATE	Date of SNAP case closure		MMDDYYYY
Closure reason	CLOSED_REASON	Reason for SNAP case closure (e.g., ABAWD time limit, sanction for failure to meet E&T requirements, missed recertification appointment, income over threshold, other)		Provide definitions of codes in separate crosswalk
E&T Activities and Outcomes (if available for E&T participants)
E&T provider serving SNAP E&T participant	PROVIDER	Name of E&T provider serving the E&T participant		Provide name(s) of providers
Type of E&T component assigned	ET_COMPONENT	Codes used to identify the type of E&T component(s) participant was assigned to receive (e.g., assessment, English as a Second Language [ESL], High School Equivalence, Institute of Higher Education, Independent Job Search, Job Search Training, Workfare, Work Experience, On-the-Job Training, Basic Education, Vocational Training, Self-Employment Training, WIOA Program, Job Retention, Unique Component)		Provide definitions of codes in separate crosswalk
Start date of E&T assignment	ET_START	Start date of E&T assignment to component for the E&T participant		MMDDYYYY
Outcome of the SNAP E&T participation	ET_OUTCOME	Outcome of the SNAP E&T participation (e.g., completed a training, educational, work experience, or on-the-job training component; exempted for good cause, quit)		Provide definitions of codes in separate crosswalk
Whether participant is in unsubsidized employment after E&T	EMPLOY_OUTCOME	Indicator of whether the participant is in unsubsidized employment after completing E&T program		0 = no 1 = yes
Start date of unsubsidized employment	EMPLOY_START	Start date of unsubsidized employment obtained after the participant completed E&T program		MMDDYYYY
Earnings of E&T participant after completing E&T program	ET_EARNINGS	Amount of monthly/quarterly earnings of E&T participant after completing E&T program		Example: 900