Attachment K. Pretest Results Memorandum

K. Pretest Results Memorandum

M emorandum

Date: August 24, 2023

To: Kristen Corey, FNS

From: April Fales, Kathy Wroblewska, Sade Akinbayo, and Jonathan Blitstein, Westat Insight

Subject: SNAP E&T Equity Study: Pretest Findings and Recommendations

This memorandum summarizes the pretest findings and recommendations for the following data collection instruments for the Assessing Equity in Work Requirements and SNAP Employment and Training (E&T) study:

1. SNAP State agency survey

2. SNAP State agency interview protocol

3. SNAP local office interview protocol

4. E&T provider interview protocol

5. Other interested parties interview protocol

The primary objective of the pretest was to ensure the instruments were clear and easy for respondents to understand. The pretest sought to—

• Identify problems related to communicating the intent or meaning of the questions.

• Determine whether respondents could provide the information requested.

• Identify problems with clarity, understandability, or ease of responding associated with the introductions, instructions, or explanations.

• Assess the amount of time needed to complete the instruments and other respondent burden issues.

Section A describes the recruitment and data collection methods for the pretest, section B provides an overview of the pretest findings, and section C summarizes the pretest findings and associated recommendations for improvement for each data collection instrument.

1. Recruitment and Data Collection Methods

1. SNAP State Agency Survey

Westat Insight sent an email May 15 to eight members of the Nutrition Assistance Research Advisory (NARA) Panel to request participation in the pretest. The NARA Panel is an independent group of former SNAP State agency staff formed by Insight Westat to serve as pretest participants to reduce pretest burdens on current State staff. Five current panel members and one newly recruited panel member agreed to participate. On June 5, Westat Insight held a kickoff meeting for the six participants to explain the approach and level of effort. After the survey was finalized and Westat Insight received institutional review board approval for the study, we emailed pretest participants a PDF version of the survey and a template for providing feedback. After each participant sent their feedback, we scheduled and conducted an interview to review the input and gain additional insight into the overall protocol and specific survey questions. The interviews lasted no more than 90 minutes.

2. Key Informant Interviews

FNS sent an email May 10 to the SNAP State Director within New Jersey’s Department of Human Services to request the State’s participation in the pretest. New Jersey was selected for pretesting because it has a mixed mandatory/voluntary E&T program, offers a large number of E&T components, and is not currently involved in other FNS data collection activities. The State confirmed its willingness to participate on May 11. The study team followed up with the SNAP Director via email and provided additional information by telephone about the pretest. We requested phone discussions with the following nine staff:

• Up to three State agency officials familiar with SNAP general and able-bodied adults without dependents (ABAWD) work requirements, SNAP E&T, and SNAP data systems

• Two SNAP local office staff at different levels familiar with SNAP general and ABAWD work requirements and SNAP E&T

• Two E&T provider staff at different levels

• Two staff from the New Jersey Department of Labor, which helps administer SNAP E&T in the State

In mid-June, New Jersey provided respondents’ contact information for four pretest interviews. We scheduled the interviews to take place between June 22 and June 30.

We allotted an additional 15 minutes for each interview for followup questions and feedback on the instruments. We took detailed notes during each interview and observed which questions respondents had difficulty answering or needed additional clarification to answer.

2. Summary of Findings

This section provides an overview of the pretest findings. Instrument-specific findings are presented in section C.

We received generally positive feedback about the survey. Participants thought the planned protocol would work well and SNAP State agency directors would take the task seriously. Most participants believed staff at the agencies would be able to provide answers to all survey questions, though they all agreed it would be a collaborative effort. The participants thought the suggestions for which staff should complete each section were appropriate and that those staff would be able to accurately complete their given survey sections. Participants found very few questions in the survey confusing, but feedback did reveal several areas for improvement.

Overall, the New Jersey pretest team thought the interview protocols flowed well, and all interviews were completed within the allotted time. However, for the State agency interview, the interviewer skipped questions related to E&T provider selection and management because that responsibility falls under the purview of the Department of Labor in New Jersey. To ensure the protocol can be completed within the allotted timeframe by all respondents, we streamlined questions related to the general and ABAWD work requirements. We also added E&T provider-related questions to the other interested parties protocol.

The E&T provider and local office respondents had difficulties answering questions related to the definition of equitable access and equitable outcomes in SNAP and SNAP E&T. We recommend restructuring this question to make it more concrete. Local office respondents had difficulties answering questions related to the general work requirements, ABAWD requirements, and data collected. Local office staff noted they do not screen for general work requirements because the State agency no longer has a mandatory E&T program. They also struggled to recall processes related to ABAWD work requirements because the time limit has not been in place since the onset of the COVID-19 public health emergency in 2020. However, because the landscape of work requirements in SNAP may be different by the time data collection begins next summer, and these questions may apply in other States that have mandatory programs, we recommend keeping these questions. Other local offices may be better able to speak to these topics because administrative processes vary across State agencies. Local office staff noted they do not collect additional data or conduct internal data analyses. Though staff were unable to answer these data-related questions in section E (Data Availability and Needs) in detail, we recommend retaining them in the protocol because it is equally important to know that local offices do not collect and/or analyze any additional data.

3. Findings and Recommendations by Data Collection Instrument

This section summarizes the pretest results for each data collection instrument and the associated recommendations for improvements.

1. SNAP State Agency Survey

To pretest the SNAP State agency survey, the team requested interviews with NARA Panel members who have experience working in SNAP State agencies. We chose this approach to avoid overburdening State agency directors who will be asked to participate in the data collection next year.

All participants have had experience serving in one or more of the following roles at SNAP State agencies: eligibility worker, SNAP E&T, and supervisor. Participants previously worked in Colorado, Nevada, Utah, and North Carolina. Therefore, the feedback represents a variety of perspectives.

Duration

Because pretest participants are not current SNAP State agency staff, we did not ask participants to attempt to complete the survey, and therefore we could not obtain an accurate estimate of survey length. However, some participants indicated the 40–60-minute estimate seemed accurate, while others thought it would take between 90 minutes and 2 hours to complete. Because of the wide variation in anticipated completion times, we increased our anticipated time to complete the survey to 75 minutes.

General findings

Participants thought the survey would be easy to answer and was straightforward.

Survey pretesting and feedback revealed inconsistent terminology at times:

• The SNAP eligibility system is referred to as the management information system (MIS), eligibility system, and database throughout the survey. For consistency, we will use the term “SNAP eligibility system” and only use MIS as a reference or example of what the system might be called.

• We will refer to the entities that provide E&T services as “E&T service providers” to encompass both in-house and external providers of various components.

Question-by-question findings and recommendations

Table 1 provides our findings and recommendations for specific questions. The findings are limited to questions that require a change and questions Westat Insight specifically probed on, regardless of whether a change is recommended.

Table 1. Item-Level Recommendations for SNAP State Agency Survey

Question From Instrument	Findings/Observations	Recommendations
5. How does your State conduct eligibility interviews? Select all that apply. In person By telephone Virtual interviews	One participant missed the “select all that apply” option. Another participant thought a definition of “virtual interviews” would be useful because she thought it might include instant messaging/chatting.	The web instrument will be programmed to ensure respondents clearly understand they can check all options that apply. We will change “virtual interviews” to “video calls,” add “instant message/text,” and also add “other; specify.” The revised question follows: 5. How does your State agency conduct eligibility interviews? Select all that apply. In person By telephone By video call By instant message or text Other; specify: ________
8. When do you require work registrants to provide verification to establish they are unfit for work? Only if client-provided information is questionable Any time the client provides information indicating they may be unfit for work Other; specify____________	One participant stated that respondents could interpret “unfit” differently and recommended to look into guidance for “unfit for work” to better inform this question and consider asking about following a script. Another participant asked if we should specify “mandatory” work registrants because voluntary registrants would not need verification.	We do not believe it is necessary to add “mandatory.” However, to gain more information, we propose to change the question response options to the following: 8. How do you decide when to require work registrants to provide verification to establish they are unfit for work? We always require verification We follow a script or set guidelines to decide when verification is needed Eligibility workers can use their discretion to decide when verification is needed Other; specify __________
11. Which of the following steps do eligibility workers take to determine if a person has good cause for not meeting the general work requirements before applying disqualification from SNAP? Select all that apply. Discuss the situation with the client Ask the employer or a collateral contact to verify the information Consult with other State agency staff before making a determination None of the above Other; specify ________	We tested this question based on FNS feedback asking if “good cause” should be defined and if employer and collateral contact should be separated into two distinct response options. Participants universally commented that “good cause” is a term that is understood universally. Participants did not think the second response option should be split, unless the distinction between employer and collateral contact is important.	Based on feedback, we do not recommend changing this question or the response options.
13. How many contact attempts are required to be made to inform individuals they have been referred to the SNAP E&T program? OPEN TEXT FIELD ______ [RANGE 1–20]	One participant asked if this question is about State agency policy or Federal policy.	Change question to the following: How many contact attempts does your State policy require in order to inform individuals they have been referred to the SNAP E&T program?
21. How are E&T participants assigned to case managers? Select all that apply. Automatically, within an E&T data management system Assignments are made based on case managers’ caseload/availability Assignments are made based on participant location Assignments are made based on skills and barrier assessment Other; specify: ________	Participants thought the question was OK as worded but agreed it would be beneficial to split it into two questions: one if assignments are made by a system or human (or by a system and can be overridden by a human) and one about what those assignments are based on. One participant thought that if the system assigns cases, only an IT person would know what they are based on. All participants agreed the three categories (caseload/availability, participant location, skills and barriers) covered everything, but one thought skills/barriers assessment could be drilled down because it covers a broad swath of situations.	Split question 21 into two questions: 21a. What factors are used to assign E&T participants to case managers? Select all that apply. Case manager caseload/availability Location of case manager and participant Case manager area of expertise Similar characteristics between case manager and participant Other; specify: ________ 21b. How are participants assigned to case managers? Computer assigned Manually assigned Computer assigned, but assignments can be manually changed
27. What type of reimbursement cap does your State have? Select all that apply. Individual expense type (e.g., auto repairs, supplies, legal services) Monthly combined cap Annual combined cap Lifetime combined cap Does not apply—no reimbursement caps Other; specify: ___________	One participant did not understand the use of the word “combined” in this question.	Because no other participants expressed concern about this item, we do not recommend a change at this time. However, we will continue to research this issue to determine if better terminology is available.
28. SNAP E&T guidelines state that participant reimbursements must be covered for support services that are “reasonable and necessary.” Does your State reimbursement policy define what “reasonable and necessary” means for any of the following? Select all that apply. Transportation assistance Other; specify: ____________ None of the above	One participant pointed out that the question as written is a yes/no question, not a check all.	We will make this question into a yes/no formatted table. Because of that change, we will also remove the “none of the above” option.
30. [If 29.1 or 29.2 selected] How are participants notified that a review of their reimbursement is available to them? Select all that apply. This information is included in the SNAP E&T orientation materials This information is available to participants through the State’s website This information is provided by case managers	One participant suggested adding either a “mailed notice” or an “other; specify” response option.	We will add an “other; specify” option to this question.
31. Does your State agency offer assistance to participants who need to submit documentation for participant reimbursements? Select all that apply. Yes, one-on-one assistance in person Yes, group assistance in person Yes, virtually through video call Yes, through email or text No, formal assistance is not provided	One participant noted that standard phone calls are not in the response options.	We will add a response option between video call and email that reads, “Yes, through phone calls.”
38. Does your State agency use any of the following data sources to tailor the types and locations of E&T services offered to meet the needs of participants? Select all that apply.	One participant was at first confused by the question because it is the only time we ask about external data.	We will consider adding an introductory/transition statement to prime respondents that this question asks about external/population-level data to inform services: This question asks about both external and internal data sources you may use to help inform your E&T program. Does your State agency use any of the following data sources to tailor the types and locations of E&T services offered to meet the needs of participants? Select all that apply.
43. [If any yes to 41 or 42] Has your State agency developed policy or guidance based on any of the data your State agency examined? Yes No	We revised this question prior to pretesting to address client concern that it was not clear. When testing the revised version of the question (shown here), participants thought the question was clear.	No additional changes are recommended.
46. For each SNAP E&T component your State agency offers, are the following participant outcomes tracked, either by your State or by providers?	One participant pointed out the phrase “each SNAP E&T component” and commented that the survey instrument would need to repeat this table for every component, which would be burdensome.	Because one State could have multiple entities offering services in a single component, even repeating this component would be difficult. Because the survey seeks to learn what data, if any, are available, we think this question would be less burdensome while still achieving the goals of the study by looking at E&T overall. We therefore recommend this question instead: Are the following participant outcomes tracked, by your State agency or your E&T service providers?
47. Does your State agency have access to the necessary data (i.e., demographic and outcome data stored on the individual level) to track outcomes by participant characteristics (for example, job placement by gender)?	This question was tested because of client concern that it is too broad and difficult to interpret. Participants generally understood this question asked about the ability to pull reports.	While no significant issues with this question are apparent, we recommend the following change to be more concise and to link the question to Q46: [IF NO TO ALL IN TRACKED AT INDIVIDUAL LEVEL IN Q46] Although outcomes are not actively tracked at an individual level, would your State agency be able to pull reports on outcomes by participant characteristics?
51. Does your State agency receive feedback on participant satisfaction (e.g., quality, fit) with SNAP E&T services and service providers? This feedback could come directly through the agency or be gathered by providers and received by the agency. [hover text on quality or fit of SNAP E&T: Perceived benefit, feels like SNAP E&T experience was helpful, E&T experience matched existing or desired skills]	One participant commented she did not like the word “fit” here because she thought of the word in the sense of physique.	This comment made us aware that the word “fit” in the context of SNAP is usually used for fit/unfit for work, so we recommend changing the wording to “relevance.”
55. Which of the following SNAP participant data elements in the first column does your State eligibility MIS capture? In the second column, indicate whether the data element is required. If it is not required, indicate in the third column whether the data element is collected	Participants commented that the instructions in the stem are confusing, and it is also unclear if “required” means by policy or by the system itself.	The data entry will be clearer in the programmed web survey, but we will also make the instructions easier to understand and clarify that “required” refers to the system: 55. Which of the following SNAP participant data elements does your State agency’s SNAP eligibility MIS capture? In the first column, indicate whether your system requires the data. If it is not required, indicate in the second column whether it is collected as optional data. Select yes or no for each.
63. At what points are these data elements collected from individuals? Select all that apply.	One participant commented it may be helpful to clarify what “these data elements” refer to.	We will change the question to the following: 63. At what points are demographic data elements collected from individuals? Select all that apply.
65. Does your SNAP State office have access to the following data elements at the individual level?	Participants were asked if any of the response options were redundant, unclear, or unnecessary. Participants had the following feedback: Items “are ABAWDS” and “are subject to ABAWD time limit” are the same If you’re exempt from ABAWD requirements, you’re not an ABAWD The “became ineligible...” option overlaps with “lost eligibility because of ABAWD time limit” and “experienced E&T sanctions” The good cause item is unclear Unclear what “completed E&T” means A different term other than “fraud detection” would be good	With the exception of the second bullet, we will implement changes based on this feedback. The revised list of items follows: Are subject to general work requirements Are exempt from general work requirements Lost eligibility because of failure to comply with general work requirements Determined to have good cause from the general work requirements Are referred to E&T Are exempt from mandatory E&T Experience E&T sanctions Completed an E&T training, educational or work experience, or an on-the-job training component Are employed after E&T Are ABAWDs Are ABAWDs living in a waived area Are at-risk ABAWDs Lost eligibility because of ABAWD time limit Are exempt from ABAWD requirements Determined to have good cause from the ABAWD work requirements Are referred to fraud detection Are sent to claims collection Did not receive timely case processing Are denied benefits at eligibility [will not allow a response for E&T column]
67. Does your State analyze differences in E&T participation or outcomes based on any of the following? Select all that apply.	One participant commented that the “select all that apply” instruction is confusing.	We will remove this instruction.
73. [If 71.3 not selected] Can individual participants be identified using these data?	Some participants thought this question was clear but could benefit from concrete examples.	We will add examples to this question and do the same for Q76: Can individual participants be identified using these data? For example, is personally identifiable information included, or are any of the data so specific that someone could identify the individual tied to a record? [Note: Q76 should have been a followup question to Q74, not Q75.]
75. Does your agency have data-sharing agreements with third parties or E&T providers regarding E&T participants?	FNS commented that this question appears to be the same as Q71. After testing, participants also agreed that these questions gather the same information.	We will remove this question.
79, 81, and 82. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position? Years of formal education/ Experience working with lower income households/ Experience working in a State agency Experience as a lower income person or living in a lower income community	One participant commented that, while she understood the idea behind the questions, they are not relevant because those doing the hiring would not know if the applicant has experience as a person with low income or living in a lower income community.	We will remove these questions.
88. [If 87.1 selected] Which of the following demographic groups are specifically named in the policy? Select all that apply	One participant was not clear if this question was about hiring or promotion.	We will rephrase the question as follows: Which of the following demographic groups are specifically named in your State agency’s policy regarding promotion of historically underrepresented groups? Select all that apply.
89. Which of the following best describes what your State is currently doing to address the role that unconscious biases may play in eligibility workers’ decisions when they interview SNAP applicants and participants from racialized or underrepresented groups?	Because of client concern, participants were asked about defining unconscious bias. Most participants thought it is not necessary to define because workers will know, especially if they were trained on it. One participant commented that biases should never come into play with eligibility workers and thought this would be more appropriate for E&T workers.	We do not recommend any changes to this question.

2. SNAP State Agency Interview Protocol

To pretest the SNAP State agency interview protocol, we requested an interview with a State agency official familiar with SNAP work requirements, E&T, and the data collected to inform the operations of these programs. One staff member from New Jersey attended this interview—a senior program manager who oversees the SNAP E&T program and is knowledgeable about work requirements and data systems.

Duration

The interview lasted 1 hour and 28 minutes, which was 2 minutes less than the estimated time for this interview. However, the interviewer skipped section D (SNAP E&T Providers) because in New Jersey the State Department of Labor is responsible for E&T provider selection and oversight. This suggests the protocol requires streamlining to ensure the interview fits within the estimated time limit for SNAP State agencies that maintain responsibility over the providers. To ensure the interview can be completed within 1 hour and 30 minutes, we restructured questions in section B related to the general and ABAWD work requirements.

General findings

The interview flowed smoothly, and the respondent was able to answer most questions with ease. The respondent noted the interview covered all the important aspects of work requirements and SNAP E&T, and no additional questions were required.

Some repetition was apparent in section B of the protocol (General and ABAWD Work Requirements). The pretest version of the guide first asked a series of questions about the general work requirements and then asked a very similar set of questions about the ABAWD work requirements. Because the processes for screening, identifying exemptions, and tracking compliance were very similar across the two types of work requirements, we suggest revising this section to avoid repetition and promote better flow. We propose asking two questions upfront specific to either the general or ABAWD work requirements. Then we propose adding a third question that asks about screening, exemptions, compliance, and tracking for both general and ABAWD work requirements. These changes will enable the study team to collect all the required information about the administration of work requirements in each State within the allotted time for the interview.

In addition to streamlining section B, we recommend five additional minor edits to the protocol questions. We recommend deleting one question (B.1.c.ii) because it is repetitive with a broader question about training later in the protocol and moving another question (B.3) for better flow. We recommend slight text revisions to two questions (B.5, G.3) to improve clarity and rewording one question (F.6) to elicit a better response.

Question-by-question findings and recommendations

Table 2 provides our findings and recommendations for specific questions.

Table 2. Item-Level Recommendations for SNAP State Agency Interview Protocol

Question Number From Instrument	Findings/Observations	Recommendations
Section B. General and Able-Bodied Adults Without Dependents Work Requirements	This section is currently structured to first ask about compliance, screening, exemptions, and challenges for general work requirements. Similar subquestions then repeat these topics for ABAWD work requirements. This structure made the interview repetitive and led to section B taking the longest time to complete at 30 minutes.	To ensure the interview can be completed within 1 hour and 30 minutes, we recommend restructuring section B to streamline questions 1 and 2. We recommend revising the questions as follows: New question B.1 will ask only about the steps work registrants need to take to comply with the general work requirement and register for work. New question B.2 will ask only about why or why not the State agency applied for an ABAWD waiver. New question B.3 will turn to screening, determining exemptions, and tracking compliance for both the general and ABAWD work requirements. Rather than asking the same questions separately, we will first ask about the State agency’s process for the general work requirements and then ask if the process is the same or different for the ABAWD work requirements. If it is different, we will ask the respondent to describe further. To make these changes, we did not remove any questions. We simply moved them to improve flow and reduce burden on the respondent by eliminating the need to repeat their responses throughout the interview.
B.1.c.ii “Do eligibility workers receive training on how to determine unfitness for work for the purposes for the general work requirements?	This question duplicates (old) question B.3.	We recommend deleting this question and adding a probe to B.3 to ensure the interviewer also asks about training for determining unfitness for work.
B.3 What training do staff receive in work requirements and ABAWD policies?	This question did not flow well in its current location in the protocol.	We recommend moving this question down in section B and making it a new question 5.
B.5 How do staff assess good cause for clients who do not meet ABAWD work requirements.?	The respondent asked a clarifying question and used the verb “determine” rather than “assess.”	We recommend revising this question as follows: How do staff determine good cause for clients who do not meet ABAWD work requirements?
F.6 How many individuals work on your State agency’s data team? What is your State agency’s capacity to analyze additional data?	This question did not provide the desired answer. While New Jersey has a large data team, there is no dedicated data analyst for SNAP E&T, which makes additional analyses difficult.	We recommend slightly revising this question as follows: Does your team have a dedicated data analyst? If no, who runs and requests data reported? If yes, what is the analyst’s capacity to analyze additional data?
G.3 What steps has your State agency taken to ensure all eligible clients can access SNAP?	The respondent asked to clarify whether we were referring to SNAP as a whole or just SNAP E&T.	We recommend revising this question as follows: What steps has your State agency taken to ensure all eligible clients can access SNAP as a whole? How about SNAP E&T?

1. SNAP Local Office Interview Protocol

To pretest the SNAP local office interview protocol, we requested an interview with a SNAP local office in the same county as the E&T provider. The State agency provided four respondents who worked at a county Board of Social Services office. The respondents included the office director; an administrator who oversees SNAP, Temporary Assistance for Needy Families (TANF), and Medicaid benefits; a supervisor who oversees ongoing SNAP and TANF cases; and a member of the county’s welfare-to-work program team.

Duration

The interview lasted 47 minutes, which is 13 minutes shorter than the allotted time. No edits are needed to the length of the protocol.

General findings

Respondents had difficulty answering questions in sections B, E, and F of the protocol. When asked about the general work requirements, staff responded they do not screen for those because the State agency no longer has a mandatory E&T program. New Jersey has not had an ABAWD time limit since the onset of the COVID-19 public health emergency, so staff had some difficulties recalling their ABAWD processes. Despite these difficulties, we do not recommend removing or revising questions under section B (General and ABAWD Work Requirements). Other local offices may operate differently, and the SNAP work requirements landscape may have altered by the time data collection occurs next summer.

In section E (Data Availability and Needs), respondents stated some questions (e.g., E.2, E.3) were difficult to answer because they were beyond the scope of their duties. The local office’s main role is to determine eligibility for program benefits rather than analyze metrics or data. We recommend retaining these questions because it is important for us to know to what extent local offices have any additional data, even if the answer is they do not.

Local office staff also expressed difficulties answering some equity-related questions in section F (Equity in Work Requirements and SNAP E&T Programs), particularly F.5. We recommend revising this question to make it more concrete for respondents.

Question-by-question findings and recommendations

Table 3 provides our findings and recommendations for specific questions.

Table 3. Item-Level Recommendations for SNAP Local Office Interview Protocol

Question Number From Instrument	Findings/Observations	Recommendations
C.3.c.i To what extent is participation tracked? What is the information used for?	A respondent asked if we were referring to how participation is tracked.	The respondents were able to answer the first question after we rephrased it to ask if they track participation and, if they do, how. We recommend rephrasing this question as follows: (New C.3.c.i) Do you track participation in SNAP E&T? If yes, how do you track it? What is the information used for?
F.3. What steps has your SNAP State agency taken to ensure all eligible clients can access SNAP and SNAP E&T? What steps has your local office taken?	Respondents had difficulty answering this question.	We recommend deleting this question and using new question F.3 instead (see below).
F.4 How does your State agency define successful participant in and completion of SNAP E&T? What steps has your State agency taken to ensure all individuals who participate in SNAP E&T succeed in the program.	Respondents had difficulty answering this question.	We recommend deleting this question and using new question F.4 instead (see below).
F.5 How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable E&T outcomes?	The respondents were unsure how to respond to this question and asked if they are defining how access is equitable.	The respondents were able to answer the first question when we rephrased it to ask how they define access. When we asked the second question, they stated they do not know how FNS should define equitable outcomes. We recommend shifting the focus of these questions and revising them as follows: (New F.3) What needs to be done to ensure all eligible individuals can participate in SNAP E&T? What data can be used to measure this? (New F.4) What needs to be done to ensure all participants who engage in SNAP E&T succeed? What data can be used to measure this?

4. E&T Provider Interview Protocol

To pretest the E&T provider interview protocol, we requested an interview with an E&T provider in the same county as the SNAP local office. New Jersey provided two respondents who worked at a county American Job Center. One respondent was the director, and the other was a supervisor who oversaw the SNAP E&T and general assistance programs at this location. The director could only stay on the call for 10 minutes, but the supervisor was knowledgeable about processes and data systems and was able to answer all our questions.

Duration

The interview lasted 58 minutes, which is 2 minutes shorter than the time allotted. No edits are needed to the length of the protocol.

General findings

The respondent was able to answer all our questions and did not have any additional feedback. Two questions in section F (Equity in SNAP E&T) led to minor confusion. We recommend adding a probe to one question (F.1) and revising the wording in the other (F.3) for greater clarity.

Question-by-question findings and recommendations

Table 4 provides findings and recommendations for specific questions.

Table 4. Item-Level Recommendations for E&T Provider Interview Protocol

Question Number From Instrument	Findings/Observations	Recommendations
F.1 We already talked about some areas in the work requirement and E&T screening and referral processes that may contribute to disparities. Are there any other administrative processes or procedures that could affect a client’s ability to access E&T services? If so, please describe.	The respondent asked us to repeat the question.	The respondent was able to answer the question after we rephrased it to ask whether any roadblocks hindered the process. We have added this language as a probe so the interviewer can use it if necessary.
F.3 How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable E&T outcomes?	The respondent was immediately confused when this question was asked and requested further specificity on what we meant by “equitable outcomes.”	We recommend breaking this question into two separate questions and rephrasing as follows: (New F.3) What needs to be done to ensure all eligible individuals can participate in SNAP E&T? What data can be used to measure this? (New F.4) What needs to be done to ensure all participants who engage in SNAP E&T succeed? What data can be used to measure this?

5. Other Interested Parties Interview Protocol

To pretest the other interested parties interview protocol, we requested an interview with staff from New Jersey’s Department of Labor. The SNAP State agency works closely with the Department of Labor to administer SNAP E&T in New Jersey. The State agency provided two respondents who were familiar with SNAP E&T: a director who oversees the Labor Department’s transitional workforce services, including SNAP E&T, and an analyst familiar with the State’s SNAP E&T data systems.

Duration

The interview lasted 58 minutes, which is 2 minutes shorter than the time allotted. No edits are needed to the length of the protocol.

General findings

The respondents were able to answer all the questions we asked and did not have any feedback on the protocol questions. The order of the questions under section B (Organization and Staffing) slightly confused the respondents. The first few questions in the section ask about the respondent’s organization and other SNAP E&T partnerships, and the last few questions ask about hiring practices. Because we discussed the organization’s SNAP E&T partners prior to inquiring about hiring practices, the respondents believed we were asking about their E&T partners’ hiring practices rather than their organization’s hiring practices. We recommend separating section B into two sections to ensure greater clarity. The new section B will focus on the organization’s role in SNAP E&T and other organizations or agencies it partners with to support or deliver E&T services; the new section C will focus on internal hiring and staffing.

We also recommend adding a set of questions about the organization’s involvement in selecting SNAP E&T providers in section C (SNAP E&T Program Operations) to determine whether the organization plays a role in this process.

Question-by-question findings and recommendations

Table 5 provides our findings and recommendations for specific questions.

Table 5. Item-Level Recommendations for Other Interested Parties Interview Protocol

Question Number From Instrument	Findings/Observations	Recommendations
Section B. Organization and Staffing	This section is currently organized with questions about the organization, its role in SNAP E&T, and other entities it works with to support or deliver SNAP E&T services, followed by questions about the organization’s staffing. By switching the topic from the organization to its partners and back to the organization, the respondents misunderstood our questions about staffing and believed we were asking about their partners’ staffing rather than their organization’s staffing.	To ensure these questions flow smoothly and respondents clearly understand whose staffing we are inquiring about, we recommend moving the following questions about the organization’s staffing to its own section after section B. (Old B.4, new C.1) Does your organization have explicit hiring targets or goals for frontline staff? [If yes] Can you tell us more about these hiring practices? What prompted your organization to implement these? (Old B.4.a, new C.1.a) What other factors does your organization consider when hiring frontline staff? [Probe: experience working with clients or a similar population, lived experience, cultural humility] (Old B.5, new C.2) [Only ask if management-level staff are participating in the interview] Does your organization have explicit hiring targets or goals for leadership and/or managerial staff? If yes, can you tell us more about these hiring practices? What prompted your organization to implement them? (Old B.5.a, new C.2.a) What other factors does your organization consider when hiring leadership or managerial staff? [Probe: experience working with clients or a similar population, lived experience, cultural humility] (Old B.6, new C.3) Can you tell us more about how [organization] recruits staff?
Section C. SNAP E&T Program Operations	This section did not include questions about the organization’s involvement in selecting E&T providers.	We recommend adding the following set of questions to this section: (New D.3) Is your organization involved in selecting SNAP E&T providers? If yes: (New D.3.a) What factors does [organization] consider when selecting SNAP E&T providers? (New D.3.b) Is your organization actively recruiting additional SNAP E&T providers? If yes, why? (New D.3.c) Does your organization provide training to providers on how to best meet the needs of participants and account for each participant’s unique circumstances? If yes, please describe. (New D.3.d) What challenges does [organization] face in selecting and contracting with SNAP E&T providers? [Probe: funding, presence in underserved areas, administrative capacity, interest, finding providers that offer relevant trainings] (New D.3.d.i) What could be done to mitigate these challenges?
E.4 How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable E&T outcomes?	The respondents were confused by the term “define.”	We recommend shifting the focus of this question, breaking it into two separate questions, and rephrasing as follows: (New F.4) What needs to be done to ensure all eligible individuals can participate in SNAP E&T? What data can be used to measure this? (New F.5) What needs to be done to ensure all participants who engage in SNAP E&T succeed? What data can be used to measure this?
E.5 What aspects of equity, specifically related to the administration of work requirements and SNAP E&T, should FNS prioritize?	Respondents are unlikely to be familiar with SNAP work requirements. We recommend limiting this question to SNAP E&T.	We recommend shortening this question and revising as follows: (New F.6) What should FNS prioritize to make sure everyone has the same access to E&T and the same opportunities to benefit from the program?

Appendix A. SNAP State Agency Survey

Sent to Pretest Panel

Westat is conducting the Supplemental Nutrition Assistance Program (SNAP) Work Requirements and Employment and Training (E&T) Data study for the U.S. Department of Agriculture’s Food and Nutrition Service (FNS). This study aims to develop a framework for data that FNS can use to monitor and assess State agencies’ progress toward equity in the future.

Your responses to this survey will provide a valuable picture of the administration of work requirements and SNAP E&T across State agencies, including areas such as hiring, case management, and participant reimbursements. This survey also asks about SNAP eligibility and SNAP E&T data.

The survey takes 40 to 60 minutes to complete. It contains the following six main sections that may be assigned to other agency staff with relevant expertise:

• About Your State’s SNAP E&T Program

• Screening

• Case Management, Initial Assessment, and Participant Reimbursements

• SNAP E&T Providers and Components

• SNAP E&T Data Use and Decision Making

• Participant Outcomes

• SNAP Eligibility and E&T Data

• State Hiring Practices

Your responses will be kept private, except as otherwise required by law. We will not share the information you provide with anyone outside the study team. Although participation is strongly encouraged, there are no penalties if you choose not to participate.  

If you have questions about the survey, please contact the Westat study team at [study email TBD] or the FNS project officer, Kristen Corey, at Kristen.Corey@usda.gov.

1. About Your State’s SNAP E&T Program

These first questions provide background information on your State agency’s SNAP E&T program.

1. Is your State agency’s E&T program mandatory or voluntary?

1. Mandatory

2. Voluntary

3. Combination of mandatory and voluntary

2. At which level is your State agency’s E&T program administered?

1. State administered

2. County administered

3. [If 1.3 selected] Approximately what percentage of the counties in your State operate a mandatory E&T program?

_______%

4. [If 1.1 is selected] How much of a priority are the following factors when your State agency develops its list of geographical-based exemptions from SNAP E&T?

	High Priority	Medium Priority	Low Priority	Not Considered
Lack of in-demand occupations
Lack of high-growth occupations
Lack of access to transportation
Area (e.g., county) unemployment rates
Median household incomes
Area demographics (e.g., race, ethnicity)

Thank you for completing these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about your State’s E&T program.

Role: _______

Years in current position: _______

2. Screening

This section includes questions about the SNAP eligibility interview process for determining whether an applicant is subject to work requirements. Someone familiar with the eligibility interview process should complete this section or be consulted before completing this section.

The next set of questions asks about the SNAP eligibility interview. For this survey, the eligibility interview is the process that takes place when a participants applies for SNAP, and an eligibility interview determines whether that person is subject to the work requirement, an able-bodied adult without dependents (ABAWD) subject to time limits, or both.

5. How does your State conduct eligibility interviews? Select all that apply.

1. In person

2. By telephone

3. We allow virtual interviews

6. How does the eligibility interviewer determine whether an applicant is subject to the SNAP work requirements (including ABAWD status)?

1. We have a script that the eligibility worker is required to follow strictly in the interview

2. We have a script that the eligibility worker can use. We allow the eligibility worker some discretion in structuring the interview

3. We do not have a script. We allow the eligibility worker to determine the structure of the interview

7. [If 1.2 or 1.3 selected]: Which of the following does your State agency consider when deciding if voluntary participants should be referred to E&T? Select all that apply.

1. Education or prior work experience

2. Current employment status

3. Age

4. County of residence

5. Housing status

6. Criminal record

7. Health status

8. Citizenship status

9. Disability status

8. When do you require work registrants to provide verification to establish that they are unfit for work?

1. Only if client-provided information is questionable

2. Any time the client provides information that indicates they may be unfit for work

3. Other; specify____________

9. Are the verification requirements to establish unfitness for work different for people subject to the ABAWD time limit?

1. No

2. Yes; specify ____________

10. [if 9.2 selected] Is the eligibility worker making the unfit-for-work determination required to file documentation supporting their decision?

1. Yes

2. No

11. Which of the following steps do eligibility workers take to determine if a person has good cause for not meeting the general work requirements before applying disqualification? Select all that apply.

1. Discuss the situation with the client

2. Ask the employer or a collateral contact to verify the information

3. Consult with other State agency staff before making a determination

4. None of the above

5. Other; specify ________

12. How does your State agency inform individuals they have been referred to the SNAP E&T program? Select all that apply.

1. In person

2. Email

3. Mail

4. Phone call

5. Text message

6. Other; specify: _______________

13. How many required contact attempts are made to inform individuals they have been referred to the SNAP E&T program?

OPEN TEXT FIELD ______ [RANGE 1–20]

The following questions are about waivers, exemptions, and policies related to ABAWDs.

14. Does your State agency currently have an ABAWD waiver?

1. Yes

2. No

15. [If 14.1 selected] Where does the waiver apply?

1. State/territory-wide

2. Specific counties or Tribal areas

16. [If 14.1 selected] Which of the following criteria is the waiver based on?

1. An unemployment rate of over 10 percent

2. Insufficient jobs in the area

17. [If 14.2 selected] Is your State agency currently using ABAWD discretionary exemptions?

1. Yes

2. No

18. [If 17.1 selected] Which of the following ABAWD discretionary exemptions are currently used in your State? Select all that apply.

1. Lack of employment opportunities in a given area

2. Veteran status

3. Youth leaving foster care

4. Persons leaving incarceration

5. Individuals experiencing homelessness

6. Rural residents with no transportation

7. Individuals with addiction on waiting lists for treatment

8. Other; specify

9. None of the above

19. [If 17.1 selected] How are your State’s ABAWD discretionary exemptions distributed?

1. First-come, first-served basis

2. Categorically; we reserve a specific number of exemptions for specific, need-based categories

3. Geographically (e.g., county); we reserve a specific number of exemptions for specific areas

4. Other; specify ______________

3. Case Management, Initial Assessment, and Participant Reimbursement

1. Case Management and Initial Assessment

These next questions ask about case management for SNAP E&T participants. For this survey, case management includes services and supports provided directly to SNAP E&T participants by a case manager or other direct-service staff person after participants are referred to E&T. Someone with case management experience should complete this section or be consulted before completing this section.

20. Which entities in your State provide E&T case management? Select all that apply.

1. Local SNAP office

2. Community college

3. Workforce Innovation and Opportunity Act (WIOA) agency or other workforce programs funded by Department of Labor (DOL)

4. Community-based organization

5. Adult basic education provider

6. Other; specify: _________

21. How are E&T participants assigned to case managers?

1. Automatically, within an E&T data management system

2. Assignments are made based on case managers’ caseload/availability

3. Assignments are made based on participant location

4. Assignments are made based on skills and barrier assessment

5. Other; specify: ________

[Programmer note: Do not allow 2–4 if 21.1 = Selected]

22. Which of the following best describes your State’s approach to case management? Select all that apply.

1. Case managers are only required to perform initial assessments

2. Case managers perform initial assessment, develop individual plans, coordinate referrals, and determine participant reimbursements

3. Case managers provide ongoing support, such as coaching, motivational interviewing, or trauma-informed case management

1. Are career navigators available to participants through either the State or E&T provider agencies?

[hover text on career navigators: These are sometimes called advisors or coaches and may or may not be the same individuals as case managers]

1. Yes, all participants are assigned a career navigator

2. Yes, participants are assigned a career navigator as needed

3. Case managers provide career navigation

4. No, career navigators are not available

2. What kinds of needs or barriers assessment tools are identified in policy and/or guidance for your State? Select all that apply.

1. Needs or barriers assessment tool developed by or for the State, territory, or provider

2. Standardized assessment tool, completed by participant on their own

3. Standardized assessment tool, administered by State or provider staff

4. Informal assessment tools, completed by participant on their own

5. Informal assessment tools, administered by State or provider staff

6. No policy or guidance for needs or barriers assessment

7. Other; specify: _______________

2. Participant Reimbursements

These next questions ask about participant reimbursements (sometimes referred to as support services) for SNAP E&T participants.

3. How are SNAP E&T participants informed of participant reimbursements or support services available to them? Select all that apply

1. This information is provided at the orientation meeting

2. This information is emailed to the participant

3. This information is provided verbally by their case manager or E&T component provider

4. This information is provided in writing by their case manager or E&T component provider

4. How does your State track reimbursements or support services at the participant level? Select all that apply.

1. Tracked in SNAP eligibility management information system

2. Tracked in State agency’s E&T data system

3. Our State does not track this, but each E&T provider tracks its own reimbursements

4. Not tracked at the participant level

5. Other; specify: ____________

5. What type of reimbursement cap does your State have? Select all that apply.

1. Individual expense type (e.g., auto repairs, supplies, legal services)

2. Monthly combined cap

3. Annual combined cap

4. Lifetime combined cap

5. Does not apply—no reimbursement caps

6. Other; specify: ___________

6. SNAP E&T guidelines state that participant reimbursements must be covered for support services that are “reasonable and necessary.” Does your State reimbursement policy define what “reasonable and necessary” means for any of the following? Select all that apply.

1. Transportation assistance

2. Uniforms, work, or interview clothing

3. Tools or other work equipment

4. Fees associated with testing, licensing, or credentialing

5. Tuition

6. Books or other supplies for classes

7. Technology (e.g., computers, hotspots)

8. Childcare assistance

9. Temporary housing (up to 2 months)

10. Other; specify: ____________

11. None of the above

7. Does your State have a formal process for SNAP E&T participants to appeal reimbursement decisions that do not fully meet their needs?

1. Yes, there is a formal review process

2. No, but participants can request a review of their reimbursement based on demonstrated need

3. No, reimbursement decisions are not subject to review

8. [If 29.1 or 29.2 selected] How are participants notified that a review of their reimbursement is available to them? Select all that apply.

1. This information is included in the SNAP E&T orientation materials

2. This information is available to participants through the State’s website

3. This information is provided by case managers

9. Does your State agency offer assistance to participants who need to submit documentation for participant reimbursements? Select all that apply.

1. Yes, one-on-one assistance in person

2. Yes, group assistance in person

3. Yes, virtually through video call

4. Yes, through email or text

5. No, formal assistance is not provided

10. Does your State agency’s reimbursement policy for transportation account for participant location?

1. Yes, for all participants

2. Yes, in certain areas

3. No, this is not part of our reimbursement policy

11. How do SNAP E&T participants receive their reimbursements or support services? Select all that apply.

1. Cash

2. Vouchers

3. Check or direct deposit

4. In-kind assistance (e.g., directly provide uniforms or supplies)

5. Other; specify: __________

12. [If 33.3 selected] How do you accommodate persons without a bank account?

1. We can provide alternative forms of reimbursement, such as gift cards or other cash equivalents, that do not require a bank account

2. We cannot provide alternative forms of reimbursement

Thank you for answering these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about case management, initial assessments, and reimbursements.

Role: ______________________

Years in current position: _________

4. SNAP E&T Providers and Components

This section is about SNAP E&T providers and the E&T components available in your State. Someone familiar with the E&T providers in your State should complete this section or be consulted before completing this section.

13. Which of the following provide E&T services in your State? Select all that apply.

1. Local SNAP office

2. Community college or university

3. WIOA agency or other DOL-funded workforce programs

4. Community-based organization

5. Adult basic education provider

6. Other; specify: _________

14. Which SNAP E&T components does your State agency offer? Select all that apply.

1. Supervised job search

2. Job search training

3. Job retention

4. Education programs (e.g., English language learners, GED, vocational education/training)

5. Self-employment training

6. Work-based learning

7. Work experience (e.g., apprenticeship, preapprenticeship)

8. Workfare

9. Other; specify: ____________

15. How much of a priority were the following factors when your State agency chose its current E&T providers and partners?

    Factor	Required	High Priority but Not Required	Medium Priority	Low Priority	Not Considered
    Includes training for in-demand or high-growth occupations
    Located in areas with in-demand or high-growth occupations
    Located in neighborhoods with historically underserved groups [hover text on historically underserved groups: people who have historically experienced discrimination or encountered barriers (e.g., racial, ethnic, sex and gender identity and expression, sexual orientation, economic, cultural, and/or linguistic) to services]
    Convenience to State agency
    Located near public transportation or offers transportation services
    Provides services based on participant feedback
    Provides services at a competitive cost
    Has a record of engaging historically underserved groups [hover text on historically underserved groups: people who have historically experienced discrimination or encounter barriers (e.g., racial, ethnic, sex and gender identity and expression, sexual orientation, economic, cultural, and/or linguistic) to services]
    Successfully worked together in the past
    Engages in culturally responsive hiring and training practices
    Employs staff with similar lived experience as participants
    Uses evidence-based practices when delivering services

16. Does your State agency use any of the following data to tailor the types and locations of E&T services offered to meet the needs of participants? Select all that apply.

    	Types of Service	Location of Service
    U.S. DOL data
    Census Bureau data
    State DOL data
    Data from other State agencies (e.g., Department of Human Services)
    Commercial data on careers growth or local labor market indicators
    Participant surveys
    Other participant feedback
    Public transit or other transportation resource data
    Other	(specify)	(specify)

17. How does your State determine which components will be assigned to each SNAP E&T participant?

1. Based on skills or barriers assessment

2. Based on case manager assessment of participant need

3. Local/regional availability

4. Other; specify ______________

Thank you for answering these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about E&T providers and E&T components

Role: ______________________

Years in current role: _________

5. SNAP E&T Data Use and Decision Making

These questions ask about how your State agency uses SNAP E&T data to make decisions about the program. Someone involved in program planning or making decisions that affect the E&T program should complete this section or be consulted before completing this section.

18. Does your State agency examine differences by race and ethnicity in rates of the following types of exemptions (e.g., compare the proportions of total SNAP participants identified as Asian versus Black/African American who are exempted from the general work requirements during the same time)? Select all that apply.

1. Exemptions from the general work requirements

2. State exemptions from mandatory E&T

3. Exemptions from the ABAWD work requirement

4. Discretionary exemptions

5. None of the above

19. Does your State agency examine differences by race and ethnicity in component assignments?

1. Yes

2. No

3. These data are not available

20. Does your State agency examine differences by race and ethnicity in use of sanctions?

1. Yes

2. No

3. These data are not available

21. Does your State agency examine differences by gender in rate of the following types of exemptions? Select all that apply.

1. Exemptions from the general work requirements

2. State exemptions from mandatory E&T

3. Exemptions from the ABAWD work requirement

4. Discretionary exemptions

5. None of the above

22. Does your State agency examine differences by gender in component assignments?

1. Yes

2. No

3. These data are not available

23. Does your State agency examine differences by gender in use of sanctions?

1. Yes

2. No

3. These data are not available

24. [If any data examined/analyzed] Has your State agency developed policy or guidance based on any of analyses mentioned in this section?

1. Yes

2. No

25. [If 46.1 selected] What policies, guidance, or practices has your State agency developed or revised based on these analyses?

_______________________________

26. [If any data examined/analyzed] How [else] has your State agency used the findings from the analyses?

_______________________________

Thank you for completing these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about SNAP E&T data and decision making.

Role: ______________________

Years in current position: _________

6. Participant Outcomes

These next questions ask about participant outcomes and participant feedback. Someone who works with your State’s SNAP eligibility data system (e.g., data manager, data analyst) should complete this section or be consulted before completing this section.

27. For each SNAP E&T component your State agency offers, are the following participant outcomes tracked, either by your State or by providers?

    Outcome	Tracked in Aggregate (by Provider or County)	Tracked at the Individual Participant Level	Not Tracked
    Overall completion
    Graduation or credential (e.g., certification) completion
    Skill gains
    Job placement
    Retention and/or attrition
    Participant salary or wages
    Reliance on SNAP after E&T completion

28. Does your State agency have access to the necessary data (i.e., demographic and outcome data stored on the individual level) to track outcomes by participant characteristics (for example, job placement by gender)?

1. Yes

2. No

29. Where do data on participant outcomes come from? Select all that apply.

    Outcome [do not show those “Not Tracked” in 49]	Collected by SNAP E&T Providers	Collected by the SNAP Agency	Collected by Other State Agencies
    Overall completion
    Graduation or credential (e.g., certification) completion
    Skill gains
    Job placement
    Retention and/or attrition
    Participant salary or wages
    Reliance on SNAP after E&T completion

30. [Repeat for each outcome tracked in Q49] How frequently does your State agency receive updated data on [fill outcome from Q49]?

1. Weekly

2. Monthly

3. Quarterly

4. Annually

5. Other; specify: _________

31. How does your State agency receive participant feedback on access to the SNAP E&T program? This feedback could come directly through the agency or be gathered by providers and received by the agency.

[hover text on access: ease of ability to participate in SNAP E&T (e.g., location, technology, language)]

1. Feedback is requested from all participants

2. Participants can provide feedback, but it is not requested

3. We have no way to receive feedback from participants

32. Does your State agency receive feedback on participant satisfaction (e.g., quality, fit) with SNAP E&T services and service providers? This feedback could come directly through the agency or be gathered by providers and received by the agency.

[hover text on quality or fit of SNAP E&T: Perceived benefit, feels like SNAP E&T experience was helpful, E&T experience matched existing or desired skills]

1. Feedback is requested from all participants

2. Participants can provide feedback, but it is not requested

3. We have no way to receive feedback from participants

Thank you for completing these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about participant outcomes.

Role: ______________________

Years in current position: _________

7. SNAP Eligibility and E&T Data

The questions in this section ask about your State’s SNAP eligibility database (sometimes called the management information system, or MIS), your State’s E&T program data, and data sharing. Someone who works with your State’s SNAP eligibility data (e.g., data manager, data analyst) should complete this section or be consulted before completing this section.

33. Which of the following best describes the association between your State’s SNAP eligibility database and SNAP E&T databases that capture information on participants and activities?

1. One integrated system or database

2. Multiple systems or databases

34. [If 55.2 selected] How are SNAP E&T data stored? Select all that apply.

1. Stored at the State agency level outside the MIS

2. Stored and maintained by county offices

3. Stored and maintained at the E&T provider level

4. Other; specify_____________

35. [if 55.2 selected] Which of the following best describes how users access and maintain SNAP eligibility data and SNAP E&T data?

1. State agency maintains an eligibility MIS system that SNAP E&T providers can view, but they cannot enter E&T data

2. State agency maintains an eligibility MIS system that SNAP E&T providers can view and enter E&T data

3. State agency maintains an eligibility MIS system that is not available to SNAP E&T providers

4. Other; specify: _________________

36. Which of the following SNAP participant data elements in the first column does your State’s SNAP eligibility MIS capture? In the second column, indicate whether the data element is required. If it is not required, indicate in the third column whether the data element is collected.

    Data Element	Required [Yes/No]	Collected (but Not Required) [Yes/No]
    Race
    Ethnicity
    Age
    Gender
    Education
    Criminal record
    Citizenship status
    Housing status
    Preferred language
    Disability status

37. [If 58.1 = Yes] Do the data allow for a participant to be associated with multiple races?

1. Yes

2. No

38. [If 58.2 = Yes] Is ethnicity a separate data element from race?

1. Yes

2. No

39. [If 50.3 = Yes] How is participant age stored in the data system?

1. Date of birth

2. Individual age

3. Age range

40. [If 50.4 = Yes] Which of the following are options for gender in the data? Select all that apply.

1. Male

2. Female

3. Transgender

4. Nonbinary

5. Other; specify: ____________

41. [If 50.6 = Yes] Are any of the following included in the criminal record data? Select all that apply.

1. Specific crime committed (i.e., theft, fraud, assault)

2. Type of conviction (i.e., misdemeanor versus felony)

3. Date of most recent conviction

4. Other; specify: ____________

42. [If 50.9 = Yes] How are the data on preferred language stored in the data system?

1. Binary variable indicating English versus non-English

2. Participant’s specific preferred language (i.e., includes specific languages other than English or Spanish)

3. Other; specify: ___________

43. How are the following data elements collected? Select all that apply.

    Data Element [Only display elements endorsed in 58]	Directly From Participant, Verbally	Directly From Participant, Written	By Observation	From Another Data Source
    Race
    Ethnicity
    Age
    Gender
    Education
    Criminal record
    Citizenship status
    Housing status
    Preferred language
    Disability status

44. At what points are these data elements collected from individuals? Select all that apply.

1. Screening

2. Initial assessment

3. During first meeting with E&T case manager

4. During subsequent meetings with E&T case manager

5. During E&T orientation

6. Periodically (e.g., monthly, quarterly)

7. Other; specify: _______

45. Does your State have data on which E&T components individuals are assigned?

1. Yes

2. No, but we could obtain the data

3. No, and we could not obtain the data

46. Does your SNAP State office have access to the following data elements at the individual level?

    Data Element	Yes, in the State MIS	Yes, in SNAP E&T Data Systems	We Do Not Have Access to the Data Element at the Individual Level
    Are subject to general work requirements
    Are exempt from general work requirements
    Are referred to E&T
    [DISPLAY IF 1.1 OR 1.3 SELECTED] Are exempt from mandatory E&T
    Experience E&T sanctions
    Completed E&T
    Are employed after E&T
    Are ABAWDs
    Are at-risk ABAWDs
    Are subject to ABAWD time limit
    Lost eligibility because of ABAWD time limit
    Are exempt from ABAWD requirements
    Became ineligible for failure to comply with general work requirements, mandatory E&T, and/or ABAWD time limits
    Are denied benefits
    Are reviewed in fraud detection
    Are sent to claims collection
    Did not receive timely case processing
    Have a good cause status pertaining to the general work requirements and/or ABAWD work requirements

47. Does your SNAP State office track the reasons for exemption from—

	Yes, in the State MIS	Yes, in SNAP E&T Data Systems	We Do Not Track This Information
General work requirements
ABAWD requirements
[if mandatory E&T] Mandatory E&T

These next questions ask about SNAP eligibility data analysis.

48. Does your State analyze differences in E&T participation or outcomes based on any of the following? Select all that apply.

    Data Element [only display those selected in 58]	Yes, Participation	Yes, Outcomes	No, Neither
    Race
    Ethnicity
    Age
    Gender
    Education
    Criminal record
    Citizenship status
    Housing status
    Preferred language
    Disability status

49. Does your State agency analyze differences by race or ethnicity in any of the following? Select all that apply.

1. Referral to E&T

2. Referral by E&T component

3. Timely case processing

4. Denial of benefits

5. Work requirements

6. Fraud detection

7. Claims collection

8. None of the above

50. Does your State agency experience any of the following barriers to analyzing SNAP eligibility data? Select all that apply.

1. We do not have staff trained to conduct these analyses

2. Staff who are trained do not have enough time

3. Not enough funding

4. Not a priority area for analysis

5. None of the above

6. Other; specify: ____________

These next questions ask about data sharing and other sources of data on SNAP E&T participants.

51. Do any of the following sources maintain or own data on SNAP E&T participants outside your State agency?

1. SNAP E&T providers

2. Other partner organizations

3. No

4. Other; specify: ______________

52. [If 73.3 not selected] Are these data available to your State agency?

1. Yes, through a formal data use agreement or a memorandum of understanding

2. Yes, through informal agreements

3. No

53. [If 73.3 not selected, display data elements not selected in 58] What data do other entities maintain that are not included in the State eligibility MIS or State E&T data system? Select all that apply.

1. Race

2. Ethnicity

3. Age

4. Gender

5. Education

6. Criminal record

7. Citizenship status

8. Housing status

9. Preferred language

10. Disability status

11. Address

54. [If 74.3 not selected] Can individual participants be identified using these data?

1. Yes

2. No

55. Does your agency have data-sharing agreements with other State agencies (e.g., State DOL, other Health and Human Services agencies) regarding E&T participants?

1. Yes

2. No

56. Does your agency have data-sharing agreements with third parties or E&T providers regarding E&T participants?

1. Yes

2. No

57. [If 78.1 selected] Can individual participants be identified using these data?

1. Yes

2. No

Thank you for completing these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about SNAP eligibility data and data sharing.

Role: ______________________

Years in current position: _________

8. State Hiring Practices

This section asks about hiring practices for the SNAP eligibility and SNAP E&T workforce in your State agency. In your responses, please consider all staff members responsible for directly reviewing SNAP participant cases and making referrals to E&T components and providers. Someone who is directly involved in making hiring decisions should complete this section or be consulted prior to completing this section.

For the next set of questions, assume that two very similar people are applying for a position in your State as a SNAP eligibility worker (i.e., merit staff).

58. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Years of formal education

2. Experience working with lower income households

59. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Years of formal education

2. Prior experience working in State agencies

60. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Years of formal education

2. Experience as a lower income person or living in a lower income community

61. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Experience working with lower income households

2. Prior experience working in State agencies

62. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Experience working with lower income households

2. Experience as a lower income person or living in a lower income community

63. Which of the following would be given more importance in making a hiring decision for a SNAP eligibility worker (i.e., merit staff) position?

1. Experience working in a State agency

2. Experience as a lower income person or living in a lower income community

64. Does your State agency use criminal background checks to make hiring decisions for the SNAP eligibility and SNAP E&T workforce?

1. Yes, as a blanket policy across all State agencies

2. Yes, as a blanket policy within the SNAP State office

3. Yes, on a case-by-case basis

4. We do not conduct criminal background checks

65. Does your State agency have a policy that addresses workforce diversity among the SNAP eligibility and SNAP E&T staff that affects hiring practices?

[hover text on policy: formalized requirements or standards for operation]

1. Yes, and specific groups are named in the policy

2. Yes, and specific groups are not named in the policy

3. No policy

66. [If 87.1 selected] Which of the following groups are specifically named in the policy? Select all that apply.

1. Racial groups (e.g., Black or African American, Asian or Pacific Islander, Native American or Alaska Native)

2. Ethnicity (i.e., Latino/a/x, Hispanic)

3. Persons with disabilities

4. Women

5. LGBTQIA+ persons

6. Former SNAP recipients

7. Other; specify: _____________

67. How does your State choose who is on the hiring panel? Select all that apply.

1. There is policy or guidance dictating diversity in demographics of the hiring panel

2. There is policy or guidance dictating diversity in the type of staff on the hiring panel

3. There is other policy or guidance dictating who is on the hiring panel

4. None of the above

5. Does not apply; we do not use a hiring panel

68. Does your State agency have a policy regarding the promotion of historically underrepresented groups into leadership roles (e.g., supervisors, directors, managers) among the SNAP eligibility and SNAP E&T staff?

[hover text on policy: formalized requirements or standards for operation]

1. Yes, and specific groups are named in the policy

2. Yes, and specific groups are not named in the policy

3. No policy

69. [If 90.1 selected] Which of the following demographic groups are specifically named in the policy? Select all that apply.

1. Racial groups (e.g., Black or African American, Asian or Pacific Islander, Native American or Alaska Native)

2. Ethnicity (i.e., Latinx, Hispanic)

3. Persons with disabilities

4. Women

5. LGBTQIA+ persons

6. Former SNAP recipients

7. Other; specify: _____________

70. Which of the following best describes what your State is currently doing to address the role that unconscious biases may play in eligibility workers’ decisions when they interview persons from racialized or underrepresented groups?

1. We have mandatory training that addresses unconscious bias for eligibility workers who interact with SNAP applicants and beneficiaries

2. We offer optional training that addresses unconscious bias for eligibility workers who interact with SNAP applicants and beneficiaries

3. We are planning or have proposed training that addresses unconscious bias but have not begun implementing the training program

4. We have no training that addresses unconscious bias and no plans to implement training

Thank you for completing these questions. Below, please provide your role and the number of years you have been in your current position. If more than one person contributed, please list the person most knowledgeable about State agency hiring and service providers.

Role: ______________________

Years in current position: _________

Appendix B. SNAP State Agency Interview Protocol

My name is [name], and I’m a researcher at Westat. Westat is conducting a study for the U.S. Department of Agriculture’s Food and Nutrition Service, or FNS. The study will examine the data FNS needs to measure equitable program access and outcomes in State agencies’ administration of work requirements and Employment and Training, or E&T, for the Supplemental Nutrition Assistance Program, or SNAP.

My colleagues and I are currently conducting interviews with SNAP State and local offices, E&T providers, and other interested parties here and in five other States. Through these interviews, we hope to collect information from a wide range of individuals involved in administering work requirements and SNAP E&T. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these processes will be very helpful to the study.

Your participation in this interview is voluntary, and your responses will be kept private, except as otherwise required by law. We will not share the information you provide with anyone outside the study team. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalties if you choose not to participate in a part or the entirety of this interview.

We will take notes over the course of the interview and would like to audiorecord the conversation to help us remember the information we collect. We will summarize the information you share with us today and combine it with information gathered from other people we interview. Nothing you say will ever be linked to your name. Direct quotes included in our reports will be presented without the speaker’s name to protect their identity.

I anticipate our conversation will take up to 1 and a half hours. Do you have any questions for me about the project in general or what we will discuss today?

Do I have your permission to record the conversation? You may stop the recording at any time.

[Confirm permission before recording starts. Do not record without respondent permission.]

[Turn on recorder] For the purpose of the recording, are you willing to participate in this interview? And are you willing to have the interview audiorecorded?

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-XXX. The time required to complete this information collection is estimated to average 1.5 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the following address: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, Alexandria, VA 22314, ATTN: PRA (0584-XXXX). Do not return the completed form to this address.

Privacy Act Statement

This information is being collected under the authority of the Food and Nutrition Act of 2008, as amended through Pub. L. 113–128, enacted July 22, 2014 [7 U.S.C. 2026], which provides the legislative authority for the U.S. Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) to administer the Supplemental Nutrition Assistance Program (SNAP). Section 17 of the Food and Nutrition Act of 2008 provides the authority to FNS to conduct research to help improve the administration and effectiveness of SNAP. Disclosure of the information is voluntary. The information collected from States and interview participants will enable FNS to understand what data are needed to ensure equitable program access and outcomes in SNAP and SNAP Employment and Training. Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifiable information collected will be kept confidential to the extent of the law. The information may be shared with SNAP contract researchers and USDA SNAP research and administrative staff, with identifying information removed.

1. Background

I’d like to start by learning about your role and responsibilities at the SNAP agency in [State].

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. General and Able-Bodied Adults Without Dependents Work Requirements

Now, let’s talk about work requirements in [State].

1. First, I want to discuss the general work requirements. SNAP regulations require that all nonexempt SNAP participants must (1) register for work, (2) accept a suitable job if offered, (3) fulfill mandatory SNAP E&T requirements or workfare if assigned, and (4) give the State agency information regarding their employment or availability for work,. SNAP participants also cannot voluntarily quit a job or reduce their hours below 30 hours a week without good cause.

1. What steps must work registrants take to comply with the general work requirements? How do they register for work? [Probe: automatic process; client must complete a form]

i. How did your State agency decide on this process? Who was involved in the decision making?

ii. Have State agency policies related to the general work requirements changed recently? How?

2. How do eligibility workers screen clients to determine if they are subject to the general work requirements or if they meet an exemption? Does the State agency eligibility system include preset questions? Do eligibility workers use any other tools? [Note: Request copies of any additional tools]

i How does your State agency track exemptions? Are some exemptions more commonly used than others? If yes, which ones?

3. How does your State agency define “unfit for work” for the purposes of the general work requirements?

i. What documentation must clients provide to demonstrate they are unfit for work?

ii Do eligibility workers receive training on how to determine unfitness for work for the purposes of the general work requirements? If yes, please describe.

4. How do staff assess good cause for clients who do not meet the general work requirements?

i. How, if at all, is this information tracked?

ii. What challenges are associated with determining whether an individual has good cause? Do you think some groups are affected differently by these policies? If yes, how? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on Indian Tribal Organizations (ITOs)]

5. What challenges does your State agency face in determining whether an individual is subject to the general work requirements?

i. Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

2. Next, I would like to discuss the work requirements and time limit for able-bodied adults without dependents, or ABAWDs. In addition to the general work requirements, ABAWDs are subject to a time limit unless they meet additional work requirements.

1. According to your survey response, [State] currently [does/does not] have a [statewide/partial] ABAWD waiver. Is that correct? [Note: See survey questions B.1 and B.2]

i. How did your State agency decide to [apply/not to apply] for the waiver? [Probe: State agency policies, consultation with advocacy groups, did not qualify for waiver]

ii. [If partial waiver] How did your State agency decide which geographic areas to include in the waiver request?

2. How do eligibility workers screen clients to determine if they are subject to the ABAWD time limit or if they meet an exemption? Does the eligibility system include preset questions? Do eligibility workers use any other tools? [Note: Request copies of any additional tools]

i. Are some exemptions used more frequently than others? How does your State agency track exemptions from the ABAWD work requirements?

ii. How does your State agency define “unfitness for work” for ABAWDs? How is this exemption operationalized?

1. Do eligibility workers receive training on how to determine unfitness for work for ABAWDs? If yes, please describe.

3. What training do staff receive in work requirements and ABAWD policies? [Probe: initially and ongoing]

1. To what extent do eligibility staff meet with their supervisor to discuss difficult cases? Please describe.

[Note: If a State agency has a waiver, you may need to ask these questions in the past tense. If the State agency has had a waiver for many years, State agency staff may have difficulty answering these questions. The Families First Coronavirus Response Act also authorized FNS to issue a temporary suspension of ABAWD time limits; this suspension expired at the end of the public health emergency in May 2023. Tailor the questions as needed prior to the interview.]

1. How does the State agency track whether clients are complying with the ABAWD work requirements? How does the State agency track ABAWD countable months?

1. How do staff assess good cause for clients who do not meet the ABAWD work requirements?

1. How, if at all, is this information tracked?

2. What challenges are associated with determining whether an individual has good cause? Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

1. What challenges are associated with determining whether an individual is subject to the ABAWD work requirements?

1. Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

2. How are clients notified they are subject to work requirements, including, as applicable, general work requirements, mandatory SNAP E&T, and ABAWD time limits? [Probe: verbally during eligibility interview, mailed notice]

1. Has your State agency ever solicited feedback from clients about the readability of the notices? If yes, please describe.

2. What challenges, if any, are there in communicating work requirements to clients? How can these be mitigated? [Probe: alternative delivery mechanisms, such as e-notices]

[Note: See State responses to survey questions B.4 and B.5. Tailor the question accordingly. If the State has a statewide ABAWD waiver, skip to question 9.]

3. According to the survey, your State agency [has/has not] used ABAWD discretionary exemptions since the end of the public health emergency in May 2023. Why or why not?

4. [If the State has used the discretionary exemptions] According to your survey responses, your State agency uses the following criteria to determine which SNAP participants receive discretionary exemptions: [list]. How did [State] decide on these criteria? Does the State agency consider other factors when deciding who receives ABAWD discretionary exemptions? If so, please describe.

5. We know [State] currently has a statewide ABAWD waiver. However, does your State agency have a policy for using discretionary ABAWD exemptions in the absence of a waiver? If yes, please describe the policy.

3. SNAP E&T Program

Next, I want to turn to your SNAP State agency’s E&T program.

1. According to the survey, [State] operates a [mandatory/voluntary/mandatory for some; see question A.1] SNAP E&T program. How did your State agency decide to operate this type of SNAP E&T program? Has the type of SNAP E&T program you operate changed recently? If yes, why?

1. [If voluntary] Thinking broadly, how does your State agency provide outreach and work to ensure access to SNAP E&T services to groups or communities that want to participate? What challenges do you encounter? Does your State agency have challenges providing SNAP E&T to any groups or communities? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

1. What outreach materials do you use?

2. Has your State agency ever solicited feedback from clients about the outreach materials? If yes, please describe.

3. [If mandatory] Does your State agency have enough SNAP E&T slots for all mandatory participants? [If not] How does your State agency determine who must participate in SNAP E&T and who receives a good cause exemption? Is this information tracked in your eligibility or SNAP E&T system?

4. According to your SNAP E&T State Plan, [State] offers the following SNAP E&T components: [list]. How did your State agency decide to offer these components? How did your State agency decide where to offer these components?

1. [If some components are not offered statewide] What factors determine where a SNAP E&T component is offered?

2. [If ITOs are located in the State] How does your State agency consult with ITOs about components and where to offer them? What changes, if any, were made based on ITOs’ feedback?

3. Are there any components your State agency would like to offer but cannot? Why?

5. [If mandatory] How does your State agency screen work registrants to determine whether they are required to participate in mandatory SNAP E&T or if they meet the State agency’s criteria for exemption?

1. Does your State agency screen all work registrants for exemptions to mandatory SNAP E&T? If not, how did you decide which clients to screen?

2. How, if at all, does the SNAP eligibility system capture data related to screening and referral of mandatory SNAP E&T participants?

3. What exemptions does your State agency allow from mandatory SNAP E&T?

i. How did your State agency decide on this list of exemptions? Are some exemptions used more frequently than others? How does your State agency track exemptions from mandatory SNAP E&T?

6. Can you please describe how your State agency screens clients who volunteer for SNAP E&T to determine if they meet the State agency’s criteria for referral? Does the eligibility system contain preset questions? Do staff use any other tools? [Note: Request copies of any additional tools]

1. Does screening happen at initial application only? Why or why not?

1. How, if at all, does the SNAP eligibility system capture data related to screening and referral of individuals who volunteer for SNAP E&T?

7. How are clients referred to SNAP E&T?

1. Are some groups of people more likely not to receive a referral? Why?

2. Is the referral automatically sent to the SNAP E&T provider, or does the client need to follow up?

3. Are some groups of clients more likely to participate in SNAP E&T than others? Which groups? Why? How did you recognize there was a difference between groups? [Probe: data analysis, anecdotal information from providers] [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

8. Can you please describe how your State agency provides case management to SNAP E&T participants? [Probe: who provides the service, how often case managers meet with clients, how case manager assignments are made, goals of case management]

1. How effective do you think case management is in guiding clients to appropriate components and helping them achieve their goals? How could case management be more helpful? Do any groups require more individualized services?

2. Does your State agency track any data related to case management? If yes, please describe.

9. According to your FY 2024 SNAP E&T State Plan, your State agency allows the following reimbursable participant expenses: [list]. Can you please describe how your State agency decided on this list? [Probe: consultation with providers, participants, other individuals/entities]

1. How are clients informed about reimbursements? [Probe: at the time of SNAP E&T referral, during screening, by the provider, via notice, during the eligibility interview]

i. How are clients reimbursed for those costs? Are costs ever paid on clients’ behalf?

2. [If State has reimbursement caps; see State Plan] How did your State agency decide to place a cap on [list type of reimbursement]?

i. What happens if a client’s needs exceed the cap?

3. Based on previous research, we know transportation and childcare are two of the biggest barriers to SNAP E&T participation. What efforts has your State agency taken to address these barriers through your reimbursement policies? [Probe for agreements with childcare agency, looking into rideshare options in areas with limited public transit, priority for childcare for E&T participants]

4. To what extent does your State agency track reimbursements at the case level?

10. Thinking broadly, are there ways that SNAP E&T policies may contribute to differences in access and outcomes among SNAP clients? If yes, which groups, if any, are more affected by these policies? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

4. SNAP E&T Providers

Now, let’s discuss the SNAP E&T providers [State] partners with.

1. According to your SNAP E&T State Plan, SNAP E&T providers in [State] include [list some examples]. How did these providers become part of the SNAP E&T program?

1. What factors does your State agency consider when selecting SNAP E&T providers? [Probe: factors such as cultural humility; whether providers and partners are staffed with people with similar lived experience or are located in neighborhoods where marginalized communities live; whether providers offer activities and components that can lead to well-paying jobs]

i. Do these factors vary by geographic area or urbanicity? If so, please describe.

2. How does a new provider join the network?

3. Is your State agency actively recruiting additional SNAP E&T providers? If yes, why?

1. How does your State agency ensure SNAP E&T providers are not perpetuating wage gaps or other hiring disparities? [Probe: alignment with State workforce development board priorities, pathways programs]

2. Does your State agency provide any equity trainings to SNAP E&T providers? If yes, please describe.

3. What challenges does your State agency face selecting and contracting with SNAP E&T providers? [Probe: funding, presence in underserved areas, administrative capacity, interest, finding providers that offer relevant trainings]

1. What could be done to mitigate these challenges?

5. Other than the SNAP E&T providers, does your State agency partner with any other organizations or entities, such as employers, to operate the SNAP E&T program? If yes, how did the partnership begin? What is the role of this organization?

5. State Hiring Practices and Staff Engagement

Next, let’s shift focus to your State agency’s internal hiring practices.

1. According to the survey responses, your State agency has the following hiring policies or guidance for frontline staff: [List from survey questions C.1–C.5]. Can you tell us more about these policies? What prompted your agency to implement these?

2. According to the survey responses, your State agency has the following hiring policies or guidance for SNAP leadership staff: [List from survey questions C.6 and C.7]. Can you tell us more about these policies? What prompted your agency to implement these?

3. What other factors does your State agency consider when hiring State and local staff? [Probe: experience working with clients or a similar population, lived experience, cultural humility]

4. Can you tell us more about how [State] recruits State agency and frontline staff? Do recruitment procedures vary among the two types of staff?

6. Available Data and Data Needs

I’d like to shift to discuss the data your State agency collects and reports.

1. Can you first tell me about your State’s capacity to analyze the data you collect on SNAP work requirements and SNAP E&T?

2. Can you please describe your SNAP State agency’s E&T system? What information is captured within the system? Who has access to it?

1. [If answer to survey question 49 is two systems] According to the survey, your State has a separate eligibility and SNAP E&T system. How do these systems interact?

[If answer to survey question 68 is “Yes, through a data use agreement or a memorandum of understanding”] According to the survey, your State has data use agreements with the following agencies: [List from survey question 72]. How do you use these data? [Probe: unemployment insurance wage data to track SNAP E&T client outcomes]

1. Currently, FNS asks State agencies to report data for the FNS-583 and the National Outcome Reporting Measures. What challenges, if any, do you encounter collecting and reporting these data? How could these challenges be mitigated?

   1. Does your State agency use these data outside of reports to FNS? If yes, how?

2. Does your State agency collect data related to work requirements and SNAP E&T that are not reported to FNS? If so, please describe the data and how the data are used.

   1. How, if at all, does your State agency use these data internally? What are your key performance indicators?

i. Have these indicators evolved? If yes, please describe.

2. What data, if any, does your State agency use to assess equitable access to SNAP and SNAP E&T and equitable outcomes?

i. How has your State agency used the findings from these data?

5. When your State agency was designing the SNAP E&T system or deciding on the key performance indicators, who was part of the decision-making process? [Probe: staff at all levels, local staff, data and nondata staff]

6. How many individuals work on your State agency’s data team? What is your State agency’s capacity to analyze additional data?

7. What data, if any, are not currently collected or reported that you would need to address disparities in the administration of work requirements and SNAP E&T?

8. What factors should FNS keep in mind when collecting State agency data and using those data to assess disparities?

7. Equity in Work Requirements and SNAP E&T Programs

Now, let’s talk about where equity issues could occur in SNAP work requirements and SNAP E&T and your ideas for how FNS could better assess equity in each.

1. We already talked about some areas in the work requirement and SNAP E&T screening and referral processes that may contribute to disparities. Could any other administrative processes or procedures affect a client’s ability to maintain compliance with work requirements or access SNAP E&T services? If so, please describe.

1. Do these procedures generally affect all clients or specific groups? [If specific groups] Which groups do these procedures affect? Why? [Note: Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

1. What factors may contribute to disparities in client outcomes? [Probe: access to childcare, access to transportation, general readiness, activities that enable clients to meet their goals]

   1. Do these factors generally affect all clients or specific groups? [If specific groups] Which groups do these factors affect? Why?

2. What steps has your State agency taken to ensure all eligible clients can access SNAP?

   1. What prevents eligible clients from accessing SNAP or SNAP E&T?

3. How does your State agency define successful participation in and completion of SNAP E&T? What steps has your State agency taken to ensure all individuals who participate in SNAP E&T succeed in the program? Can your State agency effectively capture all versions of “success” in your data system?

4. How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable E&T outcomes?

5. What aspects of equity, specifically related to the administration of work requirements and SNAP E&T, should FNS prioritize?

8. Wrap-up

Thank you for answering our questions. Before we wrap up …

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

Those are all the questions I have. Thank you very much for speaking with us!

Appendix C. SNAP Local Office Interview Protocol

My name is [name], and I’m a researcher at Westat. Westat is conducting a study for the U.S. Department of Agriculture’s Food and Nutrition Service, or FNS. The study will examine the data FNS needs to measure equitable program access and outcomes in State agencies’ administration of work requirements and Employment and Training, or E&T, for the Supplemental Nutrition Assistance Program, or SNAP.

My colleagues and I are currently conducting interviews with SNAP State and local offices, SNAP E&T providers, and other interested parties here and in five other States. Through these interviews, we hope to collect information from a wide range of individuals involved in administering work requirements and SNAP E&T. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these processes will be very helpful to the study.

Your participation in this interview is voluntary, and your responses will be kept private, except as otherwise required by law. We will not share the information you provide with anyone outside the study team. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalties if you choose not to participate in a part or the entirety of this interview.

We will take notes over the course of the interview and would like to audiorecord the conversation to help us remember the information we collect. We will summarize the information you share with us today and combine it with information gathered from other people we interview. Nothing you say will ever be linked to your name. Direct quotes included in our reports will be presented without the speaker’s name to protect their identity.

I anticipate our conversation will take up to 1 hour. Do you have any questions for me about the project in general or what we will discuss today?

Do I have your permission to record the conversation? You may stop the recording at any time.

[Confirm permission before recording starts. Do not record without respondent permission.]

[Turn on recorder] For the purpose of the recording, are you willing to participate in this interview? And are you willing to have the interview audiorecorded?

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-XXX. The time required to complete this information collection is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the following address: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, Alexandria, VA 22314, ATTN: PRA (0584-XXXX). Do not return the completed form to this address.

Privacy Act Statement

This information is being collected under the authority of the Food and Nutrition Act of 2008, as amended through Pub. L. 113–128, enacted July 22, 2014 [7 U.S.C. 2026], which provides the legislative authority for the U.S. Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) to administer the Supplemental Nutrition Assistance Program (SNAP). Section 17 of the Food and Nutrition Act of 2008 provides the authority to FNS to conduct research to help improve the administration and effectiveness of SNAP. Disclosure of the information is voluntary. The information collected from States and interview participants will enable FNS to understand what data are needed to ensure equitable program access and outcomes in SNAP and SNAP Employment and Training. Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifiable information collected will be kept confidential to the extent of the law. The information may be shared with SNAP contract researchers and USDA SNAP research and administrative staff, with identifying information removed.

1. Background

Before we get started, I would like to learn more about your role and responsibilities at [SNAP local office].

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. General and Able-Bodied Adults With Dependents Work Requirements

Next, I would like to learn more about how staff screen clients for general and able-bodied adults without dependents, or ABAWD, work requirements.

1. First, I want to discuss the general work requirements. SNAP regulations require that all nonexempt SNAP participants must (1) register for work, (2) accept a suitable job if offered, (3) fulfill mandatory SNAP E&T requirements or workfare if assigned, and (4) give the State agency information regarding their employment or availability for work. SNAP participants also cannot voluntarily quit a job or reduce their hours below 30 hours a week without good cause.

   1. How do you screen clients to determine if someone is subject to the general work requirements? [Probe: Is the process similar for new applications and recertifications?]

   2. Does the eligibility system provide any prompts or guidance for this process?

i. [If yes] Can you tell me the questions in the system? Are the prompts helpful? Is anything missing?

3. How do you determine if a client is exempt from work requirements?

i. Do you record exemptions in the eligibility system? What exemptions are most common?

4. How do you determine fitness for work?

i. Has the State agency provided guidance on how to determine fitness for work? If yes, please describe.

2. How do clients subject to general work requirements register for work? [Probe: website, paper form, automatic process]

3. What challenges do you face in determining whether an individual is subject to the general work requirements?

   1. Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on Indian Tribal organizations (ITOs)]

4. Do local office staff participate in or contribute to decision making about which areas the State agency will include in an ABAWD waiver request? If yes, please describe.

5. Next, I would like to discuss the work requirement and time limit for ABAWDs. In addition to the general work requirements, ABAWDs are subject to a time limit unless they meet additional work requirements.

   1. How do you screen to determine if a client is subject to the ABAWD time limit or is exempt? [Note: In advance of the interview, determine whether the State currently has an ABAWD waiver. If a State has a waiver, you may need to ask these questions in the past tense. If the State has had a waiver for many years, workers may have difficulty answering these questions.]

   2. Does the eligibility system provide any prompts or guidance for this process?

i. [If yes] Can you tell me what questions are in the system? Are the prompts helpful? Is anything missing?

3. How do you inform a client they are subject to the ABAWD work requirement? Do you discuss the requirement during the eligibility interview? Do they receive a letter via mail?

i. Do you find the information you provide is sufficient for clients to understand their rights and responsibilities under the ABAWD work requirement and take appropriate action? Why or why not? [Probe: client questions, reactions]

1. [If why not] What additional information or messaging would be needed?

1. What are the challenges associated with determining whether an individual is subject to the ABAWD work requirement?

   1. Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

2. How do you monitor compliance with the general and ABAWD work requirements?

   1. What happens if a client is noncompliant?

   2. In your opinion, why might some clients struggle to comply with the work requirements?

i. What supports, if any, could promote compliance?

8. How do you assess good cause for work registrants? How do you assess good cause for ABAWDs?

   1. How do clients submit information for a good cause exemption?

   2. Has the State agency provided any guidance on this process? If yes, please describe.

   3. Does the eligibility system include any prompts to guide the assessment? If yes, please describe.

9. What challenges do you face in determining whether an individual has good cause from the general work requirements? From ABAWD work requirements?

   1. Do you think these policies affect some groups differently? If yes, how? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

10. What training do staff receive in work requirements and ABAWD policies? [Probe: initially and ongoing]

    1. To what extent do eligibility staff meet with their supervisor to discuss difficult cases? Please describe.

11. Thinking broadly, are there other ways that work requirement policies may contribute to differences in access and outcomes among SNAP clients? [Probe: disqualification policies, sanctions, time limits]

    1. [If yes] Do these policies affect some groups of people more than others? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

3. SNAP E&T Program

Next, I want to discuss the SNAP E&T program in your State agency.

1. Do local office staff participate in or contribute to decision making in the State agency’s development of its SNAP E&T program? If yes, please describe. [Probe: types of components offered, where components are offered, SNAP E&T provider selection]

2. At what point during the eligibility interview do you screen clients for referral to SNAP E&T? [Probe: Are there any differences between new applicants and clients recertifying their benefits?]

1. What screening questions do you ask? Does the eligibility system include any prompts to guide the screening process? [Note: If you have not already received, ask for the list of questions.]

i. [If yes] Please describe.

ii. [If no] Do you use a checklist or some other tool? Please describe.

2. Has the State agency provided any guidance on this process? If yes, please describe.

3. How do you refer clients to SNAP E&T?

   1. Are some people more likely to not receive a referral? Why?

   2. Is the referral automatically sent to the SNAP E&T provider, or does the participant have to follow up on their own?

      1. [If sent to the provider] Are you aware of any variations in how providers reach out to referred clients?

   3. In your opinion, are some clients more likely to participate in SNAP E&T than others? What types of clients? Why? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

      1. To what extent is participation tracked? What is the information used for?

4. Do local office staff conduct the SNAP E&T assessment? If yes, please describe. [Probe: staff in the public benefits/SNAP unit and other units]

5. Who assigns clients to specific SNAP E&T components?

   1. [If local office staff] How do you make the assignments?

6. Does your office have a SNAP E&T liaison? If yes, what is their role? [Probe: collaboration with SNAP E&T providers and clients]

7. Can you please describe the relationship between your office and the SNAP E&T providers in your area?

   1. What data or information do you share with the providers?

8. Thinking broadly, are there other ways that SNAP E&T policies may contribute to inequities among SNAP clients? If yes, which groups are more affected by these policies? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

4. Hiring Practices and Partners

Let’s discuss local office staffing and the SNAP E&T providers in your area.

1. Does your office determine who is hired and staffed at your office? If yes, can you tell me about the hiring process?

   1. Does your office conduct any recruitment? If yes, please describe. [Probe: materials use, recruitment fairs]

   2. Does your office have any equity and diversity goals related to hiring and staffing? If yes, please describe.

2. [If ITOs are located in the State] How, if at all, does your office work with representatives from the nearby Indian Tribal organizations?

5. Data Availability and Needs

I’d like to shift to discuss the data your office collects.

1. We know the State agency collects the following data on work registrants and SNAP E&T participants [List responses to survey questions F.49, F.60, and F.61]. What, if any, additional demographic data do you collect on your clients? [Probe: education, disability, preferred language, housing status]

   1. How do staff collect these data? How often are these data collected or updated?

   2. Do you have any challenges collecting or updating these data?

   3. Do you share these data with the State agency?

      1. If not, why not? Could these data be shared?

   4. What factors or considerations should the State agency and FNS keep in mind when using these data?

2. How, if at all, does your office use data on work registrants and SNAP E&T participants internally? What are your key performance measures?

   1. Have these measures changed over time? If yes, please describe.

   2. To what degree does your office use these data to measure and assess lack of access to SNAP E&T? Please describe.

   3. To what extent does your office use these data to assess disparities in SNAP E&T outcomes? Please describe.

3. What data would you like to have access to that you do not? What would you be able to do with these data if you had them?

4. What, if any, additional data should the State agency or FNS collect to better assess disparities in access to and outcomes in SNAP E&T?

   1. What factors or considerations should the State agency and FNS keep in mind when using these data?

6. Equity in Work Requirements and SNAP E&T Programs

Now, let’s talk about where equity issues could occur in SNAP work requirements and SNAP E&T and your ideas for how FNS could better assess equity in each. First, I would like to share the definition of equity our study team is using for this project. This will help ground our discussion and ensure we are working from the same definition. [Insert working definition]

1. We already talked about some areas in the work requirement and SNAP E&T screening and referral processes that could contribute to disparities. Could any other administrative processes or procedures affect a client’s ability to maintain compliance with work requirements or access SNAP E&T services? If so, please describe.

   1. Do these factors generally affect all clients or specific groups?

   2. [If specific individuals or subpopulations] Which groups do these factors affect? Why?

2. What factors may contribute to disparities in client outcomes? [Probe: access to childcare, access to transportation, general readiness, activities that enable clients to meet their goals]

   1. Do these factors generally affect all clients or specific groups?

   2. [If specific groups] Which groups do these factors affect? Why? [Note: Before the interview, add specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs]

3. What steps has your SNAP State agency taken to ensure all eligible clients can access SNAP and SNAP E&T? What steps has your specific local office taken?

   1. What prevents eligible clients from accessing SNAP or SNAP E&T?

4. How does your State agency define successful participation in and completion of SNAP E&T? What steps has your State agency taken to ensure all individuals who participate in SNAP E&T succeed in the program?

5. How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable SNAP E&T outcomes?

6. What aspects of equity, specifically related to the administration of work requirements and SNAP E&T, should FNS prioritize?

7. Wrap-up

Thank you for answering our questions. Before we wrap up …

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

Those are all the questions I have. Thank you very much for speaking with us!

Appendix D. SNAP E&T Provider Interview Protocol Using During Pre-Test

My name is [name], and I’m a researcher at Westat. Westat is conducting a study for the U.S. Department of Agriculture’s Food and Nutrition Service, or FNS. The study will examine the data FNS needs to measure equitable program access and outcomes in State agencies’ administration of work requirements and Employment and Training, or E&T, for the Supplemental Nutrition Assistance Program, or SNAP.

My colleagues and I are currently conducting interviews with SNAP State and local SNAP offices, E&T providers, and other interested parties here and in five other States. Through these interviews, we hope to collect information from a wide range of individuals involved in administering work requirements and SNAP E&T. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these processes will be very helpful to the study.

Your participation in this interview is voluntary, and your responses will be kept private, except as otherwise required by law. We will not share the information you provide with anyone outside the study team. You may refuse to answer any question, and you may stop the interview at any time. There will be no penalties if you choose not to participate in a part or the entirety of this interview.

We will take notes over the course of the interview and would like to audiorecord the conversation to help us remember the information we collect. We will summarize the information you share with us today and combine it with information gathered from other people we interview. Nothing you say will ever be linked to your name. Direct quotes included in our reports will be presented without the speaker’s name to protect their identity.

I anticipate our conversation will take up to 1 hour. Do you have any questions for me about the project in general or what we will discuss today?

Do I have your permission to record the conversation? You may stop the recording at any time.

[Confirm permission before recording starts. Do not record without respondent permission.]

[Turn on recorder] For the purpose of the recording, are you willing to participate in this interview? And are you willing to have the interview audiorecorded?

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-XXX. The time required to complete this information collection is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the following address: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, Alexandria, VA 22314, ATTN: PRA (0584-XXXX). Do not return the completed form to this address.

Privacy Act Statement

This information is being collected under the authority of the Food and Nutrition Act of 2008, as amended through Pub. L. 113–128, enacted July 22, 2014 [7 U.S.C. 2026], which provides the legislative authority for the U.S. Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) to administer the Supplemental Nutrition Assistance Program (SNAP). Section 17 of the Food and Nutrition Act of 2008 provides the authority to FNS to conduct research to help improve the administration and effectiveness of SNAP. Disclosure of the information is voluntary. The information collected from States and interview participants will enable FNS to understand what data are needed to ensure equitable program access and outcomes in SNAP and SNAP Employment and Training. Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifiable information collected will be kept confidential to the extent of the law. The information may be shared with SNAP contract researchers and USDA SNAP research and administrative staff, with identifying information removed.

1. Introduction

I’d like to learn more about your role and responsibilities at [SNAP E&T provider].

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. SNAP E&T Background

First, I’d like to learn more about how your organization became a SNAP E&T provider and the services it offers.

1. Can you please describe how your organization became a provider for the SNAP E&T program in [State]? [Probe: application process, outreach from State]

   1. How long has your organization been a part of the SNAP E&T program?

   2. Does your organization work with any other Federal or State programs? [Probe: Temporary Assistance for Needy Families, Workforce Innovation and Opportunity Act, General Assistance] If yes, please describe. What, if any, challenges result from working with multiple programs?

   3. What are the challenges to being a SNAP E&T provider? [Probe: 50/50 funding, administrative requirements]

2. In what area or areas of the State do you provide SNAP E&T services?

3. Based on information from the SNAP [State] E&T Plan, [SNAP E&T provider] offers the following SNAP E&T components: [List]. Is this information correct? Is anything missing? [Interviewer note: If this information is not available in the State Plan, ask the provider which SNAP components it offers.]

   1. How did your organization decide to offer these components to SNAP E&T participants?

   2. Do you feel the components offered are reflective of the needs of the clients you serve? Why or why not?

      1. [If no] What other services would be beneficial? What are the constraints of providing these services within SNAP E&T?

3. Clients Served and Progression Through SNAP E&T

Let’s discuss the SNAP E&T clients your organization serves and how they progress through SNAP E&T.

1. What proportion of your overall service population is SNAP E&T clients?

2. How would you describe your SNAP E&T population? [Probe: gender, ethnic, and racial makeup; percentage with children or dependents; age]

   1. Are SNAP E&T clients different from other clients your organization serves? If yes, how? Are SNAP E&T clients co-enrolled in other services?

3. How do SNAP E&T clients learn about your organization and the services you provide?

   1. Do you recruit SNAP E&T clients or conduct any outreach? If yes, can you please describe the outreach you conduct?

i. What types of SNAP E&T outreach materials does your organization use? How are they distributed?

ii. Has your organization solicited feedback regarding the materials from SNAP E&T participants? If yes, what feedback did they provide?

2. [If reverse referral] How does your organization verify a client’s SNAP participation?

3. [If regular referral] Can you tell us more about how the State agency refers clients to your organization?

1. I’d like to learn more about how your clients progress through SNAP E&T. As we discuss the various points in the process, I’d like to get your perspectives on which points may have the potential to contribute to disparities in participant access and outcomes. [Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on Indian Tribal organizations.]

   1. Once the client is referred to your organization, what comes next? [Probe: assessment, orientation activities. Note: Also determine what comes next for clients who have been “reverse referred” to SNAP E&T]

i. What may prevent some clients who have been referred to SNAP E&T from participating in your services? What additional supports, if any, would be necessary to promote their participation?

2. How do you assign a case manager to clients? [Probe: randomly, based on needs assessment, whoever is available]

i. How would you describe the client-case manager relationship? Is the case manager the same person who provides SNAP E&T services to the client?

ii. How often does the client meet with the case manager? Are meetings conducted in person or virtually?

3. How are clients assigned to a SNAP E&T component or components?

i. To what extent are services tailored for individual clients?

ii. What challenges, if any, do you face in making sure clients are assigned to the appropriate component?

iii. Can clients start all activities immediately? What happens if there is a wait before the activity begins?

iv. Are SNAP E&T services provided in person? How do most clients get to the provider location?

v. When are activities offered? [Probe: evening, weekend classes]

4. How do you define successful SNAP E&T completion? Does this definition vary by a client’s individual needs and barriers? If yes, how?

5. What may prevent a client from completing the program? What additional supports, if any, would be necessary to promote completion?

6. What other challenges do your clients face in SNAP E&T participation? What are the most common reasons for dropoff? How could these challenges be mitigated?

1. According to the SNAP E&T Plan in [State], the following are allowable participant reimbursements: [List]. Does your organization offer any additional allowable participant reimbursements?

   1. Are there any caps on the reimbursements? If yes, how are those implemented?

i. Do individual clients ever meet the cap? If yes, what happens then?

2. Are reimbursements disbursed in advance or after expenses occur? Why?

i. [If after expenses occur] What steps do SNAP E&T clients take to be reimbursed for those expenses?

3. What challenges does your organization face in providing support services? [Probe: lack of available childcare/waitlists, limited transit infrastructure, reimbursement caps, administrative burden]

i. What could be done to mitigate these challenges?

4. Are there any additional support services that you think would be valuable to clients? If yes, please describe. [Probe: types of reimbursements, the amount of the reimbursement, services currently reimbursed and not reimbursed by SNAP E&T]

4. Partner Organizations and Staffing

Next, let’s talk about your organization’s partners and current staffing.

1. Does your organization collaborate with other organizations and/or agencies to provide SNAP E&T? If yes, please describe. [Probe: case management organization, refer clients based on needs assessment to partners]

   1. If yes, how did these collaborations begin?

   2. What works well with these collaborations? What are some challenges?

2. Does your organization have relationships with any local employers for job placements or training? If yes, can you please describe what these relationships look like?

   1. How were these relationships developed?

   2. What works well with these relationships? What are some challenges?

3. How does your organization ensure SNAP E&T clients find well-paid work? How do you work to minimize disparities in hiring practices and wages? [Probe: alignment with State workforce development board priorities, pathways programs]

5. Data Availability and Needs

I’d like to shift to discuss the SNAP E&T data your organization collects and reports.

1. Can you first tell me about your organization’s capacity to analyze and use the data you collect?

2. We know that [State] collects the following data on SNAP E&T participants: [List responses to survey questions F.49, F.60, and F.61]. What, if any, additional data do you collect on your SNAP E&T clients? [Probe: educational attainment, disability, language, component assignment]

   1. How do staff collect these data? Which staff collect these data? How often do staff collect or update these data?

   2. Do the data collected vary between clients who do and do not participate in SNAP E&T?

   3. What challenges do you face in how these data are collected or updated?

   4. Do you share these data with the State agency?

i. If yes, do you use a State system or your own? Do you need to complete any duplicate entries?

ii. If no, would a data-sharing agreement be possible?

5. What factors or considerations should the State agency and FNS keep in mind when using these data? [Probe: data quality issues]

1. What outcome data do you collect on your SNAP E&T clients? [Probe: wages, length of time employed, whether the client found employment, name of the employer, position]

   1. How do staff collect these data? Which staff collect these data? How often do staff collect or update these data?

   2. Are there challenges related to how these data are collected or updated? If yes, what are they? [Probe for data validity and reliability]

   3. Do you share these data with the State agency? If not, would a data-sharing agreement be possible?

   4. What factors or considerations should the State agency and FNS keep in mind when using these data? [Probe: quality issues, clients not reporting outcomes]

2. [For providers in mandatory SNAP E&T States and/or providers that serve general and able-bodied adults without dependents] What data do you collect to inform the State agency about client compliance with work requirements and/or mandatory SNAP E&T? How do staff collect these data? Which staff collect these data? How do you share these data with the State agency?

3. How, if at all, does your organization use the data we just discussed internally? What are your key performance measures?

   1. Have these measures evolved? If yes, please describe.

   2. To what degree does your organization use these data to measure and assess differences in participant access or disparities in outcomes? Please describe.

4. What data would you like to have access to that you do not? What would you be able to do with these data if you had them?

5. What, if any, additional data should the State agency or FNS collect to be better able to determine whether there are disparities in SNAP E&T access, participation rates, and outcomes?

6. Equity in SNAP E&T

Now, let’s talk about where equity issues could occur when administering SNAP E&T services and your ideas for how FNS could better assess equity in SNAP E&T.

1. We already talked about some areas in the work requirement and SNAP E&T screening and referral processes that may contribute to disparities. Are there any other administrative processes or procedures that could affect a client’s ability to access SNAP E&T services? If so, please describe.

   1. Do these procedures generally affect anyone who is eligible for SNAP E&T or specific groups? [If specific groups] Which groups and why?

2. What factors may contribute to disparities in clients’ outcomes? [Probe: access to childcare, access to transportation, general readiness, activities that help clients meet their goals]

   1. Do these factors generally affect anyone who is eligible for SNAP E&T or specific groups? [If specific groups] Which groups and why?

3. How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable SNAP E&T outcomes?

7. Wrap-up

Thank you for answering our questions. Before we wrap up …

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

Those are all the questions I have. Thank you very much for speaking with us!

Appendix E. Other Interested Parties Interview Protocol Using During Pre-Test

[Note: Depending on the structure of a SNAP State agency’s E&T program, this interview may not be conducted or may need to be tailored to the partners involved. When scheduling the site visit, determine if another State-level agency, community partner, or other interested party has a role in implementing, maintaining, and/or monitoring SNAP E&T. This protocol has been purposefully written to encompass a variety of potential interested parties, so interviewers will need to tailor questions to the respondent’s role in the SNAP State agency’s E&T program.]

My name is [name], and I’m a researcher at Westat. Westat is conducting a study for the U.S. Department of Agriculture’s Food and Nutrition Service, or FNS, to examine the data FNS needs to measure equitable program access and outcomes in State agencies’ administration of work requirements and Employment and Training, or E&T, for the Supplemental Nutrition Assistance Program, or SNAP.

My colleagues and I are currently conducting interviews with SNAP State and local offices, SNAP E&T providers, and other interested parties here and in five other States. Through these interviews, we hope to collect information from a wide range of individuals involved in administering work requirements and SNAP E&T. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these processes will be very helpful to the study.

Your participation in this interview is voluntary, and your responses will be kept private, except as otherwise required by law. We will not share the information you provide with anyone outside the study team. You may refuse to answer any question, and you may stop the interview at any time. There are no penalties if you choose not to participate in a part or the entirety of this interview.

We will take notes over the course of the interview and would like to audiorecord the conversation to help us remember the information we collect. We will summarize the information you share with us today and combine it with information gathered from other people we interview. Nothing you say will ever be linked to your name. Direct quotes included in our reports will be presented without the speaker’s name to protect their identity.

I anticipate our conversation will take up to 1 hour. Do you have any questions for me about the project in general or what we will discuss today?

Do I have your permission to record the conversation? You may stop the recording at any time.

[Confirm permission before recording starts. Do not record without respondent permission.]

[Turn on recorder] For the purpose of the recording, are you willing to participate in this interview? And are you willing to have the interview audiorecorded?

Privacy Act Statement

This information is being collected under the authority of the Food and Nutrition Act of 2008, as amended through Pub. L. 113–128, enacted July 22, 2014 [7 U.S.C. 2026], which provides the legislative authority for the U.S. Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) to administer the Supplemental Nutrition Assistance Program (SNAP). Section 17 of the Food and Nutrition Act of 2008 provides the authority to FNS to conduct research to help improve the administration and effectiveness of SNAP. Disclosure of the information is voluntary. The information collected from States and interview participants will enable FNS to understand what data are needed to ensure equitable program access and outcomes in SNAP and SNAP Employment and Training. Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifiable information collected will be kept confidential to the extent of the law. The information may be shared with SNAP contract researchers and USDA SNAP research and administrative staff, with identifying information removed.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-XXX. The time required to complete this information collection is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the following address: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, Alexandria, VA 22314, ATTN: PRA (0584-XXXX). Do not return the completed form to this address.

1. Background

I’d like to start by learning about your role and responsibilities at [organization].

1. What is your current job title or position?

2. How long have you been in this position?

3. What are your primary responsibilities?

2. Organization and Staffing

[Ask questions in section B of all respondents]

Next, I would like to learn more about your organization.

1. Please describe your organization and its mission.

2. Please describe your organization’s role in SNAP E&T. Does your organization offer services to SNAP E&T participants? Do you have a role in program monitoring?

1. How long has your organization been working with SNAP E&T?

2. How did your organization’s relationship with SNAP E&T begin?

3. Has this relationship changed over time? If yes, how? Please also describe any forthcoming changes.

3. Do you work with other organizations or agencies to [support/deliver] SNAP E&T services?

1. [If yes] What organizations or agencies do you partner with? How do you collaborate with these partners for SNAP E&T?

4. Does your organization have explicit hiring targets or goals for frontline staff? [If yes] Can you tell us more about these hiring practices? What prompted your organization to implement these?

1. What other factors does your organization consider when hiring frontline staff? [Probe: experience working with clients or a similar population, lived experience, cultural humility]

5. Does your organization have explicit hiring targets or goals for leadership and/or managerial staff? If yes, can you tell us more about these hiring practices? What prompted your organization to implement these?

1. What other factors does your organization consider when hiring leadership or managerial staff? [Probe: experience working with clients or a similar population, lived experience, cultural humility]

6. Can you tell us more about how [organization] recruits staff?

3. SNAP E&T Program Operations

[Ask questions in section C only if the respondent has a role in SNAP E&T program operations, such as a State Department of Labor that helps the SNAP State agency administer the program.]

Next, let’s discuss your organization’s role in SNAP E&T program operations.

1. The State agency operates a [mandatory/voluntary/mandatory for some] SNAP E&T program. Was your organization involved in this decision-making process? If yes, can you provide some background on how the State agency decided to operate this type of SNAP E&T program?

1. Has the type of SNAP E&T program changed recently? If yes, please describe why and how.

2. [If voluntary] Thinking broadly, how does your organization provide outreach and work to ensure access to SNAP E&T services to groups or communities that want to participate? What challenges do you encounter? Do you face challenges providing SNAP E&T to any groups or communities?

1. What outreach materials do you use?

2. Has your organization solicited feedback from SNAP E&T participants regarding the materials? If yes, what feedback did they provide?

3. [If mandatory] Does your organization have enough SNAP E&T slots for all mandatory participants? [If no] How does your State agency determine who must participate in SNAP E&T and who will receive a good cause exemption? Is this information tracked in your eligibility or SNAP E&T system?

4. According to the SNAP E&T State Plan, the State agency offers the following SNAP E&T components: [list]. Was your organization involved in this decision-making process? If yes, can you provide some background on how the State agency decided to provide these components?

1. [If some components are not offered statewide] What factors determine where a SNAP E&T component is offered?

2. Are there any components that you think the State should offer but does not? Why do you think they should be offered? Why are they not available?

5. Can you please describe how clients are screened for SNAP E&T?

6. How are clients referred to SNAP E&T? [Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on Indian Tribal organizations, or ITOs.]

1. Are some groups of people more likely to get a referral? Why?

2. Is the referral automatically sent to the SNAP E&T provider, or does the client need to follow up on their own?

3. In your opinion, are some groups of referred clients more likely to participate in SNAP E&T than others? Which groups? Why?

i. How could the State or your organization increase participation in SNAP E&T from underrepresented groups?

7. What outreach does your organization conduct for SNAP E&T? What materials do you use?

1. Has your organization solicited feedback regarding the materials from SNAP E&T participants? If yes, what feedback did participants provide?

8. Does your organization provide case management to SNAP E&T participants? If yes, can you please describe how case management is provided to SNAP E&T participants? [Probe: who provides the service, how often case managers meet with clients, how case manager assignments are made, goal of case management]

1. How effective do you think case management is in increasing retention and completion of SNAP E&T? How could case management be more helpful? Do any groups require more individualized services? What types of services?

2. Do you track any data related to case management? If yes, please describe.

9. According to the fiscal year 2024 SNAP E&T State Plan, your State agency allows the following reimbursable participant expenses: [list]. Was your organization involved in this decision-making process? If yes, can you provide some background on how the State agency decided to provide these reimbursements? [Probe: consultation with providers, participants, other individuals/entities]

1. Does your organization provide reimbursements to SNAP E&T participants? If yes, how are clients informed about reimbursements? [Probe: at the time of SNAP E&T referral, during screening, by the provider]

i. How are participants reimbursed for those costs? Are costs ever paid on behalf of participants?

2. We know that transportation and childcare are two of the biggest barriers to SNAP E&T participation. What efforts has your organization taken to address these barriers through its participant reimbursement policies? [Probe: agreements with childcare agency, looking into rideshare options in areas with limited public transit, priority for childcare for SNAP E&T participants]

3. What other reimbursements, if any, could help mitigate some of the challenges associated with SNAP E&T participation?

10. Thinking broadly, are there other ways that SNAP E&T policies may contribute to inequities among SNAP clients? If yes, which groups are more affected by these policies? [Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs.]

4. Available Data and Data Needs

[Ask questions in section D of all respondents. Tailor as needed dependent on responses on the types of data collected.]

I’d like to shift to discuss the SNAP E&T data that are collected and reported.

1. We know the State agency collects the following data on SNAP E&T participants [list responses to survey questions F.49, F.60, and F.61]. What, if any, additional demographic data do you collect on your clients? [Probe: educational attainment, disability, language]

   1. How do your staff collect these data? How often are these data collected or updated?

   2. Are there challenges related to how these data are collected or updated that may affect the quality of the data?

   3. Do you share these data with the State agency? [If no] Would a data-sharing agreement be possible?

   4. What factors or considerations should the State agency and FNS keep in mind when using these data?

2. What outcome data do you collect on your clients? [Probe: wages, length of time employed, whether the client found employment, name of the employer, position]

   1. How do staff collect these data? How often are these data collected or updated?

   2. Are there challenges related to how these data are collected or updated? [Probe: data reliability and validity]

   3. Do you share these data with the State agency?

i. [If yes] Do you use a State system or your own? Do you need to complete any duplicate entries?

ii. [If yes] Can you please describe the data-sharing agreement your organization has with the State agency? What challenges did you face in setting up the agreement? What worked well?

iii. [If no] Would a data-sharing agreement be possible? Why or why not?

4. What factors or considerations should the State agency and FNS keep in mind when using these data? [Probe: quality issues, clients not reporting outcomes]

1. How, if at all, does your organization use these data internally? What are your key performance measures?

   1. Have these measures changed over time? If yes, please describe.

   2. To what degree does your organization use these data to measure and assess participant access or disparities in outcomes? Please describe.

2. What data would you like to have access to that you do not? What would you be able to do with these data if you had them?

3. What, if any, additional data should the State agency or FNS collect to be better able to determine whether there are disparities in SNAP E&T access, participation, and outcomes?

5. Equity in SNAP E&T

[Ask questions in section E of all respondents. Before the interview, add State-specific probes that may be most relevant for groups in the State, such as rural individuals, racial/ethnic groups, or individuals living on ITOs.]

Now, let’s talk about where equity issues could occur when administering SNAP E&T services and your ideas for how FNS could better assess equity.

1. We already talked about some areas in the work requirement and SNAP E&T screening and referral processes that may contribute to disparities. Could any other administrative processes or procedures affect a client’s ability to access SNAP E&T services? If so, please describe.

   1. Do these procedures generally affect anyone who is eligible for SNAP E&T or specific groups? [If specific groups] Which groups do these procedures affect and why?

2. What factors may contribute to disparities in clients’ outcomes? [Probe: access to childcare, access to transportation, general readiness, activities that help clients meet their goals]

   1. Do these factors generally affect anyone who is eligible for SNAP E&T or specific groups? [If specific groups] Which groups do these procedures affect and why?

3. How does your organization define successful participation in and completion of SNAP E&T? What steps has your State agency taken to ensure all individuals who participate in SNAP E&T succeed in the program?

4. How do you think FNS should define equitable access to SNAP and SNAP E&T? How should FNS define equitable SNAP E&T outcomes?

5. What aspects of equity, specifically related to the administration of work requirements and SNAP E&T, should FNS prioritize?

6. Wrap-up

Thank you for answering our questions. Before we wrap up …

1. Is there anything else you would like to share with us?

2. Is there anything we did not ask about that you think is important for us to know?

Those are all the questions I have. Thank you very much for speaking with us!

M-2