Assessment of SNAP E&T Data Master Interview Discussion Guide OMB CONTROL NUMBER: 0584-XXXX
EXPIRATION DATE: XX/XX/20XX
Attachment B: Assessment of SNAP E&T Data: MASTER INTERVIEW DISCUSSION GUIDE
Introduction for respondents [tailor to meeting or interview]:
Good afternoon [morning]. My name is __________________, and I am from Mathematica. [If three-person visit, other members introduces him/herself.] FNS has contracted with Mathematica to (1) identify and describe the current State and Federal systems that collect, validate, and analyze E&T data; (2) assess the current and future E&T data needs of Federal, Regional, and State staff; and (3) recommend a plan to improve how Federal, Regional, and State staff collect and use data for E&T program improvement and reporting.
This discussion should take about <duration> minutes. Do you have any questions before we begin? Do you consent to participate in this discussion?
Your participation in this [discussion/interview] is voluntary, and your responses will be kept private to the extent provided by law. We will not share the information you provide with anyone outside of this room and the research team, [including your supervisor and State or Federal -level staff]. You may refuse to answer any question or may stop the interview at any time. There will not be any penalties if you refuse to participate in part or full.
We will take notes during the [discussion/interview] so that we can remember the information we collect. We will use this information in our reports, describing the range of responses expressed by staff. The reports might list the names of organizations that contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you.
I would like to record our discussion so I can listen to it later when I write up my notes. No one outside the immediate team will listen to the recording. We will destroy the recording after the study is complete. If you want to say something that you do not want recorded, please let me know and I will be glad to pause the recorder. Do you have any objections to being part of this interview or to my recording our discussion?
[Note for site visitors: These questions encompass the full set of questions of interest for the needs assessment. To the extent that information for some questions has already been collected through other sources, please reframe the questions to confirm the information. Similarly, please tailor other questions, as appropriate, given the purpose of the meeting. Throughout the discussion guide, we use employment and training (or E&T) to refer generally to employment and training (education) activities and services; we are not referring to a specific program.]
Topics by Respondent
This table will be used to tailor the discussion guide for each respondent.
Question |
Respondents |
||||||
State agency |
Other intermediaries |
Local office |
E&T providers |
||||
SNAP director |
E&T policy staff |
Data systems/IT staff |
Director |
Frontline staff |
|||
SNAP E&T Program overview |
|||||||
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
|
|
|
|
✔ |
✔ |
|
|
|
|
|
|
|
✔ |
✔ |
|
✔ |
✔ |
|
|
✔ |
✔ |
✔ |
SNAP E&T data collection |
|||||||
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
|
|
✔ |
|
|
|
✔ |
✔ |
✔ |
|
✔ |
|
|
|
✔ |
✔ |
✔ |
|
✔ |
|
|
|
✔ |
✔ |
✔ |
|
✔ |
|
|
|
|
|
✔ |
|
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
|
✔ |
|
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
|
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
✔ |
Master interview discussion guide
Background
What is your official job title or position?
What are your primary responsibilities?
How long have you worked at [agency/organization]? How long in this position?
What services are provided by [agency/organization]?
Are SNAP E&T services targeted to certain populations?
What geographic area does the program serve?
Are there any areas without SNAP E&T services?
Which types of clients are referred to the SNAP E&T program (or required, if mandatory)?
Eligibility/intake
What is the process for determining whether a client should be referred to the SNAP E&T program?
How do eligibility workers determine who is work exempt and who is eligible to participate in E&T?
Does the State do any kind of assessment to determine if an individual should be referred to E&T, and if so which component would be the most appropriate?
Does eligibility screening occur face to face, or by telephone, or through some other mode of communication?
How and when are clients notified about SNAP E&T, if they are eligible? Describe the process.
What happens next? Is there an orientation? Is there a direct referral connecting the client to the E&T provider by phone or in-person (a warm referral)?
How much follow-up is conducted to engage the client in a SNAP E&T program?
Where do workers input data about eligibility for E&T? What gets recorded? How? Where does that data go?
Case management, support services, and E&T services
At what point is a client referred to a case manager?
Who are the case managers (with which organization/agency)?
Are most clients referred to an E&T provider or are they served by the [agency]?
Describe the referral process or the next steps if services are offered onsite.
How does the case manager determine which provider to refer clients to?
Can you tell me about the services each provider offers? Case management? Support services? Referrals to other E&T services?
What services do E&T participants receive from providers? Case management (a specific model)? Support services? Referrals to E&T services? Describe.
Does the intake process include conducting an assessment of the client? Describe the assessment.
How long are participants engaged in services, on average?
May clients receive multiple services?
How do the State agency and E&T providers communicate to coordinate services? Does the State track E&T participants in a State MIS system? Who has access to data on E&T participants? How well does this approach work?
Engagement
How engaged are clients in receiving services?
Do they drop off initially or over time? (If mandatory, do they get sanctioned initially or over time?)
What types of information do providers report to the local office regarding client engagement? What is the content of the report, and how is it transferred?
How frequently do they provide data?
Data sources
Where do the data come from for monitoring, analysis, and reporting?
Who inputs the data about participants?
What information do they collect?
What protocols do they follow?
Where are the data inputted?
Are there any challenges in obtaining data from these sources? Please describe
Does the program collect additional performance indicators or data beyond the Federal measures for the program?
If so, how are those defined?
Who requires the program to collect them?
How did the program identify the need for and define those measures or data elements?
Does the program use these same indicators or data for other programs? If so, which programs?
Tracking and coding data
Cycle through this set of questions for each data source defined in section 5 above, data sources.
Which staff are responsible for entering data?
When does data entry happen?
Do partners or providers have the ability to input data? What about clients?
What data get entered?
Where are the data entered?
Are there protocols for collecting and entering data?
Do staff receive training on how to collect and enter the data? If so, what type of training? How frequently?
Stakeholders involved in E&T data
Whom would you consider the stakeholders for E&T data in your State or community, meaning individuals or groups that are interested in or have a use for E&T data?
What is the level of interest in E&T data?
What is the interest level in E&T data of stakeholders?
Level of data available (individual or aggregate)
What type of data (individual or aggregate) does the State, locality, or both capture data on E&T participants?
Are there any restrictions such as systems, collection mechanisms, or legal/security that prevent the State from collecting and reporting individual level data for SNAP E&T?
For reports, what level of information (individual or aggregate) is the State able to provide for E&T participants?
Does the State track in their eligibility system:
Who is a work registrant?
Reasons for work registrant exemption (Federal)?
Who is an ABAWD?
Who is referred or required to participate in SNAP E&T?
Reasons for SNAP E&T exemption (State)?
Does the State capture any of this information in a separate system from the eligibility system? If so, which information and what system? Do these systems interact or transfer information?
Who participates in an E&T component and/or case management services through E&T?
What services and activities are received (describe what is tracked)?
If the above are not tracked at the individual level, are they tracked at the aggregate level? Describe how these are tracked and reported.
How frequently are data updated and reports submitted?
When you receive new information from clients, do you update the system in real-time or afterwards for each type of data?
Are some data automatically updated? Which? How frequently?
If afterwards, when? What resources do you use to populate the data? Case notes?
Data systems
What data systems does the program tracking and reporting for E&T data?
How long has the program used this/these system(s)?
Are they off-the-shelf (OTS) products or were they developed with internal staff or contactors? If an OTS product, which one?
What data do they record?
Is the E&T data system separate from the SNAP eligibility system?
How do the data systems for E&T integrate or connect with the SNAP eligibility system and any other State systems (such as systems that provide employment data)?
Do you have a local data system in addition to the State or Federal reporting systems? What systems do you routinely use?
For each data system mentioned above, cycle through the next series of questions.
Are all the data systems automated, or are there some manual processes?
Which staff or partners have access to the data system?
To what extent do you have access to information from other programs’ data systems?
How do you obtain the information from those systems? Do you have direct access to those systems? Do you have read-write access or do you have read-only access to other programs’ systems?
Are data transferred directly across systems or are the data provided in some form of extract file?
Data validation and accuracy
What challenges are inherent in collecting, aggregating, and reporting the data you collect?
How much validation is possible given your data systems and type of data?
How much validation do you do? What is involved in the validation process?
Data reporting
Who tracks, collects, reports, and monitors the data? Describe which staff participate in each step.
Which reporting activities does the program participate in?
Prompt: FNS-583, State Plans, Annual Reports
Where are these reports submitted, and what is the process?
To what extent are the program’s data collection and reporting procedures dictated by the State or Federal government?
To what extent has the [the program/State] created its own procedures for collecting and reporting data?
Do you have these procedures written down?
Can you provide us with some examples of what these written procedures look like?
Does the program use the data for performance indicators to assess program success or manage the program?
If so, how often does the program use these reports and in what context?
Does the program share this information across programs/departments?
If not, what are the reasons for not using the collected data?
What reports does the program’s data system produce?
At what level (State or local) are these reports generated? Can you provide us with examples of the categories of information included in those reports?
Do these reports primarily focus on the Federal measures?
What about the State’s additional measures?
Can you easily customize these reports?
What reports or data do you find the most useful in day-to-day operations to help manage your time and services for participants?
Can the program get information out of the data system in a way that is useful or how you would want to access the information?
How does the State compile its reports for FNS?
How often are the reports updated?
When does the State capture the information to send to FNS?
Does the State conduct a quality/accuracy check before sending to FNS?
If an accuracy/quality check is conducted, what is involved in reviewing the data?
Is data reporting a manual or automated process?
When are the data pulled, and for which reporting activities?
What do the data reports contain? Specify for each reporting activity.
Are the data individual or aggregate?
Do you review reports before submitting them? If so, how do you review them? Are there certain checks that you perform?
What analysis is possible with the data that you collect?
Developing a State E&T plan
What is the process for developing the State SNAP E&T plan?
Do you find that staff understand all of the fields in the plan?
If not, do you find that some require discussion each year? If yes, please describe the fields that present the most challenges to staff.
Do you have any problems getting all of the data you need for the plan?
Providing FNS-583 data
What is the process for collecting and reporting the FNS-583 data?
How does the State track these measures?
Do staff understand all the fields on the form, or do they need to discuss them each year? If yes, please describe the fields that present the most challenges to staff.
Do you have any problems getting all of the needed data for the form?
Providing national reporting measures and participant characteristics
What is the process for collecting and reporting the national reporting measures and data on participant characteristics?
How does the State track these measures?
Do you find that staff understand all of the fields for the measures and characteristics?
If not, do you find that some fields require discussion each year? Please describe the fields that present the most challenges to staff.
Do you have any problems getting all of the needed data for the report?
Developing State component outcome measures
What is the process for developing the State component outcome measures?
How does the State track and extract data for these measures?
Transmitting data to local office
How do providers send information to the local office?
What data are sent to the local office? For a report? For other monitoring? Describe.
How frequently are data sent? Does this timeline every change or get delayed?
Is the process automated or manual?
Are the data reviewed by the provider before they are sent to the local office? For quality – accuracy, consistency, usefulness? For completeness? Are there specific checks required or used?
If an accuracy/quality check is conducted, what is involved in this activity?
Transmitting data to State
How do local offices send information to the State?
What data are sent to the State? For a report? For other monitoring? Describe.
How frequently are data sent? Does this timeline ever change or get delayed?
Is the process automated or manual?
Are the data reviewed by the local office before they are sent to the State? For quality– accuracy, consistency, usefulness? For completeness? Are there specific checks required or used?
If an accuracy/quality check is conducted, what is involved in this activity?
Costs and resources
What are the costs associated with SNAP E&T data collection, reporting, and analysis?
What are the costs at the local, State, Regional, and Federal levels for doing so?
What staff or other costs are required?
What costs and resources are required for changing the data collection practices, procedures, or elements for SNAP E&T?
Are there costs associated with changing the system? What about changing just discrete fields within the system?
What staff or other costs are required?
How long did this process take?
Are there costs associated with instituting new requirements for the Annual Report? Describe those.
What are the costs at the local, State, Regional, and Federal levels for doing so?
What staff or other costs are required?
Are there other barriers to these modifications?
How long did this process take?
Did you require any assistance from Regional or Federal staff to institute these changes?
If so, what assistance was required?
How much time would you estimate this took these staff?
How long did this process take?
Do you know of other States who have completed changes to their data collection practices, procedures, or elements for SNAP E&T?
Which States?
What do you know about their systems change? When did it occur? What was the result?
Data sharing between agencies
Have you received any funding to support coordinated or shared data systems at the local level?
How easy or difficult is it to modify the data systems or accept or provide data to the other system?
Have you done so in the past? What did you change?
What was the process like?
What have been the opportunities and challenges with changing metrics to data systems?
Does the State or locality share data with other agencies?
Does the State or locality have MOUs with other organizations?
What are the challenges (legal, regulatory, technical) with data sharing?
Does the State or locality have access to the necessary data for reporting?
What types of data from other agencies would be helpful that you do not currently receive? In what ways would it help?
Have you received any funding to support coordinated or shared data systems at the local level?
Guidance on data reporting
What guidance has the State received on reporting for E&T?
Is the guidance clear?
What would help improve your understanding of E&T reporting?
Using data for program improvement
Does the State or locality use data for program improvement?
How does the State or locality use data?
What data does the State and locality use for program improvement?
What additional data would be useful for State and/or locality program improvement?
Goals for future data use
What would you want to see in future reporting of E&T data?
What would you need to make this possible?
Challenges with E&T data and promising practices
What do you find challenging about SNAP E&T data collection, reporting, or analysis?
What do you find challenging about data reporting?
What priority does E&T data reporting have within your organization, relative to other data or policy priorities?
Do you have the necessary tools to submit E&T data? If not, what functionality do you need?
What are some of the practices with SNAP E&T data collection, reporting, or analysis that you would like to share with other States or localities?
What is notable about how you collect, report, or analyze E&T data?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Julie Worthington Hartnack |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |