OMB No.: 0584-0547
Expiration Date: MM/DD/YYYY
Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The OMB control number for this project is 0584-0547. Public reporting burden for this collection of information is estimated to be 108.25 hours per response including the time for participating in the interviews and providing the extant data collection. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Research and Analysis, 3101 Park Center Drive, Room 1014, Alexandria, VA, 22302, ATTN: Rosemarie Downer
My name is [X], from Mathematica Policy Research.
The Food and Nutrition Service (FNS) is sponsoring a study of advanced SNAP modernization initiatives. Mathematica will be conducting the study. To do so, we will be collecting information from state and local SNAP offices and their partners and vendors. We really appreciate the time you and your staff took to complete the survey we sent you about the SNAP modernization initiatives being implemented in your state.
Our final report will describe the range of responses expressed by staff, and may list the names of agencies and partners who contributed information, but we will not quote you or anyone by name or title.
I expect our conversation will take approximately 60 minutes.
OPTIONAL IF INTERVIEWER CHOOSES TO RECORD: I want to be sure I am keeping track of everything you are saying. May I record our discussion so that I can listen to it later when I write up my notes? No one outside of our research team will have access to the recording.
IF YES: Thank you. It will be helpful if you speak up, speak clearly, and speak one at a time.
IF NO: That’s no problem. I’ll take notes as you talk, but I may sometimes need to ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be discussing today?
States have implemented many new procedures and policies in their SNAP programs with the goal of reducing administrative costs and error rates while maintaining or increasing program access. For the purposes of this study on state efforts and experiences with modernizing their SNAP programs, we are using four broad categories to define what we mean by modernization: (1) expanded uses of technology, (2) policy changes, (3) administrative restructuring, and (4) working with community partners or vendors.
1. Although we recognize that there may be some overlap in these categories, does this definition generally capture all the recent changes to SNAP in [STATE] you would consider to be modernization?
2. Are there any changes that do not fall under these four broad categories that you think also should be included?
3. Is there anything in this definition that you would not consider to be modernization?
GO TO SECTION II
INSTRUCTIONS TO INTERVIEWER
Begin this section of the interview by confirming state profile info on the specific modernization changes undertaken by [STATE].
First, I’d like to talk about the most important modernization initiatives that took place in your state.
1. Of all the changes [STATE] has made, which do you think is (are) the most important?
PROBE IF EXPLANATION IS NOT PROVIDED:
Why?
2. Which change or changes have made the greatest difference for the way SNAP is administered?
And for clients’ experience with the SNAP program?
3. In what key ways has this change made a difference in the [SNAP LIFE CYCLE [application, certification, recertification, case management]]?
4. What other programs were included or impacted by any of the modernization changes?
And how so?
Was this by design?
INSTRUCTIONS TO INTERVIEWER
For each modernization change that the respondent has identified as one of the “most important” in the previous section, ask the following questions:
1. Now, thinking about [MOST IMPORTANT MODERNIZATION CHANGE], what was the primary impetus for making this change?
PROBE:
What were the key factors that drove or motivated [STATE] to pursue this change (e.g., desire to reduce administrative costs, political priority, part of a larger effort to streamline service delivery, or align program policy)?
2. What other kinds of changes were considered before deciding to implement this change?
Why were these alternatives ultimately not pursued?
3. In terms of decisions regarding whether to have some SNAP activities carried out by vendors or partners, what are the key factors or considerations that shape(d) your state’s approach?
1. What additional modernization changes are planned for the future?
What are the reasons for additional planned changes?
Are these changes part of the original plan or reactions to unanticipated problems/circumstances?
2. What is their current status, including anticipated implementation schedules?
Next, I would like to turn your attention back to the most important modernization initiatives you identified earlier and discuss the planning and other steps involved in their implementation.
1. Which (agency[ies]/individuals) were involved in the initial decision-making process to initiate this undertaking/change?
When did planning for this key change commence?
2. Which (agency[ies]/individuals) were most involved in the planning process?
Describe the involvement of relevant parties engaged in planning.
3. Do you think this was the right mix of agencies/expertise for this undertaking?
PROBE:
Were the right people at the table?
PROBE:
Would the decision-making/planning process benefited from having other kinds of expertise/perspectives at the table?
4. Are there any specific aspects of the decision or planning process that you think were particularly important for understanding what was ultimately implemented?
Were there any strengths or weaknesses in the design or implementation of these changes?
INSTRUCTIONS TO INTERVIEWER
For each of the most important changes, confirm the timeline and scope of implementation, including:
if change was implemented statewide or only in select areas
if implementation was/will be phased-in over time and the phase-in schedule by area/office
1. Thinking about when these changes were implemented in relation to one another, was there a broader strategy or other considerations that shaped that timing?
Please discuss.
INSTRUCTIONS TO INTERVIEWER
If the change was rolled out over time or only in select areas, ask:
1. Please describe any piloting or testing that was conducted.
What kinds of changes were (piloted/tested)?
When and where were the changes (piloted/tested)?
Why were these particular areas/offices chosen as (pilot/test) sites?
2. During testing or piloting, what kinds of problems were uncovered?
How were they resolved?
3. Were there delays between planned and actual implementation schedule?
What were the causes of the delays?
INSTRUCTIONS TO INTERVIEWER
Ask the questions below for all initiatives being discussed:
4. What were the key challenges faced during implementation of this key change?
In hindsight, do you think anything could have been done to avoid or mitigate these challenges?
5. What was the general reaction of local staff to this key change being implemented?
PROBE:
Does staff reaction vary by type of staff?
How and why?
6. How if at all would you have implemented this change/initiative differently?
7. What ongoing challenges are associated with this change?
8. What new challenges have emerged?
For the next questions, we would like to refocus on the key modernization changes you identified at the beginning of the interview.
1. How have these changes affected the way SNAP is administered?
PROBE:
How have these changed affected the [SNAP LIFE CYCLE: application, certification, recertification, case management] process?
1. How do clients currently interact [e.g. apply, recertify, report changes] with SNAP as a result of this key change?
2. How is this different from how clients interacted [e.g. applied, recertified, reported changes] with SNAP prior to this key change?
GO TO SECTION III
Next I’d like to discuss how the organization and administration of SNAP has been restructured.
1. Please describe the current state/local level organizational structure and functional responsibilities of the agency/office and any other entities (e.g., partners/vendors) as it relates to the administration of SNAP.
2. How have the organizational structure and/or functional responsibilities changed as a result of implementing any SNAP modernization changes?
3. What are the major advantages and disadvantages of restructuring functional responsibilities this way?
What are the trade-offs?
4. Is there much variation at the local level in the way functional responsibilities for SNAP are carried out?
PROBE, IF YES:
Please describe.
5. Please discuss any significant changes in staffing levels (by type) at the local level, particularly as they relate to SNAP.
PROBE:
Has the overall number (or type) of staff involved in SNAP (application, certification, recertification, and case management) increased, decreased, or stayed the same since modernization changes started?
What factors account for this change?
What effect have SNAP modernization changes had on staff levels?
Conversely, what effect have staffing levels had on SNAP modernization efforts?
INSTRUCTIONS TO INTERVIEWER
Start with generic question below, and then use additional probes only if it is not clear whether one or more key initiatives were the primary reason for the changes mentioned.
1. Please describe the staffing structure in local offices as it relates to processing and managing SNAP cases.
PROBE:
How has the staffing structure changed, if at all, since modernization changes started?
2. Has there been any change in staff turnover rates or patterns?
3. If so, what do you think accounts for these changes?
PROBE:
Are staff turnover issues/changes related to any modernization changes (e.g., call centers, online application units)?
1. How is work divided across units within your local service offices, call centers, and other units?
2. Please describe how the roles and responsibilities/activities of staff at the state level and at the local level/in your office have changed as a result of any of the modernization changes (specify by type of change).
3. What new skills are required of staff in order to carry out any of these new processes/features?
4. What if any new staff positions have been created as a result of modernization changes and what tasks/activities are they responsible for?
5. What if any staff positions have been lost as a result of modernization changes and what tasks/activities were they responsible for?
6. Are there any other factors/conditions that have led to a change in the organizational structure of the office or in staffing roles and responsibilities?
7. Please describe whether and how your role or responsibilities have changed as a result of any of these changes (specify by type of change)?
INSTRUCTIONS TO INTERVIEWER
Use the following probes for question # 8 below only if it is ambiguous to which step of the SNAP life cycle the respondent is referring.
8. How if at all has your role changed in regard to how [SNAP LIFE CYCLE PROBE] is administered?
SNAP LIFE CYCLE PROBES:
Application
Certification
Recertification
Case Management
9. What SNAP activities if any are no longer performed by staff?
10. What is your general assessment of these changes?
11. Do you consider these changes in your roles and responsibilities to be an improvement or not?
INSTRUCTIONS TO INTERVIEWER
Confirm state profile information on which functions/activities, if any are outsourced to vendors.
1. Please describe any of the most important SNAP related functions/activities that are outsourced to vendors?
2. Have the vendors or the activities outsourced to vendors changed recently?
PROBE, IF YES:
How?
Why?
3. What are the advantages of using vendors to perform these activities?
4. What are the disadvantages?
5. What have you found to be the major trade-offs of using vendors for this/these activities instead of in-house staff?
6. Do vendor staff work in-house among SNAP staff, or are their tasks completed off-site?
7. Have vendors met their contractual obligations?
8. What sorts of problems have arisen, if any?
9. Have technical products developed by vendors (such as online applications or call center technology) performed as expected?
1. Besides vendors, what outside organizations do you partner with to carry out any work related to SNAP?
2. Please describe any functions/activities that partner organizations perform in relation to [SNAP LIFE CYCLE PROBE]?
SNAP LIFE CYCLE PROBES:
Application
Certification
Recertification
Case Management
3. Where do the partner staff conduct these activities (e.g., at the SNAP office, the partner organization, elsewhere?)
4. What is the arrangement or nature of this partnership?
PROBE:
Is it a formal or informal arrangement?
INSTRUCTIONS TO INTERVIEWER
If the arrangement is formalized, ask the following:
Please describe the terms and nature of the formalized arrangement.
INSTRUCTIONS TO INTERVIEWER
if a formal contract exists with the partnering organizations, ask the following questions for each partner organization for which a formal contract exists:
Has [PARTNER] met its contractual obligations?
5. Are any partners compensated?
PROBE, IF YES:
Please identify partners that are compensated.
For which activities are they compensated?
What is the basis for this compensation (e.g. fixed fee, per hour, per application processed, reimbursement for cost incurred, etc.)?
What SNAP related activities if any are not compensated?
PROBE, IF CERTAIN ACTIVITIES ARE NOT COMPENSATED:
Why are certain activities not compensated?
How does this get decided?
6. Is the performance of the partner tracked or evaluated?
PROBE, IF YES:
Please describe.
What criteria are used to evaluate the partner’s performance?
7. Describe the partnering organization's performance in terms of obligations not met, met, or exceeded.
8. How long has this partnering relationship existed with each partner?
9. How was the relationship with this partner organization established?
10. Have there been any changes in this partnering arrangement since its inception?
PROBE, IF YES:
Please describe.
11. What are the partner organization's primary activities outside of their partnering arrangement with SNAP?
1. Please describe any training you received from or provided to SNAP staff, vendors, and community providers who have assumed new roles and responsibilities as a result of any modernization changes that have been implemented.
INSTRUCTIONS TO INTERVIEWER
For each type of training identified in previous question, ask the following:
2. Who conducted the training (e.g. in-house by staff, external training vendor brought in, etc.)?
3. Who received it?
4. Was this training voluntary or mandatory?
PROBE, IF VOLUNTARY:
If voluntary, was it well attended?
5. Were any incentives provided to encourage participation?
6. What topics did the training cover?
7. What methods are used to administer training (e.g. face-to-face classroom sessions, online learning, educational materials, etc.)?
8. Was the training helpful or useful?
9. Was the training sufficient?
10. Are there additional types of training you think are needed?
11. If mandatory training was required, has this helped you to address changing roles and responsibilities?
INSTRUCTIONS TO INTERVIEWER
These questions are specific to the roles, responsibilities and functions of call center staff.
1. Please describe the major functions of the call center(s), the types of requests it handles, its hours of operation, and the geographic area it serves.
2. Please describe the staffing structure of the call centers.
PROBE, IF VOLUNTARY:
What types and levels of staff work at this call center?
3. Which staff field client calls?
4. How is it determined to whom calls are directed?
5. How does the length of calls vary by the purpose of the call?
6. How long do callers generally wait to speak with someone from the call center?
7. How many calls does your call center field on an average day?
8. What other tasks do staff work on besides fielding calls?
9. Does the call center have multiple shifts beyond typical business hours?
PROBE, IF MULTIPLE SHIFTS:
If the call center has multiple shifts, how do staffing levels vary across shifts? (e.g., call center staffed more heavily during normal/9-5 business hours than "after hours).
10. How if at all have staffing levels changed at the call center?
INSTRUCTIONS TO INTERVIEWER
These questions are specific to the roles, responsibilities and functions of staff in specialized online application processing units and will only be asked in states that have such units.
1. Please describe the online application process for SNAP.
2. What types and levels of staff work in the processing unit?
3. Which staff types and levels with directly processing applications?
4. What if any specialization is there to determine what type of applications you may process (e.g., by geography, by purpose of the call, etc.)?
5. What proportion of applications statewide are now handled through online application processing units?
6. What if any changes have online applications had on the number of paper applications handled directly by SNAP staff in local offices?
7. Have these changes caused any shift in the tasks and/or responsibilities of local office SNAP staff?
8. DoSNAP staff in local offices offer support for clients using the online application?
9. If so, how much time is spent on a typical day offering support with the online application?
10. What if any shift has the implementation of the online application caused in the number or proportion of technical/IT staff available to provide assistance with technical aspects of the online application?
11. How if at all have staffing levels changed at both local offices or in specialized units?
12. How if at all does the length to process vary?
13. How many applications are processed on an average day?
14. What other tasks do staff work on besides processing applications?
15. Does the processing unit have multiple shifts beyond typical business hours?
PROBE, IF MULTIPLE SHIFTS:
If the application processing unit has multiple shifts, how do staffing levels vary across shifts? (e.g., unit staffed more heavily during normal/9-5 business hours than "after hours).
INSTRUCTIONS TO INTERVIEWER
These questions are specific to the roles, responsibilities and functions of outstationed staff.
1. What types and levels of staff are outstationed?
2. Where are they outstationed?
3. Are outstationed assignments permanent or temporary?
PROBE, IF TEMPORARY:
If temporary, for how long do outstation assignments typically last?
4. How do the tasks of outstationed workers differ from those performed by staff housed at the local office?
5. How was it decided what type and level of staff would be outstationed and where?
6. Please describe any issues if any with staff turnover of outstationed staff.
INSTRUCTIONS TO INTERVIEWER
These questions are specific to the roles, responsibilities and functions of outstationed staff that conduct telephone interviews.
1. What types and levels of staff conduct telephone interviews?
PROBE:
Do the same staff also conduct in-person interviews?
2. How is it determined which staff will conduct certain interviews?
3. What if any specialization is there to determine what interviews staff may conduct (e.g., by geography, by type of interview (certification, recertification))?
4. How long do certification/recertification interview calls usually last?
5. How many interview calls does a worker conduct on an average day?
6. Do interview staff work over multiple shifts beyond typical business hours?
PROBE, IF MULTIPLE SHIFTS:
If the interview staff work over multiple shifts, how do staffing levels vary across shifts? (e.g., staffed more heavily during normal/9-5 business hours than "after hours).
7. What if any changes has conducting interviews via phone instead of in person caused related to the tasks and/or responsibilities of interview workers?
8. Roughly what proportion of interviews are conducted by telephone?
GO TO SECTION IV
For this next portion of the interview, we will be discussing the implementation and performance of SNAP modernization initiatives.
1. What effect has the use of community partners for outreach or application assistance had on program outcomes?
PROBE, IF MULTIPLE SHIFTS:
Has the use of community partners for outreach or application assistance increased or decreased program participation?
Has the use of community partners for outreach or application assistance increased or decreased application errors?
2. Has the impact of changes brought about by community partners varied by client populations?
1. What criteria are used for determining whether an interview will happen over the phone?
2. Has the use of phone interviews increased program access?
3. Has the use of phone interviews affected different client populations differently?
1. Does the use of the online application vary by client population?
2. Does the use of call centers vary by client population?
3. Does the use of community partners for application assistance/submission vary by client population?
4. Does the use of kiosks or outstations vary by client population?
5. Does the tendency to apply in person in SNAP offices vary by client population?
1. What are the main factors that affect error rates?
2. Does anything else affect error rates?
3. Do the factors affecting error rates vary by application source?
PROBE:
Are applications from [SOURCE] more or less error prone?
SOURCES PROBE:
Online
Telephone
Kiosk
Community Partner
Other?
Why?
4. What explains the [INCREASING/DECREASING] error rate trend in [STATE]?
5. What is the source of most of the errors?
6. How does [STATE] use error data as a performance measure?
7. How does [STATE] respond to errors and work to minimize them?
1. What are the main factors that affect approval/denial rates?
2. Does anything else affect approval/denial rates?
3. Do the factors affecting approval/denial rates vary by application source?
PROBE:
Do applications from [SOURCE] achieve a higher approval/denial rate?
SOURCES PROBE:
Online
Telephone
Kiosk
Community Partner
Other?
Why?
4. What explains the [INCREASING/DECREASING] application denial rate trend in [STATE]?
5. Do you use approval/denial rate data as a performance measure?
1. Are there any unique state or local factors in [STATE] that contribute to the success or challenges with modernization changes?
2. What other factors might account for SNAP performance outcomes?
3. Were there concurrent policy or political changes that might have affected program performance?
4. Were there unique economic factors in [STATE] that might have affected SNAP performance?
1. How have modernization changes overall affected application timeliness?
2. Are changes in timeliness linked to specific modernization initiatives?
3. Did any other initiatives impact application timeliness?
4. Did specific aspects of the implementation of modernization initiatives drive changes in application timeliness?
5. How have modernization changes overall affected application approval/denial rates?
6. Are changes in approval/denial rates linked to specific modernization initiatives?
7. Did any other concurrent initiatives impact approval/denial rates?
8. Did specific aspects of the implementation of modernization changes drive changes in application approval/denial rates?
9. How have modernization changes overall affected error rates?
10. Are changes in error rates linked to specific modernization initiatives?
11. Did any other concurrent initiatives impact error rates?
12. Did specific aspects of the implementation of modernization initiatives drive changes in error rates?
GO TO SECTION V
Next, we’ll be discussing how modernization changes may or may not be obvious to clients and how the change may affect customer service and satisfaction.
1. Which aspects of modernization are clients aware of?
2. To what extent have modernization changes in [STATE] affected customer satisfaction?
PROBE:
How is this tracked or measured?
3. Are changes in customer satisfaction more evident in certain client populations?
4. Which aspects of modernization are most responsible for changes in customer satisfaction?
PROBE:
If respondent has not mentioned call centers, online application, changes which staff clients interact with, or changes in documentation required, ask about the effects these changes have on customer satisfaction.
5. Was there anything about how changes were implemented that caused an increase/decrease in customer satisfaction?
6. What could have been done differently in STATE that to improve satisfaction among applicants/clients?
1. What aspects of modernization have had a positive impact on your job and your level of job satisfaction?
2. Could [STATE] have implemented the changes in other ways that would have improved your experience?
3. Have modernization changes in [STATE] increased or decreased administrative burdens on staff?
4. Which aspects of modernization are most responsible for changes in staff administrative burden?
5. Did any other aspects of modernization impact staff administrative burden?
6. Was there anything about the way changes were implemented that led to changes in staff administrative burden?
7. What could have been done differently in [STATE] that might have led to greater improvements?
GO TO SECTION VI
Now, thinking from a broader perspective….
1. What were the main goals of [STATE]'s modernization changes?
PROBE:
Improving access/participation, reducing staff, reducing burdens on staff, improving customer service?
2. Did STATE have to adjust its goals or expectations about what could be achieved through the implementation of these modernization changes?
What specifically changed?
Why was this necessary?
How was this accomplished?
1. To what extent have modernization changes in [STATE] improved program access/participation?
2. Are improvements in program access/participation more evident in certain client populations?
3. Which aspects of modernization are most responsible for improvements in program access/participation?
4. Are there any other factors or reasons that account for (lack of) improvements in program access/participation?
5. What could or should have been done differently to achieve greater improvements in access and/or participation?
1. To what extent have modernization changes in [STATE] reduced administrative costs?
2. Which aspects of modernization are most responsible for reductions in administrative costs?
Are there any other factors or reasons that account for reductions/increases in administrative costs?
Please discuss.
3. What could or should have been done differently in [STATE] that might have led to greater cost reductions?
INSTRUCTIONS TO INTERVIEWER
Ask the following series of questions about any other goal mentioned above:
1. To what extent have modernization changes in [STATE] improved/affected outcomes/performance related to GOAL?
2. Are improvements in outcomes/performance related to [GOAL] more evident in certain client populations? (if relevant)
3. Which aspects of modernization are most responsible for improvements in outcomes/performance related to [GOAL]?
4. Did any other aspects of modernization impact outcomes/performance related to [GOAL]?
5. What could or should have been done differently to achieve greater improvements in outcomes/performance related to [GOAL].
1. On balance, what is your assessment of the full package of modernization changes?
PROBE:
Do you think they have had an overall positive or negative effect on [Efficiency, Access, Integrity]?
2. Which changes do you think had a positive effect?
PROBE:
Please specify which program outcomes were positively affected and any other positive effects.
3. What are the greatest successes of modernization changes?
4. Which changes do you think had a negative effect?
PROBE:
Please specify which program outcomes were negatively affected and any other negative effects.
5. Do you agree with the goals of modernization?
6. Is there anything you would add, remove, or change in [STATE]'s modernization goals?
7. Has [STATE] addressed its modernization goals in an effective way?
8. What are the biggest challenges to meeting the goals of modernization?
9. Have these challenges been resolved?
If so, how?
10. What could [STATE] have done differently to meet its goals more effectively?
1. What advice would you give to another state that was considering implementing similar modernization changes?
2. Are there particular changes you think are critical for modernization?
3. Are there aspects of modernization you would advise other states to avoid?
4. Is there a particular order in which changes should be undertaken?
5. Is there anything else states should keep in mind when planning, testing, and implementing modernization changes?
GO TO SECTION VII
Is there anything you think is important for the Food and Nutrition Service to know about [STATE]’s SNAP modernization initiatives that we did not ask about?
Thank you for your time and helpful feedback. The information you have shared will be valuable to our team as we look across states and localities for themes and ideas that we can share with FNS.
Are there any questions you have for me before we finish?
END OF INTERVIEW
B.
| File Type | application/msword |
| Author | Lisa Walls |
| Last Modified By | rdowner |
| File Modified | 2010-12-02 |
| File Created | 2010-11-16 |