01-RESEA Component Study - program manager interview protocol

OMB No.: 1290-0NEW 
Expiration: TBD

RESEA Interview Protocol – AJC Leadership

Introductory statement to respondents: We work for the research firm, Abt Global. The U.S. Department of Labor has hired us to conduct a multi-state impact evaluation of RESEA, which [INSERT NAME OF STATE] is participating in. We are examining the impact of two RESEA program components: 1) single vs. multiple RESEA one-on-one meetings and 2) meeting modality (in-person vs. remote). The study will examine the impact on Unemployment Insurance (“UI”) duration and increasing employment.

This site visit is part of the evaluation and is designed to understand how RESEA works in [INSERT NAME OF STATE], with a focus on how single and subsequent RESEAs and meeting modality are implemented.

Study Background

Over the next few months, we will be visiting [YY] [insert local name of AJCs—e.g. career source centers ] where RESEA program services are delivered. These visits are designed to help us learn about how the RESEA program and the meeting modality and subsequent RESEA components operate. We will explore topics such as, RESEA program flow, approaches to service delivery, customers’ experiences in the program.

If the interviewees ask about the study, feel free to share any of the following information:

• We plan to visit [YY] [insert local name of AJCs—e.g. career source centers ] where RESEA services are offered.
  
  
  

• At each site, we try to speak with (i) staff who deliver RESEA services, (ii) Career Center leadership, and iii) RESEA participants. We also plan to observe the implementation of RESEA service delivery (i.e., RESEA one-on-one sessions, and reemployment workshops or other referral services).

Interview Logistics

Today we’d like to ask about your activities, experiences, and perspectives as staff who work on the RESEA program. The interview will take about 60 minutes to complete.

Before we begin, I want to assure you that all your responses will be used only for this research study. Your name will not appear in any written reports we produce, and your co-workers will not know your responses. We will try to describe any responses you give in a way that will make it impossible to identify you. The interview itself is voluntary and you may choose not to answer any specific question.

Do you have any questions before we get started? If you have questions or concerns about today’s interview, specifically your rights as participant in this research, you can call the Abt Global Institutional Review Board (toll-free 877-520-6835).

We would like to record our conversation so we can have a back up to our notes. The recording will remain private. We will de-identify your information if we cite quotes from this discussion in our reports by removing references to any personal identifying information. We will erase the recording before the end of the project. Is it okay for us to record?

[If yes, thank the respondent and wait to record until you have confirmed everyone’s name, department and job title in the background section.]

Key Topics for Discussion with Local Leadership:

[Notes to Interviewer: 1) The interview is intended to be semi-structured and conversational. Though you do not need to follow the order of questions below—especially if the respondent jumps around with his/her responses—please try to collect information corresponding to each prompt. It is not necessary to ask every question in the allotted time, but please try to only exclude those questions that you determine are not relevant to your respondent. The solid black bullet points are the primary questions, and the sub-bullets identified as “Probe” are meant to encourage conversation when you determine necessary. You should not need to ask all the probing questions. 2) Please monitor the time and be sure to cover the topics in section 9 (innovations and challenges) so there is sufficient time for the respondent to respond to the topics.]

Background on Respondents

[Note to Interviewer: Collect the following information on each respondent involved in the interview; remind them that we will not use their name or any identifying information in any report or discussion with state staff.]

• Name

• Agency/division/unit

• Title and tenure in that position

[Interviewer note: Start recording after this information has been collected. Consider adding an introduction to the recording like: “This is an interview recorded on [date] by interviewer [name].”

• Role in RESEA program and tenure on that program

• Other roles at the AJC (other than related to RESEA)

1. Background on the Workforce Board and Career Center

We would like to start with a little background on the career center and this local workforce area.

State RESEA operations/leadership (if applicable)

• We know from past conversations in YYY [insert summary, if applicable, of what we know about state’s involvement in allocating RESEA funding across local areas AND guiding local implementation.] Is that accurate? Has anything changed about this process?

• Are there other ways the state involved with overseeing RESEA services at the AJC level?

Career Center Background (if applicable)

• Who/what type of organization operates the AJC (e.g., public, nonprofit)?

• How long has this organization been the operator of the AJC?

• What geographic area does the AJC serve?

• Are there any key partners/organizations co-located at the AJC?

• About how many customers does the AJC serve each month? Each year?

  • If available, what proportion of customers served are UI claimants?

Workforce Board Background (if applicable)

• In what ways is [name of local workforce board] Board involved in the RESEA program?

  • Probe: Does the Board set policies or make recommendations that affect program implementation?

  • Probe: Does the Board facilitate partnerships that affect RESEA implementation?

  • Probe: Does the Board monitor your program, observe the components of your program, ensure compliance with state policies, and address any challenges?

2. Program Context

The next several questions are about the local economy and labor market in this area. We want to understand the local context in which RESEA operates here.

• What are the local labor market conditions like in this area?

  • Probe: Which are the primary industries and major employers?

  • Probe: Have there been any major changes to the labor market, like new employers or mass layoffs?

  • Probe: Are there any seasonal patterns to employment in the area?

• Is there anything about this local area that poses challenges to serving RESEA customers?

  • Probe: Are there any barriers due to the urban/rural setting, transportation issues, or internet access?

• How do these local conditions inform implementation of the RESEA program?

  • Probe: Do you make special accommodations for customers who have to travel or have limited internet access?

  • Probe: Does the program target job referrals based on industry changes?

  • Probe: Do you have to change scheduling procedures (e.g., AJC hours, staffing) to accommodate seasonal swings in unemployment?

3. Program Goals

Now we would like to shift to more detail about the RESEA program as it operates today. First, we would like to start with a high-level discussion of the RESEA program’s goals.

• How would you describe the goals of the RESEA program?

  • Probe: How do you define success for the RESEA program?

  • Probe: In what ways does your RESEA program seek to affect participants?

4. Staffing and Supervisory Structure

The next set of questions are focused on how you staff the RESEA program.

• How many and what type of staff does the AJC have?

• How many staff are involved in implementing the RESEA program and how are they organized around program components?

  • Probe: Are staff dedicated to specific activities, like scheduling, orientations, one-on-one sessions, or eligibility reviews?

  • Probe: How are the line staff supervised?

• How many RESEA claimants does each staff person typically work within a week?

Staff Experience and Training

• What experience level do the staff that conduct the one-on-one sessions usually have?

• How long have they been conducting RESEAs? (Please let us know person by person)

• Does the AJC provide any required trainings for the staff that conduct the one-on-one training? If yes, please describe.

• Are the staff dedicated to RESEA only or are they cross-trained on other programs?

  • Probe: Why do you use that staffing model?

• Do any staff at the AJC have access to the state’s UI system, and if so, for what purpose?

Staffing Structure

• Who employs the staff who work on RESEA?

  • Probe: Are the staff who work on RESEA employees of the state or employees of a private contractor (e.g., the AJC operator) or a mix of both?

  • Probe: Are the staff who work on RESEA considered state merit staff?

• What, if any, challenges do you face with staffing the program?

  • Probe: Is there a lot of turnover among line staff? If so, what are the effects on program operation?

  • Probe: Do you have any difficulty hiring staff?

  • Probe: Are there any challenges with training new staff?

  • Probe: Did the pandemic affect staffing (e.g., higher turnover, types of staff that provide services)?

• What role, if any, do UI staff play in RESEA implementation?

  • Probe: Do they deliver eligibility reviews?

  • Probe: Do they train RESEA program staff?

  • Probe: Do they deliver a portion of the CCS orientation?

5. RESEA Program Operations

We have several questions about your perspectives on RESEA and the different component parts of the RESEA program, namely the Initial RESEA meeting and subsequent RESEA meetings [Customize as needed, 1) some states may have a different name for “initial” and “subsequent RESEA meetings” and 2) some states may have additional steps/components, too ].

• About how many RESEA participants does the program serve each week in this AJC? How consistent is that month-to-month?

• Which elements of the RESEA meeting(s) would you say are key for achieving its intended goals, and why?

[BELOW LIST IS GENERAL AND SHOULD BE CUSTOMIZED, AS NEEDED]

1. Orientation to career services

2. Customized labor market information

3. Co-enrollment in employment services

4. Creation of employment plan

5. Identification of barriers to employment?

6. Staff support or encouragement for claimants?

7. Connection to training opportunities?

   1. Approximately, how often do you refer a participant to training?

8. Connection to other services?

   1. What are the most common services?

   2. Approximately how often do you refer people to it?

• What, if anything, would you change about the different parts of the program?

  • Probe: Additions to the CCS? Initial RESEA? Subsequent RESEA?

  • Probe: Should anything be removed from CCS? Initial RESEA? Subsequent RESEA?

  • Probe: Changes to integration between UI and partner programs?

  • Probe: Longer-term follow-up?

  • Probe: More services that customers can access on their own?

6. Subsequent RESEA meetings

[Interviewer include notes here that describe the subsequent meeting content and structure that has been implemented in this state for reference.]

• Please describe the subsequent RESEA meetings.

  • Probe: What content is covered?

• Who usually conducts the subsequent RESEA meetings?

• When, relative to the Initial RESEA, are the subsequent meetings scheduled to be held?

• Approximately what portion of subsequent RESA meetings are conducted by the same staff that conducted the initial RESEA meeting?

• How long do they last?

• How many subsequent RESEA meetings could a participant be expected to attend?

• Are all claimants scheduled for subsequent meetings, and if not, how is it decided whether a claimant is scheduled for a subsequent meeting?

• What if anything would you change about the subsequent RESEA meetings? And Why?

7. RESEA meeting modality

• Which components, if any, of your program can be delivered remotely?

  • Probe: Orientation, Initial RESEA, Subsequent RESEA, Workshops

• What share of customers would you say you serve remotely vs in person?

• For remote, what portion would you estimate are conducted with video/cameras on?

• In your experience, what are the pros and cons of remote vs in-person service delivery?

  • Probe: Related to attendance?

  • Probe: Related to engagement or attentiveness at meetings?

  • Probe: Related to quality of service delivery?

• Do you think remote delivery is better suited for some services over others?

• How does remote delivery affect operational efficiency, customer service, and performance?

8. Claimant Non-compliance with RESEA Requirements

We have several questions about non-compliance, and by that we mean instances in which RESEA participants do not meet program requirements (e.g., being able, available, and actively seeking work) or RESEA program requirements (e.g., not attending a scheduled meeting).

Non-compliance with RESEA Attendance

• What are the main reasons for failure to report (FTR) for the Initial RESEA session? What about the RESEA subsequent meetings [These terms may be customized]?

• Are there any strategies in place for reducing the (FTR) to scheduled RESEA activities?

  • Probe: Reminder calls in advance of the Orientation, Initial RESEA, and Subsequent RESEA Sessions?

  • Probe: Changes to scheduling strategies and meeting times?

• About what percentage of claimants scheduled for Initial RESEA sessions fail to report? What about the RESEA subsequent meetings [These terms may be customized]?

• For RESEA sessions, is the FTR rate fairly consistent week to week or is there a lot of variation?

9. Innovations and Challenges

We would like to step back now and ask you to reflect on your perspectives on any challenges you encounter with respect to RESEA implementation and any innovations or unique elements of the program.

Innovations

• Are there any elements of your RESEA program that you feel are unique or otherwise notable/innovative?

• Are there any staffing, technology, or process innovations or practices you’ve adopted that have enabled significant improvement in RESEA processes or participant outcomes?

Challenges

• What, if any, are your biggest challenges in operating the RESEA program currently?

  • Probe: Funding, staffing, or resource limitations

  • Probe: Local economic conditions

  • Probe: UI fraud (identity issues, no shows, etc.)

  • Probe: Ongoing effects from the pandemic

• What, if any, challenges do claimants face when participating?

  • Probe: Difficulty completing any pre-orientation requirements or required forms in advance?

  • Probe: Are participants able to fully participate in remote meetings?

  • Probe: Are the program materials accessible to participants?

• Is there anything you would change about RESEA, such as how it is structured, the content provided, the duration, etc.?

10. Closing

• Is there anything about the RESEA program that we did not ask about that you think is important to discuss?

• Given our interview today, would you recommend anyone else we should talk to for any additional or relevant information?

And lastly, do you have any questions or clarifications for us?

We appreciate your time and your input! Thank you!

Public reporting burden for this data collection instrument is estimated to average 1 hour per response. The burden estimate includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing the interview. This collection of information is voluntary. You are not required to respond to this collection of information unless it displays a valid OMB control number.  Please send comments regarding the burden estimate or any other aspect of this collection of information to the U.S. Department of Labor, Chief Evaluation Office, 200 Constitution Ave NW, Washington, DC 20210, or ChiefEvaluationOffice@dol.gov and reference OMB control number 1290-0NEW.