09-RESEA Cost Study - State Office of UI staff interview protocol

OMB No.: 1290-0NEW 
Expiration: TBD 

RESEA Cost Study Project: Office of Unemployment Insurance Staff Interview Guide

This guide will be used to interview state office of unemployment insurance staff who work on various components of the RESEA program.

AIR Team: Facilitator: Note taker:	Participant Job Position(s):
Date and Time:	Location:

Note to Interviewer:

• Approach the interview with empathy and understanding

• Keep the tone conversational and comfortable

• For each question, be sure to note the time frame that you’re discussing (e.g., during an appointment, in an average week, in the past year).

Introductions

Introductory statement to respondents: We work for the research firm, [Insert Name]. The U.S. Department of Labor has hired us to help them understand the state, local, and participant costs associated with administering the RESEA program. We invited you to speak with us because of your work in the RESEA program as a/an [job title] for [state name].

Today, we will be asking you about the job duties related to the RESEA program performed by you and people you work with as they relate to the RESEA program to better understand the types of state and local resources that are used to run the RESEA program. For example, we would like to learn about your role in supporting RESEA program activities, such as orientation and initial meetings, as well as RESEA staff at this AJC.

Study Background

We will use this information to understand the costs of running the RESEA program and its components. We will also use this information to understand how the costs of running the program compare to any changes in unemployment insurance benefits paid because of claimants’ participation in the RESEA program.

[Interviewer note: Customize this paragraph to focus on the components of interest for the interview.]There are 5 program components that we are interested in learning more about today. The first is RESEA Program Administration, the second is selecting and scheduling RESEA participants, the third is AJC Orientation, the fourth is the Initial RESEA meeting, and the fifth and final component is subsequent RESEA meetings (if applicable).

If the interviewees ask about the study, feel free to share any of the following information:

• The study aims to use the information gathered from interviews, questionnaires, and other sources to understand the RESEA program’s impact on UI benefits as well as the average per-participant costs, component costs, and claimant costs of the RESEA program.

• We are conducting up to 30 interviews with State Unemployment Insurance (UI) and American Jobs Center (AJC) staff who administer, evaluate, or otherwise work on aspects of the RESEA program in your state.

Interview Logistics

Today we’d like to ask about your activities, experiences, and perspectives as staff who work on the RESEA program. The interview will take about 1 hour and 45 minutes to complete.

Before we begin, I want to assure you that all your responses will be used only for this research study. Your name will not appear in any written reports we produce. We will try to describe any responses you give in a way that will make it impossible to identify you. The interview itself is voluntary and you may choose not to answer any specific question.

We may also send you a brief follow-up survey to collect additional information or to clarify the information that you provide today.

would like to record our conversation so we can have a back up to our notes. the recording will remain private we will de-identify your information if we cite quotes from this discussion in our reports by removing references to any personal identifying information. We will erase the recording before the end of the project. Is it okay for us to record?

[If YES, thank the respondent and wait to record until you have confirmed everyone’s name, department and job title.]

Component 1: RESEA Program Administration

Note to interviewer & note-taker: It is very important to take careful note of the time unit (e.g., weekly, monthly, quarterly) used to assess role responsibilities in each question.

Great, we can go ahead and get started now. Throughout the interview we’re going to talk more about your role in the administration of the RESEA program, or your work in managing and overseeing the implementation of RESEA. We anticipate we will touch on topics like how you work with the RESEA program team to ensure program compliance with policy requirements, and eligibility assessments. .

Standard Administration

1. [If conducting the interview with more than one person, ask each individual the following question(s).] Please tell us more about yourself. Please tell us your name, your title,

[Interviewer note: Start recording after this information has been collected. Consider adding an introduction to the recording like: “This is an interview recorded on [date] by interviewer [name].”

And your role in working with the RESEA program?

1. Please tell us a little bit more about what you were doing prior to this job, including your education and employment experience.

2. Probe: Do you need any specific training to perform your job?

1. Could you tell us about your role and responsibilities specifically as they relate to the RESEA program?

   1. Probe: This may vary widely but may include high-level OUI staff liaising with the RESEA Program Administrator to facilitate the evaluation, state plan development, and on-going issues, adjudicating cases due to the claimant not attending RESEA appointments or completing AJC orientation, or data collection from AJCs regarding RESEA Failure to report, attendance, and uptake rates. \

2. Approximately what percentage of your full-time equivalent time do you spend working on RESEA program activities?

3. Do you work individually or as part of a team to carry out your RESEA-related responsibilities?

   1. [If part of a team] How many people work on your team? What are their names and titles?

      1. Approximately what percentage of their full-time equivalent time does each staff member spent working on RESEA program activities?

      2. What activities is each team member responsible for? This could include things like eligibility assessments, adjudicating RESEA-related appeals, or alignment with policy requirement and reporting activities?

      3. Who supervises the RESEA-related work of the team? In a typical week, how many hours do they spend supervising other OUI staff who work on RESEA-related activities?

         1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do they spend on supervising other OUI staff or contractors who work on RESEA-related activities?

   2. In a typical month, how many hours do you spend liaising with Employment Services staff about the RESEA program? This may include efforts to support reporting, compliance, selection and scheduling, or other program requirements.

4. [If relevant to respondent’s role and responsibilities] What kinds of RESEA communications materials do claimants who are selected for RESEA receive from OUI?

   1. In a typical week, about how many hours do [you or your team] spend creating, updating, or distributing RESEA communication materials? This can include things like RESEA selection letters or UI disqualification letters due to RESEA non-compliance.

      1. [If staff are unable to give an hours estimate] In a typical week, what percentage of time do [you or your team] spend creating, updating, or distributing RESEA communication materials? This can include things like RESEA selection letters or UI disqualification letters due to RESEA non-compliance.

      2. [If part of a team] Who on your team does this?

         1. [for each individual mentioned] About how many hours do they spend creating, updating, or distributing RESEA communication materials?

      3. What materials do [you or your team] use to do this?

         1. For each material, please describe the cost.

2. What materials do you use to develop and create-related communications?

   1. For each material, please describe the cost.

3. Are [you or your team] responsible for sending letters and/or other RESEA communications to participants?

   1. [if yes] In a typical week, about how many hours do [you or your team] spend sending communications to participants?

      1. [If staff are unable to give an hours estimate] In a typical week, what percentage of time do [you or your team] spend sending communications to participants?

   2. [if part of a team] Who on your team does this?

      1. [for each individual mentioned] About how many hours do they spend sending communications to participants?

   3. What materials do you use to send communications? This could include things like printing costs, stamps, and envelopes.

      1. For each material, please describe the cost.

6. [If relevant to respondent’s role and responsibilities] Do you or your team conduct eligibility reviews for claimants selected for the RESEA program?

   1. [If yes] What does the process for conducting a review look like?

   2. [If yes] In a typical week, what percentage of time do [you or your team] spend trying to connect with each claimant to conduct the review each week?

      1. [If staff are unable to give a percentage estimate] For a typical claimant, about how many minutes do [you or your team] spend trying to connect with the claimant to conduct the review?

      2. [If part of a team] Who on your team does this? How many people do this?

         1. [for each individual mentioned] About how many hours do they spend on this?

      3. How many claimants does each staff member typically attempt to contact per week?

   3. [If yes] In a typical week, what percentage of time do [you or your team] spend conducting eligibility reviews with claimants each week?

      1. [If staff are unable to give a percentage estimate] For a typical claimant, about how many minutes do [you or your team] spend conducting an eligibility reviews?

      2. [If part of a team] Who on your team does this? How many people do this?

         1. [for each individual mentioned] About how many hours do they spend creating, updating, or distributing RESEA communication materials?

      3. How many claimants does each staff member typically conduct eligibility reviews with each week?

   4. [If yes] In a typical week, about how many hours do you or your team spend entering data following your eligibility reviews?

   5. What materials are used to conduct RESEA-related eligibility reviews?

      1. What is the dollar value of each of these materials?

7. [If relevant to respondent’s role and responsibilities] Are you and your team responsible for adjudicating UI claims after they’ve received benefit disqualifications/payment pauses that are due to RESEA non-compliance?

   1. [If yes] For a typical claimant who received benefit disqualifications or payment pauses due to RESEA non-compliance, how many hours do you spend adjudicating their UI claim?

      1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend adjudicating benefit disqualifications or payment pauses due to RESEA noncompliance in a typical week?

      2. [If part of a team] Who does this?

         1. [for each individual mentioned] About how many hours do they spend on this?

   2. [If yes] In a typical week, how many claimants go through adjudication due to benefit disqualifications/payment pauses that are due to RESEA non-compliance?

   3. What materials are used to conduct RESEA-related adjudications?

      1. What is the dollar value of each of these materials?

8. [If relevant to respondent’s role and responsibilities] Are you and your team responsible for hearing appeals from UI claimants after they’ve received benefit disqualifications/payment pauses that are due to RESEA non-compliance?

   1. [If yes] For a typical claimant who received benefit disqualifications or payment pauses due to RESEA non-compliance, how many hours do you spend on appeals related to their claim?

      1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend on appeals related to benefit disqualifications or payment pauses due to RESEA noncompliance in a typical week?

      2. [If part of a team] Who does this?

         1. [for each individual mentioned] About how many hours do they spend on appeals related to benefit disqualifications or payment pauses due to RESEA noncompliance?

   2. [If yes] In a typical week, how many claimants appeal benefit disqualifications/payment pauses that are due to RESEA non-compliance?

   3. What materials are used to conduct RESEA-related appeals of UI disqualifications?

      1. What is the dollar value of each of these materials?

Alignment with Policy Requirements and Reporting Activities

9. [If relevant to respondent’s role and responsibilities] How does [State] ensure that your RESEA program aligns with state and federal laws?

   1. Probe: This could include conducting eligibility assessments, adjudication, engaging in required quarterly/annual federal reporting, etc.

   2. Please describe all of these activities that [you or your team] engage in to support ensure your state is aligned with policy requirements.

   3. Which staff member engages in each activity?

      1. On a monthly basis, how many hours does each individual spent on each of these activities?

         1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do each staff member spend on the activities noted above?

10. Please describe all of these activities that [you or your team] engage in to support program reporting.

    1. Which staff member engages in each activity?

       1. On a monthly basis, how many hours does each individual spend on each of the reporting-related activities they work on?

       2. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do each staff member spend on the activities noted above?

    2. What materials are used to conduct RESEA-related alignment with policy requirements and reporting activities?

       1. What is the dollar value of each of these materials?

RESEA Program Data Collection

11. [If relevant to respondent’s role and responsibilities] In a typical month, about how many hours do you spend collecting, organizing, and cleaning data for claimants who have been selected for RESEA?

    1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend collecting, organizing, and cleaning RESEA data?

       1. [If part of a team] Who does this?

          1. [for each individual mentioned] About how many hours do they spend on this?

    2. What software do you use to support data collection?

       1. What is the dollar value of any licensing fees or other costs related to these software?

12. [If relevant to respondent’s role and responsibilities] What types of data collection systems or databases do you maintain?

    1. In a typical month, about how many hours do [you or your team] spend maintaining various databases and data collection systems? This can include but may not be limited to UI application data, UI claims and payment data, RESEA services records, wage records, etc.

       1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend maintaining various databases and data collection systems?

       2. [If part of a team] Who does this?

          1. [for each individual mentioned] About how many hours do they spend on maintaining various databases and data collection systems?

       3. Of the time you just described, what percentage of that time do you think is used specifically for RESEA-related data systems?

    2. What software, cloud storage, or other materials do you use to support data collection systems or databases?

       1. What is the dollar value of any licensing fees or other costs related to these software?

13. [If relevant to respondent’s role and responsibilities] In a typical month, about how many hours do [you or your team] spend responding to data requests related to RESEA?

    1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend responding to data requests or reporting related to RESEA?

    2. [If part of a team] Who does this?

       1. [for each individual mentioned] About how many hours do they spend on this?

14. What other materials are used to conduct RESEA-related data collection?

    1. What is the dollar value of each of these materials?

Component 2: Selecting and Scheduling RESEA Program Participants

Note to interviewer & note-taker: It is very important to take careful note of the time unit (e.g., weekly, monthly, quarterly) used to assess role responsibilities in each question.

Thank you for answering those questions! The second RESEA program component we’d like to learn more about is the process of Selecting and Scheduling RESEA Program Participants. So, we’re going to be talking about your role in things like creating and maintaining the RESEA Selection Model and scheduling participants for their RESEA appointments.

15. [If relevant to respondent’s role and responsibilities] Broadly, how are claimants selected and scheduled for RESEA meetings in your state?

    1. How many staff members are involved in this process? What are their titles?

    2. What are their roles in supporting scheduling claimants’ RESEA meetings?

16. [If relevant to respondent’s role and responsibilities] What type of RESEA selection model does [state name] use? Do you use your WPRS profiling model to select claimants for RESEA or do you have a separate RESEA selection model?

    1. [If use selection model that is separate from WPRS] About how much time did [you or your team] spend creating and testing the RESEA selection model?

       1. Who was involved in creating and testing your RESEA selection model? What are their titles?

          1. Approximately how many hours did each individual spend creating and testing the RESEA selection model? An estimate is fine.

       2. When was your model last updated? How often is your model typically updated? [if unable to answer] Could you provide the rough dates of the last two updates to your model?

    2. In a typical month, how many hours do you spend maintaining your selection model and/or reviewing its output (for example, making sure it is communicating well with the scheduling system and running smoothly)?

       1. [If staff are unable to give an hours estimate] In a typical month, what percentage of time do [you or your team] spend maintaining your selection model and/or reviewing its output?

       2. [If part of a team] Who does this?

          1. [for each individual mentioned] About how many hours do they spend maintaining your selection model and/or reviewing its output?

17. What software, cloud storage, or other materials do you use to support your RESEA selection model?

    1. What is the dollar value of each of these materials?

18. Do [you or your team] work on state scheduling systems? If so, please tell us what that scheduling system looks like, and what is involved in managing it.

    1. [If yes and scheduling system is automated and maintained by state] In a typical week, how many hours do [you or your team] spend ensuring that selection model results are appropriately transmitted to your RESEA scheduling system?

       1. [If staff are unable to give an hours estimate] In a typical week, what percentage of time do [you or your team] spend ensuring that selection model results are appropriately transmitted to your RESEA scheduling system?

       2. [If part of a team] Who does this?

          1. [for each individual mentioned] About how many hours do they spend on this?

       3. What materials do you use to do this?

          1. For each material, please describe the cost.

    2. [If yes] What software, data cloud storage, or other materials do you use to support scheduling RESEA meetings in [state]?

       1. What is the dollar value of each material?

19. In a typical week, how many hours do [you or your team] spend helping AJCs troubleshoot scheduling issues? This may include issues pulling eligible claimants from the scheduling pool, helping them figure out technology issues, etc.?

    1. [If staff are unable to give an hours estimate] In a typical week, what percentage of time do [you or your team] spend helping AJCs troubleshoot scheduling issues?

    2. [If part of a team] Who does this?

       1. [for each individual mentioned] About how many hours do they spend on this?

20. In a typical week, how many hours do [you or your team] spend providing other guidance, training, and/or scheduling support to AJCs?

    1. [If staff are unable to give an hours estimate] In a typical week, what percentage of time do [you or your team] spend providing general scheduling support to AJCs?

    2. [If part of a team] Who does this?

       1. [for each individual mentioned] About how many hours do they spend on this?

    3. What materials do you use to provide this support?

       1. For each material, please describe the cost.

Wrap up

1. What factors do you consider when allocating RESEA funds?

a. How do you decide which specific RESEA activities or interventions to invest in or scale up/down?

b. How does this decision-making process take into account fiscal effectiveness, efficiency, and expected impact?

c. Who is involved in the allocation decision making process?

2. What metrics or data do you rely on to guide your allocations?

a. What challenges have you faced in recording or using metrics/data related to costs or cost-effectiveness? [If yes: what steps did you take to resolve these challenges?]

3. In what ways, if any, do you braid funding from multiple sources?

4. What external factors (like state budgets) impact your resource allocation?

5. How does your organization assess return on investment for the RESEA program?

6. Is there anything else you would like to share with us that we have not already covered?

Thank you very much for your time! We found your input very helpful. If you have any questions or concerns, or if you would like to share anything with us that we did not discuss, please feel free to contact us.

Public reporting burden for this data collection instrument is estimated to average 1 hour and 45 minutes per response. The burden estimate includes the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and submitting the survey. This collection of information is voluntary. You are not required to respond to this collection of information unless it displays a valid OMB control number.  Please send comments regarding the burden estimate or any other aspect of this collection of information to the U.S. Department of Labor, Chief Evaluation Office, 200 Constitution Ave NW, Washington, DC 20210, or ChiefEvaluationOffice@dol.gov and reference OMB control number 1290-0NEW.