U.S. DEPARTMENT OF LABOR
REEMPLOYMENT SERVICES AND ELIGIBILITY ASSESSMENTS (RESEA) – STATE SURVEY
DRAFT INSTRUMENT
In the last year, legislative and administrative action has significantly changed the RESEA program. With the FY 2019 RESEA operating guidance (UIPL 07-19), the U.S. Department of Labor (DOL) afforded states greater flexibility in targeting RESEA service delivery to claimant populations. The Bipartisan Budget Act of 2018 (Public Law 115-123), which amended the Social Security Act, authorized substantially increased funding for RESEA programs, providing states with new opportunities to rethink the types and levels of services provided to claimants through the RESEA program. The statute also introduced a requirement that the funding be used for evidence-based interventions, and requires evaluation of interventions that are not evidence based.
DOL’s Chief Evaluation Office, in close collaboration with the Office of Unemployment Insurance, has contracted with Abt Associates and its partners the Urban Institute, Capital Research Corporation, and NASWA, to help implement these statutory requirements. In order to understand how well existing evidence supports current interventions used by states, this project is conducting an implementation study to give DOL a detailed, systematic understanding of the components of states’ current RESEA programs and how programs have changed in response to new statutory rules and requirements. That understanding will help DOL identify what evidence needs to be built in support of those program components.
We are asking state RESEA program leads to complete the following survey, as part of the implementation study, to help develop that understanding. The survey will ask you to reflect on your understanding of current RESEA program operations and recent changes you have made to the program’s design. It also asks you to identify operational challenges you have faced and recommendations you would make to other RESEA program leads. Last, you will be asked about your state’s evaluation experience and capacity, in order to help guide DOL decisions about strategies for developing new evidence.
We value your privacy and will not share any answers related to your personal perceptions, opinions or recommendations. Those answers will be kept private and will not be shared with other staff at your program or organization, with staff from other states, or with DOL. Only the evaluation team will have access to that kind of subjective information you provide through this survey. Objective responses that you provide describing your state’s RESEA program may be used in reports that do identify your state, but will be reported in aggregate. For example, a report may list the states that plan to change how they target claimants for selection for RESEA.
Your name will not be listed in any reports published and no comments will be attributed to you. Your responses to these questions are also completely voluntary. We hope you will choose to complete all of the questions on the survey, but you may choose to skip any question you do not feel comfortable answering. If you are unable to answer all of these questions on your own, you may request assistance from colleagues as necessary. The email you received included a link to the web-based survey; that link can be used by others to access and complete all or portions of the survey. If you have questions about completing this survey, you can contact the study team at resea@abtassoc.com. Thank you in advance for your assistance and providing important information to the study.
This section asks you to identify any changes your state has made to your RESEA program in response to recent legislative and administrative action that affects the RESEA program. You will also be asked about upcoming changes your state plans to make.
Topic 1.a. Changes Made to RESEA
1.a.1 Thinking about the recent changes in RESEA program rules and funding – for example, increased funding from Congress, new evidence-based requirements, and new flexibility in targeting services to claimants – which aspects of your RESEA program have you changed in FY2019 or FY2020 in response?
(Select all that apply.)
1 Changed approach to targeting RESEA services to claimants
2 Changed service delivery design
3 Changed size and/or geographic coverage area of RESEA program
4 Changed RESEA staffing or resources
5 Other (please specify) ________
6 We have not changed the RESEA program
[IF RESPONDENT ANSWERED “1” TO 1.a.1, CONTINUE TO 1.a.2, ELSE SKIP TO 1.a.3]
1.a.2 You said your state has changed your approach to targeting RESEA services to claimants. Please identify the changes you have made:
(Select all that apply.)
1 Focused on claimants with a lower range of profiling scores/risk of benefit exhaustion
2 Implemented a random selection process
3 Changed rules for selection of veteran claimants
4 Incorporated local labor market information and economic trends
5 Other (please specify) ____________
[IF RESPONDENT ANSWERED “2” TO 1.a.1, CONTINUE TO 1.a.3, ELSE SKIP TO 1.a.4]
1.a.3 You said your state has changed your RESEA service delivery design. Please identify the changes you have made:
(Select all that apply.)
1 Offered more individualized reemployment services to claimants who are selected
2 Increased the number of RESEA meetings with claimants who are selected
3 Assign more staff to work on each case
4 Made more referrals to workshops or other services for claimants who are selected
5 Conduct more follow-up with claimants to monitor progress finding a job
6 Performed more intensive verification of eligibility for claimants who are selected
7 Added or expanded virtual services
8 Other (please specify) ____________
[IF RESPONDENT ANSWERED “3” TO 1.a.1, CONTINUE TO 1.a.4, ELSE SKIP TO 1.a.5]
1.a.4 You said your state has changed the size and/or geographic scope of your RESEA program. Please identify the changes you have made:
(Select all that apply.)
1 Expanded RESEA to new locations in your state
2 Increased the number of participants served in existing RESEA locations
3 Other (please specify) ____________
[IF RESPONDENT ANSWERED “4” TO 1.a.1, CONTINUE TO 1.a.5, ELSE SKIP TO 1.b.1]
1.a.5 You said your state has changed your RESEA program’s staffing and/or resources. Please identify the changes you have made:
(Select all that apply.)
1 Hired new RESEA staff
2 Allocated more staff time to RESEA
3 Purchased new equipment
4 Developed or revised data systems for RESEA
5 Other (please specify) ____________
Topic 1.b. Changes Under Consideration for RESEA
1.b.1 Does your state have definitive, official plans that you have not yet implemented to change your RESEA program in response to changes in RESEA program rules and funding?
(Please select one.)
1 Yes
2 No [SKIP TO 2.a.1]
1.b.2 Which of the following changes are you considering?
(Select all that apply.)
1 Changing approach to targeting RESEA services to claimants
2 Changing service delivery design
3 Changing size and/or geographic coverage area of RESEA program
4 Changing RESEA staffing or resources
5 Other (please specify) ________
6 We are not considering changes to the RESEA program
[IF RESPONDENT ANSWERED “1” TO 1.b.2, CONTINUE TO 1.b.3, ELSE SKIP TO 1.b.4]
1.b.3 You said your state plans to change your approach to targeting RESEA services to claimants. Please identify the changes you plan to make:
(Select all that apply.)
1 Focus on claimants with a lower range of profiling scores/risk of benefit exhaustion
2 Implement a random selection process
3 Change rules for selection of veteran claimants
4 Incorporate local labor market information and economic trends
5 Other (please specify) ____________
[IF RESPONDENT ANSWERED “2” TO 1.b.2, CONTINUE TO 1.b.4, ELSE SKIP TO 1.b.5]
1.b.4 You said your state plans to change your RESEA service delivery design. Please identify the changes you plan to make:
(Select all that apply.)
1 Offer more individualized reemployment services to claimants who are selected
2 Increase the number of RESEA meetings with claimants who are selected
3 Assign more staff to work on each case
4 Make more referrals to workshops or other services for claimants who are selected
5 Conduct more follow-up with claimants to monitor progress finding a job
6 Perform more intensive verification of eligibility for claimants who are selected
7 Add or expand virtual services
8 Other (please specify) ____________
[IF RESPONDENT ANSWERED “3” TO 1.b.2, CONTINUE TO 1.b.5, ELSE SKIP TO 1.b.6]
1.b.5 You said your state plans to change the size and/or geographic scope of your RESEA program. Please identify the changes you plan to make:
(Select all that apply.)
1 Expand RESEA to new locations in your state
2 Increase the number of participants served in existing RESEA locations
3 Other (please specify) ____________
[IF RESPONDENT ANSWERED “4” TO 1.b.2, CONTINUE TO 1.b.6, ELSE SKIP TO 1.b.7]
1.b.6 You said your state plans to change your RESEA program’s staffing and/or resources. Please identify the changes you plan to make:
(Select all that apply.)
1 Hire new RESEA staff
2 Allocate more staff time to RESEA
3 Purchase new equipment
4 Develop or revise data systems for RESEA
5 Other (please specify) ____________
1.b.7 Do you expect the size of your RESEA grant to increase further in the coming two-to-three years?
(Please select one.)
1 Yes
2 No
3 I don’t know
[IF RESPONDENT ANSWERED “Yes” TO 1.b.7, CONTINUE TO 1.b.8, ELSE SKIP TO 2.a.1]
1.b.8 How do you expect to use any increased RESEA funding in the coming two-to-three years?
(Select all that apply.)
1 Serve more claimants
2 Expand RESEA to more geographic areas
3 Hire more staff
4 Purchase new equipment
5 Evaluate or conduct other research on the RESEA program
6 Other (please specify)______________
7 I don’t know
This section asks several questions about your state’s policies regarding selecting claimants to participate in the RESEA program. It also includes questions about selection practices at local offices.
2.a.1 Excluding UCX claimants, which of the following most accurately reflect the types of claimants that your state’s RESEA selection process currently targets?
(Please select one.)
1 Claimants who are deemed most likely to exhaust UI benefits
2 Claimants who are deemed somewhat more likely than average to exhaust UI benefits
3 Claimants who are deemed somewhat less likely than average to exhaust UI benefits
4 Claimants who are deemed least likely to exhaust UI benefits
5 Claimants who have both a high risk of UI benefit exhaustion and other characteristics that make them an appropriate participant
6 Selection for RESEA does not depend on how likely they are to exhaust UI benefits
2.a.2 The logic of selecting claimants with high risk of benefit exhaustion is that RESEA can have a larger impact on the outcomes of those claimants than on other claimants. In your opinion, which of the following types of claimants would benefit most—in terms of better employment outcomes and reduced UI duration—from being selected for RESEA?
(Please select one.)
1 Claimants who are most likely to exhaust UI benefits
2 Claimants who are somewhat more likely than average to exhaust UI benefits
3 Claimants who are somewhat less likely than average to exhaust UI benefits
4 Claimants who are least likely to exhaust UI benefits
5 All claimants are similarly likely to benefit, regardless of how likely they are to exhaust UI benefits
2.a.3 In some states, AJCs may be referred more claimants than they can serve right away. Does this occur in your state?
(Please select one.)
1 Yes
2 No [SKIP TO 3.a.1]
2.a.4 In your experience, which of the following most accurately characterizes how AJCs typically decide which referred claimants to schedule an initial RESEA for?
(Please select one.)
1 Local offices schedule those with the highest profiling scores first
2 Local offices schedule those with lower profiling scores first
3 Local offices choose claimants using some approach that is not related to their background or risk of benefit exhaustion (e.g., in alphabetical order, by last four SSN digits, using a random number generator in Excel, etc.)
4 I don’t know
2.a.5 In your experience, what other factors do AJCs most often use in deciding which referred claimants to schedule an initial RESEA for if they do not have the capacity to schedule all of them?
(Select all that apply.)
1 Veteran status
2 Prior participation in AJC services
3 Geographic proximity of claimants’ residence to the AJC
4 Claimants’ prior occupation or industry
5 Other (please specify) _____________
6 I am not aware of any other factors
2.a.6 In your experience, to what extent do AJCs vary in their approach to selecting participants for whom to schedule an initial RESEA meeting?
(Please select one.)
1 There is no local variation. All local offices select RESEA participants in the same way. [SKIP TO 3.a.1]
2 There is some local variation. Most local offices follow similar procedures, with slight differences.
3 There is substantial variation. Local offices follow a wide range of selection procedures.
4 I don’t know [SKIP TO 3.a.1]
2.a.7 In what ways do local offices vary in their approach to selecting claimants to participate in RESEA?
(Select all that apply.)
1 Local offices set different policies for using likelihood of exhausting UI benefits to select claimants
2 Local offices set different policies for selection of specific groups (e.g., Veterans)
3 Some local offices select claimants based on arbitrary factors like alphabetical order or SSN, while other offices do not
4 Other (please specify) __________
5 I don’t know
This section asks a series of questions related to the how your RESEA program is currently implemented, including topics related to scheduling claimants, conducting RESEA meetings, and delivering in-person services.
Topic 3.a. Claimant Notification
3.a.1 How many weeks into the claim does selection for RESEA typically occur?
(Please select one.)
1 Week 2
2 Week 3
3 Week 4
4 Week 5
5 Week 6 or later
3.a.2 Once selected, how are UI claimants notified of that selection and their participation requirements for RESEA?
(Select all that apply.)
1 Email
2 Telephone
3 Mailed letter
4 Online (UI website, mobile app)
Topic 3.b. Meeting Scheduling
3.b.1 How soon after notification does the initial meeting occur?
(Please select one.)
1 1 week
2 2 weeks
3 3 weeks
4 4 weeks or more
3.b.2 On average, in which claim week would you estimate that initial RESEA meetings are held?
(Please enter an integer.)
Week _______
3.b.3 Which of the following most closely describes your state’s method for scheduling the initial RESEA meeting?
(Please select one.)
1 State or AJC staff schedule the initial meeting. Claimants can notify staff if there are extenuating circumstances that prevent them from attending at that time. Staff then reschedule the meeting for a different time. [SKIP to 3.b.5]
2 State or AJC staff schedule the initial meeting. Claimants can reschedule the meeting for a different date/time if there are extenuating circumstances.
3 State or AJC staff schedule the initial meeting. Claimants are invited to reschedule for a different time if another time would be more convenient.
4 Claimants self-schedule the meeting, within a window of a week or less provided by the state.
5 Claimants self-schedule the meeting, within a window more than a week provided by the state.
3.b.4 Which of the following options are available to claimants to schedule or reschedule the initial RESEA meeting?
(Select all that apply.)
1 Online (e.g., UI website, mobile app)
2 Telephone
3 In-person (e.g., at AJC)
4 Other (please specify) __________________
[IF ONLY ONE RESPONSE WAS CHOSEN FOR 3.B.4 THEN SKIP TO 3.C.1,
ELSE 3.B.5]
3.b.5 Which of the following options is the method that claimants most commonly use to schedule or reschedule the initial RESEA meeting?
(Please select one.)
1 Online (e.g., UI website, mobile app)
2 Telephone
3 In-person (e.g., at AJC)
4 Other (please specify) __________________
Topic 3.c. Initial Meeting Details
3.c.1 Where do initial RESEA meetings occur?
(Select all that apply.)
1 In-person at an AJC
2 In-person at a location other than an AJC (please specify) __________________
3 Remotely by phone call (audio, not video)
4 Remotely by videoconference
5 Other remote option (please specify) __________________
[IF ONLY OPTIONS 1 OR 2 ARE SELECTED FOR 3.c.1, THEN SKIP to 3.c.4;
ELSE CONTINUE TO 3.c.2]
3.c.2 Is the option to hold the initial RESEA meeting remotely, rather than in-person, an option for all claimants or only some?
(Please select one.)
1 All claimants [SKIP to 3.c.4]
2 Only some claimants
3.c.3 Which types of clients have the option to hold the initial RESEA meeting remotely?
(Please select one.)
1 Claimants who live far from an in-person meeting location
2 Claimants who live within travelling distance of an in-person location, but who have some other type of barrier to attending (please specify acceptable barriers) ________
3 Other (please specify) __________________
3.c.4 Which of the following types of staff participate in the initial RESEA meeting?
(Select all that apply.)
1 UI staff
2 Wagner-Peyser staff
3 WIOA staff
4 Other staff (please specify) __________________
3.c.5 Which of the following must occur as part of the initial RESEA meeting?
(Select all that apply.)
1 Assess compliance with UI requirements (e.g., work search, able and available)
2 Schedule (but do not conduct) orientation to AJC services
3 Conduct orientation to AJC services
4 Register with Employment Service (if claimant has not done so already)
5 Conduct assessments (of skills, barriers, interests, etc.)
6 Provide general labor market information
7 Provide individualized labor market information
8 Create reemployment plan
9 Review and, if necessary, create or revise a résumé
10 Refer to any appropriate employment workshops
11 Refer to any appropriate individualized career services
12 Make employment matches and job referrals
13 Other (please specify) __________________
[IF RESPONDENT SELECTS ONLY “1” (UI STAFF ONLY), NOT ANY OTHER OPTION FOR 3.c.4 THEN CONTINUE TO 3.c.6, ELSE SKIP TO 3.d.1]
3.c.6 How are referrals typically made to staff who provide reemployment services?
(Please select one.)
1 UI staff contact reemployment services staff via an online system.
2 UI staff contact reemployment services staff by phone.
3 UI staff contact reemployment services staff through some other means. (please specify) __________________
4 UI staff do not contact reemployment services staff, but rather tell RESEA participants how to contact them.
Topic 3.d. AJC Services
3.d.1 How are AJC orientations most commonly conducted?
(Please select one.)
1 One-on-one
2 Group, including RESEA claimants only
3 Group, including UI claimants, whether RESEA or non-RESEA
4 Group, both UI and general AJC populations
3.d.2 Beyond the basic information and resources, in which individualized career services does your state make particular efforts to engage RESEA participants?
(Select all that apply.)
1 Assessments of job-related qualifications and interests
2 Individualized reemployment plan
3 Career planning assistance
4 Individualized labor market information
5 Information on occupational training opportunities
6 Referrals to particular job openings
7 Other individualized employment service: ______________
3.d.3 Which RESEA participants typically receive individualized career services?
(Please select one.)
1 All RESEA participants
2 Only RESEA participants identified through assessments as needing individualized services
3 Few or no RESEA participants
4 I don’t know
3.d.4 After the helping the claimant create an individual reemployment plan, how often do staff follow up with claimants to check on their progress carrying out the plan and provide further help that the claimant might need?
(Please select one.)
1 Regularly, every week or two
2 Regularly, every month or so
3 Once or twice, but not on a regular, on-going basis
4 This type of staff follow-up is not a specific part of RESEA
3.d.5 After helping the claimant create an individual reemployment plan, does your state use any automated methods (e.g., automated emails or text messages) to check with claimants about their job search progress?
(Please select one.)
1 Yes
2 No
3.d.6 If AJC staff who provide employment services become aware of potential claimant non-compliance with eligibility requirements, what feedback mechanisms have been set up to communicate non-compliance to the UI system?
(Select all that apply.)
1 There is an online portal for those AJC staff to report UI compliance issues.
2 There is a dedicated email address for those AJC staff to use to report UI compliance issues.
3 There is a dedicated telephone number for those AJC staff to use to report UI compliance issues.
4 None of the above dedicated mechanisms exist for those AJC staff to report UI compliance issues. Communication occurs through less formal channels.
Topic 3.e. Subsequent RESEA Meetings
3.e.1 Are subsequent RESEA meetings a feature of your RESEA program?
(Please select one.)
1 No, we do not conduct subsequent RESEAs [SKIP to 4.a.1]
2 Yes, for at least some claimants we conduct a second RESEA, but never a third
3 Yes, we conduct a third RESEA for at least some claimants
3.e.2 At what point in the claim does the second RESEA typically occur?
(Please select one.)
1 4 weeks or less
2 5-6 weeks
3 7-9 weeks
4 10 weeks or more
3.e.3 Which RESEA participants are required to attend a second RESEA?
(Select all that apply.)
1 All RESEA participants
2 RESEA participants who staff deem to be in need of more help in finding a job (e.g., based on assessment results at the first RESEA)
3 RESEA participants who have had some non-compliance with eligibility requirements during the claim
4 RESEA participants who request a follow-up meeting
5 Other (please specify) __________________
This section asks a variety of questions related to other aspects of your RESEA program, including claimant noncompliance with program requirements and integration with other workforce programs in your state.
Topic 4.a. Failure to Report
4.a.1 Which of the following most accurately reflects what occurs after a claimant fails to report to a scheduled RESEA meeting?
(Please select one.)
1 Benefits are immediately and indefinitely suspended until the failure to report is remedied or until an adjudication shows that the failure to report was for cause.
2 Benefits are immediately suspended for a week.
3 The case is adjudicated. If the adjudication finds that the failure to report was not for cause, then benefits are suspended indefinitely until the failure to report is remedied.
4 The case is adjudicated. If the adjudication finds that the failure to report was not for cause, then benefits are suspended for a week.
5 Other (please describe)________________
4.a.2 What steps has your state taken to reduce rates of failure to report?
(Select all that apply.)
1 Remind claimants by letter
2 Remind claimants by phone
3 Remind claimants by email
4 Remind claimants by text
5 In RESEA notification communications, highlight penalties for failure to report
6 Increase the consequences of failure to report (e.g., lengthier suspensions of benefits)
7 Give claimants greater flexibility to schedule or reschedule meetings
8 Other (please specify) __________________
9 No steps have been taken to reduce failure to report
Topic 4.b. Eligibility Enforcement
4.b.1 Which of the following most accurately describes your state’s primary motivation when reviewing work search through the RESEA program?
(Please select one.)
1 The work search review is intended to help claimants understand and comply with work search requirements.
2 The work search review is intended to identify and penalize those claimants who are not complying with work search requirements.
3 The work search review is intended to improve claimants’ work search practices and help them return to work more quickly.
4.b.2 Which of the following most accurately characterizes the work search review efforts that occur as part of the eligibility assessments?
(Please select one.)
1 Staff review claimants’ log of reported work search contacts
2 Staff may ask about how many contacts were made, but do not review specific reported contacts
3 Other (please specify)______________
4.b.3 For claimants who, during the initial eligibility assessment, are found to not have met work search or able and available requirements, which of the following is the most common outcome?
(Please select one.)
1 Claimants are found ineligible for benefits for the period(s) covered and benefits are suspended indefinitely until the claimant is shown to be in compliance
2 Claimants are found ineligible for benefits for the period(s) covered and benefits are suspended for a definite period (e.g., a week)
3 Claimants may be warned, but after adjudication do not lose benefits
Topic 4.c. WIOA and Wagner-Peyser Integration
4.c.1 Current RESEA program guidance requires enrollment in Wagner-Peyser Act-funded Employment Services as a core component of RESEA. When does your state’s program complete enrollment for each claimant?
(Please select one.)
1 When claimants are enrolled into the RESEA program
2 When claimants are notified of the requirement to attend an RESEA meeting
3 During the initial RESEA meeting
4 As a follow-up activity after completion of the RESEA meeting
5 Other (please specify) __________________
6 I don’t know
4.c.2 How does your state enroll each RESEA claimant into Wagner-Peyser Act-funded Employment Services?
(Please select one.)
1 An automated system conducts Wagner-Peyser enrollment
2 An RESEA staff member – who does not meet with the claimant – manually completes enrollment
3 The RESEA staff member who meets with the claimant manually completes enrollment
4 Other (please specify) __________________
5 I don’t know
4.c.3 What steps has your state taken to more fully integrate RESEA programs with WIOA and Wagner-Peyser-funded services?
(Select all that apply.)
1 Include WIOA or Wagner-Peyser staff in the RESEA meeting where eligibility assessment occurs
2 Work with the State Workforce Board to promote integration
3 Work through the WIOA State Plan to promote RESEA integration
4 Integrate data systems to allow staff access to both UI and workforce data
5 Train WIOA or Wagner-Peyser staff on RESEA procedures and requirements
6 Work with local workforce development boards to create or implement an AJC orientation tailored to RESEA claimants
7 Align internal RESEA reporting with WIOA performance measures
8 Other (please specify) __________________
9 No significant steps have been taken to integrate programs
Topic 4.d. Using Behavioral Insights
4.d.1 In which ways, if any, has your state made the following small changes to your program in order to alter claimants’ behavior?
(Select all that apply.)
1 Changed a default program requirement
2 Use positively themed messaging in communications with claimants
3 Invoke social norms to motivate claimants
4 Simplified communications with claimants due to limited attention
5 Send automated reminders (emails, letters, calls, texts, etc.) to claimants about upcoming meetings
5 Have staff contact claimants to remind them about upcoming meetings
6 Other (please specify) __________________
7 No, we have not explicitly incorporated behavioral insights into our RESEA program
The questions in this section ask you to identify challenges your state faces with RESEA program operation and what recommendations you would make to other states to improve operation of their programs.
Topic 5.a. Operational Challenges
5.a Based on your experience, to what extent is each of the following a challenge to your state in operating your RESEA program?
(READ EACH ITEM FOLLOWED BY ANSWER CATEGORIES)
|
|
Major Challenge |
Moderate challenge |
Slight challenge |
Not a challenge |
1. |
Lack of sufficient funds to provide appropriate reemployment services |
1 |
2 |
3 |
4 |
2. |
Lack of sufficient funds to rigorously verify eligibility for UI |
1 |
2 |
3 |
4 |
3. |
Lack of AJC capacity to serve enough RESEA claimants |
1 |
2 |
3 |
4 |
4. |
Claimants’ failure to report to RESEA meetings |
1 |
2 |
3 |
4 |
5. |
Claimants’ lack of full engagement with reemployment services |
1 |
2 |
3 |
4 |
6. |
Problems coordinating with local workforce development boards and the WIOA system |
1 |
2 |
3 |
4 |
7. |
Lack of data systems to fully support appropriate feedback loops |
1 |
2 |
3 |
4 |
8. |
Lack of funds to comply with evidence and evaluation requirements |
1 |
2 |
3 |
4 |
9. |
Lack of technical expertise to comply with evidence and evaluation requirements |
1 |
2 |
3 |
4 |
10. |
Lack of technical expertise to update profiling models |
1 |
2 |
3 |
4 |
5.a.11 Please list any other challenges to your state in operating an RESEA program, beyond those listed previously.
_______________________________________
Topic 5.b. Recommendations
5.b How strongly would you recommend each of the following to other states and DOL as ways to make RESEA programs more effective? Some options may include increasing emphasis on existing program activities.
(READ EACH ITEM FOLLOWED BY ANSWER CATEGORIES)
|
|
Strongly recommend |
Somewhat Recommend |
Do not recommend |
Recommend against |
1. |
Increasing the use of individualized career services |
1 |
2 |
3 |
4 |
2. |
Increasing the provision of customized labor market information to inform claimants’ decisions |
1 |
2 |
3 |
4 |
3. |
Increasing the use of earn-while-you-learn training (e.g., on-the-job training, apprenticeship) |
1 |
2 |
3 |
4 |
4. |
Using a broader range of assessments to identify claimants’ reemployment needs |
1 |
2 |
3 |
4 |
5. |
Using more intensive case management |
1 |
2 |
3 |
4 |
6. |
Increasing the use of behavioral nudges to support behaviors that will help claimants meet employment goals and increase compliance with UI requirements |
1 |
2 |
3 |
4 |
7. |
Selecting claimants other than those who are most likely to exhaust UI benefits |
1 |
2 |
3 |
4 |
8. |
More intensively verifying eligibility |
1 |
2 |
3 |
4 |
9. |
Strengthening penalties for failure to report or comply with UI eligibility requirements |
1 |
2 |
3 |
4 |
10. |
Conducting more RESEA follow-up meetings per claimant |
1 |
2 |
3 |
4 |
11. |
Improving the feedback loops between UI and AJC staff |
1 |
2 |
3 |
4 |
5.b.12 What other changes would you recommend to other states and DOL as ways to make RESEA programs more effective in reducing UI duration by improving employment outcomes for program participants?
_______________________________________
[IF 5.b.4 = 1 or 5.b.4 = 2, ASK 5.b.13;
ELSE, SKIP TO 5.b.14]
5.b.13 You said you would recommend using a broader range of assessments to identify claimants’ reemployment needs. Which kinds of assessments would you recommend be more commonly used with RESEA claimants?
__________________
[IF 5.b.7 = 1 or 5.b.7 = 2, ASK 5.b.14
ELSE, SKIP TO 6.a.1]
5.b.14 You said you would recommend selecting claimants other than those who are most likely to exhaust UI benefits. States are no longer required to select RESEA claimants based on profiling scores. Please list any other factors that you personally would recommend using to select claimants. Which claimants would you recommend selecting, rather than focusing solely on those with the highest profiling scores?
__________________
This section covers a range of topics related to evaluating RESEA programs. It includes questions related to your state’s previous experience with program evaluation, plans for future evaluations, and capacity to carry-out evaluations of your RESEA program.
Topic 6.a. RESEA Funding Requirements for Evidence of Effectiveness and Performance
6.a.1 How well do you feel your state understands the new RESEA requirements for evidence of effectiveness and performance?
(Please select one.)
1 Very well
2 Somewhat well
3 Not well at all
6.a.2 What suggestions, if any, do you have to improve support for states implementing these new requirements?
_______________________
Topic 6.b. Previous Evaluation Experience
6.b.1 To your knowledge, has your office participated in an evaluation of any kind in the past five years?
(Please select one.)
1 Yes
2 No [skip to 6.c.1]
3 I don’t know [skip to 6.c.1]
6.b.2 What kind of evaluations have you participated in?
(Select all that apply.)
1 Impact evaluation using random assignment (aka, an experimental impact evaluation)
2 Impact study with a comparison group that did not use random assignment (aka, a quasi-experimental impact evaluation)
3 Outcomes study (e.g., program performance study)
4 Implementation study
5 Cost study (i.e., a systematic disaggregation of the costs of different program elements)
6 Other __________
6.b.3 How were these evaluations conducted?
(Select all that apply.)
1 Part of larger national or multi-state evaluation
2 Our office contracted or partnered with a local evaluator
3 Our office conducted the evaluation using our own staff
4 I don’t know
6.b.4 How would you rate your past experiences participating in evaluations?
(Please select one.)
1 Mostly positive
2 Somewhat positive
3 Not at all positive
4 I don’t know
Topic 6.c. Evaluation Plans
6.c.1 What has your state done to plan for the new RESEA program evaluation and evidence requirements?
(Select all that apply.)
1 Held internal discussions with RESEA staff
2 Discussed with State ES, UI, WIOA, or LMI office
3 Consulted other states one-to-one or through regional, NASWA, or other national meetings
4 Consulted with outside partners or consultants
5 Consulted DOL resources and publications
6 Other activity ___________
7 Nothing yet
8 I don’t know
6.c.2 Are you currently planning to conduct any evaluations of your RESEA program?
(Please select one.)
1 Yes [Skip to question 6.c.4]
2 No
3 Maybe [Skip to 6.c.4]
4 I don’t know [Skip to question 6.d.1]
6.c.3 Why not? [Skip to 6.d.1]
(Select all that apply.)
1 Evaluations require too much staff time to conduct or contract out
2 Staff do not have the expertise needed to conduct or oversee evaluations
3 Evaluations are too expensive
4 Evaluation is not necessary at this time
5 The state’s RESEA program already has a sufficient evidence base
6 Other reasons __________
7 I don’t know
6.c.4 Do you expect to use any RESEA grant funds for any evaluation of your RESEA program?
(Please select one.)
1 Yes, we intend to use the full 10 percent that is allowed by statute
2 Yes, we intend to use some funds, but less than 10 percent of our grant
3 No
4 I don’t know
6.c.5 Which type of evaluations will you [or might you] conduct? (Select all that apply)
(Select all that apply.)
1 Random assignment impact evaluation (aka, an experimental impact evaluation
2 Impact evaluation with a comparison group that does not use random assignment (aka, quasi-experimental impact evaluation)
3 Outcomes study (e.g., program performance study)
4 Process study
5 Implementation study
6 Cost study (i.e., a systematic disaggregation of the costs of different program elements)
7 Other __________
8 I don’t know
6.c.6 Which aspects of the RESEA program—if any—might you evaluate? (Select all that apply.)
(Select all that apply.)
1 Career and labor market information
2 Criteria used to select RESEA claimants (e.g., likelihood of exhaustion)
3 Ways to develop a reemployment plan
4 Job search assistance
5 Approaches to reduce failure to report
6 Penalties for non-compliance/failure to report
7 Providing more individualized career services
8 Adding or removing subsequent RESEA meetings
9 Other (please specify) ___________________
10 I don’t know
Topic 6.d. Thoughts on Pooled Evaluations
6.d.1 States may choose to cooperate on their evaluations with other states to pool grant funds and conduct larger evaluations. Does your state plan on participating in a multi-state evaluation?
(Please select one.)
1 Yes [Skip to 6.d.3]
2 Maybe
3 No
4 I don’t know
6.d.2 How interested would your state be in participating in a multi-state evaluation like this?
(Please select one.)
1 Very interested
2 Somewhat interested
3 Not at all interested [Skip to 6.d.4]
4 I don’t know
6.d.3 What do you see as the most important benefits of participating in a multi-state evaluation?
(Please select up to three.)
1 Opportunity to work with a lead administrative organization (USDOL or other)
2 Opportunity to pool evaluation to get a larger sample
3 Access to the expertise of other states UI and WIOA offices
4 Access to a larger pool of external evaluators
5 Opportunity to learn about successes in other states
6 Opportunity to take advantage of existing state data partnerships
7 Other (please specify) __________
8 I don’t know
6.d.4 Beyond any reservations you might have about participating in evaluations in general, what reservations, if any, do you have about participating in evaluation that involves multiple states?
____________________
6.d.5 How interested would your state be in participating in future DOL-sponsored studies of RESEA programs?
(Please select one.)
1 Very interested
2 Somewhat interested
3 Not at all interested
4 I don’t know
6.d.6 What do you see as the most important benefits of participating in future DOL-sponsored studies of RESEA programs?
(Select all that apply.)
The opportunity to:
1 improve the state’s RESEA program
2 help other states improve their RESEA program
3 develop the state’s evaluation capacity
4 learn from other states’ experiences with RESEA
5 reduce the logistical and technical challenges involved in designing an evaluation
6 get assistance with covering the cost of evaluation
7 Other __________
8 I don’t know
6.d.7 Beyond any reservations you might have about participating in evaluations in general, what reservations, if any, do you have about participating in DOL-sponsored studies?
_______________________
Topic 6.e. In-house Evaluation Capacity
6.e.1 How would you rate your staff’s expertise in designing formal evaluations?
(Please select one.)
1 Excellent
2 Good
3 Satisfactory
4 Needs improvement
5 I don’t know
6.e.2 How would you rate your staff’s expertise in contracting with research partners and overseeing evaluations?
(Please select one.)
1 Excellent
2 Good
3 Satisfactory
4 Needs improvement
5 I don’t know
6.e.3 How much does your office currently use and analyze the data it already collects on individual RESEA participants to assess how the program is working?
(Please select one.)
1 A lot
2 Some
3 A little
4 Not at all
5 I don’t know
6.e.4 How would you rate the quality of your state’s data on individual RESEA participants?
(Please select one.)
1 Excellent
2 Good
3 Satisfactory
4 Needs improvement
5 I don’t know
6.e.5 How would you rate your state’s technical ability to link RESEA participant data (i.e., UI claims data) to wage records, workforce data, and any other important data?
(Please select one.)
1 Excellent
2 Good
3 Satisfactory
4 Needs improvement
5 I don’t know
6.e.6 What methods do you find most effective for receiving evaluation technical assistance?
(Select all that apply.)
1 Webinars
2 Conferences, workshops, or other in-person group meetings
3 One-on-one coaching and support
4 Written guidance (e.g., toolkits, checklists, etc.)
5 Other (please specify) ________________
6 I don’t know
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Andrew Clarkwest |
| File Modified | 0000-00-00 |
| File Created | 2021-01-15 |