Att 3.7 Local AJC Operator Protocol

OMB No. 1290-XXXX
Exp. Date XX/XX/2021

3.7 PROTOCOL FOR ONE STOP OPERATOR
Instructions to site visitors: Bullets below each question represent probes for important details;
depending on how well you are doing on time, try to make sure respondents touch on each of
these issues. Text in brackets [ ] should be tailored based on the state or local area. Sections in
gray are the high priority sections for this respondent.

Introduction
Thank you for taking the time to meet with me today. My name is [NAME] and I work for
[Mathematica Policy Research/Social Policy Research Associates]. I am part of an independent
research team conducting a study of WIOA implementation on behalf of the U.S. Department of
Labor. Thank you very much for agreeing to participate in this conversation.
The purpose of this three-year study is to help DOL understand how states are implementing
changes to the core workforce programs authorized under Titles I and III of WIOA, as well as
how those changes are influenced by integration with other partners such as adult education,
vocational rehabilitation, and TANF. To help us better understand WIOA’s implementation, we
are conducting site visits to 14 states, including two local areas in each state, to learn about their
experiences. The information you share will help us understand the range of experiences that
states have had, including key successes and challenges.
We have about [duration] for our conversation. I want to let you know that all interview data will
be reported in the aggregate and your name will never be mentioned in any report that we write,
though we might use quotes from your interview to illustrate findings, without using your name.
I would also like to record our conversation so that I can listen to it later as I complete my notes,
and I will not share the recording with anyone outside of the research team. Being part of this
discussion is voluntary, and you may choose not to answer a question if you wish, or to pause the
recording at any time. Do you have any objections to being part of this interview or to my
recording our discussion?
Okay, I’m going to turn on the audio recorder now.

Respondent Background
1. [If not already known for each person participating in the interview]
a. What is your official title and role?
b. What are your main responsibilities related to implementation of WIOA?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection
displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour and 15 minutes
per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the
burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to
Chiefevaluationoffice@DOL.gov and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of
Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview
guide to the email or mailing address.

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

A. Governance and Planning
1. Local Plan Development
1.

Have there been changes in federal or state funding for the workforce system that
have affected your plan or how the plan will be implemented?
a. If there have been significant cuts to funding, how are you adjusting the system in
response to these cuts?
b. What will be the impacts of these cuts on infrastructure, programming, and
service delivery?

2. Regional Designations and Regional Plans
1.

How satisfied are you with your state-designated region under WIOA?
a. Do you think it accurately reflects your regional labor market?

2.

Has being part of a region helped you to better engage with regional employers? With
regional economic development entities?

3. Local Workforce Development Board Role
1.

[If operator was in place prior to WIOA]: How, if at all, have the role and approaches
of the board changed under WIOA?
a. Is it more strategic, more innovative, more focused on continuous improvement,
more focused on improving service integration?
b. Why has the role of the board changed?
i. New board mandates under WIOA?
ii. Required changes in membership?
iii. Some other reason?

B. AJC System
1. Infrastructure Funding and MOUs
1.

How does your MOU (excluding resource sharing components) differ from the last
one you had under WIA?

2.

What was in your local One-Stop operating budget(s)?
a. Other than IFAs, what did you include?
b. Why did you include those components? [Make sure to get a copy or copies.]

3.

What process did the LWDB follow to negotiate the required MOU and One-Stop
operating budget(s)?
a. Did they negotiate umbrella budgets or separate budgets for each AJC?

3.70

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

4.

What was the role of board staff in negotiating MOUs and budgets? How did this
compare with the role played your organization?

5.

What methodologies did the LWDB choose to use for determining their various types
of costs?
a. Why did the LWDB choose those methodologies?
b. What are the advantages and disadvantages of these methodologies?

6.

How did the LWDB determine proportionate use and relative benefit for each
partner?
a. Why did the LWDB choose these methods?
b. How well do these methods work?
c. Are there differences in their appropriateness for on-site vs. off-site partners?

7.

What non-cash contributions do partners make?
a. Why were they not able to contribute cash?
b. How did the LWDB determine how to value non-cash contributions? How well
did that work?

8.

How has the WIOA RSA requirement affected partnerships (either positively or
negatively)?

9.

How often is the LWDB reviewing your agreement (for issues like value of non-cash
contributions, proportionate use and relative benefit)?
a. Are there any challenges related to this review process?

10.

What challenges did you face in the process of developing your MOU and One-Stop
operating budgets?

11.

Has your state has taken any steps to facilitate negotiation of One-Stop operating
budgets (e.g. meetings at the state level, with state partners, to facilitate their
participation)?

12.

What TA have you participated in on MOUs and resource sharing?
a. How helpful was this TA?
b. What could have been more helpful?

13.

What additional guidance or technical assistance do you need from the state or
USDOL on MOUs and resource sharing?

2. One-Stop Operator
1.

Are you the operator for the entire local area?
a. If not, what other entity/entities serve as operator?

2.

Operators have different responsibilities in different areas; for your local area, what is
your role as operator it relates to:

3.71

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

a. Partner meetings?
b. Cross-training sessions?
c. Coordinating AJC operations & policies?
d. Managing AJC facilities?
e. Implementation of MOUs/RSAs?
3.

Do you have any other significant roles in this local area?

4.

How and why, if at all, has your role as operator changed with the implementation of
WIOA?

5.

What is the role of the AJC manager?

6.

How and why has the AJC manager’s role changed, if at all, with the implementation
of WIOA?

7.

How did you learn about the operator competition bid?

8.

[If operator is new under WIOA]: How, if at all, do you think changes in what entity
serves as the operator will affect AJC operations?

3. Collocation at AJCs
1.

Across all the AJCs in the local area, what partners are always or almost always
collocated at comprehensive centers? At affiliate sites?
a. What do you think are the primary reasons for this?

2.

Across all the AJCs in the local area, what partners are never or almost never
collocated at comprehensive centers? At affiliate sites?
a. What do you think are the primary reasons for this?

3.

What partners are collocated at this AJC?

4.

If any required partners that are active in the local area are not collocated at
comprehensive AJC(s), are they providing access to services via training an onsite
staff member or via a “direct linkage”?

5.

Why did they opt for that approach and how is it being operationalized?
a. What challenges and benefits have you experienced with each approach?

6.

Has partner collocation at AJCs changed since WIOA implementation? If so, how
and why?

7.

What guidance have you received from the state on collocation of partners at AJCs
under WIOA?
a. Did that guidance change from what was provided under WIA? If so how?
b. Is any additional guidance needed?

3.72

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

C. Overall AJC Streamlining & Accessible Services
1. Accessibility
1.

How integrated and streamlined is service delivery in the local area?

2.

Please describe any new state or local policies and actions that promote a more
seamless, accessible and customer-focused workforce service delivery network?

3.

Has the state or LWDB encouraged or required any changes to your AJC customer
intake process? What changes did this involve?

4.

To what extent has the state or LWDB emphasized customer or human-centered
design as a key method for making your state’s One-Stop career center system more
customer-focused?
a. What problems or goals do you plan to address through this approach?
b. What challenges and benefits have you encountered in operationalizing it?

5.

Please describe any state or local efforts to improve the accessibility of One-Stops for
individuals with disabilities (e.g. policies, additional funding, or training).

2. Referral Process
1.

How frequently are referrals are made between partner programs?
a. Which partners are most likely to initiate referrals?
b. Which partners are customers most frequently referred to?

2.

How are referrals made to each agency?
a. Do staff provide contact information for the program to a customer?
b. Do staff connect directly with the program they are referring a customer to?
c. Do staff submit a referral via shared MIS?
d. Or, is there some other referral process?

3.

Are referrals tracked? Is information shared about referred customers with the
program the customer is being referred to?

4.

What do you think is the most successful aspect of your referral process?

5.

Is there anything about the referral process you would like to improve?

3. Co-enrollment
1.

About how frequently are local customers are co-enrolled in partner programs?
Which programs and for what reasons?

2.

How, if at all, do co-enrollment processes differ from referral processes?

3.

How, if at all, is information about co-enrolled customers shared with partner
programs?

4.

What is the most successful aspect(s) of local co-enrollment processes?

3.73

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

a. What requirements particularly facilitate co-enrollment?
b. Are there factors that impede co-enrollment?
5.

Is there anything about local co-enrollment processes you would like to improve?

4. AJC Partner Meetings
1.

Do you have regular AJC partner meetings? (If no, skip to cross-training)

2.

Are these meetings for individual AJCs or only across the local area?

3.

What partners attend regularly?

4.

What level of staff attend these meetings? Program managers? Line staff? Both?

5.

Who organizes and facilitates these meetings?

6.

How often do they occur? For how long?

7.

What topics are covered in these meetings?

5. Cross-training
1.

Have you organized or participated in any cross-training sessions across different
programs? (If not, skip to 9)

2.

How often have these sessions occurred? At what locations?

3.

Are these in-person or virtual?

4.

How long did/do they last?

5.

Who attends regularly? Are these local or state-wide trainings?

6.

Who organizes/facilitates?

7.

What is covered in these sessions? [try to get an agenda for the last 1-2 cross-training
sessions.]

8.

How helpful have these sessions been?
a. Is there any way they could have been improved?

9.

In what other ways—if any—do you share information on partner programs?
a. How well do these other methods work?

6. Use of shared MIS
1.

Do any core or required partners use the same management information system
(MIS)? If so, what systems? (If not, skip to #5)
a. Which partners use them?
b. For what purposes?
i. Case management?
ii. Participant tracking?

3.74

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

iii. Performance reporting?
iv. Fiscal reporting?
2.

Are there differences in how partners can use any of these common systems?
a. Can all partners view data? Are there specific fields that are restricted?
b. Can all partners enter data into the system? Are there specific fields that only
some partners can enter?
c. Can all partners revise data? Are there specific fields than only some partners can
modify?

3.

How do you ensure that the data being entered into these systems by different
partners is accurate?

4.

How are you protecting the security of the data being entered and stored?

5.

If not all programs use the same MIS, why not?
a. Are there any plans to add more partners to shared MIS?
b. Which ones?
c.

With what data entry or viewing privileges?

d. If there are no plans to add partners, why not?
e. Have you developed any workarounds to allow partners to share data in the
absence of a shared MIS?
6.

Has the state or LWDB encouraged, required, or led the development of a shared case
management system?

7.

How important is it for seamless services that partner programs use the same MIS?

D. Performance Accountability
1. ETPL Changes
1.

How and why, if at all, have new ETPL requirements impacted the number of
providers available for your local area?

2. Common Measures
1.

Have you faced any challenges implementing the WIOA common performance
measures?

2.

Which challenges have you faced in measuring the effectiveness of employer
services?
a. Do you have thoughts on measures other than the ones being used that might
better capture the effectiveness of serving employers?

3.75

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

3.

Have you faced any challenges in implementing state-specific performance
measures?

4.

Are you developing any local measures that you and your local partners will be
reporting on?

5.

Have you faced any challenges in implementing local performance measures?

E. Services for Job Seekers and Employers
1.

Has WIOA changed your approach to providing services to businesses? In what way?

2.

Please describe your efforts to improve access and services to more barriered
customers, including disabled customers and those with low basic skills.

3.

Have you received any guidance or TA from the state on improving services to
customers with barriers? How helpful was it?

4.

What, if any, additional guidance should USDOL or your state should provide on
serving customers with barriers?

G. Overall Questions
1. Partnerships
1.

How would you assess relationships among core partners? Required partners? Other
partners?
a. Has this changed because of WIOA?
b. If not, why do you think WIOA’s focus on enhanced partnerships has not affected
your partnerships?

2.

Are there sufficient systems in place to ensure regular and effective communication
between partners?

3.

What have been the key challenges and successes related to increasing collaboration
across partners?
a. Are there differences in the extent of collaboration with your core partners versus
other required partners, such as UI, SCSEP, Veterans programs, and TANF?

4.

Overall, how integrated are workforce services now?
a. Has this changed because of WIOA?
b. What changes reflect this more integrated service delivery? For example, are case
managers of different programs meeting more often to discuss co-enrolled clients?

5.

[If respondent feels workforce system is not integrated]: Why do you think WIOA’s
focus on enhanced partnerships has not affected the integration of services?

2. Systems Change
1.

Overall, do you think the workforce system here has changed as a result of WIOA?

3.76

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.7

a. If so, what are the major changes you can attribute to WIOA? E.g. employer
engagement, partnerships, services more streamlined, data sharing, others?
b. If not, do you feel systems had already begun to change prior to WIOA, or that
efforts under WIOA have not yet been effective in changing the system?
2.

Have there been any negative, unintended consequences of implementing the WIOA
regulations?
a. Are there ways in which you feel the new regulations actually make it more
difficult for you to achieve your goals?

3.

[If respondent doesn’t think WIOA has changed the system]: Do you feel systems had
already begun to change prior to WIOA, or that efforts under WIOA have not yet
been effective in changing the system?

4.

What additional changes are planned or needed to transform your workforce system
to meet the goals of WIOA?
b. What changes would you like to see to make the workforce system better overall?

3. Guidance and Technical Assistance
1. Of the TA provided by DOL national and regional offices on WIOA implementation,
which assistance has been most helpful?
a. Which TA formats and types of materials have been most useful for you?
2. Are you satisfied with the level of TA DOL has provided on WIOA implementation?
3. Have you accessed the ION site to obtain TA materials or guidance on specific issues? If
so, which issues?
4. In what areas would you like to receive additional TA?

Wrap-up
1. How would you describe your overall progress to date with WIOA implementation?
a. Where do you still have plans to make significant changes?
2. Beyond what we’ve discussed today, are there other areas you would like to highlight?
3. Are there any other areas of WIOA implementation our study should explore?

3.77

This page has been left blank for double-sided copying.