Att 2.3 State Title I Youth Program Protocol

OMB No. 1290‐XXXX 
Exp. Date XX/XX/2021 

2.3 PROTOCOL FOR STATE TITLE I YOUTH STAFF
Instructions to site visitors: Bullets below each question represent probes for important details;
depending on how well you are doing on time, try to make sure respondents touch on each of
these issues. Text in brackets [ ] should be tailored based on the state or local area. Sections in
gray are the high priority sections for this respondent.

Introduction
Thank you for taking the time to meet with me today. My name is [NAME] and I work for
[Mathematica Policy Research/Social Policy Research Associates]. I am part of an independent
research team conducting a study of WIOA implementation on behalf of the U.S. Department of
Labor. Thank you very much for agreeing to participate in this conversation.
The purpose of this three-year study is to help DOL understand how states are implementing
changes to the core workforce programs authorized under Titles I and III of WIOA, as well as
how those changes are influenced by integration with other partners such as adult education,
vocational rehabilitation, and TANF. To help us better understand WIOA’s implementation, we
are conducting site visits to 14 states, including two local areas in each state, to learn about their
experiences. The information you share will help us understand the range of experiences that
states have had, including key successes and challenges.
We have about [duration] for our conversation. I want to let you know that all interview data will
be reported in the aggregate and your name will never be mentioned in any report that we write,
though we might use quotes from your interview to illustrate findings, without using your name.
I would also like to record our conversation so that I can listen to it later as I complete my notes,
and I will not share the recording with anyone outside of the research team. Being part of this
discussion is voluntary, and you may choose not to answer a question if you wish, or to pause the
recording at any time. Do you have any objections to being part of this interview or to my
recording our discussion?
Okay, I’m going to turn on the audio recorder now.

Respondent Background
1. [If not already known for each person participating in the interview]
a. What is your official title and role?
b. What are your main responsibilities related to implementation of WIOA?
2. Briefly describe your agency and department/division.
a. What is your department/division’s role in the agency? In WIOA implementation?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection 
displays a valid OMB control number.  Public reporting burden for this collection of information is estimated to average 1 hour and 18 minutes 
per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and 
completing and reviewing the collection of information.  The obligation to respond to this collection is voluntary. Send comments regarding the 
burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to 
Chiefevaluationoffice@DOL.gov and reference the 0MB Control Number 1290‐XXXX.  Comments can also be mailed to:  U.S. Department of 
Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S‐2312, Washington, DC  20210.  Note: Please do not return the completed interview 
guide to the email or mailing address. 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

b. Do any of your agency’s other divisions/department play a significant role in
WIOA implementation? Which ones? What role do they play?
c. Have there been any major changes in administrative structure or responsibilities
for the core workforce partners? If so, what changes and why did they come
about?
d. Do you have local or regional staff? How does your agency supervise/monitor the
implementation of WIOA at the local and regional levels?

A. Youth
1. Are local boards serving more, about the same, or fewer youth under WIOA as compared
to under WIA?
a. What factors are affecting the percentage of youth being served under WIOA?
2. Are local boards typically serving more older youth than under WIA? If so, how has this
changed their Youth programs?
a. Has being able to serve youth over 21 changed local Youth programs? How?
i. Recruitment and marketing?
ii. Partner relationships?
iii. Types of services offered?
3. In the most recent program year (PY17), did any local areas fail to meet the requirement
to spend 75% of their youth funding on out-of-school youth (OSY)?
a. If so, how many? Which ones? What percentage are they spending on OSY?
b. What are the primary reasons they failed?
c. How is the state working with these areas to help them meet this requirement?
d. Have any of these local areas requested waivers of the 75% requirement?
4. Have you identified any promising practices at the local level for identifying, recruiting,
and retaining OSY in the WIOA Youth program?
a. Are any of these strategies new under WIOA?
b. Are successful recruitment strategies different for rural areas than for urban areas?
c. Are successful strategies different for different populations of OSY?
5. What relationships and approaches, if any, do you feel have been compromised in the
shift away from focusing on in-school youth (ISY)?
a. Have the types of organizations/entities that local areas partner with changed?
b. Are there sufficient workforce services available for ISY now that the Youth
program cannot serve large numbers of ISY?
c. What entities or programs are serving ISY instead?

2.38 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

6. Under WIOA, are local boards directly providing more youth services than under WIA
(rather than through competitively procured contractors)? If so, what services are boards
now more likely to directly provide?
7. Where are Youth program staff members typically located? Are they co-located in the
state’s AJCs? If not, why not?
a. Have there been changes since WIOA in the extent to which Youth program staff
members are co-located in AJCs? Why?
b. If Youth staff members are not co-located in AJCs, how difficult was it for local
boards to train AJC staff to serve youth and connect them to appropriate services
(as required by WIOA)?
8. Has the Youth program developed new partnerships since WIOA with core programs?
a. With VR? Describe. Why was this partnership developed?
b. With AEFLA? Describe. Why was this partnership developed?
9. Have local Youth programs had trouble spending 20% of funding on work experience?
a. Did any of your local areas fail to meet this requirement? How many and why?
b. What has worked well at the local level for identifying work experience
opportunities? Are any of these strategies new under WIOA?
10. Describe your state’s current use of work-based training—such as OJT, apprenticeships,
and transitional employment—for youth. How common is each type? Why?
c. Are certain approaches used more frequently for particular youth, such as OSY?
d. Has the use of work-based-based training for youth increased with WIOA? Why?
11. What is your state’s overall approach for Youth programs to engage with and provide
services to businesses?
e. Has WIOA changed this approach? How?
i. Are business services a greater focus for the Youth programs?
ii. If so, why? (e.g., because of the requirement that 20% of funds be spent
on work experience? The new business services performance indicator?)
12. How much do Youth program staff who engage with employers collaborate with other
workforce system who work with employers? With which partners?
f. Has collaboration on business services/engagement improved with WIOA? Why?
13. Have local boards encountered any challenges providing new youth program service
elements under WIOA (such as financial literacy education, etc.)?
14. What challenges have local boards reported related WIOA’s greater focus on serving
youth with specific barriers (e.g. youth with disabilities, out-of-school youth (OSY),
etc.)?
15. What service delivery strategies have worked well for serving youth with barriers?

2.39 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

16. Overall, have services for youth become more comprehensive and integrated under
WIOA? In what ways?
17. Is more guidance/TA needed from DOL provide on Youth program requirements under
WIOA?

B. Governance and Planning
1. State Workforce Development Board Role and Operations
1. How much of a focus does the board have on youth programs/populations vs. adult
programs/populations?
a. Has this changed as a result of WIOA? How? Why?
2. Does the state board have any committees related to youth?
a. Are those committees different from those the board had under WIA? Why?

2. State Plan Development
1. What are the state’s workforce priorities for youth (e.g. Governor’s priorities)?
2. Were you involved in state partner staff meetings to discuss/facilitate WIOA
implementation or to develop the state plan? Describe.
a. What other partners were involved in discussion around serving youth?
b. How often did they meet? Are they still meeting? How often? What did they
discuss? What do they now discuss?
3. Does your state’s WIOA plan have a greater or lesser focus on youth than under WIA?
4. Has your WIOA state planning process been helpful for the Youth program? If so, how?

3. Local Workforce Development Board Role and Focus

1. Do most local boards in the state have a youth committee? Why or why not?
a. If yes, is membership on these committees about the same as it was on WIA’s
Youth Councils? If not, how has membership changed? How has the role, focus,
or activities of Youth Councils changed under WIOA?
2. In general, have WIOA’s changes to local board membership and roles shifted the
priorities/operations of local boards? If so, how?
a. Has this affected how Youth programs are operated locally?
3. How much do local boards focus on youth versus adult populations?

2.40 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

C. Overall AJC Streamlining & Accessible Services
1. Co-enrollment
1. [If not already known from the DOL Data Book] What programs are Youth customers
co-enrolled in? Which are the most common, and why?
2. Do any new WIOA requirements particularly facilitate or impede co-enrollment?
3. Are you encouraging co-enrollment and if so, how? [Skip the following probes if the
state is not encouraging co-enrollment]
a. [If not already known from pre-visit call] Has your state provided any TA or
guidance on increasing co-enrollment? [Get copies of guidance or tools.]
b. What challenges have you encountered in your efforts to increase co-enrollment?
What has worked well?

D. Performance Accountability
1. Performance Measurement
1. Have you had any challenges with the youth performance indicators? Which ones?
a. Measurable skill gains?
i. Challenges related to youth who begin participating late in the PY?
b. Credential Attainment?
i. Challenges related to determining what credentials count?
ii. Challenges related to youth who get a high school diploma also needing to
be in unsubsidized employment/post-secondary education?
2. Are the WIOA indicators appropriate for Youth participants? Why or why not?
3. Are you concerned about the Youth program meeting state negotiated targets for the
WIOA performance indicators? Why or why not?

E. Services for Job Seekers and Employers
1. Performance Measurement
1. Describe your state’s current use of career pathways. How commonly do Title I Youth
participants access them?
a. Are career pathways efforts for youth generally happening at the state, regional,
or local levels?
2. For any statewide career pathways initiatives focused on youth:
a. How are they funded?
b. Who are they targeted at (e.g., all youth, youth with specific barriers, etc.)

2.41 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

c. What are the main targeted occupations/industries?
3. Please describe your state’s efforts to develop and support career pathways initiatives,
especially for youth.
4. How involved are VR and adult education partners in career pathways efforts for youth?
5. What role has WIOA played in your implementation or expansion of career pathways?
a. Did your efforts related to career pathways precede WIOA? If yes, how are you
building on your pre-existing work?
6. What challenges have you faced in developing/supporting career pathways for youth?
What has worked well?
7. Is there a need for more guidance/TA from DOL on youth career pathways?

F. Overall Questions
1. Partnerships
1. How would you assess the strength of your state’s relationships among core partners?
Required partners?
a. Has this changed because of WIOA?
b. If not, why do you think WIOA’s focus on enhanced partnerships has not affected
your partnerships?
2. Are there sufficient systems in place to ensure regular and effective communication
between partners?
3. What have been the key challenges and successes, if any, related to increasing
collaboration across partners?
a. Are there differences in the extent of collaboration with your core partners versus
other required partners, such as UI, SCSEP, Veterans programs, and TANF?
4. Overall, how integrated are workforce services now?
a. Has this changed because of WIOA?
b. What examples can you give that show this more integrated service delivery?
5. [If respondent feels workforce system is not integrated]: Why do you think WIOA’s
focus on enhanced partnerships has not affected the integration of services?

2. Systems Change
1. Overall, do you think the workforce system in your state—especially related to youth-has changed as a result of WIOA?
a. If so, what are the major changes you can attribute to WIOA?

2.42 

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.2

b. If not, do you feel systems had already begun to change prior to WIOA, or that
efforts under WIOA have not yet been effective? What would need to happen to
enable WIOA’s changes to take effect?
2. Have there been any negative, unintended consequences of implementing WIOA,
especially for youth in the state?
a. Are there ways in which you feel the new law makes it more difficult to provide
effective, integrated workforce services for youth? If so, how and why?
3. Have there been positive, unexpected benefits from implementing WIOA?
4. What additional changes are planned or needed to transform your workforce system to
meet the goals of WIOA?
a. How do you think those changes would make the system better, especially for
youth participants?
5. Are there any other areas that we haven’t already discussed where you would like to
receive additional TA or guidance from DOL?
6. Beyond what we’ve discussed today, is there anything else you want to share about your
experience implementing Title I WIOA Youth programs?

3. Guidance and Technical Assistance
1. Of all the TA provided by DOL national and regional offices on WIOA implementation,
which assistance has been most helpful?
a. Which TA formats and types of materials have been most useful to you?
2. Are you satisfied with the level of TA DOL has provided on WIOA implementation?
3. Have you accessed the ION site to obtain TA materials or guidance on specific issues? If
so, which issues?
4. In what areas would you like to receive additional TA?

Wrap-up
1. How would you describe your overall progress to date with WIOA implementation?
a. Where do you still have plans to make significant changes?
2. Beyond what we’ve discussed today, are there other areas you would like to highlight?
3. Are there any other areas of WIOA implementation our study should explore?

2.43 

This page has been left blank for double-sided copying.