Protocols for Focus Groups

APPENDIX E
VSS SITE VISIT PROTOCOLS

This page has been left blank for double-sided copying.

VETERANS’ SUPPLEMENTAL STUDY OF
WIA GOLD STANDARD EVALUATION
State Staff
State:
Respondent Name/title:
Discussion Date/Time:

A. INTRODUCTION TO THE STUDY
Hi. My name is [xxx]. I am a researcher with Mathematica Policy Research/Social Policy
Research. Thank you very much for agreeing to talk with us.
As you know [local area] has been participating in the National WIA Evaluation. As part of that
study, we are exploring how the workforce system identifies and provides services to the area’s
veterans. Your input will be really useful.
Just a couple of things…
I would like you to know that the project team has received approval from the Office of
Management and Budget to conduct this study (OMB Control Number xxxx-xxxx). We could
not conduct this session without that approval. Your responses are completely voluntary and will
be used for research purposes only.
We will not share your comments today with anyone outside the immediate study team. Our
report will describe the experiences and viewpoints expressed, but comments will not be
attributed to specific individuals.
I would like to record our discussion so I can listen to it later when I write up my notes. No one
outside the immediate team will listen to the tape. If you want to say anything that you do not
want taped, please let me know and I will be glad to pause the tape recorder. Do you have any
objections to being part of this interview or to my taping our discussion?
This discussion will last about 60 minutes. Once again, thank you for participating. Do you have
any questions before we get started?

B.

INDIVIDUAL BACKGROUND

I’d like to start by briefly talking about your job and your overall responsibilities.
1. How long have you held your current position? What was your previous position?
2. What are your major responsibilities in your current position?
State Staff

1

3. Do you have a military background?
C. STAFFING FOR VETERAN PROGRAM
I’m interested in learning how Disabled Veteran Outreach Program (DVOP) specialists or
Local Veterans’ Employment Representatives (LVERs) are allocated across the LWIAs in
your state.
4. How do you determine the location of the state’s DVOPs? LVERs? What factors
determine where these staff are assigned?
5. How is their full- or part-time status determined? Have there been recent changes in
the distribution of full- and part-time staff? Which arrangement do you find
preferable and why?
6. Within local areas, who determines at which local American Job Center(s) these staff
should be staffed? Is that a local or state decision? If the latter, based on what?
7. [If not already mentioned:] What is your role in working with staff working with
veterans? Do you oversee state staff, including the DVOP, LVER, and ES staff
stationed at American Job Centers?

D. WORKING WITH VETERANS
8. Please describe the policies and guidance provided to local areas by the state
regarding providing services to Veterans.
Probes:
- Are local areas provided the federal guidance on implementing POS in its
original form?
- What state guidance is provided on implementing POS? How and why does it
differ from the federal guidance?
- In what format is the guidance provided by the state? (Written directives,
presentations, webinars, etc.) Have different approached been tried? Why is
this method used?
- What, if any, guidance has been provided with regard to implementing the
Gold Card?
9. How do local staff, including staff at American Job Centers and LVERs and DVOP
specialists, learn about the guidance?
Probes:
- In what format is the guidance provided by the state? (Written directives,
presentations, webinars, etc.)
- Why is this method used?
- Have different approached been tried?

State Staff

2

10. Does the state provide any specific training about working with veterans or POS for
veterans? If so, please describe.
11. What are the roles of DVOP and LVER staff? Who defines these? Who sees LVER
and DVOP staff?
12. Typically, how do DVOP/LVER staff coordinate work and delivery of services with
their ES and WIA colleagues? Do staff tend to coordinate more with one than the
other? Please explain.
13. Do you believe it is easier to get veterans or nonveterans to use American Job Center
services? Why?
14. Do the American Job Centers use the same outreach methods to inform veterans and
nonveterans about available services?
Probes:
- Are there any special efforts to inform veterans?
- Do your American Job Centers receive referrals of veterans from other
programs, military facilities, employers, schools, or other sources?
15. How would you say the characteristics and service needs of the veteran population
served through the American Job Centers compare with those of nonveterans? Are
there unique characteristics and service needs in any of the Centers?
16. Is it easier or more difficult to provide services to veterans compared to nonveterans?
Why? (E.g. skills translation, language and presentation format, computer literacy)
17. What challenges has the state encountered in serving veterans? POS? For example,
what challenges have you encountered in identifying veterans and eligible spouses?
18. In what ways do you believe that the state and local areas have been successful in
serving veterans? How do you know?
19. In your opinion, have the state or particular LWIAs within the state developed
innovative/unique ways of identifying and/or working with veterans? Please
describe.

E. CLOSING
Thank you again for taking the time to speak with me. I appreciate and value your input.
20. Do you have anything you would like to tell me or questions you would like to ask
me?

State Staff

3

This page has been left blank for double-sided copying.

VETERANS’ SUPPLEMENTAL STUDY OF
THE WIA GOLD STANDARD EVALUATION
American Job Center Staff Discussion Guide
LWIA/American Job Center:

____________________________________

Respondent Name/Position:

____________________________________

Interview Date/Time:

____________________________________

Interviewer:

____________________________________

A. INTRODUCTION TO THE STUDY
Hi, my name is [xxx] from Mathematica Policy Research/Social Policy Research. Thanks for
agreeing to participate in this discussion.
As you know [local area] has been participating in the National WIA Evaluation. As part of that
study, we are exploring the partnerships developed as a result of WIA and how the workforce
system identifies and provides services to the area’s veterans. Your input will be really useful.
Just a couple of things before we start…
I would like you to know that the project team has received approval from the Office of
Management and Budget to conduct this study (OMB Control Number xxxx-xxxx). We could
not conduct this session without that approval. Your responses are completely voluntary and will
be used for research purposes only.
We will not share your comments today with anyone outside the immediate study team at
Mathematica. Our report will describe the experiences and viewpoints expressed, but comments
will not be attributed to specific individuals.
I would like to record our discussion so I can listen to it later when I write up my notes. No one
outside the immediate team will listen to the tape. If you want to say anything that you do not
want taped, please let me know and I will be glad to pause the tape recorder. Do you have any
objections to being part of this interview or to my taping our discussion?
This full discussion will last about 60 minutes. Once again, thank you for participating. Do you
have any questions before we get started?

B. VETERAN MODULE
1. When would a customer walking in the center for the first time be identified as a
veteran? Who is responsible for making this determination?

Center Staff Guide

1

2. Once identified as a veteran, what are the next steps for that customer? How does
this differ from the process for a non-veteran customer?
Probes:
- Is the customer informed about POS? If so, when and by whom?
- Does the veteran meet with a different sequence of staff than a non-veteran
customer? If so, please explain.
3. What about POS eligible spouses? How and when are they typically identified and
certified, and what are their next steps?
4. What information/documents are required to certify the customer as a veteran? For
POS?
Probe:
- Does verification/certification of veteran status depend on the services the
customer is requesting? Please explain.
5. Do veterans receive an orientation to the services available to them? If so, please
describe. If not, how do veterans learn of the available services?
Probe:
- Do veterans receive an orientation to services that is different from what
others receive? If so, please describe how it is different.
6. Do veterans and eligible spouses typically come to the American Job Center with
some familiarity with services available to veterans, including POS? If so, from
where/who do they tend to receive the information?
Probes:
- To what extent do they receive information from their military base (for those
recently discharged), veterans’ service organizations [VSOs], the internet,
and/or friends?
- Please tell me about any outreach you do (out-stationing, brochures, and so
on).
- Do other staff at the American Job Center conduct any outreach? Please
describe.
7. Once enrolled, do veterans work with an assigned case manager or specific types of
staff? How are these assignments made? How does this differ from the process for
other American Job Center customers?
8. How is POS implemented in practice? Please explain how a veteran receives priority
of service when he/she first contacts the workforce system. (This could be at a
center, at home on a computer, or by some other means.)
Probes:
- How is the intake and flow different for veterans than for non-veteran
customers?
- What is your role in POS?

Center Staff Guide

2

- Do veterans have priority for job openings that are posted? If so, please
describe how this works. Are veterans’ resumes flagged in the system? Do
employers see an indicator of any kind that the person is a veteran?
- If a veteran requests training services, how are they served? What occurs if the
LWIA/center has a waiting list?
9. To what extent do you (as American Job Center staff member) work with the local
area’s LVER and/or DVOP to meet the needs of your veteran customers? With WIA,
ES, or other partners? In what ways do you work together?
Probes:
- To what extent do you serve the same veteran customers that the center’s
DVOP, LVER, and/or WIA staff serve?
- Do you communicate about individual veterans? About what? How often?
10. Do you provide services to veteran customers that differ from the services you
provide to civilian customers? If so, please describe.
11. How do you help veterans translate their military experiences, training, and jobs to
civilian opportunities? Do you use specific tools? What are they? How do you find
out about these tools?
12. How do you or your American Job Center track services provided to veterans and
POS eligibles and the services they receive? What information is tracked? Who has
access to this information?
Probe:
- Are veteran services tracked separately for ES, Veterans’ State Grants, and
WIA?
- Is information about veterans reported in separate systems for ES, WIA, and
Veterans’ State Grants? If so, are there any links between systems?
- Are staff from the various programs able to see each other’s notes?
13. To what extent do you believe you are providing services to veterans using unique or
innovative strategies? Please explain. Have these strategies been developed at the
local level or state? Please explain.
14. What have been the challenges to working with veterans?
15. How do you know if you are successful with your veteran customers? What is the
process for following-up with them?

Center Staff Guide

3

This page has been left blank for double-sided copying.

VETERANS’ SUPPLEMENTAL STUDY OF
THE WIA GOLD STANDARD EVALUATION
Local DVOP/LVER Staff Discussion Guide

American Job Center Location:

____________________________________

Respondent Name/Position:

____________________________________

Interview Date/Time:

____________________________________

Interviewer:

____________________________________

A. INTRODUCTION TO THE STUDY
Hi, my name is [xxx] from Mathematica Policy Research/Social Policy Research. Thanks for
agreeing to participate in this discussion.
As you know [local area] has been participating in the National WIA Evaluation. As part of that
study, we are exploring how the workforce system identifies and provides services to the area’s
veterans. Your input will be really useful.
Just a couple of things before we start…
I would like you to know that the project team has received approval from the Office of
Management and Budget to conduct this study (OMB Control Number xxxx-xxxx). We could
not conduct this session without that approval. Your responses are completely voluntary and will
be used for research purposes only.
We will not share your comments today with anyone outside the immediate study team at
Mathematica. Our report will describe the experiences and viewpoints expressed, but comments
will not be attributed to specific individuals.
I would like to record our discussion so I can listen to it later when I write up my notes. No one
outside the immediate team will listen to the tape. If you want to say anything that you do not
want taped, please let me know and I will be glad to pause the tape recorder. Do you have any
objections to being part of this interview or to my taping our discussion?
This discussion will last about 60 minutes. Once again, thank you for participating. Do you have
any questions before we get started?

DVOP/LVER Guide

1

B. INDIVIDUAL BACKGROUND
I’d like to start by briefly talking about your job and your overall responsibilities.
1. Please tell me a little about your work experience. How long have you worked as a
DVOP/LVER? Have you held other positions within the American Job Center? What
kind of work did you do before?
2. Do you work full- or part-time as a DVOP/LVER? Has this always been the case?
(If there has been a shift in the organization between full- and part-time, probe for
the reasons for this shift, how the staff feel about it, how it has changed work
flow, and the impacts it has on how many veterans are getting what kinds of
services.)
3. What are your major responsibilities in this position?
(Site visitor: If interviewee is DVOP and you know LVER is also located on site, ask
about any differentiation in their responsibilities/tasks)
4. I’m interested in learning how you spend your time. What percentage of your time
would you say you devote to [Site visitor: tailor list based on response to #3]:
-

One-on-one meetings with veterans

-

Coordinating work or services with WIA staff?

-

Coordinating work or services with other partners?

-

Conducting outreach/informational activities

-

Completing paperwork (including case notes and other computerized
“paperwork”)

5. Are you located only at this center? If not, please describe your other locations and
your schedule. Do you think this is the arrangement that best serves veterans? Why
or why not?
C. INITIAL STEPS FOR A VETERAN VISITING THE CENTER
I’m interested in learning about the steps that occur when a veteran walks into the center for
the first time.
6. When would a customer walking in the center for the first time be identified as a
veteran? Who is responsible for making this determination?
7. Once identified as a veteran, what are the next steps for that customer? How does
this differ from the process for a non-veteran customer?
Probes:
- Is the customer informed about POS? If so, when and by whom?
- Does the veteran meet with a different sequence of staff than a non-veteran
customer? If so, please explain.
DVOP/LVER Guide

2

8. What about eligible spouses? How and when are they typically identified and
certified, and what are their next steps?
9. Do veterans receive an orientation to the services available to them? If so, please
describe. If not, how do veterans learn of the available services?
10. What information/documents are required to certify the customer as a veteran? For
POS?
Probe:
- Does verification/certification of veteran status depend on the services the
customer is requesting? Please explain.
11. Do veterans and eligible spouses typically come to the American Job Center with
some familiarity with services available to veterans, including POS? If so, from
where/whom do they tend to receive the information?
Probes:
- To what extent do they receive information from their military base (for those
recently discharged), veterans’ service organizations [VSOs], the internet,
and/or friends?
- Please tell me about any outreach you do (out-stationing, brochures, and so
on).
- Do other staff at the American Job Center conduct any outreach? Please
describe.
12. Under what circumstances would you (as DVOP/LVER) first meet with the
customer? How is it determined if the customer will meet with the DVOP, LVER, or
a WIA counselor?
Probes:
- Is the determination based on availability?
- Is the determination based on the client’s needs or request for particular
services? Please explain.
- Can they be working with both a DVOP/LVER and a WIA counselor?

D. PROVISION OF SERVICES
I’m now interested in learning how veterans receive services at the center, and how it may or
may not differ from how other customers receive services.
13. Once enrolled, do veterans work with an assigned case manager or specific types of
staff? How are these assignments made? How does this differ from the process for
other American Job Center customers?

DVOP/LVER Guide

3

14. How is POS implemented in practice? Please explain how a veteran receives priority
of service when he/she is first in contact with the workforce system. (This could be
at a center, at home on a computer, or by some other means.)
Probes:
- How is the intake and flow different for veterans than for non-veteran
customers?
- What is your role in POS?
- Do veterans have priority for job openings that are posted? If so, please
describe how this works. Are veterans’ resumes flagged in the system? Do
employers see an indicator of any kind that the person is a veteran?
- If a veteran requests training services, how are they served? What occurs if the
LWIA/center has a waiting list?
15. To what extent do you (as LVER/DVOP) collaborate with others also providing
services to veterans? To what extent do you work with the WIA and ES staff to meet
the needs of your veteran customers? With other partners?
Probes:
- To what extent do customers on your caseload overlap with customers on WIA
counselors’ caseloads? On the ES caseload?
- Do you communicate about individual veterans? About what? How often?
16. What services do you as a LVER/DVOP most often provide to veterans? How, if at
all, does this differ by the type of veteran customer? (This can include things like
working with employers who are then interested in hiring veterans.)
Probes:
- What differences, if any, by age of veteran, or time since on active duty?
- What differences, if any, by gender?
17. How do you help veterans translate their military experiences, training, and jobs to
civilian opportunities? Do you use specific tools? What are they? How do you find
out about these tools?
18. How do you or the American Job Center track services provided to veterans and POS
eligibles and the services they receive? What information is tracked? Who has access
to this information?
Probe:
- Are veteran services tracked separately for ES, Veterans’ State Grants, and
WIA?
- Is information about veterans reported in separate systems for ES, WIA, and
Veterans’ State Grants? If so, are there any links between systems?
- Are staff from various programs able to see each other’s case notes?
19. To what extent do you think you are providing services to veterans using unique or
innovative strategies? Please explain. Have these strategies been developed at the
local level or state? Please explain.
DVOP/LVER Guide

4

20. What have been the challenges to working with veterans?
21. How do you know if you are successful with your veteran customers? What is the
process for following-up with them?

E. CLOSING
Thank you again for taking the time to speak with me. I appreciate and value your input.
22. Do you have anything you would like to tell me or questions you would like to ask
me?

DVOP/LVER Guide

5

This page has been left blank for double-sided copying.

VETERANS’ SUPPLEMENTAL STUDY OF
THE WIA GOLD STANDARD EVALUATION
Veteran and Eligible Spouse Focus Group Discussion Guide

LWIA/American Job Center:

_________________________________

Focus Group Date/Time:

_________________________________

Focus Group Moderator:

_________________________________

A. INTRODUCTION TO THE STUDY
Hi, my name is _______________ and I work for Mathematica Policy Research/Social Policy
Research, an independent social policy research firm. Thank you very much for agreeing to
participate in this discussion. As you might know, we are conducting a study for the U.S.
Department of Labor (DOL) concerning services provided to veterans through One-Stop Career
Centers/American Job Centers. Your participation is very important to the study. During my
visit, I will also be talking with state and center staff about how services are provided.
I would like you to know that the project team has received approval from the Office of
Management and Budget to conduct this study (OMB Control Number xxxx-xxxx). We could
not conduct this session without that approval. Your responses are completely voluntary and will
be used for research purposes only.
We will not share your comments today with anyone outside the immediate study team at
Mathematica. Our report will describe the experiences and viewpoints expressed, but comments
will not be attributed to specific individuals.
I would like to record our discussion so I can listen to it later when I write up my notes. No one
outside the immediate research team will listen to the tape. If you want to say anything that you
do not want taped, please let me know and I will be glad to pause the tape recorder. Do you have
any objections to being part of this discussion or to my taping our discussion?
This discussion will last about 60 minutes. There are no right or wrong answers. We are
interested in hearing about your experiences and opinions. You also do not have to answer
questions that make you uncomfortable.
Once again, thank you for participating. Do you have any questions before we get started?

Veterans

1

B. BACKGROUND
First I’d like to find out a little about each of you. If you are a veteran, please tell us what branch
you were in and when you separated. If you’re a spouse, please tell us about your spouse’s
branch and how long you’ve been an eligible spouse.
C. AWARENESS
1. How did you first learn about the services available to veterans at the Center?
Probes:
- Did you hear about the services in relation to your status as a veteran (or
covered spouse)? If so, please explain where and when.
- Did you receive an orientation about available services when you first came to
the Center? If so, please describe.
2. What is your understanding of the services that you are eligible to receive through
the One-Stop Career Center/American Job Center as a veteran (or eligible spouse)?
How is that different from non-veterans?
Probes:
- Are you familiar with Priority of Service (POS)? If so, what is that?
- Are you familiar with the Gold Card? If so, what is that?
3. [If familiar with POS and/or Gold Card:] When did you first learn about veterans’
priority of service? From whom? How about the Gold Card initiative?
Probe:
- Do you recall who at the center first mentioned POS to you?
4. When and how was your status as a veteran determined by the One-Stop Career
Center/American Job Center staff?
Probes:
- What documents did you need to provide?
- At what point in the process did you need to provide documents to verify your
veteran status?
D. SERVICES
5. When you first came to the One-Stop Career Center/American Job Center, were
there particular services you were seeking? Please describe.
6. What kinds of services have you received or participated in through the center?
Please tell me about them.
Probes:
- Have you participated in workshops? Assessments?
- Have you met with a counselor? Have you developed an IEP?
- Are you in training?
Veterans

2

7. To what extent have you received the services you needed/expected to receive?
Please describe how your expectations have or have not been met.
Probes:
- What services/activities have worked best for you at the Center? Why?
- What has worked least well? Why?
8. Have there been any circumstances in which you feel you did not receive timely
services at the center? If so, please describe what happened and how, if at all, it was
resolved.
9. Please tell me about the staff who you’ve met here at the center.
Probes:
- When you come to the Center, do you meet regularly with one or two
particular staff members? Who?
- To what extent have you found Center staff to be knowledgeable about and
experienced with working with veterans?
- Which staff were less knowledgeable? Which more? Please describe a
particular situation.
- What effect did any differences in staff knowledge about veterans have on
your experience with the One-Stop Career Center/American Job Center?
10. Have you found that staff understand your military skills and experiences? Have
they been able to help translate your military skills and experiences to civilian
terminology and resume formats?
Probes:
- Did they use any specific tools to try to do this? If so, what kinds of tools?
- If they were not able to understand or translate your skills, please describe a
situation where this was evident. What could staff have done differently?
E. CLOSING
11. Do you have any suggestions for how the One-Stop Career/American Job Center
System could improve the services it provides to veterans? Please describe.
12. Those are my questions. Do you have anything you would like to tell me or
questions you would like to ask me?

Thank you again for taking the time to speak with me. I appreciate and value your input.

Veterans

3

This page has been left blank for double-sided copying.

OMB Control No.: xxxx-xxxx
Expiration Date: xx/xx/20xx

Veterans’ Supplemental Study
Focus Group Information Sheet
Background
1.

Name:

_______________________________________________

2.

Date of Birth:

|

3.

Gender:

4.

What is the highest level of education you have completed?

| |/| | |/| 1 | 9 | |
Month
Day
Year
1

 Male

2

 Less than high school
 High School Diploma or GED
5.

|

 Female

 Some college, no degree
 Associate’s degree

 Bachelor’s degree
 Graduate degree

Are you currently enrolled in training or education?
 Yes, please describe: ________________________________________________________________
 No

Veteran Background
If you are a veteran, please answer the following questions about yourself. If you are a POS eligible spouse,
please answer them about your spouse.
 Army  Marine Corps  Navy  Air Force  Coast Guard

6.

In what branch(es) did you serve?

7.

What was your Military Occupational Specialty (MOS) or equivalent? ____________________________________

8.

Do you have a service connected disability?

 No  Yes ==>

9.

Have you been deployed in the past two years?

 No  Yes

10. When did you last separate from the military?
11. What was your rank at separation?

8a. If yes, what percentage? ______%

________________________
(mm/dd/yyyy)
_______________________

Employment
12. Since separating from the military, have you worked for pay in the civilian job market?  No=>END of form
Yes
 No Yes

13. Are you currently employed?

ANSWER NEXT QUESTIONS ABOUT YOUR CURRENT OR MOST RECENT JOB. (If you currently have more than one
job or had more than one job recently, give answers about your job with the most hours.)
14. When did you start work at this current or most recent job?

___________________________
(mm/dd/yyyy)
15. How many hours per week do (or did) you usually work at your main job? | | | HOURS PER WEEK

THANK YOU FOR YOUR TIME.
Public reporting burden for this collection of information is estimated to average 3 minutes per respondent, including the time for reading and
responding to questions. Send comments concerning this burden estimate or any other aspect of this collection of information to the U.S.
Department of Labor, Employment and Training Administration, WIA Evaluation Room N-5641, 200 Constitution Ave. NW, Washington, DC,
20210. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a
collection of information unless it displays a valid OMB control number. The OMB control number for this information collection is xxxx-xxxx.
Expiration Date xx/xx/20xx.

This page has been left blank for double-sided copying.

VETERANS’ SUPPLEMENTAL STUDY OF
THE WIA GOLD STANDARD EVALUATION
Case File Review
LWIA/American Job Center:

____________________________

Site Visitor Name:

____________________________

Date:

______________________________

Customer First Name:

_____________________________

Customer Status:

DW

Adult

(can check both)

Background: Veteran Status and Other
When was this customer on active duty? When was he/she most recently
discharged? (For covered spouse, why/when did the spouse become eligible?)
What was this customer’s rank at discharge? (Or: What was the last rank of the
spouse who created eligibility?)
What documentation was provided to verify the customer’s eligibility? At what
point(s) in the process was it requested and provided?
What is the customer’s educational background?
What is his/her history in the civilian labor market?

Services
Is there any evidence that the customer received POS? Please describe.
Where is this customer in the customer flow process?
Has an IEP been developed? If so, what are the customer’s goals?
Is the customer working with a counselor on a training plan?
Does the customer have an approved training plan? If so, describe.
If the customer is in training, are there grade reports in the file? Is
there information on the ITA?
Who has met with the customer? Differentiate between LVER/DVOP, WIA, and
ES case manager staff. How many times?
For this customer, in what ways did staff at the American Job Center –
LVER/DVOP, WIA, and ES--coordinate delivery of service or implementation of
POS?
Case Review

1

What assessments, (if any), has the customer completed? Were they completed,
and feedback given, online, in person, or some other way? How were those
assessments used to determine appropriate services?
What key barriers, (if any) does the customer face?
Is the customer co-enrolled in any other programs? If yes, which ones?
What other services has the customer accessed prior to this meeting?
Workshops? Which ones?
Supportive services? What types? How many times?
Other?
Has the customer been denied any services to which he was referred?

Case Review

2