Attachment L - Question by Question Justification - Staff Survey

DRAFT

ATTACHMENT L

STAFF SURVEY Question-BY-Question JUSTIFICATION

This page intentionally left blank for double sided copying

This document provides the source and justification for each question on the Staff Survey (Attachment E).

Question #	Question text	Source	Justification
SECTION A. BACKGROUND AND PROGRAM INVOLVEMENT
1	What is your title in your current position with [program name or agency]?	STED/ETJD (OMB No. 0970-0413)	These items measure staff background demographic characteristics.
2	How long have you been working in the position of [title from Q1] at [program name or agency]?
3	How long have you worked for [organization name]?
4	How would you describe yourself?	SOGI Report
5	What is your age?	STED/ETJD (OMB No. 0970-0413)
6	Are you Hispanic, Latino, or of Spanish origin?	OMB
7	What is your race?
8	What is the highest level of education you have completed?	Adapted from cobra (OMB No. 1291-0001)
9	In your position of [title from Q1] at [program name or agency], are you a full-time employee, part-time employee, full-time contractor or part-time contractor?	Adapted from STED/ETJD (OMB No. 0970-0413)
10-M	Which of the following are among your responsibilities in your position of [title from Q1]?	Adapted from HPOG-NIE (OMB No. 0970-0394)	These items measure the role of the staff member.
10-S	Which of the following are among your responsibilities in your position of [title from Q1]?
11-M	On average, which of your responsibilities do you spend the most time on in a typical week?
11-S	On average, which of your responsibilities do you spend the most time on in a typical week?
12	How much total work experience (including your current and prior positions) do you have in performing responsibilities similar to those you carry out as part of [program name or agency]?	JSA (OMB No. 0970-0400)	These items measure the staff member’s employment background.
12a	How much total work experience (including your current and prior positions) do you have in coaching and/or supervising coaching?	Adapted from JSA (OMB No. 0970-0400)
13a-M	In your position of [title from Q1] at [program name or agency], do you formally manage/supervise staff on an ongoing basis?	Adapted from HPOG-NIE (OMB No. 0970-0394)	These items measure the role the staff member plays in the organization.
13b-M	If yes, how many staff do you typically manage/supervise?
13c-M	Do you supervise: coaches, case managers, workshop instructors, employment assistance frontline staff (e.g., job developers, trainers), administrative staff, and/or other?
14a	In your position of [title from Q1] at [program name or agency], are you responsible for working with a number of participants on an ongoing basis (i.e., do you carry a “dedicated caseload” or fill in for frontline staff when needed)?
14b	Do you work with: only participants in [treatment group], only participants in [control group], both, do not work with participants, or don’t know?
14c	On average, how many participants do you typically work with (i.e., what is your caseload) each month?

14d	During a typical session with a participant, what percentage of your time do you spend supporting the participant to set goals or work on plans to achieve goals?	New Developed by Mathematica/Abt	These items measure the extent to which staff use the goal setting approach.
14e	During a typical session with a participant, what percentage of your time do you spend describing and directing the participant to support services?
15a	In your position of [title from Q1] at [program name or agency], have you participated in coaching training?		These items measure the extent to which staff have been trained in coaching.
15b	If yes, approximately when did you complete your initial coaching training at [program name or agency]?
15c	If yes, approximately when did you complete your most recent coaching training at [program name or agency]?
15d	What method best describes your initial coaching training and your most recent coaching training?
15e	How many hours of coaching training have you participated in, including initial and follow-up sessions?
15f	Did your coaching training include a manual or other written materials?
15f-S	Do you think the manual or other written materials are useful?
15g	Has your training included follow-up sessions or check-ins after your initial coaching training through the following methods? Peer-to-peer coaching meetings Peer-to-peer observations Case reviews Webinars Other method (please specify)
16a	Does your supervisor or manager reinforce coaching techniques and strategies with you?
16b	If yes, please select all that apply.
16c	How frequently does your supervisor or manager use coaching techniques and strategies when meeting with you?
SECTION B. TYPES OF [PROGRAM NAME] SERVICES PROVIDED
17-M	Based on your discussions with coaches, what is your perception of how often participants in [program name or agency] are able to accomplish the following: Identify a long-term goal Identify short-term goals Break down goals into small achievable steps on their own Use an identified goal to motivate themselves Identify potential challenges that may get in the way of accomplishing a goal Make adjustments to steps if unable to accomplish goal Celebrate their achievement of a goal	New Developed by Mathematica/Abt	These items measure staff beliefs in program participants’ capacity to set and achieve goals.
17a-S	Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often participants who have been in [program name or agency] for less than one month are able to work with their [coach/case manager] to accomplish the following: Identify a long-term goal Identify short-term goals Break down goals into small achievable steps on their own Use an identified goal to motivate themselves Identify potential challenges that may get in the way of accomplishing a goal Make adjustments to steps if unable to accomplish goal Celebrate their achievement of a goal
17b-S	Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often participants who have been in [program name or agency] for more than one month are able to work with their [coach/case manager] to accomplish the following: Identify a long-term goal Identify short-term goals Break down goals into small achievable steps on their own Use an identified goal to motivate coaches Identify potential challenges that may get in the way of accomplishing a goal Make adjustments to steps if unable to accomplish goal Celebrate their achievement of a goal
18	Please indicate about how often you use the following [resources/process from curricula] with participants. Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often these resources are used during one-on-one sessions with participants: [TOOL A] [TOOL B] [TOOL C] [TOOL D] [TOOL E] [TOOL F] [TOOL G] [TOOL H] Tangible incentives (e.g., gift cards, tokens used to buy items, other items) Other (please specify)	JSA (OMB No. 0970-0400)	These items measure staff use of treatment intervention materials.
19	Please indicate about how valuable you find each of these resources in helping participants gain and retain employment: [TOOL A] [TOOL B] [TOOL C] [TOOL D] [TOOL E] [TOOL F] [TOOL G] [TOOL H] Tangible incentives (e.g., gift cards, tokens used to buy items, other items) Other (please specify)
20	Please indicate how much time it takes on average to complete each of the following [program name or agency] resources with a participant: [TOOL A] [TOOL B] [TOOL C] [TOOL D] [TOOL E] [TOOL F] [TOOL G] [TOOL H] Tangible incentives (e.g., gift cards, tokens used to buy items, other items) Other (please specify)
SECTION C. AMOUNT OF STAFF SUPERVISION AND MONITORING
21-M	On average, how often do you Communicate with case managers about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)? Communicate with coaches about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)? Communicate directly with participants about their individual situations (e.g., participant progress, strengths, challenges to participation)? Observe case managers conduct sessions with participants? Observe coaches conduct sessions with participants?	Adapted from HPOG-NIE (OMB No. 0970-0394)	These items measure staff frequency and mode of contact with participants and supervisors.
21a-S	On average, how often do you have contact with participants through each of the following methods? In person, one-on-one session In person, group session Over the phone By email or other electronic communication Other method (please specify)	Adapted from JSa (OMB No. 0970-0400)
21b-S	On average, how often do you work with your supervisor on an ongoing basis to practice or review coaching techniques through each of the following methods? In person, one-on-one session In person, group session Over the phone By email or other electronic communication Other method (please specify)
22-S	On average, how often do you Communicate with program management or supervisors about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)? Communicate with other staff about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)?	Adapted from HPOG-NIE (OMB No. 0970-0394)
23a-S	Are follow-ups conducted as part of [program name or agency] with participants after they have found a job?	Adapted from STED/ETJD (OMB No. 0970-0413)
23b-S	On average, how often do you use the following contact methods with participants after they have found a job? In person, one-on-one session In person, group session Over the phone By email or other electronic communication Other method (please specify)
SECTION D. CHALLENGES TO EMPLOYMENT
24	Based on your experience, on average, how frequently do participants experience the challenges listed below? Limited education Limited prior work or volunteer experience Limited relevant vocational skills Child care or dependent care issues Transportation problems Low motivation to find employment Mental health condition(s) Physical health condition(s) Limited problem-solving abilities Drug and/or alcohol addiction Learning disabilities Intimate partner violence issues Other domestic issues (e.g., divorce, child custody) Homelessness or housing problems Criminal history Legal problems Unable to focus on what’s important Procrastination Unable to follow through to the completion of a goal Inability to set goals Limited soft skills (e.g., communication skills, social skills, teamwork) Limited number of jobs that match the education, skills, and abilities of participants Limited number of good jobs (e.g., well-paying, benefits) Participant doesn’t know where to find jobs Participant afraid to approach employers Other (please specify)	Adapted from JSA (OMB No. 0970-0400)	These items measure perceived challenges to participant employment.
SECTION E. PROGRAM PARTICIPANTS AND NONPARTICIPATION
25	Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please indicate how much you agree or disagree with the following statements about staff at [program name or agency]: Staff in this program closely monitor the progress of participants assigned to [program name or agency] Staff in this program learn quickly about participants who did not attend planned activities Staff in this program learn quickly about a participant who quit or lost a job Staff in this program explain program expectations and consequences of nonparticipation to participants Staff in this program consistently use tangible incentives (e.g., gift cards, tokens used to buy items, other items) to encourage participants Staff in this program use positive reinforcement to encourage participation (e.g., praise, clapping) Staff in this program impose sanctions on participants who do not participate or comply with program rules Staff in this program are consistent in their use of a sanctions (e.g., apply consistent criteria) Staff implement sanctions quickly after a participant stops participating in program activities Staff work hard to reengage sanctioned participants in program activities	Adapted from JSA (OMB No. 0970-0400)	These items measure staff efforts to increase program participation.
SECTION F. [PROGRAM NAME or AGENCY] AND ORGANIZATIONAL PERFORMANCE
26	Do you think coaching is an effective approach within [program name or agency]?	New Developed by Mathematica/Abt	These items measure staff perception of program effectiveness.
27a	On average, how many participants benefit from the coaching approach?
27b	In your opinion, what are the characteristics of participants who benefit most from the coaching approach?
28a	Do you think a directive and prescriptive approach with participants in [program name or agency] is effective?
28b	In your opinion, what are the characteristics of participants who benefit most from a directive and prescriptive approach?
29	Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please indicate how much you agree or disagree with the following statements about [program name or agency]: Staff make an effort to learn about participants’ personal and family situations Staff make an effort to learn about participants’ career and employment goals Staff make an effort to learn about participants’ motivation to work Staff provide information about available support services Staff assign activities for participants to do before sessions Staff help participants set personal goals Staff offer tangible incentives (e.g., gift cards, tokens used to buy items, other items) to increase participant motivation and persistence Staff check-in with participants between sessions	Adapted from JSA (OMB No. 0970-0400)	These items measure staff perception of organizational environment.
30	Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please rate how strongly you agree or disagree with each of the following statements about [organization name] and your experience in your position: Frequent staff turnover is a problem for your organization Staff are able to spend the time needed with participants Staff have the skills they need to do their jobs The [program name or agency] has enough staff to meet current participants’ needs [Program name or agency] staff are well-trained A larger support staff is needed to help meet needs in the [program name or agency] Staff training and professional development are priorities in the [program name or agency] The [program name or agency ] holds regular in-service training The budget of the [program name or agency] allows staff to attend professional training The [program name or agency ] is managed well The [program name or agency] has supervisors who are capable and qualified When needed, [program name or agency] supervisors devote much time and attention to staff supervision Management decisions for the [program name or agency] are well considered You have confidence in how decisions in the [program name or agency] are made	JSA (OMB No. 0970-0400)
30	You meet frequently with supervisors about participants’ needs and progress Staff concerns are ignored by management when making decisions about the [program name or agency] Some staff members seem confused about the main goals for the [program name or agency] The [program name or agency] operates with clear goals and objectives Your job duties are clearly related to the goals for the [program name or agency] Management for the [program name or agency] has a clear plan for its future
31	Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please rate how strongly you agree or disagree with each of the following statements about [organization name] and your experience in your position: The [program name or agency] encourages and supports professional growth for the staff Keeping your knowledge and skills up-to-date is a priority for you You do a good job of regularly updating and improving your skills You regularly seek to learn new techniques or updates in the field You are satisfied with your present job You feel appreciated for the job you do You give high value to the work you do You are proud to tell others where you work You like the people you work with You would like to find a job somewhere else The heavy staff workload reduces the effectiveness of the [program name or agency] You are under too many pressures to do your job effectively Staff members at the [program name or agency] often show signs of high stress and strain Staff frustration is common where you work Staff performance measures do not align with the coaching approach

Sources: geniuss (gender-related measures overview), National implementation evaluation of the Health profession opportunity grants (HPOG-NIE), jsa (job search assistance), subsidized and transitional employment demonstration (STED) / enhanced transitional jobs demonstration (ETJD).