Form IVA Staff Survey IVA Staff Survey IVA Staff Survey

OMB No.: xxxx-xxxx
Expiration date: xx/xx/xxxx

Staff Survey
Regional Partnership Grants National
Cross-Site Evaluation
November 5, 2013

Public reporting burden for this collection of information is estimated to average 25 minutes per response, including the time for reviewing
instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of
information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it
displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of
information, including suggestions for reducing this burden, to: XXX ATTN: XXX (xxxx--xxxx). Do not return the completed form to this address.

INTRODUCTION

The Children’s Bureau within the U.S. Department of Health and Human Services, Administration for Children
and Families (ACF) has contracted with Mathematica Policy Research to complete the national cross-site
evaluation of the Regional Partnership Grants (RPG) program. The evaluation will describe the interventions
that were implemented, the nature of the partnerships, the types of services provided, and their impacts.
You are asked to complete this survey because you were identified as a front-line staff member who works
directly with RPG participants. Your participation is important to helping us understand the characteristics of
the staff and organizations implementing RPG-funded programs.
The length of this survey is different for different people, but on average it should take about 25 minutes. Not
all response options may apply to you or your organization. Please choose the best answer to each question.
You may also choose not to answer any question.
The evaluation focuses on specific evidence-based programs (EBPs), and many questions in the survey will
reference a specific EBP. Please answer the questions about the specific program that is listed and not other
programs that your organization may operate.
Your responses will be kept private and used only for research purposes. They will be combined with the
responses of other staff and no individual names will be reported. Participation in the survey is completely
voluntary.
If you have any questions about the survey, please contact the team at Mathematica by calling
1-xxx-xxx-xxxxx (toll-free) or emailing xxxxxxx@mathematica-mpr.com.
Before starting the survey, please read and answer the statement below.
i1.

I have read the introduction and understand that the information I provide will be kept private and used
only for research purposes. My responses will be combined with the responses of other staff and no
individual names will be reported.

□
0 □
1

i2.

I agree with the above statement and will complete the survey
I do not agree with the above statement and will not complete the survey

END

Could you please confirm whether you work for [RPG PROGRAM] at [ORGANIZATION]?
MARK ONE ONLY

□
0 □
d □
1

Yes, I work for [RPG PROGRAM] at [ORGANIZATION]
No
Don’t know

END

i

A. YOUR WORK ROLE AND EXPERIENCE

A1.

Which of the following is closest to your job title?
MARK ONE ONLY

□ Mental health counselor, therapist, or psychologist
2 □ Early intervention or child development therapist
3 □ Substance abuse counselor
4 □ Family advocate
5 □ Child welfare case manager
6 □ Other case manager
7 □ Social worker
8 □ Recovery coach
9 □ Child development specialist
10 □ Other (Specify)
1

A2.

How long have you been employed at [ORGANIZATION]?
Please include the total time you have been employed at the organization, not just the time you have been
in your current position.
|

|

| MONTHS OR |

|

| YEARS

1

A3.

The next questions are about your work activities at [ORGANIZATION]. Which of the following
activities do you take part in on this job at least once every two weeks?
Please answer thinking about your job as a whole, not just activities related to implementing RPG.
MARK ONE PER ROW
AT LEAST
ONCE
EVERY
TWO
WEEKS

a.

Screen or assess potential participants for program eligibility .................................

b.

Conduct participant intake .......................................................................................

c.

Conduct substance abuse screening ......................................................................

d.

Conduct substance abuse assessment ...................................................................

e.

Conduct risk assessment for child abuse, neglect, and other risk factors ...............

f.

Screen children for prenatal substance exposure, developmental delays,
emotional or mental health problems, or substance use disorder ...........................

g.

Provide parenting education ....................................................................................

h.

Provide case management services ........................................................................

i.

Develop coordinated care plans ..............................................................................

j.

Monitor the implementation and the quality of screening and assessment
protocols ..................................................................................................................

k.

Conduct group therapy sessions .............................................................................

l.

Conduct individual therapy sessions .......................................................................

m. Conduct motivational interviewing sessions (conversations to elicit and
strengthen motivation for change) ...........................................................................
n.

Conduct parent-child therapy sessions....................................................................

o.

Coordinate services for participants with other partner agencies ............................

p.

Manage or supervise other individuals at your organization ....................................

q.

Train other staff at your organization .......................................................................

r.

Hold family team conferences, multidisciplinary team meetings, or joint client
staffing .....................................................................................................................
Work with clients to accomplish designated treatment goals (for example, job
searching, housing applications) .............................................................................

s.
t.

Conduct administrative activities (for example, paperwork) .....................................

u.

Other activities (Specify)..........................................................................................

2

□
1 □
1 □
1 □
1 □
1

□
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1 □
1

□
1 □
1 □
1

NOT AT
LEAST
ONCE
EVERY
TWO
WEEKS

□
0 □
0 □
0 □
0 □
0

□
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0 □
0

□
0 □
0 □
0

DON’T
KNOW

□
d □
d □
d □
d □
d

□
d □
d □
d □
d □
d □
d □
d □
d □
d □
d □
d □
d □
d

□
d □
d □
d

A4.

How long have you been providing services to child welfare involved children and families?
Please account for all work you have done for current and past organizations related to providing services
to child welfare involved children and families.
d

□

|

A5.

I have not done any work related to providing services to child welfare involved children and families
|

| MONTHS OR |

|

| YEARS

How long have you been providing substance abuse assessment or treatment services?
Please account for all work you have done for current and past organizations related to substance abuse
assessment or treatment services.
d

|

□

I have not done any work related to substance abuse assessment or treatment services
|

| MONTHS OR |

|

| YEARS

3

B. IMPLEMENTING AN EVIDENCE-BASED PROGRAM

B1.

The following statements are about feelings someone might have about using new types of therapy,
interventions, or treatments. To what extent do you agree with each statement?
Manualized therapy, intervention, or treatment refers to any intervention that has specific guidelines and/or
components that are outlined in a manual and/or that are to be followed in a structured or predetermined
way.
MARK ONE PER ROW

NOT AT
ALL

a. I like to use new types of therapy/interventions to
help my clients ...........................................................

0

b. I am willing to try new types of therapy/interventions
even if I have to follow a treatment manual ...............

0

c.

I know better than academic researchers how to
care for my clients ......................................................

0

d. I am willing to use new and different types of
therapy/interventions developed by researchers .......

0

e. Research based treatments/interventions are not
clinically useful ...........................................................

0

f.

Clinical experience is more important than using
manualized therapy/interventions ..............................

0

g. I would not use manualized therapy/interventions .....

0

h. I would try a new therapy/intervention even if it were
very different from what I am used to doing ...............

0

4

□
□
□
□
□
□
□
□

TO A
SLIGHT
EXTENT

1

1

1

1

1

1

1

1

□
□
□
□
□
□
□
□

TO A
MODERATE
EXTENT

2

2

2

2

2

2

2

2

□
□
□
□
□
□
□
□

TO A
GREAT
EXTENT

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□

TO A
VERY
GREAT
EXTENT

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□

B2.

If you received training in a therapy or intervention that was new to you, how likely would you be to
adopt it if…
MARK ONE PER ROW
NOT AT
ALL

a. it was intuitively appealing?. .................................

0

b. it “made sense” to you?. .......................................

0

c.

it was required by your supervisor?. .....................

0

d. it was required by [ORGANIZATION]?. ................

0

e. it was required by your state? ...............................

0

f.

it was being used by colleagues who were happy
with it? ...................................................................

0

g. you felt you had enough training to use it
correctly? ...............................................................

0

5

□
□
□
□
□
□
□

TO A
SLIGHT
EXTENT
1

1

1

1

1

1

1

□
□
□
□
□
□
□

TO A
MODERATE
EXTENT
2

2

2

2

2

2

2

□
□
□
□
□
□
□

TO A
GREAT
EXTENT
3

3

3

3

3

3

3

□
□
□
□
□
□
□

TO A VERY
GREAT
EXTENT
4

4

4

4

4

4

4

□
□
□
□
□
□
□

B3.

Organizations have a “personality” that is reflected in the day to day operations of the organization
and the way staff members view their work. These items ask about some dimensions that relate to the
use of [EBP NAME] in organizations. For each item, please indicate the extent to which you disagree or
agree the statement is true for [ORGANIZATION]. Within the past six months…
MARK ONE PER ROW
STRONGLY
DISAGREE

a.

b.
c.

d.

e.

f.

g.

h.

i.

Staff members are adequately trained
to implement [EBP NAME] at this
organization ........................................

1

Top administration strongly supports
the implementation of [EBP NAME] ....

1

Staff members get positive feedback
and/or recognition for their efforts to
implement [EBP NAME] .....................

1

Top administrators minimize
obstacles and barriers to
implementing [EBP NAME] at this
organization ........................................

1

This organization established clear
and specific goals related to the
implementation of [EBP NAME]..........

1

There are performance-monitoring
systems in place to guide the
implementation of [EBP NAME]..........

1

Training and technical assistance are
readily available to staff members
involved in implementing [EBP
NAME] ................................................

1

Adequate resources are available to
implement [EBP NAME] as
prescribed...........................................

1

Staff members have been
encouraged to express concerns that
arise in the course of implementing
[EBP NAME] .......................................

1

DISAGREE

□

2

□

2

□

2

□

2

□

2

□

2

□

2

□

2

□

2

□
□
□
□
□
□
□
□
□

AGREE

3

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□
□

STRONGLY
AGREE

4

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□
□

DOES NOT
EXIST IN OUR
ORGANIZATION

n

n

n

n

n

n

n

n

n

DON’T
KNOW

□
□
□
□
□
□
□
□
□

If you are not a supervisor, please go to question C1.
If you are a supervisor, please continue to question B4. The next questions in this section are about your
experiences implementing [EBP NAME].

B4.

When implementing a program, it often happens that changes get made to meet the needs of
participants, the timeline, organizational resources, or some other factor. Has [ORGANIZATION]
adapted [EBP NAME] for any reason?

□
0 □
d □
1

Yes
No

GO TO C1

Don’t know

GO TO C1

6

d

d

d

d

d

d

d

d

d

□
□
□
□
□
□
□
□
□

B5.

What kinds of adaptations to [EBP NAME] were made?
MARK ALL THAT APPLY

□
2 □
3 □
4 □
5 □
6 □
7 □
8 □
9 □
1

d

B6.

□

Changed procedures
Changed the sequence of sessions
Increased the number of sessions
Decreased the number of sessions
Changed the length of sessions
Changed the target population
Changed program content
Changed for cultural relevance
Other (Specify)
Don’t know

There are several possible reasons why an organization might choose to make changes to a program.
To what extent did the following factors contribute to any changes being made to [EBP NAME]?
MARK ONE PER ROW
NOT
AT
ALL

a. Difficulty recruiting participants ..........................

1

b. Difficulty retaining or engaging participants .......

1

c.

Difficulty finding adequate staff ..........................

1

d. Lack of or limited resources (such as space
or time) ...............................................................

1

e. Lack of time or competing demands on time .....

1

f.

Resistance from implementing staff ...................

1

g. Need for a more culturally appropriate
program ..............................................................

1

h. Requests for changes by participants ................

1

7

□
□
□
□
□
□
□
□

PRIMARY
REASON FOR
CHANGE
2

2

2

2

2

2

2

2

□
□
□
□
□
□
□
□

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□

4

4

4

4

4

4

4

4

□

5

□

5

□

5

□

5

□

5

□

5

□

5

□

5

□
□
□
□
□
□
□
□

DON’T
KNOW
d

d

d

d

d

d

d

d

□
□
□
□
□
□
□
□

C. SUPERVISION AND SUPPORT

The next questions ask about supervision you may receive as a staff member for [RPG PROGRAM]. If you
have more than one supervisor, please answer these questions about the supervisor you work with the most
in the [RPG PROGRAM].
C1.

Is there at least one person at [ORGANIZATION] whom you regard as your supervisor?
MARK ONE ONLY

□
0 □
d □
1

C2.

Yes
No
Don’t know

GO TO C5

In the past 12 months, how often did you have formal, one-on-one supervision meetings?
MARK ONE ONLY

□
2 □
3 □
4 □
5 □
6 □
7 □
d □
1

C3.

Never
Daily
Weekly
Twice per month
Monthly
Once every few months
Yearly
Don’t know

In the past 12 months, how often did you have group supervision meetings with other staff members?
MARK ONE ONLY

□
2 □
3 □
4 □
5 □
6 □
7 □
d □
1

Never
Daily
Weekly
Twice per month
Monthly
Once every few months
Yearly
Don’t know

8

C4.

In the past 12 months, how often did you participate in meetings, trainings, or other joint activites with
staff from RPG partner agencies?
MARK ONE ONLY

□
2 □
3 □
4 □
5 □
6 □
7 □
d □
1

Never
Daily
Weekly
Twice per month
Monthly
Once every few months
Yearly
Don’t know

9

C5.

Please read the following statements and decide how strongly you disagree or agree with each
statement. My supervisor…
MARK ONE PER ROW
STRONGLY
DISAGREE

a.

encourages staff to spend time
mentoring new employees?.............

1

encourages staff to help each other
with work problems? ........................

1

c.

cares about me as a person? ..........

1

d.

provides emotional support to me in
difficult situations with RPG
program participants? ......................

1

is appropriately flexible when it
comes to applying rules? .................

1

has an attitude that helps me be
enthusiastic about working in social
services? .........................................

1

supports me in balancing the
demands of my job with my
personal life? ...................................

1

provides the help I need to do my
job? .................................................

1

knows effective ways to work with
RPG program participants? .............

1

is willing to help me complete
difficult tasks? ..................................

1

k.

encourages creative solutions? .......

1

l.

reinforces the training I receive? .....

1

m. helps me learn and improve? ..........

1

n.

is available when I ask for help? ......

1

o.

has expectations for my work that
are challenging but reasonable? .....

1

gives me clear feedback on my job
performance? ..................................

1

has helped staff develop into an
effective team? ................................

1

b.

e.

f.

g.

h.

i.

j.

p.

q.

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

DISAGREE
2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

10

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

SOMEWHAT
DISAGREE
3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

SOMEWHAT
AGREE
4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

AGREE
5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

STRONGLY
AGREE
6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

DON’T
KNOW
d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

C6.

Overall, how supported do you feel by the other staff working at [ORGANIZATION]?
MARK ONE ONLY

□
2 □
3 □
d □
1

C7.

Very supported
Somewhat supported
Not very supported
Don’t know

How strongly do you agree or disagree that overall, the staff at [ORGANIZATION] works as a team?
MARK ONE ONLY

□
2 □
3 □
4 □
d □
1

C8.

Strongly agree
Agree
Disagree
Strongly disagree
Don’t know

How strongly do you agree or disagree that overall, the your organization’s RPG program and its
partners work as a team?
MARK ONE ONLY

□
2 □
3 □
4 □
d □
1

Strongly agree
Agree
Disagree
Strongly disagree
Don’t know

11

C9.

Please read the following statements and rate how dissatisfied or satisfied you are with each with
regard to [EBP NAME]. Overall, how satisfied are you that...
MARK ONE PER ROW

VERY
DISSATISFIED

a.

b.

c.

d.

e.

f.

g.

h.

i.

the information you received during
your hiring process reflects the work
you are being asked to do? ....................

1

the training you are receiving is
preparing you to work effectively with
families and children? .............................

1

the coaching you are receiving is
improving your skills and abilities to
work effectively with families and
children? .................................................

1

the challenges you encounter in
providing effective services are
understood in your organization? ...........

1

the challenges you encounter in
providing effective services are being
actively addressed by your
organization? ..........................................

1

the challenges you encounter in
providing effective services are
understood by the RPG program
leadership? .............................................

1

the challenges you encounter in
providing effective services are being
actively addressed?................................

1

your immediate supervisor helps you
develop your [EBP NAME] skillset?........

1

your organization’s administrators
effectively develop the supports and
conditions that make it possible for you
to work effectively with children and
families? .................................................

1

□

SLIGHTLY
DISSATISFIED

2

□

2

□

2

□

2

□

2

□

2

□

2

□

2

□

2

12

□
□
□
□
□
□
□
□

□

NEITHER
SATISFIED
NOR
DISSATISFIED

3

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□

□

SLIGHTLY
SATISFIED

4

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□

□

VERY
SATISFIED

5

5

5

5

5

5

5

5

5

□
□
□
□
□
□
□
□

□

D. ORGANIZATIONAL CLIMATE

D1.

Please read the following statements and decide how strongly you disagree or agree with each
statement with regard to [ORGANIZATION].
MARK ONE PER ROW
STRONGLY
DISAGREE

a.

The mission of this organization is
clear to me.......................................
b. My work reflects the organization’s
purpose ...........................................
c. I feel good about what this
organization does for RPG
participants ......................................
d. In this organization, there is more
emphasis on the quality of services
than on the number of participants
served..............................................
e. I am satisfied with the salary I
receive from this organization..........
f. I am paid fairly considering my
education and training .....................
g. I am paid fairly considering the
responsibilities I have ......................
h. I am satisfied with the physical
work environment at this
organization .....................................
i. I am proud to tell others that I am
part of this organization ...................
j. The administration shows concern
for staff ............................................
k. Employees of this organization are
respected by other community
professionals ...................................
l. This organization is committed to
my personal safety in the office .......
m. This organization is committed to
my personal safety when working
off-site..............................................
n. My professional opinions are
respected in this organization ..........
o. I have sufficient input in formulating
policies that govern my work ...........
p. There are strong, positive
relationships between this
organization and other community
resource providers ...........................
q. I have the support to make workrelated decisions when appropriate .
r. Organizational management shares
leadership roles with staff ................
s. This organization effectively
responds to public criticism when it
occurs ..............................................

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

DISAGREE
2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

13

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

SOMEWHAT
DISAGREE
3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

SOMEWHAT
AGREE
4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

AGREE
5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

STRONGLY
AGREE
6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

DON’T
KNOW
d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

d

□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□

E. DEMOGRAPHICS

These next questions ask about your background.
E1.

Are you Hispanic or Latino?
MARK ONE ONLY

□
1 □
d □
0

E2.

No
Yes
Don’t know

What is your race?
MARK ALL THAT APPLY

□
2 □
3 □
4 □
5 □
6 □
1

d

E3.

□

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White
Other (Specify)
Don’t know

What is the highest level of education you have completed?
MARK ONE ONLY

□
2 □
3 □
4 □
5 □
6 □
7 □
8 □
9 □
d □
1

Did not complete high school or General Educational Development
High school diploma
General Educational Development
Some college/some postsecondary vocational courses
2-year or 3-year college degree (Associate’s degree)
Vocational school diploma
4-year college degree (Bachelor’s degree)
Some graduate work/no graduate degree
Graduate or professional degree (for example, MA, MBA, Ph.D., JD, or MD)
Don’t know

14

E4.

What is your profession or area of work?
MARK ALL THAT APPLY

□ Substance abuse counseling
2 □ Other counseling
3 □ Education
4 □ Vocational rehabilitation
5 □ Juvenile justice
6 □ Psychology
7 □ Social work/human services
8 □ Medicine
9 □ Administration
10 □ Student
11 □ Other (Specify)
1

□ None of these
d □ Don’t know

12

E5.

Are you male or female?

□
2 □
1

E6.

Male
Female

Is there anything else about your experiences implementing RPG that you would like to add?

______________________________________________________________________________________________
______________________________________________________________________________________________
______________________________________________________________________________________________
(End of survey for those who opt out in the first screen)
Thank you for considering participation in this survey. Please click the “Submit survey” button in the lower
right hand corner so that we have a record of your desire NOT to participate. This will result in your removal
from our contact list.
(End of survey for those who are ineligible in the first screen)
Thank you for considering participation in this survey. Please click the “Submit survey” button in the lower
right hand corner and we will remove you from our contact list.
(End of survey for respondents)
Thank you for completing the Regional Partnership Grant Staff Survey! Please click the “Submit survey”
button in the lower right hand corner to submit your completed survey.

15