Improving Child Welfare Outcomes through Systems of Care - Stakeholder Survey

Public reporting burden for this collection of information is estimated to be 0.5 hours to complete this questionnaire. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The control number for this project is 0970-0288. The control number expires on X/X/XXXX.

National Technical Assistance and Evaluation Center

Improving Child Welfare Outcomes Through Systems of Care Program

System of Care (SOC) Collaborative/Steering Committee Survey

INFORMED CONSENT

This survey is designed to help us learn more about the Systems of Care initiative in your community, specifically around issues related to continued planning and implementation, the resources available to the community, the level of collaboration among systems in your community, and changes you are seeing as a result of the initiative. You were selected to complete this survey by your local project director based on your level of activity in the Systems of Care initiative. To determine the extent to which change has occurred over time, this survey will be administered a total of three times: once in 2005, once in 2006, and now (2008).

Before you get started, there are a few other issues of which you need to be aware. First, it is important that you know that your participation in this survey is completely voluntary and that no one associated with your Systems of Care initiative will know whether or not you complete it. Also, you may skip any questions you do not wish to answer, without consequence. However, because your input is invaluable in helping the evaluation team understand the types of changes that occur as a result of the Systems of Care initiative, we would like to ask that you consider answering each question. Finally, if you do not wish to participate in this survey, please simply return the blank survey to the project coordinator/evaluator.

The survey should take about 30 minutes to complete. To protect your confidentiality, surveys will be collected by the local project coordinator/evaluator and placed in sealed envelope and mailed to Caliber, an ICF International Company. No one in your community will see your responses, including the project coordinator/evaluator or other collaborative members. All analyses and reports will only present data in aggregate form – either by County, State, or cross-site, that is by all nine grantee communities. Please do not put your name on the survey so your answers can remain completely anonymous.

As part of our agreement with the local evaluation team, Caliber, an ICF International Company, will be providing the raw data from this survey to them. However, since the survey is anonymous, there is not way to track individual respondents. Your local team, is also bound by the same confidentiality regulations as the national evaluation team (i.e., the team conducting this survey). Therefore, they are required to keep confidential the information they receive from the national team, and to report findings in summary form only. To ensure compliance to these regulations, the national evaluation team has a signed agreement with the local evaluation team.

Thank you in advance for completing this survey. You input will be integral in helping the national evaluation team understand how the Systems of Care initiative is working in communities nationwide.

SOC Collaborative/Steering Committee Survey

STATE: _____________________________________________________(PRE_FILL)

COUNTY/TRIBE: ______________________________________(PRE_FILL) DATE: _____________

BACKGROUND

1. When did you first begin to actively participate in the Systems of Care Initiative?

____(Mo.)/____(Yr.)

2. What type of organization do you represent? Check ONE response category that most closely fits your organization.

Child Welfare Mental Health Legal Services (e.g., judge, public defender, etc.) Job in legal services: __________________ Juvenile Justice Substance Abuse Physical Health/Medicine Child Care / Child Development Education Food /Nutrition Housing Law enforcement Domestic Violence	Court-employed advocate/CASA Foster Families Birth Families Adoptive Families Youth Community Member Business and Private Sector Local Government (e.g. Mayor’s office) Cultural/Ethnic Group Other: ______________________________ ____________________________________

3. How long have you worked in the field you identified in question 2?

____ Years ___ Months

4. If you represent an organization/agency, please select ONE of the following that BEST describes your position within your agency:

A. Agency Director

B. Administrative

C. Supervisor/Manager

D. Direct Services (primarily work directly with clients)

E. Not applicable (do not represent an agency/organization)

F. Other

5. Approximately how many employees are in your entire organization?

____ a) 1 - 5 ___d) 21 - 50 ___g) Not Applicable

____ b) 6 - 10 ___e) 51 - 100

____ c) 11 - 20 ___f) More than 100

6. Are you male or female? (circle one)

A. Male B. Female

7. Are you Hispanic/Spanish/Latino?

A. No, not Hispanic/Spanish/Latino

B. Yes, Hispanic/Spanish/Latino

8. Select one of more of the following categories to best describe your race:

1. White

B. Black or African American

3. Asian

4. American Indian or Alaska Native

5. Native Hawaiian or Other Pacific Islander

9. How many times have you taken this survey?

A. This is my first time

2. Two times

3. Three times

4. Not Sure/Don’t Know

10. How long have you participated on the System of Care (SOC) collaborative?

___ Years ___ Months

11. Approximately how many people regularly attend your SOC collaborative meetings?

____ a) 1 - 5 ___d) 16 - 20 ___g) 31 - 35

____ b) 6 - 10 ___e) 21 - 25 ___g) 36 - 40

____ c) 11 - 15 ___f) 26 - 30 ___g) 41 >

12. On average, how many hours per month do you work on SOC Collaborative activities, including regularly scheduled collaborative meetings? ________ Hours

13. Have you participated in a collaborative initiative prior to the Systems of Care Initiative?

____a) Yes ___b) No

14a. Please indicate the role(s) you play (if any) in SOC interagency collaborative activities.

(For each item, circle Yes (Y) or No (N))

Roles	Yes	No
Attend meetings regularly	Y	N
Serve as member of workgroup (subcommittee)	Y	N
Work on activities outside of meetings	Y	N
Help organize activities (other than meetings)	Y	N
Direct the implementation of a particular program	Y	N
Chair/lead a workgroup	Y	N
Serveas an officer other than chair	Y	N
Chair/co-chair the entire group	Y	N

14b. Please indicate if your current SOC collaborative…
(For each item, circle Yes (Y) No (N)

	Yes	No
Has regularly scheduled meetings	Y	N
Has workgroups or subcommittees	Y	N
Conducts work on activities outside of meetings	Y	N
Has bylaws	Y	N
Has an agenda for each meeting	Y	N
Distributes minutes from meetings	Y	N
Has a vision statement	Y	N
Has a mission statement	Y	N
Has a written strategic plan	Y	N
Tracks progress on strategic plan (goals, objectives)	Y	N
Revisits strategic plan (at least once annually)	Y	N
Has formalized rules and procedures	Y	N

15. The following statements refer to your community and agency readiness for

this systems of care effort over the past year. For each statement, please indicate the extent to which you agree with the statement, using the following scale: “1” means you strongly disagree, “2” means you disagree, “3” means you are neutral (neither disagree nor agree), “4” means you agree, and “5” means you strongly agree. If you do not know, please circle the X in the right hand column (Don’t Know).

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don’t Know
There is widespread knowledge about the SOC Initiative in the community.	1	2	3	4	5	X
There is widespread support for the Systems of Care Initiative in the community.	1	2	3	4	5	X
There is widespread knowledge about the SOC Initiative among participating agencies, organizations, and individuals.	1	2	3	4	5	X
There is widespread support for the Systems of Care Initiative among participating agencies, organizations, and individuals.	1	2	3	4	5	X
Senior managers and directors of key organizations are prioritizing the improvement of the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X
Leaders of participating organizations are willing to commit resources, including staff time, for the Systems of Care Initiative.	1	2	3	4	5	X
Community leaders are concerned about improving the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X
Community members and stakeholders are concerned about improving the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X
Local data on the safety, permanency, and well-being of children and families in the child welfare system are available.	1	2	3	4	5	X
There is a high level of expertise available to improve the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X
Local policies are conducive to developing interagency collaborative relationships.	1	2	3	4	5	X
State policies are conducive to developing interagency collaborative relationships.	1	2	3	4	5	X
Existing programs within the community are conducive to developing interagency collaborative relationships.	1	2	3	4	5	X
There is a history of productive interaction among the stakeholders involved in designing and implementing our Systems of Care Initiative.	1	2	3	4	5	X
Financial resources are readily available to support efforts to improve the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X
Services and supports – formal and informal - are readily available in the community to support efforts to improve the safety, permanency, and well-being of children and families in the child welfare system.	1	2	3	4	5	X

16. Please rate the extent to which each of the following served as challenges to your local Systems of Care initiative over the past year. Please use the following scale where “1” means not at all an obstacle and “5” means very much an obstacle. If you do not know, please circle the X in the right hand column (Don’t Know).

	Not at all	A little	Somewhat	Moderately	Very Much	Don’t Know
Poor understanding of Systems of Care Initiative.	1	2	3	4	5	X
Taking longer than expected.	1	2	3	4	5	X
Lack of time by participants.	1	2	3	4	5	X
Lack of resources (financial, staff, etc.).	1	2	3	4	5	X
Burn-out of participants.	1	2	3	4	5	X
Confidentiality issues.	1	2	3	4	5	X
Existence/accessibility of data.	1	2	3	4	5	X
No clearly defined leader.	1	2	3	4	5	X
Turf issues (e.g., conflict over ownership of tasks/resources).	1	2	3	4	5	X
Lack of commitment from stakeholders and/or senior managers from key organizations.	1	2	3	4	5	X
Conflicting organizational cultures (e.g., child welfare and mental health viewing the needs of children and families differently).	1	2	3	4	5	X
Lack of leadership buy-in from key organizations.	1	2	3	4	5	X
Staff turnover in key positions of leadership.	1	2	3	4	5	X
Disagreements over what changes should occur.	1	2	3	4	5	X
Disagreements over what activities to implement.	1	2	3	4	5	X
Lack of accountability among initiative members for projects or tasks.	1	2	3	4	5	X
Too much focus on collaborating, not enough on individuals served.	1	2	3	4	5	X
Other (specify and rate) ____________________________	1	2	3	4	5	X
Other (specify and rate) ____________________________	1	2	3	4	5	X
Other (specify and rate) ____________________________	1	2	3	4	5	X

17. Please rate the extent to which each of the following factors has contributed to the success of the Systems of Care initiative in your community over the past year. Please use the scale below where “1” means not at all a success factor and “5” means very much a success factor. If you do not know, please circle the X in the right hand column.

	Not at all	A little	Somewhat	Moderately	Very Much	Don’t Know
Collaborative member agreement about the nature of the problem.	1	2	3	4	5	X
Input from frontline workers.	1	2	3	4	5	X
The partners in the project work well together.	1	2	3	4	5	X
Having the right people at the table.	1	2	3	4	5	X
Strong leadership.	1	2	3	4	5	X
Well-defined roles among collaborative members.	1	2	3	4	5	X
Well-specified activities.	1	2	3	4	5	X
Accountability for meetings, tasks, and activities.	1	2	3	4	5	X
Agencies and organizations having necessary resources.	1	2	3	4	5	X
Court involvement.	1	2	3	4	5	X
Commitment of key leaders.	1	2	3	4	5	X
Commitment/time availability of participants.	1	2	3	4	5	X
Involvement of certain key agencies/groups.	1	2	3	4	5	X
Individual relationships among collaborative members and agency staff.	1	2	3	4	5	X
Having infrastructure in place to support the initiative.	1	2	3	4	5	X
The partners have the needs of children and families in mind.	1	2	3	4	5	X
Other (specify and rate) ____________________________	1	2	3	4	5	X

18. The following statements refer to the Systems of Care planning and implementation process in your community over the past year. Please indicate the extent to which you agree with each statement with “1”indicating Strongly Disagree and “5” means Strongly Agree. If you do not know, please circle the X in the far right hand column (Don’t Know).

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don’t Know
There are key players missing from the collaborative.	1	2	3	4	5	X
Collaborative members are committed to working together to improve conditions for children, youth, and families.	1	2	3	4	5	X
The communication procedures are clearly understood among collaborative members.	1	2	3	4	5	X
The collaborative has a feeling of cohesiveness and team spirit.	1	2	3	4	5	X
There is a shared vision for desired outcomes for children and families throughout the system.	1	2	3	4	5	X
Roles and responsibilities of members are clear.	1	2	3	4	5	X
Decision-making within the collaborative is participatory and inclusive.	1	2	3	4	5	X
The collaborative routinely reviews and shares information about the status of families.	1	2	3	4	5	X
The collaborative regularly reviews its own effectiveness.	1	2	3	4	5	X
The collaborative obtains input for planning from the broader community.	1	2	3	4	5	X
The collaborative has developed positive working relationship with the media.	1	2	3	4	5	X
The collaborative is successfully changing policies and practices within child-serving agencies.	1	2	3	4	5	X
The collaborative makes every effort to engage missing players and bring them to the table.	1	2	3	4	5	X
Stakeholders have adequate time to commit to the Systems of Care initiative.	1	2	3	4	5	X
There is a formal process for resolving conflicts among participating organizations.	1	2	3	4	5	X
Collaborative members feel valued and important.	1	2	3	4	5	X
There are clearly defined, attainable goals for the initiative.	1	2	3	4	5	X
The skills and expertise of members are utilized effectively by the collaborative.	1	2	3	4	5	X
The collaborative is promoting results-based accountability throughout the system.	1	2	3	4	5	X
The collaborative seeks regular input from the community and resource providers	1	2	3	4	5	X
The collaborative shares its successes with key decision-makers.	1	2	3	4	5	X
The collaborative is fostering positive changes in the lives of families.	1	2	3	4	5	X
Representation from key players within the initiative is adequate.	1	2	3	4	5	X
The collaborative enjoys the commitment of key leaders.	1	2	3	4	5	X
Differences among collaborative members are recognized and worked through.	1	2	3	4	5	X
Conflicts arise frequently among participating organizations in the collaborative.	1	2	3	4	5	X
There is a shared vision of what the collaborative should accomplish.	1	2	3	4	5	X
The collaborative is disorganized and inefficient	1	2	3	4	5	X
The collaborative is flexible enough to accept diversity in members’ views and backgrounds.	1	2	3	4	5	X
The collaborative shares its successes with the community	1	2	3	4	5	X
Each member has an equal voice in the partnership	1	2	3	4	5	X
The number of stakeholders involved in the initiative is adequate.	1	2	3	4	5	X
The collaborative has a strong commitment from the policy-making level of each organization that is represented.	1	2	3	4	5	X
Communication between member organizations is closed and guarded.	1	2	3	4	5	X
The collaborative includes representatives from the community.	1	2	3	4	5	X
The collaboration needs more formalization and structure.	1	2	3	4	5	X
Family members assume active representation/leadership throughout collaborative efforts.	1	2	3	4	5	X
Stakeholders working on the initiative have many competing responsibilities.	1	2	3	4	5	X
The collaborative has a strategy for seeking regular/continuous input from families.	1	2	3	4	5	X
The community is aware of the System of Care goals and mission.	1	2	3	4	5	X
Within the Systems of Care collaborative, there is too much talking and not enough doing.	1	2	3	4	5	X
The partnership includes members representative of the cultural/ethnic diversity of the community.	1	2	3	4	5	X

19. The following statements refer to the effectiveness of your Systems of Care (SOC)

activities and efforts. For each statement, please indicate the extent to which you agree with the statement, using the following scale: “1” means you strongly disagree, “2” means you disagree, “3” means you are neutral (neither disagree nor agree), “4” means you agree, and “5” means you strongly agree. If you do not know, please circle the X in the far right hand column (Don’t Know).

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don’t Know
SOC activities and efforts have been successful in creating positive changes in the safety of the SOC identified target population.	1	2	3	4	5	X
SOC activities and efforts have been successful in creating positive changes in the well-being of the SOC identified target population.	1	2	3	4	5	X
SOC activities and efforts have been successful in creating positive changes in the permanency of the SOC identified target population.	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing community awareness of SOC.	1	2	3	4	5	X
SOC activities and efforts have been successful in creating positive changes in child welfare practices.	1	2	3	4	5	X
SOC activities and efforts have been successful in creating positive changes in child welfare policies and procedures.	1	2	3	4	5	X
SOC activities and efforts have been successful in addressing issues related to cultural competence in the child welfare system.	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing individualized strengths-based approaches in the child welfare system.	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing community-based approaches in the child welfare system.	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing family involvement in the child welfare system.	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing accountability in the child welfare system	1	2	3	4	5	X
SOC activities and efforts have been successful in increasing interagency collaboration in the child welfare system.	1	2	3	4	5	X

The final set of questions refer to your collaborative’s sustainability plans. that is, strategies to continue and/or expand the SOC initiative BEYOND FEDERAL grant funding.

20. When did your collaborative start planning for sustainability? (circle one)

A. From the beginning of the grant (2003)

B. 2004

C. 2005

D. 2006

E. In the past year (2007/8)

21. The following statements refer to your collaborative’s sustainability planning efforts. For each statement, please indicate the extent to which you agree with the statement, using the following scale: “1” means you strongly disagree, “2” means you disagree, “3” means you are neutral (neither disagree nor agree), “4” means you agree, and “5” means you strongly agree. If you do not know, please circle the X in the far right hand column (Don’t Know).

Each of the statements REFERS to continuing efforts beyond federal grant funding. Please select Not Applicable if your community has not yet implemented the activity in question.

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree	Don’t Know	Not Applicable
A. The collaborative has specific plans to provide ongoing training on SOC principles to staff of child serving agencies.	1	2	3	4	5	X	X
B. The collaborative has plans for continuing to hold SOC steering committee/collaborative meetings.	1	2	3	4	5	X	X
C. The collaborative has plans for continuing family involvement on the SOC steering committee/collaborative.	1	2	3	4	5	X	X
D. The collaborative has plans to continue compensating family members for their participation on SOC collaborative groups.	1	2	3	4	5	X	X
E. The collaborative has plans for compensating family members who train staff of child serving agencies.	1	2	3	4	5	X	X
F. The collaborative has plans for continuing the family navigator/parent partner program in the child welfare system.	1	2	3	4	5	X	X
G. The collaborative has specific plans for increasing family centered practices in child serving agencies.	1	2	3	4	5	X	X
H. The collaborative has specific plans for increasing an individualized, strengths-based approach in child serving agencies.	1	2	3	4	5	X	X
I. The collaborative has plans for continuing to provide training on cultural competence to child serving agencies.	1	2	3	4	5	X	X
J. The collaborative has specific plans to increase the ethnic and racial diversity of staff of child serving agencies.	1	2	3	4	5	X	X
K. The collaborative has specific plans to continue to increase the safety, well-being, and permanency of children in the target population.	1	2	3	4	5	X	X
L. The collaborative has specific plans to continue to increase the availability of community-based services.	1	2	3	4	5	X	X
M. The collaborative has plans to continue to assess the effectiveness of services for children and families in the child welfare system.	1	2	3	4	5	X	X
N. Key child serving agencies will continue to have access to each other’s data systems.	1	2	3	4	5	X	X
O. Key child serving agencies have signed MOU’s agreeing to continue to collaborate in serving children and families.	1	2	3	4	5	X	X
P. Procedures for continuing to share case information across agencies have been established.	1	2	3	4	5	X	X
Q. The collaborative has developed strategies to continue to combine agency resources to better serve children and families (e.g., blended funding, identification of alternative funding, etc.).	1	2	3	4	5	X	X

Thank You For Your Help With This Important Study

Please return the Survey to your Project Coordinator/Evaluator

2