Evaluation of the Child Welfare Capacity Building Collaborative
Leadership Interview—Tribes
The purpose of the interview is to obtain information from Tribal Child Welfare (CW) Directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Center for Tribes; contextual factors that influence the Tribal CW systems; and perceptions of the capacity building services received and outcomes achieved.
Name of Tribal CW Agency:
Respondent Name:
Position Title:
Interviewer:
Date/time of interview:
Introduction:
Good morning/afternoon. My name is ______________. I am calling from James Bell Associates (or ICF International), an evaluation firm based in Arlington, VA. As you may know, James Bell Associates and ICF International have been contracted by the Children’s Bureau to conduct an independent evaluation of the Children’s Bureau’s Capacity Building Centers.
Directions to interviewer – please inform the interviewee:
This evaluation addresses the role of the Children’s Bureau Capacity Building Centers in helping States, Tribes, and Court Improvement Programs (CIPs) to enhance the capacity of their child welfare systems. The Leadership Interview is an important component of the evaluation. We are conducting this survey with all State CW directors and Tribal directors and CIP directors who receive capacity building services from the Centers.
Your participation in this interview is voluntary but critical to this evaluation. We anticipate that our discussion with you today will require at most 75 minutes. Your privacy is important to us. In order for us to have an accurate account of your responses, we would like to record this interview. The recording will be destroyed as soon as we transcribe it. We will remove all identifying information from the transcript. Responses that you provide will be combined with answers from other respondents and will not be associated with your agency/Tribe or you as an individual respondent. Do we have your permission to record this conversation?
This interview will include two parts. I will let you know when we get to the end of the first part of this interview. The later part II of this interview includes a separate set of discussion topics that we would like to cover on behalf of the Center for Tribes. Your responses on part II will be de-identified and shared with the Center for Tribes evaluators; your responses will not be tied to your name or Tribe.
We are going to ask you about the factors that influence your decisions to use the services of the CB’s Capacity Building Center, contextual factors that influence your CW system, and your experiences with the services you have received from the Capacity Building Center for Tribes.
Do you have any questions or concerns before we begin?
PART I
We would like to first start out by asking you some basic information on your background and history with the agency.
RESPONDENT BACKGROUND
What is your official position title?
How long have you been in this position?
_______years or _____months
How long have you been with the agency?
_______years or _____ months
CHARACTERISTICS OF THE CHILD WELFARE SYSTEM
Your child welfare system is:
Prefill |
|
Tribally administered/governed |
Prefill |
|
Bureau of Indian Affairs (BIA) administered |
Prefill |
|
State Administered |
Prefill |
|
County Administered, State Supervised |
Is this correct?
Does your Tribe receive Title IV-E funding?
No |
|
Yes – a planning grant to develop needed policies, procedures, and infrastructure |
As of [year] |
Yes – through Tribal/State agreement |
As of [year] |
Yes – through Direct IV-E |
As of [year] |
Yes – Tribe has IV-E waiver |
As of [year] |
CAPACITY BUILDING SERVICES RECEIVED
Interviewer: if an agency has received services from more than one Center, please focus the interview on the primary Center that is assigned to the agency (i.e., Center for Tribes).
Now, let’s talk about the services that your agency has received from the Capacity Building Center for Tribes to date. Please note, that this interview focuses on topics related to your experiences with the Capacity Building Center for Tribes, and does not include topics related to your past experiences with NRCs or ICs.
According to the Children’s Bureau’s records of capacity building activities in the past year, the services that your agency received or participated in are listed in the document I sent you. Is that list of services accurate?
Prefill (from CapTrack) Universal/Constituency Services Received
Center Name |
Month/year |
Brief description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Prefill (from CapTrack) Tailored Services Received
Center Name |
Month/year |
Brief description of services (include brief vs intensive) |
|
|
|
|
|
|
|
|
|
Have you noticed any changes in how technical assistance is now delivered by the Children’s Bureau’s new Capacity Building Centers? (In the past, the National Resource Centers (NRCs) and Implementation Centers (ICs) were the CB-supported T/TA providers.)
If No move to C3
If Yes What is different now, and what is your opinion of the new approach?
What factors influenced your decision to utilize these services from the Center for Tribes? Please describe.
Have there been particular types of services and/or assistance that your agency needed that were not available from the Center?
Which types of services and/or assistance from the Center would you say have been most successful at meeting your agency’s needs?
Are there other TA providers (outside the Children’s Bureau supported Capacity Building Centers – such as local universities, Casey Family Programs, NICWA, National Congress of American Indians, your State CW agency, or other private TA providers) that are contributing to your CW system’s capacity building efforts? Please describe.
Now we have some specific questions related to the types of assistance that your Tribe has received from the Center for Tribes:
ASSESSMENT AND WORK PLANNING
Interviewer: In round I (year 2) ask these questions of all Tribal agencies that are receiving tailored services. In round 2 (year 4), ask these questions of Tribal agencies that have newly (since round I) begun to receive tailored services.
First, a few questions about the assessment process:
Did your agency participate in the Center’s assessment process?
If Yes Why? [then move to D2]
If No Why not? [then move to D8]
During the Tribal Organizational Assessment (TOA), did the Center for Tribes staff have sufficient expertise to conduct meaningful assessment of your CW system? Please describe.
Was the assessment comprehensive, taking into account all relevant aspects of your Tribal CW system? Please explain.
Did your agency’s capacity to self-assess its capacity building needs increase through the assessment process? Please briefly explain.
Did your agency’s understanding/awareness of the strengths and gaps in your Tribal CW system increase through the assessment process? Please briefly explain.
Did the assessment prioritize the areas of “need” for building capacity in your Tribal CW system? Please briefly explain.
How beneficial, or not, was the assessment process to your agency?
Now a few questions about the integrated capacity building plan that was developed as the result of the assessment process and about a work plan with the Center for Tribes that may have also been developed.
Did the Capacity Building Plan identify appropriate resources and partners (e.g., consultants, agencies, or organizations – such as Casey Family Programs) to address your Tribal child welfare system’s capacity building needs? Please briefly describe.
Did the Capacity Building Plan include a work plan that details the planned delivery of services by the Center for Tribes?
If Yes Please briefly describe the work outlined in the work plan. [then move to D10]
If No Why not? [then move to D11]
Please describe whether or not the work plan clearly outlined tasks, responsibilities, milestones, timelines, and desired short and long-term outcomes?
Do you have additional thoughts that you would like to share regarding your agency’s experiences with the assessment process or the Capacity Building Plan [if the Tribe has a work plan with the Center ] or the work plan that was developed with the Center?
[Interviewer: Continue to section E for Tribes that participate in tailored services OR skip to section F if the Tribe has only participated in universal and/or constituency services.]
TAILORED SERVICES PROVIDED BY THE CENTERS
The next questions (section E in the interview guide that we sent you) pertain to: [Prefill from item C1 a brief description of the Tailored services]
In your opinion, was your agency ready to receive the services that the Capacity Building Center provided? [e.g., sufficient resources including staff resources, equipment and materials, evaluation resources]
Please explain.
Did your agency revise existing policies and procedures or develop new ones with the support of the Center for States in order to address [Prefill from C1 a few words to describe the focus of the capacity building work]?
Please describe.
Now, let’s talk about the outcomes that were expected:
Along with the interview questions, we also sent you a brief description of the outcomes that were expected from the project supported by the Center’s services. Let’s first talk about which of these outcomes have been achieved to date:
[Interviewer: please go over each expected outcome and ask if it has been achieved. Please make a note of what was achieved and those outcomes that were not (or not yet) achieved.]
[Prefill from the work plan brief descriptions regarding intended outcomes/results]
Outcomes/results related to: |
Brief description: |
|
|
|
|
|
|
|
|
|
|
|
|
[Interviewer: For each outcome that was achieved, ask the next questions, focusing on one outcome at a time]
Outcome 1:
Please describe how this outcome was achieved?
To what extent did the services provided by the Center contribute to achieving this outcome?
How would you describe the quality of the Center’s services/materials?
Please describe what other factors contributed to this outcome?
Have these changes influenced your agency’s practice(s)? Please describe.
How will the changes that were achieved be sustained?
[If another outcome was achieved, ask E4]
Outcome 2:
Please describe how this outcome was achieved?
To what extent did the services provided by the Center contribute to achieving this outcome?
How would you describe the quality of the Center’s services/materials?
Please describe what other factors contributed to this outcome?
Have these changes influenced your agency’s practice(s)? Please describe.
How will the changes that were achieved be sustained?
[If a third outcome was achieved, ask E5]
Outcome 3:
Please describe how this outcome was achieved?
To what extent did the services provided by the Center contribute to achieving this outcome?
How would you describe the quality of the Center’s services/materials?
Please describe what other factors contributed to this outcome?
Have these changes influenced your agency’s practice(s)? Please describe.
How will the changes that were achieved be sustained?
[Interviewer: If more than three key outcomes were achieved, please add the items as additional sets of the questions in E5]
Many factors can either help or hinder your State’s ability to achieve the intended changes that the Center is supporting. These factors could be internal to the organization as well as external – and might include organizational or agency factors, community factors, or other environmental/contextual factors.
In your opinion, what were the key factors that helped your CW system to achieve the outcomes we discussed?
In your opinion, what key factors hindered your CW system from achieving the intended outcomes?
In your opinion, what have been some of the key benefits of working with the Center?
Have there been any unintended consequences or unforeseeable outcomes as a result of working with the Center? If yes Please describe.
Based on your work with the Center, do you have any suggestions that you would like to share for improving the Center’s services? [Probes: e.g., regarding requesting services, service delivery, extent to which the services addressed the needs of the agency, etc.]
Now we have a few questions regarding more general services that the Capacity Building Centers (Center for States, Center for Courts, and Center for Tribes) provide, such as products and materials, webinars, presentations, etc.
UNIVERSAL/CONSTITUENCY SERVICES
Has your Tribe used products/materials from the Capacity Building Centers or accessed content from the Centers’ websites?
What was useful?
What was not useful?
Overall, how effective has the content of the Centers’ products and materials been in addressing the needs of your agency?
Has your Tribe viewed or listened to any webinars, podcasts, or presentations offered by any of the Centers? If yes which ones?
What was useful?
What was not useful?
Overall, how effective has the content of the Centers’ webinars, podcasts, and presentations been in addressing the needs of your agency?
Could you talk about whether or not the strategies that the Centers use to disseminate information have been useful to your agency?
Could you provide a specific example of what has been particularly useful and how?
Could you provide an example of what has not been useful?
Has your Tribe participated in peer-to-peer networking activities provided by the Center?
What was useful?
What was not useful?
Do you have any suggestions that you would like to share for improving the more general services that the Centers provide (e.g., peer-networking activities, webinars, materials)?
TRIBE’S RELATIONSHIP WITH THE CENTER
Now a few questions regarding your State’s opinion of working with the Center. Please provide the degree of your agreement with the following statements, using the seven response categories, from 7 to 1: Strongly agree; Agree; Agree somewhat; Undecided; Disagree somewhat; Disagree; Strongly disagree. These response categories are also listed in your copy of the interview questions document.
Our Tribal CW agency is satisfied with the level of accessibility of the Center for Tribes:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could accessibility be improved?
When working with the Center for Tribes on issues related to our CW system, our Tribe has been satisfied with the frequency of communication:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
Our Tribe is comfortable disclosing areas of concern or weaknesses of our CW agency to the Center for Tribes staff:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
When working with the Center for Tribes, our Tribe’s experience is that we play an active part in decision-making:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
Our Tribe is satisfied with the level of follow-through from the Center for Tribes:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could their follow-through be improved?
Overall, our Tribe is satisfied with our agency’s relationship with the Center for Tribes:
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could the relationship be improved?
Overall, the services that the Center for Tribes provided to our agency reflect an understanding of how my Tribal CW agency operates.
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
The services provided by the Center were appropriate and aligned with our Tribal agency’s needs.
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
The services provided by the Center took into account the characteristics and unique context of our Tribal CW agency.
7-Strongly Agree |
6-Agree |
5-Agree Somewhat |
4-Undecided |
3-Disagree somewhat |
2-Disagree |
1-Strongly disagree |
If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?
IN CONCLUSION:
[Conclusion question is at the end of Part II (item 38) for Tribal respondents]
This concludes the cross-center evaluation interview questions (part I).
***
THANK YOU
WE GREATLY APPRECIATE YOUR PARTICIPATION!
| File Type | application/msword |
| File Modified | 2016-02-08 |
| File Created | 2016-02-08 |