Leadership Interview - CIPS

Evaluation of the Child Welfare Capacity Building Collaborative

Leadership Interview—CIPs

The purpose of the interview is to obtain information from Court Improvement Program (CIP) Directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Centers (specifically the Center for Courts); contextual factors that influence the CIPs; and perceptions of the capacity building services received and outcomes achieved.

Name of CIP:

Respondent Name:

Position Title:

Interviewer:

Date/time of interview:

Introduction:

Good morning/afternoon. My name is ______________. I am calling from James Bell Associates (or ICF International), an evaluation firm based in Arlington, VA. As you may know, James Bell Associates and ICF International have been contracted by the Children’s Bureau to conduct an independent evaluation of the Children’s Bureau’s Capacity Building Centers.

Directions to interviewer – please inform the interviewee:

This evaluation addresses the role of the Children’s Bureau Capacity Building Centers in helping States, Tribes, and Court Improvement Programs (CIPs) to enhance the capacity of their child welfare systems. The Leadership Interview is an important component of the evaluation. We are conducting this survey with all State CW directors as well as Tribal directors and CIP directors who receive capacity building services from the Centers.

Your participation in this interview is voluntary but critical to this evaluation. We anticipate that our discussion with you today will require at most 60 minutes. Your privacy is important to us. In order for us to have an accurate account of your responses, we would like to record this interview. The recording will be destroyed as soon as we transcribe it. We will remove all identifying information from the transcript. Responses that you provide will be combined with answers from other respondents and will not be associated with your agency/CIP or you as an individual respondent. Do we have your permission to record this conversation?

We are going to ask you about the factors that influence your decisions to use the services of the CB’s Capacity Building Center, contextual factors that influence your CIP, and your experiences with the services you have received from the Capacity Building Center for Courts.

Do you have any questions or concerns before we begin?

We would like to first start out by asking you some basic information on your background and history with the CIP.

1. RESPONDENT BACKGROUND

1. What is your official position title?

2. How long have you been in this position?

_______years or _____months

3. How long have you been with the CIP?

_______years or _____ months

2. CHARACTERISTICS OF THE SYSTEM

1. The structure of your CIP and courts is:

Prefill 		Statewide court system
Prefill 		County based court system, overseen by the State

Is this correct?

3. CAPACITY BUILDING SERVICES RECEIVED

Interviewer: if the CIP has received services from more than one Center, please focus the interview on the primary Center that is assigned to CIP.

Now, let’s talk about the services that your agency has received from the Capacity Building Center for Courts to date. Please note, that this interview focuses on topics related to your experiences with the Capacity Building Centers, and does not include topics related to your past experiences with NRCs or ICs.

1. According to the Children’s Bureau’s records of capacity building activities in the past year, the services that your CIP received or participated in are listed in the document I sent you. Is that list of services accurate?

Prefill (from CapTrack) Universal/Constituency Services Received

Center Name	Month/year	Brief description

Prefill (from CapTrack) Tailored Services Received

Center Name	Month/year	Brief description of services (include brief vs intensive)

2. Have you noticed any changes in how technical assistance is now delivered by the Children’s Bureau’s new Capacity Building Centers? (In the past, the National Resource Centers (NRCs) and Implementation Centers (ICs) were the CB-supported T/TA providers.)

   1. If No  move to C3

   2. If Yes  What is different now, and what is your opinion of the new approach?

3. What factors influenced your decision to utilize these services from the Center for Courts? Please describe. [e.g., CFSR findings, etc.]

4. Have there been particular types of services and/or assistance that your CIP needed that were not available from the Center?

5. Which types of services and/or assistance from the Center would you say have been most successful at meeting your CIP’s needs?

6. Are there other TA providers, outside the Children’s Bureau supported Capacity Building Centers (such as local universities, Casey Programs, foundations, philanthropic organizations, or other private TA providers) that are contributing to your CIP’s capacity building efforts? Please describe.

Now we have some specific questions related to the types of assistance that your State has received from the Capacity Building Center:

4. ASSESSMENT AND WORK PLANNING

First, a few questions about the CIP self-assessment process:

1. Was the self-assessment comprehensive, taking into account all relevant aspects of your CIP? Please explain.

2. Did your CIP’s understanding/awareness of the strengths and gaps in your system increase through the self-assessment process? Please briefly explain.

3. Did the self-assessment prioritize the areas of “need” for building capacity in your CIP?

   1. Please explain.

4. Did the Center for Courts’ staff have sufficient expertise to help support a meaningful self-assessment of your CIP? Please describe.

5. How beneficial, or not, was the self-assessment process to your CIP?

6. In addition to the CIP self-assessment, did your CIP also participate in any way in the State assessment process that was conducted with the Center for States?

   1. If Yes  Why? [then move to D7]

   2. If No  Why not? [then move to D9]

7. During the State assessment, was your CIP an active participant? Please explain.

Now a few questions about the State’s Capacity Building Plan and about a work plan with the Center for Courts that may have also been developed.

8. Did the State’s Capacity Building Plan identify appropriate resources and partners (e.g., consultants, agencies, or organizations – such as Casey Programs) to address your CIP’s capacity building needs? Please briefly describe.

9. Was a work plan developed (with the Center for Courts) that details the planned delivery of services by the Center?

   1. If Yes  Please briefly describe the work outlined in the work plan. [then move to D10]

   2. If No  Why not? [then move to D11]

10. Please describe whether or not the work plan clearly outlined tasks, responsibilities, milestones, timelines, and desired outcomes?

11. Do you have additional thoughts that you would like to share regarding your CIP’s experiences with the self-assessment process, participation in the State assessment, or the State’s Capacity Building Plan [if the CIP has a work plan with the Center ] or the work plan that was developed with the Center?

[Interviewer: Continue to section E for CIPs that participate in tailored services OR skip to section F if the CIP has only participated in universal and/or constituency services.]

5. TAILORED SERVICES PROVIDED BY THE CENTERS

The next questions (section E in the interview guide that we sent you) pertain to: [Prefill from item C1 a brief description of the Tailored services]

1. In your opinion, was your CIP (and the courts) ready to receive the services that the Capacity Building Center provided? [e.g., sufficient resources including staff resources, equipment and materials, evaluation resources]

   1. Please explain.

2. Did your CIP recommend/influence the courts to revise existing statutes, policies and procedures or to develop new ones with the support of the Center for Courts in order to address [Prefill from C1 a few words to describe the focus of the capacity building work]?

1. Please describe.

Now, let’s talk about the outcomes that were expected:

Along with the interview questions, we also sent you a brief description of the outcomes that were expected from the project supported by the Center’s services. Let’s first talk about which of these outcomes have been achieved to date:

[Interviewer: please go over each expected outcome and ask if it has been achieved. Please make a note of what was achieved and those outcomes that were not (or not yet) achieved.]

[Prefill from the work plan brief descriptions regarding intended outcomes/results]

Outcomes/results related to:	Brief description:

[Interviewer: For each outcome that was achieved, ask the next questions, focusing on one outcome at a time]

3. Outcome 1:

1. Please describe how this outcome was achieved?

2. To what extent did the services provided by the Center contribute to achieving this outcome?

2. How would you describe the quality of the Center’s services/materials?

3. Please describe what other factors contributed to this outcome?

4. Have these changes influenced your agency’s practice(s)? Please describe.

5. How will the changes that were achieved be sustained?

[If another outcome was achieved, ask E4]

4. Outcome 2:

1. Please describe how this outcome was achieved?

2. To what extent did the services provided by the Center contribute to achieving this outcome?

2. How would you describe the quality of the Center’s services/materials?

3. Please describe what other factors contributed to this outcome?

4. Have these changes influenced your agency’s practice(s)? Please describe.

5. How will the changes that were achieved be sustained?

[If a third outcome was achieved, ask E5]

5. Outcome 3:

1. Please describe how this outcome was achieved?

2. To what extent did the services provided by the Center contribute to achieving this outcome?

2. How would you describe the quality of the Center’s services/materials?

3. Please describe what other factors contributed to this outcome?

4. Have these changes influenced your agency’s practice(s)? Please describe.

5. How will the changes that were achieved be sustained?

[Interviewer: If more than three key outcomes were achieved, please add the items as additional sets of the questions in E5]

Many factors can either help or hinder your CIPs/courts ability to achieve the intended changes that the Center is supporting. These factors could be internal to the organization as well as external – and might include organizational or agency factors, community factors, or other environmental/contextual factors.

6. In your opinion, what were the key factors that helped to achieve the intended outcomes we discussed?

7. In your opinion, what key factors hindered your CIP/courts from achieving the intended outcomes?

8. Have there been any unintended consequences or unforeseeable outcomes as a result of working with the Center? If yes  Please describe.

9. In your opinion, what have been some of the key benefits of working with the Center?

10. Based on your work with the Center, do you have any suggestions that you would like to share for improving the Center’s services? [Probes: e.g., regarding requesting services, service delivery, extent to which the services addressed the needs of the CIP/courts, etc.]

Now we have a few questions regarding more general services that the Capacity Building Centers (Center for States, Center for Courts, and Center for Tribes) provide, such as products and materials, webinars, presentations, etc.

6. UNIVERSAL/CONSTITUENCY SERVICES

1. Has your CIP used products/materials from the Capacity Building Centers or accessed content from the Centers’ websites?

   1. What was useful?

   2. What was not useful?

   3. Overall, how effective has the content of the Centers’ products and materials been in addressing the needs of your CIP?

2. Has your CIP viewed or listened to any webinars, podcasts, or presentations offered by any of the Centers? If yes  which ones?

   1. What was useful?

   2. What was not useful?

   3. Overall, how effective has the content of the Centers’ webinars, podcasts, and presentations been in addressing the needs of your CIP?

3. Could you talk about whether or not the strategies that the Centers use to disseminate information have been useful to your CIP?

   1. Could you provide a specific example of what has been particularly useful and how?

   2. Could you provide an example of what has not been useful?

4. Has your CIP participated in peer-to-peer networking activities provided by the Center?

   1. What was useful?

   2. What was not useful?

5. Do you have any suggestions that you would like to share for improving the more general services that the Centers provide (e.g., peer-networking activities, webinars, materials)?

7. CIP’S RELATIONSHIP WITH THE CENTER

Now a few questions regarding your CIP’s opinion of working with the Center. Please provide the degree of your agreement with the following statements, using the seven response categories, from 7 to 1: Strongly agree; Agree; Agree somewhat; Undecided; Disagree somewhat; Disagree; Strongly disagree. These response categories are also listed in your copy of the interview questions document.

1. My CIP is satisfied with the level of accessibility of the Center for Courts:

7--Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could accessibility be improved?

2. When working with the Center for Courts on issues related to my CIP/courts, my CIP/courts has been satisfied with the frequency of communication:

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

3. My CIP/courts is comfortable disclosing areas of concern or weaknesses of our agency to the Center for Courts’ staff:

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

4. When working with the Center for Courts, my CIP’s experience is that our program plays an active part in decision-making:

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

5. My CIP is satisfied with the level of follow-through from the Center for Courts:

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could their follow-through be improved?

6. Overall, my CIP is satisfied with our relationship with the Center for Courts:

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could the relationship be improved?

7. Overall, the services that the Center provided to my CIP reflect an understanding of how my CIP and our court system operates.

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

8. The services provided by the Center were appropriate and aligned with my CIP’s needs.

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

9. The services provided by the Center took into account the characteristics and unique context of my CIP and the court system.

7-Strongly Agree

6-Agree

5-Agree Somewhat

4-Undecided

3-Disagree somewhat

2-Disagree

1-Strongly disagree

If “disagree somewhat”, “disagree”, or “strongly disagree”: How could this be improved?

7. IN CONCLUSION:

Do you have any additional comments you would like to share regarding CB’s Capacity Building Centers or your experiences with the services, products, and assistance they provide?

THANK YOU

WE GREATLY APPRECIATE YOUR PARTICIPATION!

12