OMB
Control No. ________
Expiration
Date: ________
Instrument 4:
Site Visit Discussion Guide for Project Directors and Leaders from Partner Organizations – Interview #1
Introduction to Discussion Guide 3
Discussion Guide for Project Directors and Leaders from Partner Organizations – Interview #1 3
Introduction: We are from the research firms Abt Associates and Child Trends and we are conducting a study of the Child Welfare Community Collaborations to Strengthen and Preserve Families (CWCC) grant initiative. The study is overseen by the Office of Planning, Research, and Evaluation (OPRE), in collaboration with the Children’s Bureau, in the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS). We are conducting a cross-site process evaluation study to learn how grantees receiving funding under the CWCC initiative are approaching their collaborative work.
As part of this study, we are talking with leaders and staff at the lead and partner organizations involved in [insert name of local CWCC-funded initiative]1 to learn more about how grantees and their partners are collaborating to prevent child abuse and neglect in their communities. Discussion topics include: partnership approaches; data use; implementation of CWCC-funded activities or strategies; factors influencing implementation of your CWCC grant; and sustainability plans beyond the CWCC grant.
Before beginning our discussion, we want to thank you for agreeing to talk with us today. We know you are busy and we appreciate the valuable time you are spending with us today. The interview will take about 120 minutes. Your participation is voluntary, and there are no penalties for choosing not to take part in the interview. You can refuse to answer any questions or stop the interview at any time. Our aim is to learn from your insights and experience. There are no right or wrong answers. Your answers will be kept private to the extent permitted by law. Information you provide will not be shared with other staff at your agency or organization. Only the study team will have access to the information you provide through this interview. Your name will not be listed in any published reports, and comments will not be attributed to you. Instead, your information will be combined with information provided by others. However, because of the relatively small number of agencies/organizations participating in the study, there is a possibility that a response could be correctly attributed to you. We may archive the data collected by this discussion and allow access to the de-identified dataset to qualified researchers. Your data will only be used for research purposes.
We would like to record this discussion with a digital recorder so we can listen to it later when we write up our notes. No one besides our research team and the transcription vendor will listen to the recording. If you want to say anything that you don’t want recorded, please let us know and we will be glad to pause the recorder. Do you have any objections to being part of this interview or to us recording our discussion?
The Paperwork Reduction Act Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is _____, which expires ______. Please send comments about this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Abt Associates, 6130 Executive Blvd., Rockville, MD, 20852, Attn: Allison Hyra OMB-PRA ______. If you have questions or concerns about your rights as a participant in this evaluation, please call Abt’s Institutional Review Board at 877-520-6835.
Do you have any questions before we begin?
First, I’d like to get an idea of the background of your organization and your work in it.
Please provide an overview of the type of work your organization performs.
What is your title and role at [name of agency/organization]?
How long have you been at [name of agency/organization]?
How long have you been in your current role?
What role does [name of agency/organization] play in [name of site’s Child Welfare Community Collaborations initiative]?
Please describe your level of involvement within the [name of site’s Child Welfare Community Collaborations initiative], including the specific roles or responsibilities that you perform.
Now, we have some questions about partnerships in [initiative name].
For Project Directors from Lead Grantee Organization:
How did you identify partners to participate in your CWCC grant?
Who was involved in the process of identifying partners? (Probes: Were parents/families engaged? Businesses? Faith organizations? Service providers? Law enforcement?)
At what point did you identify partners (Probes: prior to the proposal (i.e., existing partners), during proposal development, after award/during planning phase)?
Are you still identifying partners?
How did you reach out to the local, tribal, and state CW agencies and local or tribal services? Probe: Why (did you reach out in that way)?
How did you reach out to local or tribal partners, including service providers, schools, law enforcement, courts and the legal community, businesses, faith organizations, and parents/families targeted by CWCC services? What strategies did you employ to engage partners in the [collaborative]?
Which organizations are involved?
Are there any organizations that you wanted to involve, but couldn’t?
For Leaders from Partner Organizations:
How did your organization get involved with the CWCC grant?
At what point was your organization invited to participate in the grant (Probes: prior to the proposal, during proposal development, after award (September 2018)/during planning phase)?
Why did your organization decide to get involved?
What does your organization expect to get out of its participation in the grant?
Are there any organizations that you wanted to see involved at any stage, but weren’t?
Did you (do you) have any concerns about partnering? (Probe: for example the amount of time or staff effort that would be diverted to the [collaboration], or concerns about how the [collaboration] would affect your organization’s autonomy?)
Has your level of engagement in this partnership changed over time? Do you expect it will change going forward?
For Project Directors from Lead Grantee Organization:
Which data, if any, informed how you approached developing your CWCC partnerships?
(Probe if not answered): Which data sources did you rely upon? In what ways did the data lead you to invite specific partners to join the grant?
(if not answered in 7, above): Were the voices or perspectives of frontline staff (from the lead and partner organizations) considered when identifying and creating new partnerships? What about the voices of families?
For Leaders from Partner Organizations:
Are you familiar with the extent to which data, if any, informed [lead organization’s name] approach to developing CWCC partnerships? Was there any data or research that inspired you to join this [collaborative]?
Were the voices or perspectives of frontline staff (from the lead and partner organizations) considered when identifying and creating new partnerships? What about the voices of families?
Have there been any changes to the composition of partnering organizations since launching the grant? Why did those changes happen?
Are there any types of organizations that don’t exist in your community but you see a need for in order to prevent child maltreatment? (Probe: For example, is there a need for more advocacy organizations versus service providers? Or providers of different types of services? Are there organizations that focus on preventing child maltreatment but have waiting lists?)
For Project Directors from Lead Grantee Organization:
What successes, if any, has your organization experienced in establishing the CWCC partnerships?
For Leaders from Partner Organizations:
What are some of the successes, if any, you have noticed in establishing the CWCC partnerships?
For Project Directors from Lead Grantee Organization:
What challenges, if any, did your organization face when establishing the CWCC partnerships?
Pause for answer.
Was it difficult getting the “right” partner organizations to come to the table?
Was it difficult to identify the right people from those organizations to participate (for example, mid-level managers instead of executives or administrators)?
Were there any challenging dynamics between partners?
Was it difficult to identify common priorities and strategies to address them?
Were there any challenges in developing an implementation plan, such as deciding which partners would be responsible for delivering which services?
What steps, if any, were taken to resolve the challenges?
For Leaders from Partner Organizations:
What are some of the challenges, if any, you noticed or experienced when the CWCC partnerships were being established?
Pause for answer.
Was it difficult getting the “right” partner organizations to come to the table?
Was it difficult to identify the right people from those organizations to participate (for example, mid-level managers instead of executives or administrators)?
Were there any challenging dynamics between partners?
Was it difficult to identify common priorities and strategies to address them?
Were there any challenges in developing an implementation plan, such as deciding which partners would be responsible for delivering which services?
What challenges, if any, did your organization face when joining the CWCC partnerships?
Do you recall any steps taken to resolve the challenges?
For Leaders from Partner Organizations: Have you or staff involved at your organization faced any difficulties finding time to participate in the [collaborative]? (Probe for time outside of work hours, staff already overburdened with existing efforts)
For Project Directors from Lead Grantee Organization:
Have any of the initial challenges of setting up the partnership carried over to your efforts to maintain the CWCC partnerships?
Have any new challenges arisen amongst the partnership since partners started to implement the planned strategies, activities, and/or services of the [CWCC grant]?
What steps, if any, have been taken to resolve the challenges?
For Leaders from Partner Organizations:
Have you noticed or experienced whether any of the initial challenges carried over to the efforts to maintain the [CWCC grant] partnerships?
Have any new challenges arisen amongst the partnership since partners started to implement the planned strategies, activities, and/or services of the [CWCC grant]?
Have any new challenges arisen within your organization since partners started to implement the planned strategies, activities, and/or services of the [CWCC grant]? Are you aware of steps that have been taken to resolve the challenges?
For Project Directors from Lead Grantee Organization:
How would you describe your experience partnering with other organizations on the [CWCC grant]?
For Leaders from Partner Organizations:
How would you describe your experience partnering with the lead organization and other partner organizations on the [CWCC grant]?
In your opinion, how effectively has [the CWCC grant] been organized?
What structures and processes, if any, are in place to support the collective work of the partners?
Are all voices heard? Is there a shared sense of responsibility?
Are there adequate supports in place to assist implementation of the [CWCC grant]’s [collaborative] activities or strategies?
What, if any, challenges have arisen with the provision of services or activities?
What are the various functions or roles that partners play in the work of the [CWCC grant]? (Probe for role/function of the lead organization, CW agency, direct service providers, courts and legal community, community-based organizations, faith-based organizations, parent/community members)
Are there clear expectations about what each partner organization is expected to do?
If so, to what extent are they contributing or performing as expected?
Which partners (or types of partners) do you consider to be most essential to the work of the [CWCC grant], and why? Are there any partners who seem less essential to this work? (If needed, clarify: Essential could mean that these partners provide many services to families or provide direct services to client families as opposed to partners outside of this core set.)
The next few questions are about data use and data management.
For Project Directors from Lead Grantee Organization:
Please describe the data sources, assessments, and screening tools, if any, that are used by your organization and your partners to determine family and community needs.
Who are organizations screening?
When are screenings and assessments conducted?
Is screening conducted systematically across partners? (Probe: If not, are there any efforts to make screening more systematic across partners?)
Who is responsible for the collection and entry of screening and assessment data? What, if any, challenges do your organization and your partner organizations face in collecting and entering data in a timely manner?
How, if at all, are assessment or screening results used? (Probe: Are these data shared across partners? If so, in what ways are the shared data used?)?
Since implementing the CWCC grant, has your organization or any of your partner organizations made any changes to the data sources, assessments, and screening tools that are used to determine family and community needs? (Probe: Why or why not?)
For Leaders from Partner Organizations:
Please describe the data sources, assessments, and screening tools, if any, that are used by your organization to determine family and community needs.
Who is your organization screening?
When are screenings and assessments conducted?
Do you know whether screening is conducted systematically across partners? (Probe: If not, are there any efforts to make screening more systematic across partners?)
Who is responsible for the collection and entry of screening and assessment data? What, if any, challenges does your organization face in collecting and entering data in a timely manner?
How, if at all, does your organization use assessment or screening results? (Probe: Do you share these data with other partners? If so, in what ways are the shared data used?)
Are you aware of how other partner organizations use assessment or screening results? (Probe: Do you receive these data from other partners?
Since implementing the CWCC grant, has your organization made any changes to the data sources, assessments, and screening tools that are used to determine family and community needs? (Probe: Why or why not?)
For Project Directors from Lead Grantee Organization:
To what extent, if any, do your organization and your partner organizations use needs assessment processes to prioritize which services participants receive?
Are these needs assessment processes used to determine the order in which participants receive services? (Probe: Why or why not?)
For Leaders from Partner Organizations:
To what extent, if any, does your organization use needs assessment processes to prioritize which services participants receive?
Are these (needs assessment processes) used to determine the order in which participants receive services? (Probe: Why or why not?)
To what extent, if any, does [CWCC grant] use data to inform its work?
Does data inform collaborative strategies and activities? (Probe: Why or why not?)
Are some strategies and activities more data-informed than others?
In what ways, if any, has the [CWCC grant]’s use of data changed since the CWCC initiative began? (Probe for explanations and examples)
For Leaders from Partner Organizations Only: To what extent, if any, does your organization use data to inform its work within the [CWCC grant]? Has this changed over time?
For Leaders from Partner Organizations Only: Have you noticed any changes in data usage by other partners since the [CWCC grant] began?
Prior to [CWCC grant], what kind of data infrastructure did your organization have? What kind of infrastructure did your partner organizations have? Were the data shared amongst partners? Linked across partners?
Is there an expectation that partners participating in [CWCC grant] share data with [name of lead agency’s organization] as the lead organization?
If so, what are the purposes or goals of partners sharing data with the lead organization?
Are partners also expected to share data with other partner organizations? (Probe: Why or why not?)
Have expectations around data sharing changed since the [CWCC grant] began? (Probe: In what ways and why?)
What is the process, if any, for partner organizations to share data (either with the [name of lead agency’s organization] as lead organization or with other partner organizations)?
Do partners share data summaries or raw data?
Are partners entering data into an integrated system (available across multiple partner organizations)?
If so, who is responsible for hosting/maintaining the integrated data system?
Are data shared at the individual participant level (e.g., the parent/caregiver or child level) or in the aggregate (e.g., all caregivers, all children)?
How frequently are data shared?
What kind of training, if any, did staff receive related to data collection or entry?
What kinds of security measures, if any, are in place to safeguard the data?
Have data sharing processes changed since the [CWCC grant] began? (Probe: In what ways and why?)
Which, if any, data from partner organizations are linked?
In what ways are data linked?
In what ways are data linked and used to do the following:
identify families in need of child abuse and neglect (CAN) prevention services
identify the specific needs of families
make informed decisions about service provision
inform continuous quality improvement efforts
track outcomes
To date, how successful have the organizations partnering in [CWCC grant] been at sharing and using client data? Specifically, probe for whether they are doing the following (and ask why not, if applicable):
Using data to identify target populations?
Using data to identify client needs at intake?
Using data to decide on services to provide/refer to?
Using data to assess client service receipt and program participation?
Using data to assess client outcomes?
Using data to self-assess performance on above activities?
Are there particular processes or tools that have helped facilitate data sharing and use? (Listen for information about data systems, training/support to use those systems, templates, clear expectations about data sharing and use, regularly scheduled opportunities to share and discuss data)
For Project Directors from Lead Grantee Organization:
Have partners received guidance on how to interpret data and apply it to their work?
For Leaders from Partner Organizations:
Has your organization received guidance on how to interpret data and apply it to your work?
What are some of the barriers, if any, that hinder efforts for partners to share and use data? (Probe for barriers related to cost, man power, data security, legislation on data sharing/use; barriers to sharing across programs, across systems, etc.)
For Project Directors from Lead Grantee Organization:
How are these barriers being addressed?
For Leaders from Partner Organizations:
Are there any barriers that your organization has faced? How are the barriers faced by you and other partner organizations being addressed?
Next, we have some questions about implementation of the [CWCC grant]-funded activities.
For Project Directors from Lead Grantee Organization:
What were some of the initial activities that were prioritized when developing the [CWCC grant] during proposal writing?
Were these initial activities things that were already taking place (through the lead organization or partner organization), or were they newly planned activities? How did you prioritize new and existing activities?
For Leaders from Partner Organizations:
Are you aware of how the initial activities for the [CWCC grant] were prioritized when developing the proposal?
If so, do you know whether these initial activities were things that were already taking place (through the lead organization or partner organization), or were they newly planned activities? Do you know how new and existing activities were prioritized?
If so, do you know whether planning started with existing partners (to determine what they could do/offer) or did planning start with what the partners thought the community needed (in other words, what the [collaborative] needed to do) and then seek out organizations to deliver activities to help meet those needs?
After the grant award, what were some of the subsequent priorities, if any, that emerged during the 10-month planning phase?
Have priorities shifted during the implementation phase? (Probe: If so, in what ways and why?)
For Project Directors from Lead Grantee Organization:
Please tell me about the continuum (or menu) of services or activities that families served by [CWCC grant] can receive.
How did your partners determine what should be included in the continuum of services offered to participants?
In what ways, if any, did partners come together to offer or deliver these services?
Which services are offered by organizations that partner in the [CWCC grant], and which are offered by other organizations?
Since the [CWCC grant] was awarded, have you noticed any shifts in the array of social service organizations available to support families in your community?
If so, please describe the changes. (Probe for changes in the number of service providers, types of services offered, target populations.)
For Leaders from Partner Organizations Only: In what ways, if any, have these changes impacted the work of your organization?
Do you anticipate any additional shifts in the array of social service organizations available to support families in your community? Why or why not?
For Leaders from Partner Organizations:
Please tell me about the continuum (or menu) of services or activities that families served by [CWCC grant] can receive.
Do you know how the partners determined what should be included in the continuum of services offered to participants? Were you or your organization involved in this process?
In what ways, if any, did partners come together to offer or deliver these services?
Are all of these services offered by organizations that partner in the [CWCC grant], or are some offered by other organizations?
Thinking about [CWCC grant] as a whole, which services or activities, if any, are provided through collaborative efforts? (For tribal grantees, probe for information about cultural or spiritual activities and interventions included as part of their programming.)
Does this way of delivering services or activities differ from how services were delivered prior to the [CWCC grant]? (Probe: If so, how and why?)
For Leaders from Partner Organizations Only: Are there any services or activities that your organization started delivering collaboratively as a result of the [CWCC grant]?
If so, what are some of the pros and cons of working collaboratively to deliver these services? (Probe for issues related to organizational identity, changes in mission and/or scope of work)
Have these new approaches required changes in terms of activities, staffing, reallocation of resources, etc.?
For Leaders from Partner Organizations Only: Are staff on board with these changes?
Which, if any, of the collaboratively provided services offered through [CWCC grant] pre-dated the grant? (Probe: Were they offered as part of a prior collaborative initiative?)
Which services and/or partners, if any, are new because of the grant?
Have additional changes been made since the grant was awarded?
Which services or activities offered by [CWCC grant], if any, are delivered by a single agency (as opposed to multiple agencies)?
Does this way of delivering services or activities differ from how services were delivered prior to the [CWCC grant]? (Probe: If so, how and why?)
For Leaders from Partner Organizations Only: Are there any services or activities that your organization delivers as a single agency?
For Leaders from Partner Organizations Only: If so, what are some of the pros and cons of being the sole partner responsible for delivering these services?
Do you believe there are services or types of services that are more efficiently and effectively provided through a community collaboration effort?
If so, which services and why?
Are there any circumstances in which collaborative service delivery is not feasible?
Do you believe there are services or types of services that are more efficiently and effectively provided through a single-agency approach (as opposed to by multiple agencies)?
If so, which services and why?
Are there any circumstances in which single-agency service delivery is not feasible?
To date, are there any promising practices that have emerged from implementation of [CWCC grant]?
What implications, if any, might these practices have for the future of this particular initiative?
For Leaders from Partner Organizations Only: What implications, if any, might they have for your organization?
What implications, if any, might they have for other grantees or sites interested in developing a CWCC initiative?
To date, have any new or modified policies (legislation or internal organizational procedures/policies) emerged from implementation of [CWCC grant]?
If so, which policies?
Have these policies been created or changed within particular agencies or organizations (state, local, or tribal)?
To what extent, if any, have these policies made their way into practice?
In what ways, if any, do these policies impact your organization and your partner organizations?
In what ways, if any, do these policies impact the families and children that your organization serves? Those served by other partner organizations?
Now I’d like to ask a few questions about some of the factors that may be influencing the implementation of [name of initiative].
For Project Directors from Lead Grantee Organization:
When you think about planning your [CWCC grant] (during the proposal phase and post-award planning phase), what were some of the factors that influenced your work?
We know that often, contextual factors may facilitate or make your work easier Have any of the following contextual factors done this? (Probe: If so, can you provide some examples?)
state, local, and tribal policies
geographical location (rural vs. urban)
resources
staff and organizational capacity
training
cross-partnership coordination
existing infrastructure
Did any of those contextual factors hinder your work?
For Leaders from Partner Organizations:
When you think about the process for planning the [CWCC grant] (during the proposal phase and post-award planning phase), what were some of the factors that influenced the work of the lead and partner organizations?
Are there any factors that influenced your organization in particular?
Did any of the following contextual factors facilitate or make the [collaborative’s] work easier? (Probe: If so, can you provide some examples?)
state, local, and tribal policies
geographical location (rural vs. urban)
resources
staff and organizational capacity
training
cross-partnership coordination
existing infrastructure
Did any of those contextual factors hinder your [collaborative’s] work?
Do you have any suggestions for how to leverage contextual factors to support implementation of CWCC initiatives?
Do you have any suggestions for how to handle negative contextual factors?
For Leaders from Partner Organizations Only: Since beginning to implement the [CWCC grant], have there been any changes to the target population(s) typically served by your organization?
If so, what informed or led to these changes?
Do you agree with those changes? Why or why not?
When were the changes made?
Do you anticipate any additional changes to target population? Why or why not?
For Project Directors from Lead Grantee Organization:
Since beginning to implement the [CWCC grant], have there been any changes to the target population(s) typically served by the partner organizations?
If so, what informed or led to these changes?
Do you agree with those changes? Why or why not?
When were the changes made?
Were these changes adopted by all partner organizations, or just some of them?
Do you anticipate any additional changes to target population? Why or why not?
Since the [CWCC grant] was awarded, have there been any changes to the CPS agency or any other major agency related to preventing child abuse and neglect? Any changes to state, local, or tribal policies?
Have there been changes in leadership at any of these agencies functioning at the state, local, or tribal level? If so, have these changes affected the implementation of the [CWCC grant]?
Have you noticed a difference in the guidance and/or policies coming out of state, local, or tribal agencies? (Probe: If so, is this impacting the work of the [CWCC grant]?)
Have there been any changes as a result of the [collaborative’s] work (e.g., review/removal of unnecessary service delivery or funding barriers)?
In these last few questions, we would like to know more about your perceptions of the progress your team is making towards achieving the project goals and also lessons learned thus far.
What are the ultimate goals and desired outcomes of the [CWCC grant]? (Probe for intermediate system- and participant-level outcomes, as well as long-term outcomes)
What role did partners have in creating the goals?
How does the [initiative] track or measure progress towards meeting its goals?
To what extent do you believe the [CWCC grant] is meeting its goals? (Probe for specific examples)
Can you describe some of the early “wins” or successes from the [collaborative]?
Are there any goals that the [initiative] is falling short in at this time? (Probe for specific examples)
Are there any outcomes that the [initiative] is falling short in at this time? (Probe for specific examples)
In your opinion, is there anything that has happened since the [CWCC grant] started that would not have happened without the grant?
To date, what lessons, if any, have you learned about cross-agency collaboration?
In what ways, if any, have these lessons confirmed that the approach to collaboration is on-track?
In what ways, if any, have these lessons led you to change how you and your partners are approaching the CWCC work?
What advice would you give to other organizations preparing to launch or participate in a CWCC effort?
1 Throughout this document, wherever terms are in brackets, the grantee’s own term will replace the bracketed terms in order to facilitate comprehension. Potential terms include: partnership, collaboration, coalition, collective impact, collaborative, collective, program, or project.
Abt
Associates ▌pg.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2022-09-19 |