Models of Coordination and Technical Assistance to Achieve Outcomes in Communities

OMB Control #: 0990-0421
Expiration Date: 10/12/2020

Attachment A: Key informant semi-structured discussion guide on using policy levers to enable local cross-sector collaboration

A. Introduction and outline purpose of call

Thank you for speaking with us today. My name is [NAME] and I am joined by my colleague, [NAME]. We are from Mathematica, and we are conducting a project called Models of Coordination and Technical Assistance to Achieve Outcomes in Communities, on behalf of the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, or ASPE.

The purpose of today’s discussion is to better understand the policy levers or strategies that the federal government and philanthropies have used to foster cross-sector collaboration at the local level. By cross-sector, we mean supports that improve the ability of organizations to cooperate with each other. Examples of relevant policy levers and strategies include convening stakeholders across various sectors, providing grants that encourage cross-sector collaboration, conducting a formal evaluation of an initiative, or coordinating cross-sector TA to maximize local integration of an initiative. We are interested in learning about specific factors that may facilitate or hinder local cross-sector collaboration, including the relative advantages, cost, and speed of the policy levers used; any interactions between policy levers; challenges and barriers encountered; and recommendations for future initiatives.

We hope to frame today’s discussion specifically around your experience with [INITIATIVE].

Nothing you say will be linked to your name or the organization that you work for. Participation in the study is voluntary. You may choose not to answer a question and may stop or leave the discussion at any time. This discussion will take approximately 1 hour.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-0421. The time required to complete this information collection is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.

With your permission, to make sure I hear everything you have to say, I would like to record the discussion. The recording is for research purposes only, will only be available to the study team, and will be destroyed at the end of the study. Could you each please tell me if I have your permission?

B. Respondent and initiative background

Let’s start with some background about you and [INITIATIVE].

1. What role or roles do/did you play in [INITIATIVE]?
   
   
   

2. Interviewer: With only the first respondent interviewed for each initiative, ask them to confirm the information Mathematica gathered prior to the interview using federal and/or public sources. Specifically, confirm the timeline (ongoing or complete) and setting (place-based or national) for the initiative.
   
   
   

3. How would you describe [INITIATIVE] from your perspective?

Potential Probes:

• What is/was it trying to accomplish?

• Who would you consider primary players or stakeholders involved in [INITIATIVE]?
  
  
  

4. Can you tell me how [INITIATIVE] started?

Potential Probes:

• How was the need for [INITIATIVE] identified?

• Who designed [INITIATIVE]? Were multiple agencies or organizations involved in designing [INITIATIVE]?

• Was [INITIATIVE] based on a model already implemented in one or more communities?

• What criteria were used to select communities for [INITIATIVE]? Do you think these criteria led you to the types of communities you intended to reach?
  
  
  

5. Did cross-sector collaboration for these same purposes regularly occur prior to the initiative? If so, to what extent?

1. FOR LOCAL PROGRAM STAFF ONLY: Did [INITIATIVE] build on earlier work in your community? How did your community become involved in [INITIATIVE]

6. Were there any pre-existing conditions or factors in the local environment that helped get the initiative off the ground?
   
   
   

7. Were there any barriers or challenges that either prevented cross-sector collaboration or made it difficult to achieve prior to [INITIATIVE]?

C. Use of levers

Next, I want to talk about the specific policy levers and strategies employed as part of [INITIATIVE] that help/helped facilitate cross-sector collaboration. As a reminder, policy levers and strategies include activities like convening stakeholders across sectors, disseminating research and evaluation findings, issuing regulatory waivers to facilitate coordination, or using TA to convene peer learning groups across sectors and local areas.

8. What types of policy levers or strategies are/were used to facilitate cross-sector collaboration as part of [INITIATIVE]?

Potential Probe:

• That is, what did/has the federal government/philanthropy done to facilitate cross-sector collaboration?

9. Now I’d like you to think, overall, about all of the policy levers or strategies [INITIATIVE] used.

1. Which policy lever or strategy seems(ed) to be the most impactful? Which seems(ed) to be least impactful? Why?

2. Which policy levers or strategies do you think were the quickest or easiest to start implementing? By that I mean, which helped produce the quickest early “wins” or results, and which took/is taking longer to have an impact?

3. How responsive are/were the policy levers or strategies to the needs of those implementing the initiative?

Potential Probes:

• How did the initiative assess needs in recipient/designee sites? How did members of the initiative prioritize and determine how to meet these needs?

• In what way(s) are/were the policy levers responsive to these needs?

1. To what extent do/did the set of policy levers or strategies work together as part of a coordinated effort?

Potential Probes:

• Which levers work well together, and which don’t?

5. Did any of the policy levers or strategies bring about unintended consequences?

6. Were there one or more policy levers or strategies that were not used that you wish had been used or that you think would have been helpful?

10. Of all of these policy levers, which 3-4 do you think were most important to [INITIATVE]?

11. Now I’d like to ask you a couple of questions about each of the 3-4 most important policy levers or strategies you identified.
    For each of the 3-4 most important individual policy levers identified, ask the following:

1. How helpful is/was [LEVER] to cross-sector collaboration? What does/did it help to achieve?

2. Are/were there any challenges associated with [LEVER]?

3. Now I’d like to hear about how cost effective [LEVER] is/was. That is, to what extent has it been/was it a good investment of staff time and other resources. Thinking about your time commitments relative to the benefits reaped from participation, how cost-effective do you think [LEVER] is/was?

D. Outcomes/Perceptions of Outcomes (7 minutes)

Next, I want to talk about the outcomes of [INITIATIVE]. Please consider any conclusions from formal evaluations or other reporting, where available.

12. Thinking about the overall impact of [INITIATIVE], what were the initiative’s most important successes or accomplishments in improving cross-sector collaboration?
    
    
    

13. How does/did this initiative affect local program staff? Does/Did it make people’s jobs easier or harder in the short run? What about in the long run?
    
    
    

14. How does/did this initiative affect funders? Does/Did it make people’s jobs easier or harder in the short run? What about in the long run?
    
    
    

15. How does/did this initiative affect participating agencies? Does/did it make people’s jobs easier or harder in the short run? What about in the long run?
    
    
    

E. Lessons Learned

Now I want to talk about any lessons that have been learned from [INITIATIVE].

16. How would you change [INITIATIVE] if you could start over? (How) would you re-prioritize resources for better results?

    • If respondent replies that they would not change their approach, ask:
      If you had more funding for [INITIATIVE], how would you have used the additional funds?
      
      
      

17. What advice would you give to practitioners or policymakers looking to design an initiative to increase cross-sector collaboration on the local level?

F. Conclusion

18. Thank you for taking the time to speak with us today about [INITIATIVE].

We would love to see any briefs or reports on [INITIATIVE] that you’re able to share.

1. Anything that is available confidentially/internally or in draft form, if you’re willing to share it, will be used only for the purposes of the project and we will check with you before citing it even indirectly/anonymously.

2. We would happily accept any excerpts of key takeaways that might contribute to our general understanding of [INITIATIVE].

A-3