12. MUSE Program Coordinator-Manager and Program Director Interview Questions

July 2020 OMB Control No.: 0970-0521

Expiration Date: 12/31/2021

Length of time for instrument: 1 hour and 30 minutes

MULTI-SITE IMPLEMENTATION EVALUATION OF TRIBAL HOME VISITING (MUSE)

PROGRAM COORDINATOR/MANAGER AND PROGRAM DIRECTOR INTERVIEW QUESTIONS

This collection of information is voluntary. Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0521, Exp: 12/31/2021. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Kate Lyon, James Bell Associates; 3033 Wilson Blvd. Suite 650, Arlington, VA 22201; MUSE.info@jbassoc.com.

Qualitative Interview – Program Coordinator/Manager and Program Director

Thank you for taking part in today’s interview. The purpose of this study is to learn about tribal home visiting programs and the experiences of families receiving home visiting services.

We have asked you to take part in this interview because you are a Program [Coordinator/Manager/Director] for one of the home visiting programs participating in MUSE. We are interested in hearing about the work you and your staff do at [PROGRAM NAME]. We will ask you questions about your experience with the planning and on-going implementation of [Program Name] as well as questions about the agency where you work, your staff and the families who you serve.

Your participation in this interview is voluntary. If you choose to participate, it will take about 1 hour and 30 minutes. Please let me know if you need clarification on a question or if you need to take a break at any time.

Your answers will be kept private. Only the MUSE study team will have access to this information. What you tell me here today will not be shared with anyone at your program or any other agencies. We will not report information collected in this study in a way that could identify you or your program.

Please note that there are no right or wrong answers to the questions I am going to ask. I am interested in learning about your particular thoughts and experiences.

Universe of Possible Questions—Program Coordinator/Manager and Program Director

1. Please tell me about what led you to enter the field of home visitation.

   1. How long have you worked in this field?

   2. What did you do before you became [Title] here?? (Probe: What other fields have you worked in?)

   3. What keeps you working in home visiting?

2. What’s it like to work as a [Title] here at [program/organization]?

   1. What does a typical workday look like for you? (Probe: What are your work roles and responsibilities?)

   2. What other programs do you work on at [Agency]?

   3. What do you like best about working for [Agency]?

   4. What is difficult about working for [Agency]

   5. How do home visitation services for [program/organization] differ from other places in which you may have worked or you know about?

3. What do you hope families get out of the home visitation services they receive from [program/organization]?

   1. How good of a job does [program/organization] do in helping families get these benefits?

   2. What else could [program/organization] do to meet these expectations.

4. Based on your experience, what would you say are the skills that any parent needs to have in order to raise a happy, healthy and successful child?

   1. Of these skills, which ones are most important for parents in this community [Urban sites: Native American/American Indian families in XYZ city]? Why?

   2. How do parents in this community [Urban sites: American Indian/Native American parents in XYZ (i.e. “the bay area”)] generally gain these skills?

   3. What makes it hard for parents to learn about or do these things? (Probe: How might these barriers differ for Native American families?)

   4. How does [program] support families in gaining or strengthening these skills?

5. Why do you think parents sign up for services through [program/organization]?

   1. How do you decide if the program is a good fit for a family?

   2. In general, what are the major strengths of families who have joined the program?

   3. What do you see as the biggest needs of parents who participate?

   4. How does [program/organization] support families in meeting these needs?

   5. What are the biggest challenges that [program/organization] faces in meeting these needs?

   6. In what ways do you think the families who [program/organization] serves have similar or different reasons for seeking out home visiting services than families in other communities?

6. What types of staff make up your home visiting program? (Probe: professional backgrounds, typical length of employment, whether or not they are from community, Native/Non-Native)

   1. What do you look for when hiring a home visitor?

   2. What do you do to retain home visitors?

      1. How is that working?

      2. What make this challenging?

7. How is your evaluation staff structured?

   1. What did you look for when finding an evaluator?

   2. Please describe your relationship with the evaluator for [Program/organization]

   3. How do you communicate with him/her?

   4. What is his/her primary role?

   5. What are the most valuable things that you evaluator does for the program?

   6. What do you wish they would do differently or in addition to what they do now?

      1. What makes this difficult to put in place?

8. Please tell me a little about the home visiting services you provide here at [program]?

   1. What is the program trying to accomplish?

   2. How are these services tailored to the local context?

   3. If program existed prior to MIECHV: How has your program changed since you received MICHVE funding?

   4. What other services does this agency provide? How are these services connected to this program?

9. How has COVID-19 and related responses impacted your program?

   1. Can you tell me about any changes you all have made?

   2. How have the changes gone?

   3. What was the impact on staff?

   4. What effects have these adjustments had on home visits? Any effect on quality?

   5. Why do you think it was important for your program to continue to serve families during this time?

   6. What have you learned, if anything, from these adjustments that you will continue to use once home visits are in person again?

10. Please tell me about a typical caseload for a home visitor.

    1. How do you make decisions about who to assign new families?

    2. How do you ensure it is manageable?

    3. How do you support home visitors when they are overwhelmed by their caseload?

11. How do you know your program is working for a given family?

    1. What do you mostly attribute this success to? (Probe: the work your home visitors do? Good program planning? Implementation supports? The families themselves?

    2. How do you learn when things about the program are not working well?

12. How have enrolled families responded to [program]?

13. How do you support home visitors in tailoring home visits to a family's goals? Their needs? Their interests? (Probe: when is it important to provide tailored services?)

1. How do you ensure that home visitors understand/empathize with the families who they serve? (Probe: Hiring, training?)

14. What are some of the most challenging things your staff encounter in home visits? (Probe: how do you handle it when these things come up?)

15. What supports are in place for when home visitors become overwhelmed with their work? (Probe: what do you typically do to help? Outside supervision? Peer support)

16. How do you determine how much home visiting an individual family needs?

    1. When can you tell that a family no longer needs home visits?

17. From your perspective, what most influences a home visitor's ability to provide successful home visits to your families? (Probe: Understanding how to use the model? Feeling as though the model will work families? The amount of work required to use the model? The size/characteristics of caseloads? The supervision received?)

18. What kinds of supervision do your home visitors receive? (Probe: is it group or individual supervision?

    1. How often do they participate in supervision?

    2. Who provides the supervision?

    3. If interviewee is supervisor: What does a typical supervision session look like? (Probe: What kinds of things do home visitors usually bring up or talk about in supervision?

    4. What do you think is the most helpful aspect of supervision?

    5. What do you wish you could do differently as a supervisor?

    6. Please describe the relationship you have with your supervisees generally? (Probe: Do you think they confide in you when they have made a mistake? How do you demonstrate to your staff that you are invested in their professional development? In their personal growth?)

19. How does the quality of the relationship between the home visitor and the family impact home visits?

    1. How can you tell when a home visitor has a good relationship with a family? When they do not? (Probe: What do you do when you can tell it is challenging?)

    2. What do you think accounts for positive working relationships between a home visitor and a family?

    3. What do you think accounts for more challenging working relationships between a home visitor and a family?

    4. How have changes related to COVID-19 affected home visitor-family relationships?

20. What do home visitors do after a home visit? (Probe: What kinds of tracking do they do? Data entry? How do you account for the time needed for these activities? What kinds of support do they have in completing required documentation, etc.?)

    1. How do you facilitate home visitors having time in their workday to reflect on what happens during specific home visits?

21. Please tell me anything you know about the model selection process? (Probe: If relevant, what was your involvement in the process? Who else was involved? What other models did you consider?)

    1. What kinds of things did folks consider while selecting a model?

    2. When and how did you first hear about [name of model]? (Probe: how did agency leadership explain why this model was selected to you and your coworkers?)

    3. Why do you think [name of model] was selected? (Probe: Why did planners think it would work here? Do you have a sense that it has worked in other communities?)

    4. What were your initial impressions of [model/curriculum]? (Probe: How did you first become aware of [model/curriculum] about it? How long have you been using it? Did you think the model was going to be difficult or easy to use? What other expectations did you have? How has the model met your expectations?)

    5. Would you choose this model again?

    6. What experience had your agency had with implementing "evidence-based" interventions before MIECHV?

22. How is the model working out for your program?

    1. How do home visitors learn to use [model/curriculum]?

    2. What do you do to help them gain confidence in using the model?

    3. How confident do you think staff are with using the model?

    4. What do they need to feel more confident?

    5. Do you think your coworkers feel similar to you?

    6. What supports did you receive from the model developers as you adjusted to COVID-19? What was most/least helpful?

23. How supportive are agency leadership of the model?

    1. Evidence-based interventions more generally? (Probe: how do you know?)

    2. What kinds of supports do you receive in your implementation work?

    3. What could the agency do better to support your team in using the model?)

24. How well does [model/curriculum] fit in with the norms or values of the families you serve?

    1. How well does it fit in with the norms or values of the communities that you serve?

    2. How well does it fit in with your own norms or values?

    3. How does the overall fit of [model/curriculum] influence when and with whom to use it?

25. To what extent does [model/curriculum] do a good job in meeting needs of the families you serve?

1. To what extent does it do a good job in addressing the needs of the communities you serve?

2. To what extent does [model/curriculum] consider your own needs as a home visitor?

3. How have you supplemented or augmented the program to do a better job?

4. What would you change about [model/curriculum]? (Probe: Why would make these changes.)

26. How do you [the program] support home visitors in gaining skills around making necessary tweaks to the model as they tailor services to families in a particular home visit?

27. What enhancements, adaptations or supplements does your program include?

    1. Please describe the process your site went through to develop this [name of EAS]? (Probe: how did you develop it? Who was involved with decision making?)

    2. What do you see as the overall effect of [name of EAS]?

       1. Has this impact been worth the extra work?

       2. How does it augment the home visiting model?

28. What besides [name of model] impacts how helpful home visiting services are for the families you serve? (Probe: connection to community resources/events, parent groups, cultural activities)

29. How important do you think [name of model] specifically is to the success of home visits? (Probe: Do you think you would you achieve similar outcomes with any home visiting model? Better ones with a different model?)

30. What supports are in place at your agency to help staff do a good job as home visitors? (Probe: Supervision? Training? Coaching? Administrative support/resources?)

    1. Any other on-going support not related to the model that helps home visitors be great at their jobs? (Probe: mental health trainings? Cultural training?)

    2. What other kinds of support do home visitors most need to feel prepared? (Probe: cultural trainings? Data? Coaching from model developers? Do they have access to these supports?)

    3. What kinds of things does your agency do to support implementation more generally?

       1. How are policy and procedures manuals used by staff?

       2. What revisions have been made to the P&P?

       3. How does your supervisor know how you are doing using the model?

       4. What kind of feedback do you receive?

       5. Is using the model well something that is considered in your performance reviews?)

31. What supports do you rely on to do your best at [TITLE]?

    1. How has PATH supported you in your role?

    2. How has TEI supported you in your work?

    3. How could TA for evaluation be enhanced or improved?

    4. What about Program Directors from other THV programs, how have you connected with your peers and has that been helpful? How would this type of peer to peer support be best facilitated?

32. How did/do larger systems, including Tribal governments, state home visiting programs, education, health and mental health systems impact program decision making?

33. What makes managing this federally funded program different than providing other kinds of services? (Probe: What are the advantages? Disadvantages?

34. How do you share your program's story with community members? (Probe: How has this helped to ensure the sustainability of the program?)

35. What is the biggest benefit of home visiting for the families you serve through program/organization]?

    1. What makes it easier for families to get this benefit?

    2. What makes it harder?

    3. How would you mitigate these challenges in an ideal world?

36. What most influences whether a family will get that benefit out of your program? (Probe: My success as a manager? The home visitors using the model well? Fit of model to family? Fit of home visitor and family? Family’s investment in program? Supplemental programming? Connection to community resources?)

37. How do you know when you are doing a good job as a manager?

    1. How does home visitor performance impact how successful you feel in your role?

    2. Thinking about the last progress report you prepared, were there areas where having to write up what the program accomplished made you recognize a success that had gone unnoticed?

    3. Were there areas that you realized had been neglected and needed more of your attention?

    4. What kinds of information do you use to assess how well you are doing as a manger? (Probe: performance data—internal, MIECHV, supervisor feedback, home visitor feedback, participant feedback). How do you access this information?

    5. Can you describe any supports from agency leadership that seem to make the most difference in your feeling successful? What about inspired?

38. Is there anything that we have not talked about regarding your experience as a manager here at [name of program/organization], the [model/curriculum] or the home visitation services you provide that you would like to share?

Multi-Site Implementation Evaluation of Tribal Home Visiting OMB Supporting Documents: Program Coordinator/ Manager and Program Director Interview Questions

11