Supporting Family Economic Well-Being through Home Visiting (HomeEc) Formative Evaluation

Supporting Family Economic Well-Being through Home Visiting (HomeEc)

SEMI-STRUCTURED STAFF INTERVIEW GUIDE FOR LEARNING CYCLES

In 2024, the HomeEc team will use this interview guide to:

• Collect data from program staff and partners after each learning cycle to understand practice implementation

• Understand what went well and what was challenging with learning about and using the practice with caregivers and/or other staff members

• Learn how programs should refine the practice to improve service delivery and caregiver and/or staff receptiveness

A. Introduction and Informed Consent

Hello, my name is [NAME], and my colleague’s name is [NAME]. Thank you for taking the time to speak with us today. We are from [ORGANIZATION(S)], and we are here to learn about your experience with [HOME VISITING PROGRAM].

As you know, [PROGRAM NAME] has been testing [PRACTICE NAME] to better support family economic well-being through home visiting. [Insert description of practice.] Today we’d like to hear about your experience learning about and using [PRACTICE NAME]. We would like to hear about what has gone well, what has been challenging, and how the practice can be refined to work better for you and for the families your program supports.

The information you share today will help us understand your experience using [PRACTICE NAME] and help us decide on next steps. We expect this discussion to take about 60 minutes. Before we start, I want to let you know your participation in this interview is voluntary, and your responses will be kept private to the extent permitted by law, except if you say something that suggests you are very likely to harm yourself, that you are planning to hurt another person or child, or that someone is likely to harm you. We might summarize the information in a public final report; however, we will not use your name in any of our reports. There are no right or wrong answers, and you can skip any questions you do not want to answer. You have the option to skip any question or end the interview at any time without any reason and consequences. The answers you give today will not affect your participation in the study or your home visiting grant. There are no direct benefits and no risks associated with participation in the study. We would like to record this discussion to supplement our notes. Only the study team will have access to those materials. We will destroy the recording and the notes at the end of the study. Do we have your permission to record this discussion?

[If yes, press record. If no, take written notes only.]

Do you agree to participate in this interview?

Do you have any questions before we get started?

If you want more information or have questions about the study or your privacy rights, you can call the project director, Katie Eddins at 202-838-3614.

INTERVIEWER NOTE: Customize the questions in Section B before the interview based on your understanding of the program, the practice being tested, and the staff member you are speaking to.

B. Interview questions

Topic

Feedback on training, guidance, or materials

I’d like to start by discussing [the training, guidance, or materials] you received as part of using [PRACTICE] for the HomeEc project. As a reminder, your program [used practice in TIMEFRAME/you received the training, guidance, or materials in TIMEFRAME].

1. What did you think about the [training, guidance, or materials]?

1. What parts of the [training/guidance/materials] were particularly helpful, relevant, or engaging?

2. What parts of the [training/guidance/materials] were not helpful, relevant, or engaging?

2. What were the most important things you learned from the [training/guidance/materials]? Why?

3. Do you have suggestions for improving [training, guidance, or materials]? (Probe for changes and additions.)

Use of practice

Now I’d like to learn more about how you used [PRACTICE] with the families you support.

4. Walk me through how you used/implemented [PRACTICE]. Pick a typical case of when you used/implemented [PRACTICE] [with a caregiver] and explain how you used it. (If the practice has multiple parts, ask about each part.)

   1. Please describe why this case is a good example of how you used [PRACTICE].

5. What has worked well with using [PRACTICE]?

a. What factors made [PRACTICE] work well?

6. What hasn’t worked well with using [PRACTICE]?

1. What factors have made using/implementing [PRACTICE] more difficult?

2. Have any parts of [PRACTICE] felt unnatural or uncomfortable? Which ones?

3. How did you overcome those difficulties?

7. Have you been able to use/implement the [PRACTICE] as intended (for example, as was taught in the training)?

1. What about [PRACTICE] was easy to do as planned?

2. What about [PRACTICE] was challenging to do as planned?

8. Have you made any modifications to [PRACTICE] while implementing it?

a. If yes, please describe these modifications.

b. Why did you make these modifications?

c. How common were these modifications (or how often was modification needed)?

d. Do you feel the modifications were effective or helpful for implementing [PRACTICE]?

9. If more than one caregiver participated in a home visit, how did you use [PRACTICE] with multiple caregivers?

   1. Do you made any adjustments to [PRACTICE] if using with multiple caregivers compared to using [PRACTICE] with one caregiver?

10. Have you used [PRACTICE] with [type of families, such as families whose primary language is not English or with families of diverse backgrounds]?

    1. IF YES: What has gone well with using [PRACTICE] with [type of family]?

    2. IF YES: What has been challenging with using [PRACTICE] with [type of family]?

11. Would you say [PRACTICE] has made it easier or more difficult to address economic well-being with families? Please describe.

12. In what ways is [PRACTICE] similar to your previous work with families and in what ways does [PRACTICE] differ?

13. Has implementation of [PRACTICE] affected the delivery of other services or time spent on other activities?

14. For supervisors: Have you had a chance to observe the practice being used?

1. If yes, what are your takeaways from those observations?

2. Have your staff used [PRACTICE] according to how they were trained to use it?

15. For supervisors: What feedback have you received from the staff you supervise about [PRACTICE]? What have they told you about how comfortable they are using [PRACTICE]?

16. What data are you responsible for related to [PRACTICE]? (for example, self-assessments, a log or tracker, or other data)

1. How do you track use of [PRACTICE]? What is easy or difficult about it?

17. How has the use of [PRACTICE] affected the burden on home visitors?

a. Has the burden increased, decreased, or not changed from using [PRACTICE]?

b. If the burden increased, how could staff adjust the practice to reduce the burden on home visitors?

18. How has the use of [PRACTICE] affected the burden on participating families? By burden, we mean additional time, effort, or responsibility.

a. Have families experienced increased, decreased, or no change in burden from the use of [PRACTICE]?

b. If the burden increased, how could home visitors adjust the practice to reduce the burden on families?

Proposed changes or improvements (tailor based on previous responses)

Now I’d like to hear about any changes you’d make to [PRACTICE].

19. How would you adjust [PRACTICE] to make it work better or make it easier for you to use?

1. [If practice was used with caregivers]: How would you adjust [PRACTICE] to make it easier to use with caregivers?

2. How about adjustments to make caregivers better buy into the practice? (for example, use it more, think it’s more useful, etc.)

20. How could your program adjust its policies or procedures to improve its adoption and use of [PRACTICE] [with caregivers]?

1. How would these adjustments improve your use of [PRACTICE] [with the caregivers you support]?

21. What additional supports would be helpful for [home visitors] to use [PRACTICE]? (for example, supervision, feedback and coaching, reminders to use practice, or other supports)?

Participants’ responsiveness and extent to which the practice met families’ needs regarding economic well-being

Now I’d like to hear how families you support engaged with [PRACTICE].

22. On a scale of 1 to 5, with 1 being “very ineffective” and 5 being “very effective,” how effective do you think [PRACTICE] has been in helping families meet their needs or goals regarding economic well-being?

1. Why did you choose that rating?

2. Have you observed an instance of [PRACTICE] helping (directly or indirectly, depending on practice) families meet their economic well-being needs or goals? If so, please provide an example.

23. Over the past [TIME INTERVAL], have you seen a change in how receptive families are to participating in [PRACTICE] or how engaged they are with [PRACTICE]? (Probe by asking about participant reaction/engagement we expect to see.)

24. Have there been situations in which [PRACTICE] worked better or worse in helping families meet their economic well-being needs or goals?

1. Please describe some examples of when [PRACTICE] worked well and when [PRACTICE] didn’t work as well.

2. What factors helped or hindered [PRACTICE] in meeting family economic well-being needs or goals?

25. Was [PRACTICE] particularly well-suited for some families?

    1. Please describe these families and why [PRACTICE] was appropriate for them.

    2. Do you think more families could benefit from [PRACTICE]?

26. What other data or information would help show whether [PRACTICE] is working?

If partner organizations are involved in implementing the strategy, ask this set of questions for each partner.

Now I’d like to hear how [PROGRAM] worked with [PARTNER] to implement [PRACTICE].

27. [IF PROGRAM STAFF]: How often do you [meet/talk] with [relevant partner] about [PRACTICE]?

1. How do you typically communicate with the partner staff? (Probe for standing meetings, ad hoc, phone, in-person, and email.)

[IF PARTNER STAFF]: How often do you [meet/talk] with the home visiting staff about [PRACTICE]?

1. How do you typically communicate with the home visiting staff? (Probe for standing meetings, ad hoc, phone, in-person, and email.)

28. [IF PROGRAM STAFF]: What topics and issues related to [PRACTICE] do you discuss with [relevant partner]? (Probe for general implementation, data, specific troubleshooting, coordination, and other topics.)

[IF PARTNER STAFF]: What topics and issues related to [PRACTICE] do you discuss with the home visiting staff? (Probe for general implementation, data, specific troubleshooting, coordination, and other topics.)

29. [BOTH PROGRAM AND PARTNER STAFF]: What is working well with using [PRACTICE] in the partnership? What is not working well?

1. [If applicable] What might improve the partnership’s use of [PRACTICE]?

Lessons learned (final test only)

Now I’d like to hear any lessons learned that came from using [PRACTICE] with the families you support.

30. Do you plan to continue using [PRACTICE]? If so, how will [PRACTICE] be helpful for you?

31. Are there any other materials or supports that would have been helpful to implement [PRACTICE]?

32. What advice or information would you share with other programs interested in implementing [PRACTICE]? (Probe for building buy-in, what might need to be tailored in other settings, and other tips)

Paperwork Reduction Act Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0531 and it expires XX/XX/XXXX.