Supporting Family Economic Well-Being through Home Visiting (HomeEc) Formative Evaluation

Supporting Family Economic Well-Being through Home Visiting (HomeEc)

The HomeEc team will use this guide to:

• Learn about the home visiting programs participating in the formative evaluation

• Understand the need each program will focus on for the formative evaluation

• Design a solution to address the need and determine how to implement and test the solution in iterative learning cycles

• Review the findings from each learning cycle and determine next steps

• Note: This is a guide, not a script. Facilitators will tailor the guide for the program, the phase of the evaluation, and the mix of staff we expect to attend the meeting. Facilitators can refer to Table 1 for objectives for each meeting. Facilitators will tailor questions in Table 2 to the specific program and add probes to explore the experiences and perspectives of meeting participants.

• The strategic planning meetings are meant to be collaborative discussions between the HomeEc team and program staff. During these meetings, program staff will contribute their knowledge and expertise of their program’s unique circumstances and the caregivers they work with, and the HomeEc facilitators will contribute information learned from available research and evidence. During the discussions, the HomeEc facilitators will probe, follow-up, and clarify questions and responses as appropriate. The HomeEc facilitators may use human-centered design activities during the meetings to provide staff with an interactive way to answer the questions.

DISCUSSION GUIDE FOR STRATEGIC PLANNING MEETINGS

A. Introduction and Informed Consent

Thank you for joining us today. Today’s meeting will focus on [objectives(s) from Table 1]. The meeting will last about 90 minutes. Your participation to the meeting is voluntary. Being part of this meeting is up to you, and it will have no consequences if otherwise. [If applicable: We will use an online whiteboard called Mural (https://mural.co/) to facilitate today’s activities and discussion. Mural allows everyone to participate anonymously. There are no right or wrong answers to our questions; you only have to share information you want to share, and you don’t have to answer any questions you don’t want to answer. Information you provide during these meetings will not affect your home visiting grant. There are no direct benefits and no risks associated with participation in the study. The information you share will be kept private to the extent permitted by law, except if you say something that suggests you are very likely to harm yourself, that you are planning to hurt another person or child, or that someone is likely to harm you. The information will be used to help decide the next steps for the formative evaluation in your program. We might summarize the information in a public final report on what we learn; however, we will not use any of your names in our reports. If you want more information or have questions about the study or your privacy rights, you can call the project director, Katie Eddins at 202-838-3614.

Do you agree to participate in this discussion? [Obtain verbal consent from each person individually]

Do you have any questions before we get started?

Great, let’s start with a recap of our last meeting and what has happened since we last met. [Insert recap here.]

Facilitator note: Customize this discussion guide ahead of the meeting based on your current understanding of the program, the objectives for the meeting, and activities and questions you will use to facilitate the discussion. Refer to Table 1 for possible objectives and Table 2 for guiding questions for each meeting.

Table 1. Meeting objectives and outputs, based on status in the formative evaluation process

Objectives	Output	Number of meetings per site
Create a shared understanding of program’s goals and its current services to support participants’ economic well-being Clarify motivations for change Identify the specific challenges involved in providing services that support economic well-being Explore root causes and effects of challenges	Potential focal need to address	1
Explore creative ideas for developing or refining economic well-being practices that might be a good fit for the program	Prioritized list of practices to address need	1
Create a shared understanding of plans to implement the selected practice, including: 1) the people that will implement the practice; 2) processes and policies to support the practice; 3) products, tools or resources needed to support the practice; and 4) principles that will guide proper implementation Create a shared understanding of plans to test the practice, including information collection plans for who will collect the information, how will it be collected, and how it will be analyzed and used	Implementation and testing plans	1
Review findings from the learning cycle by sharing findings with relevant staff and discussing the implications for the practice Created a shared understanding of next steps for implementing the practice, such as if program intends to maintain or scale the practice, adapt it and test it again, or abandon the practice	Plan for next steps for practice	Up to 3

Table 2. Guiding questions for meetings

Guiding question for meetings

Meeting 1: Identify potential focal need to address

1. What do we do well as a program to support economic well-being for the families we serve? [Or, if program doesn’t currently address family economic well-being: What are your goals in supporting economic well-being within your home visiting program?]

2. What are the challenges in what we are currently doing to support family economic well-being? [Or, if program doesn’t currently address family economic well-being: What family economic well-being needs should the program address?]

3. What opportunities are there to improve our services to support economic well-being?

4. What economic well-being need(s), challenge(s), or opportunity(ies) for growth is/are highest priority to address?

5. How does the need affect the people involved (home visitors, program staff, families)? Why is it important?

6. What root causes contribute to the need?

7. What are the consequences and effects resulting from this need (for the program and/or participants)? What are the benefits (for families, home visiting staff, the program) if we can overcome the need?

8. Of the root causes and effects that the program could influence, which are most important to address? Which of them are within your home visiting program’s control?

Meeting 2: Brainstorm and prioritize practices to address focal need

1. Reflecting on our last discussion about what the program does well, challenges, and opportunities to improve, how might we address [focal need identified in previous meeting]?

2. What practice(s) can we create or adapt to address the need we chose?

3. How might each practice help families improve their economic well-being? (For example, meet and sustain basic human needs, have present and future financial security, gain control over financial decisions, and/or achieve security and satisfaction in employment)

4. What practices are the highest priority? (Probes: Consider likelihood of success, target priority outcomes for families, practices staff are most motivated to try, etc.)

5. How might we structure the practices (for example, the tool and delivery method) to limit additional burden and stress on home visitors, other program staff, partner staff, and/or participants?

6. How can we use research and evidence (such as home visiting research, behavioral science, or implementation science) to inform the design of these practices? (The HomeEc team will provide information about available research and evidence relevant to the practice.)

Meeting 3: Create implementation and testing plan

1. Who will implement the practice, how often, and with whom?

2. How will the program build its knowledge and capacity to implement the practice well? What tools or resources (such as manuals, supervision, or coaching) are needed to support implementers?

3. What is the plan for testing the practice in this learning cycle? (By learning cycle, we mean the process where staff and/or caregivers try using the practice and provide feedback on it over a short period of time.) When will the test start and end?

4. What data will be collected during the test? How can data collection be integrated into the design of the practice? How will the data be collected? Who will collect them?

5. Can we leverage any existing data collection efforts during the test? If so, how will we incorporate this data into the test?

6. How will we review data collected during the test? How often? Who will be included in the review?

7. How will we share data collected during the test with other important audiences? Who are those audiences?

8. Who will oversee the test and make sure it is executed as planned?

Meetings 4-6: Review findings and discuss next steps

1. What worked well in the implementation of the practice? What was challenging?

2. Did staff implement the practice with fidelity to the implementation plan and training?

3. How did families receive the practice? Did they understand its purpose and how to use it?

4. How well did staff think that training and associated supports prepared them to implement the practice?

5. Did staff have enough support to implement the practice? If not, what other supports would be useful? Were any of the supports provided during the test unnecessary? (If yes, explain.)

6. How could we improve the practice’s implementation going forward? What do staff suggest improving about the practice? How do families suggest improving it?

7. What are our next steps with the practice (for example, tweak its implementation and test it again)?

Paperwork Reduction Act Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0531 and it expires XX/XX/XXXX.