Implementation Planning Guide
Tribal Maternal, Infant, and Early Childhood Home Visiting Program Implementation Plan Guidance
Implementation Planning Guidance for IEG_FINAL 04.04.2023
OMB: 0970-0611
|
|
|
OMB Control No,:
Expiration Date: xx/xx/xxx
Tribal
Maternal, Infant,
and
Early Childhood Home Visiting (MIECHV)
Implementation
Plan Guidance
Implementation
and Expansion Grants
PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: The purpose of this information collection is to provide guidance for Tribal Home Visiting Grantees when they are developing their Implementation Plans. Public reporting burden for this collection of information is estimated to average 1,000 hours per grantee, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This collection of information is required to retain a benefit of Title V of the Social Security Act. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0XXX and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact:
Anne Bergan
Senior Policy Analyst
Tribal Home Visiting Program
Administration for Children and Families, HHS
Mary E. Switzer Building
330 C Street, SW. Suite 3014F
Washington, DC 20201
202-260-8515
INTRODUCTION
This document guides the submission of the Implementation Plan (IP) for Tribal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) Implementation and Expansion (IEG) grantees.
Each grantee will work closely with the Administration for Children and Families (ACF) and Technical Assistance (TA) providers in the development of the IP. There will be TA opportunities to support each grantee with the development of a quality plan, including monthly calls with your Tribal Home Visiting (THV) Federal Project Officer (FPO) and TA providers, webinars, individual TA, the grantee kickoff meeting, site visits, including on-site TA, as well as additional tools and resources on the Implementation Plan Guidance (IPG) Resource Library. (This includes an “IPG Examples Guide,” which provides select examples for some sections in the IPG.) Grantees will receive a link to this library from their FPO and TA providers. In addition, the submission of the plan will be an iterative process with feedback being given incrementally.
The visual on the next page demonstrates the process that will lead to implementing a high-quality tribal home visiting program. The work begins with planning the Community Needs and Readiness Assessment (CNRA) process, moves to the collection and analysis of CNRA data, and prioritizes what has been learned from the CNRA. Next is program design, which will lead to developing the program blueprint, and setting the program up to collect and track data.
Also included in this IPG is a conceptual framework of home visiting implementation quality, intended to offer a way to think about quality in a holistic way and evoke questions for implementation, continuous quality improvement, and evaluation.
As you go through the CNRA process and implementation planning, intentionally consider the promotion of equity in the design, development, and implementation of your home visiting program. You’ll find some resources to support you as you have these conversations, within the IPG Toolkit.
Grantees are expected to respond to every section of the guidance and each element listed under each section. The text boxes included within this guidance expand so that each response can fully address each element. The IPG was developed with the goal that by responding to each section and area, grantees will have developed a comprehensive plan that will outline critical activities required to execute their Tribal MIECHV grant successfully.
The IPG is designed in a way such that each section builds on the previous. Grantees will follow an iterative process and timeline to submit sections of the IP in accordance with submission milestones provided by ACF, and support provided by ACF and TA providers.
Grantees will be permitted to implement program services (including data collection) upon approval of the IP.
What is the Implementation Plan?
The IP is a blueprint for THV program implementation. Grantees should implement their program as laid out in their IP, but to some extent, the plan is a living document. The IP should be aligned with the program budget, and grantees are encouraged to keep Year 2-5 budgets while planning for implementation. Developing the IP is a long and thorough process, and the end product, while a living document, includes detailed program context and information that will serve as a reference to current staff and new staff throughout implementation of the grant.
How often is it updated?
Grantees are encouraged to regularly review and update their IP to ensure it accurately reflects activities throughout the grant. As grantees begin implementation, anticipate adjustments to the plan that reflect what is effective for the program. At a minimum, the IP should be reviewed and updated annually to reflect current practices as part of Non-Competing Continuation (NCC) application grantees submit to receive the next year’s funding. Although the NCC application time provides an opportunity to review and update the plans, grantees are not limited to making adjustments during this time alone. It is not expected that section 1 (the CNRA) will be updated after its submission.
Components of the IP that will require conversations with FPO and approval by ACF before grantees move forward in making changes include communities served; program design, such as the Home Visiting (HV) model, adaptations, enhancements, supplements; caseload/number of families served; and the performance measurement plan. Grantees should not make changes to the selected home visiting model mid-course, but there may be an opportunity to consider additional adaptations, enhancements and supplements as the model is implemented with families.
Table of Contents
A. NEEDS ASSESSMENT METHODOLOGY 11
B. TEAM ROLES AND RESPONSIBILITIES 11
C. COMMUNITY Participation and Coordination 11
D. Other Needs Assessments 12
SECTION 1.1: Organizational Capacity and Readiness 13
A. Organizational Capacity And Readiness for Implementing Tribal Home Visiting 13
B. Organizational and Program Capacity and Readiness Reflection 17
SECTION 1.2: Community Context 17
A. Community Factors Likely to Influence Program Design and Implementation 17
B. Secondary Data 19
C. Primary Data 19
D. Community Context Reflection 20
SECTION 1.3: Community Assets 21
A. Home Visiting and Early Childhood Systems Asset Mapping 21
B. Services and Systems Quality and Capacity Assessment 21
C. Community Asset Mapping Reflection 22
SECTION 1.4: Successes, Challenges, and Lessons Learned 23
SECTION 2.1: Data Analysis and Prioritization 24
A. Data Analysis and Reflection 24
B. Prioritization for Program Design 24
SECTION 2.2: Program Design 26
A. Intended Target Population and Community 26
B. Vision, Goals, and Objectives 26
C. Home Visiting Model 27
D. Adaptations, Enhancements, and Supplements 28
E. Integration of Model, Adaptations, Enhancements, and Supplements 29
SECTION 3: PROGRAM BLUEPRINT 31
SECTION 3.1: Leadership, Governance, and Administration 31
SECTION 3.2: Program and Workforce Management 32
A. ROLES AND RESPONSIBILITIES 32
B. Consultant/Contract Positions 33
C. Recruitment and Hiring of Staff 33
D. New Employee Orientation and Training 34
E. Ongoing Professional Development 34
F. Home Visiting Team Support and Supervision 35
G. IECMHC 36
H. Staff Engagement Retention and Transition Planning 36
SECTION 3.3: Community and Partner Engagement 37
A. Advisory Committee 37
B. Community Partners 37
C. Incorporating Family Voice 38
SECTION 3.4: Recruitment, Enrollment, and Engagement of Families 39
A. Estimated Number of Families to be Served (including the total proposed caseload) 39
B. Family Recruitment and Enrollment 39
C. Home Visiting Program Dosage 40
D. Family Engagement to Ensure Retention and Appropriate Dosage of Services 40
E. Virtual Home Visiting 41
SECTION 3.6: Early Childhood Systems Building 45
SECTION 3.7: Policies and Procedures 46
A. Policy and Procedure Development Plan 46
B. Policy and Procedure Engagement Process 47
SECTION 3.8: Sustainability Planning Readiness 48
SECTION 4: PLAN FOR DATA COLLECTION, MANAGEMENT, AND PERFORMANCE MEASUREMENT 49
SECTION 4.1 : DATA MANAGEMENT PLAN 50
A. PLAN FOR STAFFING AND TRAINING 50
B. PLAN FOR DATA MANAGEMENT AND INFORMATION SYSTEM (MIS) 51
C. PLAN FOR DATA SECURITY 53
A. KEY DEFINITIONS FOR DATA COLLECTION AND REPORTING 53
B. PLAN FOR DSUR DATA COLLECTION AND ENTRY 53
C. PLAN FOR PMR DATA COLLECTION AND ENTRY 57
D. PLAN FOR QPR DATA COLLECTION AND ENTRY 78
SECTION 5: PLAN FOR USING DATA TO INFORM FIDELITY MONTIORING, PROGRAM MANAGEMENT, AND IMPROVEMENT 80
SECTION 5.1: Fidelity Monitoring and Quality Assurance Plan 80
A. Identify data elements that you will use on an ongoing basis for quality assurance 80
B. Approach to engaging staff in quality assurance 81
C. Approach to monitoring model fidelity 82
implementation planning process
a conceptual framework for implementation quality in home visiting
Home
visiting provides individualized services with the intent of
improving outcomes for families with young children. Implementing
evidence-based home visiting programs in alignment with model
fidelity and implementation standards is important for achieving
these expected outcomes.
This framework highlights the ways in which quality implementation at each level of the home visiting system influences and informs implementation at other levels. Broadly speaking, the underlying assumption of this framework is that support for these quality threads across each level of the home visiting system promotes quality program implementation, which in turn leads to high-quality service delivery and intended program outcomes.
The family is at the center of the framework. Next is the home visitor, followed by the local implementing agency and the home visiting program. The community systems and contexts, like other local agencies and services that support families is the next level. Finally, the outer level includes state agencies, tribal organizations, model developers, funders, researchers and the policies and political environments at the national, tribal, or state levels.
The conceptual framework, which was developed through input from many interested parties including tribal home visiting programs, identifies nine potential aspects of quality, that are referred to as “threads” across the levels:
Value and respect for family context, culture, and voice
Commitment to promoting racial equity
Flexibility and responsiveness to emerging needs and circumstances
Understanding home visiting theory and content
Strong fiscal support, compensation structures, and sustainability
Systems that support monitoring, accountability, and continuous quality improvement processes
Qualified, stable, and supported workforce
Adequate resources, tools, and time
Coordination, collaboration, and relationship development
The
conceptual framework1
provides a holistic way of thinking about quality. Each aspect of
these “threads” and asks grantees to think critically
and strategically about them as they conduct the CNRA and then use
the findings to design and plan for home visiting implementation.
GRANTEE INFORMATION
Table 1
Implementing Organization |
|
Program Name |
|
Program Lead |
|
Program Lead’s Contact Information |
|
ABOUT THE CNRA
The goals of the CNRA are to identify program and community strengths and needs and based on this, prioritize goals and strategies to meet needs through home visiting programs. The assessment will give grantees the opportunity to assess the quality and capacity of existing community services to meet the needs of young children and families in the community and develop and sustain partnerships with a comprehensive array of services at the community, tribal, and state level. The needs and readiness assessment must use quantitative and qualitative data to complete the following:
Identify the at-risk tribal community (or communities) in the recipient's target area by collecting data on the health and well-being of individuals and families in these communities, including both strengths/protective factors and risk factors such as: premature births; low birth weight; infant mortality, including infant death due to abuse and neglect or other indicators of at-risk prenatal, maternal, newborn, or child health; poverty; crime; domestic violence; high-school dropout; substance abuse; unemployment; and child maltreatment.
Assess the quality and capacity of any existing maternal, infant, and early childhood home visiting programs in the at-risk community.
Assess the community’s capacity for providing substance abuse treatment and counseling services to individuals and families in need of such treatment or services.
Assess the community’s status and capacity to implement and integrate home visiting services into an early childhood system of support for families, including an assessment of existing or ongoing efforts or resources to develop a coordinated network of supports for expectant families and families with young children at the community level.
The CNRA must involve and engage community members and partners. Through conducting the CNRA, grantees will set the stage for strengthened cooperation and coordination and promote linkages among various programs that serve expectant families, young children, and families in the community. Coordination across programs helps ensure that high-quality, evidence-based home visiting programs are part of a comprehensive, aligned strategy for improving child and family well-being in tribal communities.
Another critical portion of the CNRA is assessing your organizational capacity to implement a home visiting program, both identifying what is already in place, as well as where there might be gaps that will need to be addressed before the organization is ready to successfully implement the program.
CNRA PLANNING
In this section, you will begin the CNRA by first outlining a plan, who from your team will be involved, the types of data collection methods you will use, and how you will engage community partners and stakeholders.
In Table 2, please use a short bullet format to describe the plan for collecting the CNRA data.
Data Collection Method: Enter the approach or method you will use to collect the data (e.g., focus group, survey, or publication). Include how you will obtain the information (e.g., participants, community elders, source of publication).
Type of Data: Enter all applicable types of data, primary (collected firsthand specific for the needs assessment), secondary (someone else already collected the information for a different purpose and will be re-analyzed), and under primary or secondary, whether the data is qualitative (words and narratives), or quantitative (data which can be measured and expressed numerically).
Purpose: Enter the information you hope to gain (e.g., community’s vision about home visiting services and program outcomes, elder perception about community needs).
Timing: Enter the approximate planned date and length of the data collection process (e.g., July-September 2023).
Table 2
Data Collection Method (survey, focus group, name of existing data, etc.) |
Type of data (quantitative, qualitative, secondary, primary) |
Purpose (What information are you hoping to gain?) |
Timing (The approximate time frame/length for collection or analysis) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Include a brief description of team members within your home visiting program, such as staff members, contractors, and other key staff who will play a role in the CNRA process; and identify the specific role they will have in the CNRA process.
Table 3
Staff Member Name and Position |
Intended role |
|
|
|
|
|
|
A CNRA process can include a combination of activities, such as planning; review of other needs assessments; review of existing data; data collection, such as fielding surveys and facilitating focus groups; data analysis; and engaging in conversations with community members and partners. It is important to develop a process and structure that involves the program, organization, and various community groups in guiding all CNRA efforts, including planning, implementing, and evaluating the process. In Table 4, please describe your plans for engaging and coordinating with a diverse set of partners, including articulating your initial plan for the Community Advisory committee.
Individual or group: Enter the name of the organization, group or the name and title of the individual partner who will be involved.
Intended role: Enter the role of the group or individual (e.g., will be part of the core CNRA committee, will assist with the planning of the CNRA, will assist with the review and analysis of the data, will assist with the recruitment of participants, or will assist with writing the CNRA sections).
Primary contact person: Enter the name and contact information for the primary contact or individual.
Table 4
Name of individual or group |
Intended role |
Primary contact person |
|
|
|
|
|
|
|
|
|
|
|
|
Plan for community advisory committee involvement in the CNRA – who will be involved, how often will the committee meet, ways to engage the council and topics for initial meetings: |
||
|
||
Your community or other organizations in your community may have conducted needs assessments that include data relevant to home visiting implementation planning (e.g., Head Start programs conduct needs assessment every five years, or your community may engage in a population health assessment and health plan development). These needs assessments comprise a portion of your secondary data. If applicable, in Table 4, please list:
The title of the existing needs assessment and when the assessment was facilitated. Please add the link to the assessment whenever possible.
Who conducted it
Data points that are relevant to home visiting implementation planning
How you will integrate the results in your CNRA
Table 5
Title and timeframe of the needs assessment (please add link) |
Name of the organization that conducted the needs assessment |
Relevant data collected that applies to home visiting design and implementation |
How will you consider the results, in the assessment process? |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SECTION 1: CNRA
In this section, you will document the results of your CNRA, including describing your organizational capacity and readiness to implement a home visiting program, detailing your community context, articulating the results of your primary and secondary data collection, documenting your community asset mapping process and findings, as well as reflections and findings from your conversations with your team and community partners.
As you work through your plan to engage community partners and begin data collection to understand your community context, consider starting off with thinking through your organizational capacity and readiness to implement a home visiting program. A lot of this information should be available in your grant application, and you can turn to your application as a starting point.
SECTION 1.1: Organizational Capacity and Readiness
Assess the organization’s capacity to implement a high-quality, culturally grounded, evidence-based home visiting program. A lot of this information is available in your grant application
Please describe and assess your readiness and capacity and identify any programmatic concerns that should be addressed before program implementation. Under each area, consider how your organization operates and how this will relate to the functioning of your home visiting program. Areas within your organization to consider include (but are not limited to) the following:
DOCUMENT HOW THESE AREAS WILL, OR WILL NOT,
MEET HOME VISITING IMPLEMENTATION NEEDS
Table 6
Leadership and governance |
||
Reciprocal relationship, guidance, support, and reporting structure between tribal/organizational leadership and your program |
|
|
Program integration within organizational or tribal structures and support systems that can help support and strengthen the program |
|
|
Organizational Management Practices: Human Resource practices |
||
Organizational hiring processes (including timelines) |
|
|
Flexible work schedule practices or other practices that support your workforce |
|
|
Availability of qualified staff, including supervisors, home visitors, evaluation and data staff (to lead data entry, collection, tracking) within the target community |
|
|
Organizational Management Practices: Financial practices |
||
Fiscal management, accounting, budgeting, and fiscal reporting practices |
|
|
Availability of resources (e.g., office space, IT, agency vehicles) |
|
|
Organizational Management Practices: Technology and data system infrastructure |
||
Organizational capacity for data collection, analysis, record retention, and management, including existing data systems to collect participant-level data |
|
|
Availability and use of technology (e.g., cell phones, computers, laptops) |
|
|
Organizational Experience: QA |
||
Capacity to support QA practices (monitoring, data tracking) |
|
|
Organizational Experience: Service delivery |
||
Organizational capacity and experience implementing evidence-based programs or practices |
|
|
Organizational capacity and experience implementing early childhood services |
|
|
Organizational capacity and experience to implement reflective practice and deliver reflective supervision |
|
|
Organizational Experience and Capacity to Provide IECMHC |
||
Explore the IECMHC Toolbox and discuss the extent to which such support systems and qualified staff are available to the organization and community (e.g., in-kind or financial support available) |
|
|
Other |
||
Other |
|
|
B. Organizational and Program Capacity and Readiness Reflection
After collecting the data, gather your team, relevant leadership, and stakeholders and discuss:
The overall story the data are telling
Surprises or unexpected results
Organization’s greatest strengths and challenges
Other things to be considered
Please list the outcomes of this discussion as bullets in Table 7. Include sufficient detail to clearly articulate the point you are making and include applicable concerns that should be addressed prior to program implementation.
Table 7
Discussion results |
|
SECTION 1.2: Community Context
In Table 8, please list your most relevant community characteristics or aspects that will impact your home visiting program's design, implementation, and sustainability. You may add additional factors that are not listed. Add protective factors, strengths, and challenges, as applicable, in each section. You can present this information in multiple forms (narrative, charts, graphs, tables, images, photos, infographics, etc.).
Table 8
Community demographics |
|
Historical, cultural, traditional, tribal laws, ordinances (as appropriate, include community practices and traditional child-rearing practices) |
|
Geographic and environmental factors |
|
Community’s established goals, visions, and priorities related to young children and families |
|
Other relevant items not already described above |
|
In Table 9, please provide community health, well-being, economic, and developmental indicators that will affect your home visiting program's design, implementation, and sustainability. Include:
The specific indicator (e.g., percent of premature births or percent of pregnant women receiving prenatal care).
The data source and year
The applicable population race and ethnicity
The applicable comparison indicator (if able to locate, depending on the level of data available). You might decide to compare to the general population in the area, county, national or state numbers, or other similar tribal communities or counties for specific indicators.
How your community compared to the comparison (e.g., the rate for the AIAN population is 50 percent lower)
You may present data in tables, charts, graphs, or other visuals as needed. Provide narrative explanations when appropriate.
The following are required indicators. Please add other relevant indicators (e.g., rate of teen births, rate of well-baby checks, number of children ages 0-5, cultural and traditional practices) as desired to describe your community context.
Premature births
Low birth weight
Infant mortality
Other prenatal, maternal, newborn, or child health and mental health conditions
Child maltreatment
Poverty and use of public assistance
Unemployment and underemployment
Crime, including juvenile delinquency and incarceration
Domestic or intimate partner violence
High school dropout and graduation rates
Substance use /abuse, including alcohol, tobacco, prescription drugs, illicit drugs
Table 9
Secondary data
|
In Table 10, list the summarized results of the primary data you collected for your community. In column one, list the method of data collection. In column two, summarize the main and most important results. You can present this information in multiple forms (narrative, charts, graphs, etc.). Please provide the raw data in an appendix. Add protective factors, strengths, and challenges, as applicable, in each section.
Table 10
Method of data collection (focus group, survey, etc.) and overview of sample |
Summarized Results (Most important implications and discoveries for home visiting program design and implementation you discovered through each of the primary data collection methodologies) |
|
|
|
|
|
|
|
|
After collecting the data, gather your team and relevant stakeholders (e.g., community members, elders, partners, families) and discuss. (The IPG Toolkit has ideas on how to conduct this discussion):
Results of primary and data secondary collection (including results noted in Tables 6 and 7)
Strengths, challenges, and opportunities
The overall story the data is telling
Surprises or unexpected results
What is important to address in program design and implementation planning
Other things to be considered
List the outcomes of this discussion as bullets in Table 11. Include sufficient detail to clearly articulate the points you are making. Add percentages, rates, and comparison statements where needed, and include applicable concerns that should be addressed before program implementation. You can add detailed notes in the Appendix.
Table 11
Discussion results |
|
SECTION 1.3: Community Assets
This section will help identify the breadth of early childhood, behavioral health, domestic violence, and other community services, resources, and programs that currently exist to support prenatal families and families with children up to kindergarten entry.
The community asset mapping exercise is essential to understanding what formal and informal resources exist to support families that you will enroll in your home visiting program. It provides an opportunity to identify how resources are connected to your home visiting program (and families being served) and where gaps may exist. It also illustrates the relationships between organizations that can be used in your project’s design and development.
Multiple perspectives and insights are important when creating a community asset map. Consider inviting a broad selection of early childhood systems partners, elders, family members, Tribal Council members, and CNRA planning team members to the discussion. Such participatory mapping creates a tangible visual display of the people, places, and experiences that make up your tribal community.
The community asset mapping process:
Provides a framework for discussing the location of resources,
Highlights resources of importance,
Helps analyze current programmatic and family access to resources,
Raises awareness of existing or gaps in resources, and
Creates a visual representation of existing and potential resources.
Instructions and tools for completing the mapping exercise are included in the IPG toolkit. Please insert a copy or picture of your asset map in Table 12 and include summarized discussion points in Table 13.
Table 12
Insert the community asset map |
|
Table 13
In Table 10, please insert summarized discussion points (insert other relevant planning or discussion outcome documents in the appendix as needed) |
|
Assess the quality and service capacity of existing programs or initiatives for maternal, infant, and early childhood home visiting and behavioral health services, including but not limited to substance abuse treatment and mental health services, in the tribal community.
Describe the capacity of home visiting services and systems in the community:
The home visiting programs or initiatives that currently serve your tribal community or communities, including the existing home visiting program the organization is currently operating
The funding sources for these programs and who administers them
The extent to which are the services high-quality and meet the needs of individuals and families who are eligible for home visiting services (e.g., are accessible and culturally relevant)
How many families currently receive services through these programs
The characteristics of individuals and families who are receiving services
The existing mechanisms for screening, identifying, and referring families and children to home visiting programs serving the at-risk tribal community (e.g., coordinated intake procedures)
The referral resources currently available to support families enrolled in home visiting programs residing in the tribal community
Table 14
Capacity of home visiting services and systems |
|
Describe the capacity for behavioral health services, including substance abuse treatment and mental health services, to individuals and families in need of treatment or services and who are eligible for home visiting programs:
List the existing investments in providing behavioral health services through various funding streams
The numbers and characteristics of individuals and families who are receiving these services in the community
The extent to which are the services high-quality and meet the needs of individuals and families who are eligible for home visiting services (e.g., are accessible and culturally relevant)
The factors limiting additional investment and capacity for providing needed services to individuals and families who are eligible for home visiting programs
Table 15
Capacity for behavioral health services |
|
After facilitating the community asset mapping activity, gather your CNRA planning team to:
Review and add relevant details and changes to the asset map, and
Discuss the discoveries you made during the community asset mapping process that have implications for your home visiting program.
What is the overall story?
What were the surprises or what was unexpected?
In this map or visual, where are the strengths and opportunities?
In this map or visual, where are the weaknesses and barriers?
How do the programs relate, enhance, support, or impede the home visiting program?
What else should be considered?
Please list the outcomes of this discussion as bullets in Table 16 (Include sufficient detail to clearly articulate your point and include applicable concerns that should be addressed before program implementation.)
Table 16
Discussion results |
|
SECTION 1.4: Successes, Challenges, and Lessons Learned
In Table 17, list the successes, challenges, and lessons learned and experienced related to conducting the CNRA. Reflect on what can apply to future needs assessments.
Table 17
CNRA Process Reflection |
|
Successes of your CNRA process |
|
What changed from your original plan for conducting the CNRA?
|
|
Lessons learned and how these might be relevant to future needs and readiness assessments |
|
SECTION 2: PROGRAM DESIGN
In this section, you will design your home visiting program, responding to the community strengths and needs identified in Section 1. You will prioritize the data analyzed to make decisions around target population for services, select a home visiting model to meet the needs identified, determine if any adaptations and enhancements are necessary, set program goals and objectives, and create a logic model to visually describe the program.
SECTION 2.1: Data Analysis and Prioritization
In the table below, summarize the important discussion results from CNRA Sections 1.1, 1.2, and 1.3 and identify some key takeaways by addressing the following questions.
What are the most pressing needs of your community, specifically families with children ages 0-5?
What strengths does your community bring to home visiting program implementation? What are your community’s most significant opportunities for success?
What are internal program and organizational capacities, resources and infrastructure that can help with successful implementation of home visiting?
Where are there gaps in resources and infrastructure and limitations in the organization that will need to be considered in program design, budgeting, and implementation?
Table 18 |
|
In this step, convene your CNRA and program team, relevant colleagues and organizational leadership, community partners, and community advisory council, and engage them to help prioritize the listed items above to help make decisions about your vision, goals, objectives, home visiting program design, and implementation activities.
The IPG toolkit has several frameworks, approaches, and methods to help you have these conversations as well as synthesize, organize, and prioritize data to help you make decisions for program design. These approaches include the SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis, SOAR (Strengths, Opportunities, Aspirations, Results) analysis, and others. Your PATH TA specialist can direct you to these tools and assist in facilitating these exercises with your team and partners, as needed.
What was narrowed down or prioritized? (This will help you make decisions about your vision, goals, objectives, program design, and implementation activities.) Please list these in the table below. Include results or visuals of your facilitated approach or discussion below, or in the Appendix.
Table 19 |
|
SECTION 2.2: Program Design
For
this section, you will need to connect with various home visiting
models in selecting one and then ensuring the information below is
thorough.
The
HomVEE website has information on evidence-based models:
https://homvee.acf.hhs.gov/
The
IPG Toolkit has several tools and resources to help with model
selection.
Based on your analysis, please define the target population (e.g., ages) and community the program will serve (e.g., county, reservation) and enter the information into Table 20. If you are modifying the target population or expanding it based on your current program, please provide additional details to explain these adjustments.
Table 20
Target population and target community |
|
Define your proposed program's vision, goals, and objectives. The vision, goals, and objectives should then drive the design of the program.
Analysis
Program
Design
Vision:
In Table 21, please insert your program vision. A vision is a one-sentence change-inspiring statement describing the clear and inspirational long-term desired future state resulting from your program’s work. This vision should be a picture of the future, about how the home visiting program will change the community.
Table 21
Program Vision |
|
Goals:
In Table 22, please insert your program goals. The number should not exceed 3-5 goals. Goals are statements that explain what you want to achieve with the program. They are the intended specific results of your program, should it be effective.
Objectives:
In Table 22, please insert your program objectives that align with each goal. The number should not exceed more than 2-3 objectives per project goal. Objectives are strategies of implementation that outline the “who, what, when, where, and how” of reaching the goals. Unlike goals, objectives are Specific, Measurable, Achievable, Results-oriented, and Time-bound (SMART).
Table 22
Goal |
Objectives |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note:
Connect with the model to ensure responses below are thorough.
Based on your data analysis and prioritization, please describe the home visiting model selected for implementation.
Table 23
Basic description of the home visiting model |
|
The model’s theory of change |
|
A description of how the model, including what is known about the model’s evidence of effectiveness, will meet the program’s vision, goals, and objectives listed above. |
|
Anticipated challenges related to model implementation and how they will be addressed |
|
The home visiting model may not be able to address all of the needs identified and fully meet the program’s visions, goals, and objectives. In this case, you may consider adaptations, enhancements, and/or supplements to the evidence-based home visiting model selected. If so, please describe them below. If creating more than one adaptation, enhancement, or supplement, make sure to address all of the information requested below. (Definitions for adaptations, enhancements, and supplements are in the appendix.)
Some supplements may already exist and be implemented in other settings. In this case, you would implement an “existing” supplement, and may not create one specific to your program. Connect with your FPO and TA provider for any questions or clarifications needed.
In Table 24, please describe your proposed adaptations, proposed enhancements, and proposed supplements).
Table 24
Enhancements, Adaptations, or Supplements |
Select One: Existing ☐ Program will create ☐ Select One: Adaptation ☐ Enhancement ☐ Supplement ☐ |
A general description and overview of the component and its intended purpose (e.g., how it will meet the program’s identified vision, goals, and objectives as listed above). |
|
FOR EXISTING COMPONENTS Based on the CNRA discoveries, list any planned changes to the existing component(s) and list any anticipated TA and supports needed (enter N/A if there are no planned changes). |
|
FOR NEW COMPONENTS
Strategies and steps for creating the new program component |
|
|
|
Partners or staff involved in the development |
Partner or staff role |
|
|
|
|
FOR ALL COMPONENTS Anticipated challenges related to the implementation of the component and how they will be addressed |
|
|
|
Note:
Connect with
the home visiting model to ensure your description is thorough and
the model approves of the proposed additional components.
Table 25
Description for how the model and proposed components will work in concert to collectively meet the proposed vision, goals, and objectives |
|
SECTION 2.3: Logic Model
The logic model helps design results-based programs. It is a great tool for describing your program to partners and supporting collaboration. It is a tool that can and should be adapted and utilized throughout the life of the home visiting project.
Logic models are effective tools to assist in program planning, implementation, management, evaluation,
and reporting. They help define a program’s intended impact and goals; the sequence of intended effects;
which activities are to produce which effect; and where to focus outcome and process evaluations. (W.K. Kellogg Foundation, 2014).
The purpose of logic models is to help conceptualize your change effort. It does this through articulating the understanding of the current situation, the changes you hope to achieve through the home visiting program effort, with and/for whom, the activities that will contribute toward this change, the resources needed to put into the effort, assumptions you are making, and external factors that could influence results.
Logic model templates and resources are available in the IPG Toolkit.
Below, insert an image/screenshot of the Logic Model for the home visiting program. Please insert a full representation of the logic model below in the appendix. Minimum components of the Logic Model include vision, goals, program design, resources, inputs, activities, outputs, and outcomes.
Table 26
Logic Model |
|
SECTION 3: PROGRAM BLUEPRINT
This section articulates your action plan for effectively implementing your vision, goals, objectives, and program design as laid out in Section 2. Section 3 includes:
Leadership, Governance and Administration
Program and Workforce Management
Community and Partner Engagement
Recruitment, Enrollment and Engagement of Families
Dissemination
Early Childhood Systems Building
Policies and Procedures
Sustainability Planning Readiness
The results documented in Section 1 (CNRA) and your Program Design in Section 2 will assist you in developing an action plan and blueprint for successfully implementing your program in years 2-5 of your grant.
SECTION 3.1: Leadership, Governance, and Administration
Organizational Leadership, Governance, and Administration
In the table below, list how the organization or tribe will manage, guide, and support the home visiting program’s implementation. Describe roles and responsibilities below, including how each of these functions are carried out, who is responsible, and major process steps.
Table 27
Describe who is responsible for oversight of successful implementation within the lead agency, including specifics about the level and type of support provided and details about how the implementation is monitored. |
|
Describe who is responsible for and the process for fiscal oversight for the program within the lead agency (e.g., who develops and approves the budget, who approves/authorizes expenditures, who manages the Payment Management System (PMS) account, who draws down money, and who does the fiscal reports (i.e., SF-425). |
|
Describe the monitoring activities and communication between the program and fiscal staff and who participates (include frequency). |
|
SECTION 3.2: Program and Workforce Management
In the table below, list all positions for your home visiting program, which includes administrative, supervisory, data, home visiting, and other relevant roles and positions. This can include staff positions as well as those contracted out. Indicate what type of role they fill (some positions may have multiple role categories). Provide the job descriptions and resumes in the appendix.
Note:
Connect with the home visiting
model to ensure the descriptions below are thorough
and align
with model guidelines around staffing.
Organizational chart: |
|
Table 28
Position Title |
Roles and responsibilities |
Type
of Role |
|
|
☐ Supervisory ☐ Administrative ☐ Home Visiting ☐ Data ☐ Other __________________ |
|
|
☐ Supervisory ☐ Administrative ☐ Home Visiting ☐ Data ☐ Other __________________ |
|
|
☐ Supervisory ☐ Administrative ☐ Home Visiting ☐ Data ☐ Other __________________ |
|
|
☐ Supervisory ☐ Administrative ☐ Home Visiting ☐ Data ☐ Other __________________ |
|
|
☐ Supervisory ☐ Administrative ☐ Home Visiting ☐ Data ☐ Other __________________ |
Below, list the positions that are contracted, and describe your plan for oversight and contract management. List the title and role of the contracting organization, the frequency of oversight, and who is responsible for overseeing the contract.
Table 29
Contracted position, if applicable, the name of the organization and role |
Frequency and process of oversight |
Position responsible for oversight |
|
|
|
|
|
|
|
|
|
Describe how you will assure staff roles and responsibilities are well planned and crafted, match with job descriptions and align with the interview process, including but not limited to interview panels and questions.
Table 30 |
|
Insert a process map of the recruitment and hiring process for the program. Include who is responsible, the overall timeframe for hiring (from start to finish), and the duration for each step.
Table 31
Recruitment and hiring process map, including timing and the length of activities |
|
Note:
Connect with
the home visiting model to ensure your descriptions for model
trainings are accurate and thorough.
The Implementation Plan, or certain portions of the plan (dependent on staff roles), will be important to share with your staff as part of orientation. Please include when and how will you incorporate during orientation, onboarding and training.
Table 32
Orientation/training topic and format |
Staff and applicable contracted positions receiving training
|
Timing (e.g., how many weeks after hire) |
Person or entity responsible for providing the orientation |
Organizational orientation and training activities |
|||
|
|
|
|
|
|
|
|
Home visiting program orientation and training activities |
|||
|
|
|
|
|
|
|
|
Model, adaptation, enhancement, and supplement training |
|||
|
|
|
|
|
|
|
|
Other |
|||
|
|
|
|
|
|
|
|
Note:
Remember to include relevant cost in budget years 2-5
Table 33
Topic or opportunity for ongoing, core, or mandated professional development needs |
Staff or contracted positions involved |
Frequency of occurrence |
|
||
|
|
|
|||
|
|
|
|||
|
|
|
|||
In the table below, please describe your plan for regularly assessing other professional development needs that help increase home visiting competencies for program staff (e.g., an annual competency survey, performance evaluations, reflective team discussion), including how often this will occur.
Table 34
Assessing professional development needs |
|
Note:
Connect with
the home visiting model to ensure your descriptions for supervision
are accurate and are in line with model guidelines.
Below, please list your plan for reflective supervision, administrative supervision, and clinical supervision. Include positions who participate, positions responsible for providing the applicable supervision, the frequency, and the format. Note that in addition to home visitors, it is recommended that supervisors are well supported as well.
Table 35
Positions that receive the relevant supervision
|
Position responsible for providing the supervision
|
Frequency
|
Format (e.g., individual, group) |
|
Administrative supervision |
||||
|
|
|
|
|
|
|
|
|
|
Reflective supervision |
|
|||
|
|
|
|
|
|
|
|
|
|
Clinical supervision |
|
|||
|
|
|
|
|
|
|
|
|
|
Other supports and support structures and other reflective practices |
|
|||
|
||||
In Section 1 under the organizational capacity and readiness, you assessed the organizational capacity for providing IECMHC. In the table below, list your next steps for building, enhancing, or maintaining this capacity in years 2-5.
Table 36
IECMH next steps |
|
|
|
Below, list your plan for engaging and retaining staff positions (i.e., what steps your program will take to avoid unnecessary departures). In the second part of the table, include your plan for transitioning each staff position in the event of a planned or unplanned leave or departure.
Table 37
Plan for engaging and retaining staff |
|
|
|
Position title (e.g., home visitor, coordinator) |
Transition plan for a planned or unplanned leave or departure (note: transition planning for families is addressed in the family engagement section) |
|
|
|
|
SECTION 3.3: Community and Partner Engagement
Below, describe the plan for engaging your advisory committee to support planning and oversight of the program. In the appendix, include a current membership list (organizations/roles that are involved, e.g., Director of Head Start program, parent representative) if there is a committee that already exists that will serve as the home visiting program’s advisory committee, and include a proposed list for a committee that will be formed.
Table 38 Discuss the selection and duration of the membership |
|
|
|
The role of the Advisory Committee |
|
|
|
Frequency of meetings |
|
|
|
Describe the plan for engaging elders, community members, partners, and other interested parties. Collaboration, including developing partnerships in your community, is an integral part of delivering an effective home visiting program. Effective programs enhance the efficiency and effectiveness of their own efforts by developing partnerships with other agencies. Such efforts promote the sharing of ideas, resources, tasks, and even staff members.
Before completing the table below, review the community asset mapping process results documented in Section 1.2 and consider the assessment and reflection results in your plan.
Table 39
Community partner
|
Intended role and/or the level of engagement (e.g., inform, consult, involve, collaborate2)
|
Characteristic of the relationship (e.g., formal memorandum of understanding, other agreement, informal relationship) |
|
||
|
|
|
|||
|
|
|
|||
Consider how to engage families as leaders and incorporate their perspective and voice as you implement services. Please articulate a plan below, including how often you plan to engage them and what this process will look like during implementation.
Table 40
|
|
SECTION 3.4: Recruitment, Enrollment, and Engagement of Families
Note:
Connect with
the home visiting model to ensure caseload numbers, information
about dosage, and policies around recruitment and enrollment are
accurate and in line with model guidelines.
In the below table, enter the annual number of caseload slots (i.e., the number of families your program can serve at any given time) the program anticipates serving in years two through five. Please document the factors you considered in the decision (e.g., number of staff, supervisor taking a caseload, full time or part-time staff, amount of travel, new vs. seasoned staff, risk factors of the families, anticipated turnover of staff).
Table 41
Estimated number of families to be served |
|
|
Year 2 (Note when you anticipate starting to serve families within Year 2.) |
|
|
Years 3-5 |
|
|
Considerations and how the estimated number of caseload slots was calculated |
|
|
|
|
|
Insert a process map that describes the plan for recruiting and enrolling families. Include the person responsible and the timing and length of each process step.
Table 42
Family recruitment and enrollment process map |
|
Note:
You will also document this
information in a slightly different format later in the IP related
to performance measures and fidelity monitoring (sections 4 and 5).
Table 43
Frequency and duration of services – how often are home visits provided, how long is each home visit?
|
|
What constitutes service completion? When do families complete or graduate from the program?
|
|
Below, describe the plan for engaging families. Engagement is the collaboration between the parent and home visitor to achieve family-oriented goals to improve the health and well-being of their family. It occurs when there is an ongoing, reciprocal, strength-based partnership between the family and the home visitor, and a consistent application of parental learning.
Table 44
Plan for building effective relationships with families |
|
Engagement strategies during the recruitment period |
|
Engagement strategies during the enrollment period (after families agree to participate in the program) |
|
Once enrolled, activities that ensure engagement in the home visits, curricula, learning, application of knowledge, and other program activities |
|
Engagement activities when a home visitor transition occurs |
|
Plans for maintaining or developing an incentive program (note: grantees are not required to have an incentive program). |
|
A virtual home visit is a visit conducted solely by use of electronic information and telecommunications technologies. Virtual home visits help extend the reach of home visiting, prevent service delivery disruptions due to illness or weather, and provide flexibility in scheduling for families and home visitors. The COVID-19 pandemic introduced and accelerated the use of virtual home visiting. Virtual home visiting has allowed families to maintain connections to their home visitors and home visiting programs in unprecedented and emergency circumstances.
While virtual home visiting will continue to remain an option for Tribal Home Visiting, per legislation, grantees will be expected to prioritize in-person home visits and ensure that at least one in-person home visit is conducted annually for each participant in the program, except if a public health emergency is declared by Federal, State, or Tribal law.
Note:
Connect with the model to discuss virtual home visits in relation
to model fidelity.
Table 45
Description of Factor to Consider to Determine Appropriateness of Virtual Home Visits:
|
How the Factor Will be Considered to Decide on Modality of Home Visit (Virtual or In-person)
|
|
|
|
|
Provide a brief description of your chosen model’s guidelines for virtual home visits to ensure model fidelity.
|
Describe a plan for encouraging in-person home visits with families. This could include internal communication, policies and procedures with home visiting staff and communication, as well as setting expectations between staff and families on when virtual visits may be used. Consider the information you include in the tables above when articulating this plan.
|
SECTION 3.5: Dissemination
Dissemination is an intentional process to communicate information relevant to program implementation, performance measures, program outcomes, policies, practice, and research from your home visiting program to well-defined and multiple early childhood and tribal community audiences for a particular purpose. It is the delivery and receipt of intentional information and data to a target audience and the active engagement and subsequent action of that target audience.
The Tribal Home Visiting Dissemination Toolkit houses a variety of dissemination resources that can support your dissemination efforts. It provides guidance, fact sheets, worksheets, planning step descriptions, and examples intended to support your planning and development process. Your PATH TA specialist can share this toolkit with you.
Below, list your overall dissemination goal(s) and your plan for reviewing the effectiveness of the dissemination plan.
Table 45
Your goal(s) for dissemination |
|
How often and with whom will you review the effectiveness of your dissemination activities? What indicators will you track? |
|
In the table below, list your strategies for dissemination. Add major milestones and activities to your project timeline. Please note that you were linked to this table throughout the document. Review the previous entries and complete the table as applicable. Include dissemination activities for program outcomes and evaluation results described in Section 4 and Section 6.
Table 46
Audience
|
Types of information needed by audience (e.g., outcome data, eligibility information, families served, success stories) |
Dissemination products |
Dissemination plan for the products
(who, through what channel, and when) |
Referral partners
|
|
|
|
Leadership and administration |
|
|
|
Families |
|
|
|
Sustainability partners |
|
|
|
Broader community |
|
|
|
Advisory Committee |
|
|
|
Peers |
|
|
|
Other |
|
|
|
SECTION 3.6: Early Childhood Systems Building
In Section 1.2, you completed a community asset map and discussed the results with a community stakeholder group. Below, develop two to four next steps for building and/or strengthening the target community’s early childhood system.
Table 47
What is the long-term vision for your early childhood system? |
|
Concrete next 2-4 steps for strengthening the current system (who, what, when and how) |
|
|
|
|
SECTION 3.7: Policies and Procedures
In the table below, describe your plan for developing a programmatic policy and procedure manual. Check the applicable policies and procedures that (c) need to be developed, or (d) already exist on an organizational level. You will be submitting your policies and procedures to ACF for review starting in year 3. (Note: ACF will review but will not be approving your policies and procedures).
Table 48
As applicable with your planning team, please identify which topic area needs a policy and/or a procedure. Then, using the chart, mark if the policy/procedure already exists at the program level, needs to be developed, needs to be revised, or already exists at an organizational level. (Note: policies and procedures related to data and fidelity monitoring are included in sections 4 and 5.) |
Exists on a program level |
Revise or adapt |
Develop |
Exists at tribal /organizational level |
Leadership, Governance and Administration |
||||
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
Program and Workforce Management |
||||
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
|
|
|
|
Community and Partner Engagement |
||||
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
Recruitment, Enrollment, and Engagement of Families |
||||
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
|
☐ |
☐ |
☐ |
☐ |
Dissemination |
||||
|
☐ |
☐ |
☐ |
☐ |
Early Childhood System Building |
||||
|
☐ |
☐ |
☐ |
☐ |
Promoting Sustainability |
||||
|
☐ |
☐ |
☐ |
☐ |
Other |
||||
|
☐ |
☐ |
☐ |
☐ |
Describe your approach to the policy and procedure manual development and review process. Describe how you plan to engage partners and how you will assure that the team regularly reviews and updates the policies and procedures.
Table 49
Plan for engaging program staff, partners, leadership, advisory committee, and service recipients in the development |
|
Plan for disseminating the program manual (e.g., published digitally on the organization share drive and link emailed to current and future staff) |
|
Plan for regular utilization and review |
|
SECTION 3.8: Sustainability Planning Readiness
In Section 1.3, you began looking at what opportunities might be important to consider for sustainability planning. During years 2-5, it will be important to continue working on the sustainability of your program. Consider what you have learned during the assessment process. Below, describe your vision for sustainability and the next key steps to increasing the program’s readiness for sustainability planning.
Table 50
Long term vision for sustaining the program |
|
Concrete next 2-4 steps for to increase sustainability planning readiness |
|
|
|
|
Please link to the Dissemination Table and describe your plan for disseminating to your sustainability audience. |
|
SECTION 4: PLAN FOR DATA COLLECTION, MANAGEMENT, AND PERFORMANCE MEASUREMENT
Introduction
to data reporting requirements for Tribal Home Visiting
Data collection and management is an essential building block for ensuring that your program is providing quality home visiting services. All Tribal Home Visiting Grantees are required to collect and report demographic, implementation, and performance data on a quarterly and annual basis. This section of your Implementation Plan will help you thoughtfully plan for successful data collection and management, to ultimately support your ability to use data to improve service delivery and submit required Tribal Home Visiting reports.
Tribal Home Visiting grantees collect data on a regular basis to be reported in three reports described below. All reports are submitted into the Tribal Home Visiting Reporting System (THVRS) to be reviewed and approved by ACF. Your TEI liaison will support you in preparing for, submitting, and finalizing your data reports. Visit the THV Reporting Resource for more information about the reporting process and timeline.
Demographic and Service Utilization Data Report (DSUR)
The DSUR is used to annually report demographic and service utilization information. This includes data such as numbers of newly enrolled and continuing participants, educational level and poverty status of participants, gender and ethnicity of staff, and the number of home visits. This type of data helps grantees, ACF and TA providers understand whom the program serves, who is staffing the program, and how many families are being served.
Performance Measurement Data Report (PMR)
The PMR is used to annually report performance measurement data. This includes data for 12 “core” measures and three “flex” measures across six benchmark areas. Grantees select their “flex” measures from a list of 11 potential measures. Both the “core” and “flex” performance measures are standardized. Performance data help grantees, ACF, and TA providers understand program improvement, strengths, challenges, and TA needs.
Quarterly Performance Data Report (QPR)
The QPR is used to report information, quarterly, program capacity, place-based services, family engagement, staff recruitment and retention, and staff vacancies. Quarterly program data helps grantees and ACF track program implementation.
There are major areas to plan for when collecting and reporting data, they are:
In this section you will describe how your team will collect, monitor, manage, and report all required Tribal Home Visiting data, and specify your plan for each of these major areas.
In addition, the following symbols in this section provide supplemental information to help you along the way:
Toolkits: TEI resources to support data collection, management, and reporting decision making
Attention: important points to remember
SECTION 4.1 : DATA MANAGEMENT PLAN
In Table 52, please describe who will support your data collection, entry, management, and reporting activities. Ensure staff identified in this table align with those included in Section 3.2. The table includes the following columns:
Staff role – Identify individual staff roles on your home visiting team with data-related responsibilities (e.g., home visitor, evaluator). If more than one role has responsibilities related to the same activity, create multiple rows (one for each staff role).
Data responsibilities – Specify the data-related responsibilities for each staff role. Be specific. For example, “Collects intake forms from families during enrollment; Enters family data into the data system.” Use bullet points to list multiple responsibilities for each role. Be sure to include responsibilities related to annual and quarterly reporting. Include frequency of the responsibility. For example, “Completes data quality checks each quarter; Supports HVs each week with intake.”
Training plan – Explain how the staff member will be trained to complete their data responsibilities. Please include details such as who will provide the training, how often the training will occur, etc.
Table 52
Staff role |
Data responsibilities |
Training plan |
Example: Program Coordinator |
|
Needs to be trained on completing data quality checks and pulling reports from data system. Will receive training from evaluator by January 1, 2023. |
|
|
|
|
|
|
|
|
|
In Table 53, please describe what your program will have in place to support continued data-related activities during staff transitions and turnover (e.g., policies, documentation). Ensure this aligns with Section 3.2, Part H.
Table 53
Strategies for sustaining data activities during staff transitions |
|
Please select the option that best reflects your plan for obtaining a data management and information system (MIS) for your Tribal Home Visiting program and complete the tables that follow to describe how you will obtain, build, and/or improve that system(s).
☐ |
Our organization has an existing MIS that the Tribal Home Visiting program will use to store, manage, and report data. |
☐ |
Our organization plans to build a MIS for the Tribal Home Visiting program or use a commercial off-the-shelf (COTS) system not owned by the model developer. |
☐ |
Our Tribal Home Visiting program plans to use the model developer data system. |
In Table 54, please describe your plan to build a data system or obtain a commercial off-the-shelf system. Include the vendor you will work with and timeline. If you plan to use a model developer data system, please skip to Table 55.
Table 54
Plan for building a MIS or obtaining a COTS system |
|
In Table 51, please describe the MIS your program will use.
Name of data system — Identify the name of the system supported by the model developer.
Contractor/Vendor (if applicable) — Provide the name of the company or individual supporting your data system development and ongoing implementation.
Costs for system use – Identify all costs associated with using this MIS. Include regular access costs and costs associated with training, support, and accessing reports.
Process for accessing reports – Describe how you will work with the system and/or vendor to pull your data from the system.
Process for accessing MIS support – Describe the process for obtaining technical support for the MIS.
Process for training staff – Describe what services the model provides to support staff training to use the MIS. Consider any relevant trainings described in Section 3.2.
Process for making changes to the MIS - Describe how changes are made to the system (e.g., is there a period where system locks and all changes are made? Or do changes occur on an ongoing basis?).
Table 55
Name of data system |
|
Contractor/Vendor (if applicable) |
|
Costs for system use (e.g., monthly, annual, extra) |
|
Process for accessing reports |
|
Process for accessing MIS support |
|
Process for training staff (e.g., online vendor-led training) |
|
Process for making changes to the MIS |
|
(If applicable) In Table 56, please identify goals for improving your existing data system to align with THV program needs.
Table 56
Goals for improving the existing data system |
|
(If applicable) If you’re using more than one MIS, please describe in Table 57 how the systems will be used together for data management and reporting. Please include how the systems will differ in terms of their purpose, what data they will store, who will have access, and how reports will be created.
Table 57
Plan for using multiple data systems |
|
In Table 58, please describe how staff will interact with the system. Ensure this table aligns with staffing plans described in Section 3.2.
Staff role — Identify each staff role that will access the data system. Examples include Home Visitor, Program Coordinator, Evaluator, Program Director
Level of access — If your program grants different levels of access based on program role, describe those here. Examples include Full Administrator Access, Read Only Access, Data Entry Access, etc.
Data system responsibilities — Identify the activities that each role is responsible for related to the data system. Examples include making system changes, overseeing system changes, data entry, etc.
Table 58
Staff role |
Level of access |
Data system responsibilities |
Example: Program Coordinator |
Full Administrator Access |
Data entry quality checks; oversight of data entry; communication with system vendor; report generation |
|
|
|
|
|
|
In Table 59, please describe your plan for data security, including privacy of data, administration procedures that do not place individuals at risk of harm (e.g., questions related to domestic violence and child maltreatment reporting), and compliance with applicable regulations related to Tribal oversight and approval of strategies for protection of human subjects, data safety and monitoring, and compliance with applicable regulations, other Institutional Review Board/human subject protections, Health Insurance Portability and Accountability Act (HIPAA), and Family Educational Rights and Privacy Act (FERPA).
Table 59
Plan for data security |
|
SECTION 4.2: PLAN FOR THE REPORTING OF REQUIRED DEMOGRAPHIC, SERVICE UTILIZATION, IMPLEMENTATION, AND PERFORMANCE MEASUREMENT DATA
Accurate and reliable data reporting within and across Tribal Home Visiting grantee programs requires the development and consistent use of clear definitions for key home visiting concepts. In this section you will provide your definition of key terms. Some of these terms may have been defined in other sections of your Implementation Plan. Please refer to those sections and ensure that definitions are consistent.
In Table 60, provide definitions for home visiting concepts that are relevant to the DSUR, the PMR, and the QPR. These are concepts where grantees have flexibility in the definition. There are many other key terms and concepts that ACF has defined for consistency across grantee teams. Please refer to the Key Definitions Guide for the complete list of defined terms and associated ACF guidance.
Table 60
Home visiting concept |
Grantee definition |
Home visit |
|
Inactive status |
|
Active status |
|
Hold status |
|
Enrolled |
|
Completion/graduation |
|
Overview of the Demographic and Service Utilization Data Report
Demographic and Service Utilization Data should be collected at enrollment (as defined by grantee or per model developer guidelines) for newly enrolled participants/households and once during the reporting period (as determined by grantee) for continuing participants/households.
Grantees may determine the method of and individuals responsible for data collection based on their own policies and procedures, and as guided by model developers and in consultation with ACF. ACF will use this data to better understand the population receiving services from Tribal MIECHV grantees and the degree to which they are using services. All data submitted to ACF through this report is aggregated across participating families and de-identified. Technical assistance is available to grantees to determine the method and timing of data collection and to ensure high quality data collection and reporting.
Grantees submit the DSUR on an annual basis. The draft submission of this report is due by October 31st. The grantee then works with their TEI liaison and FPO to revise and finalize their report, and the final submission is due on December 31st.
The DSUR includes 26 Tables of demographic indicators for adult participants (newly enrolled and continuing), index children (newly enrolled and continuing), households (newly enrolled and continuing), home visits, and staff.
The DSUR is organized into two sections.
Section A includes Participant Demographics and Service Utilization and contains three sub-sections:
A.1: Participant Demographics during Reporting Period (Newly Enrolled and Continuing)
A.2: Participant Demographics during Reporting Period (Newly Enrolled Only)
A.3: Participant Service Utilization during Reporting Period (Newly Enrolled and Continuing).
Each section requests demographic and service utilization data for program participants and households.
When reporting on the DSUR, grantees may decide to report on data through the following data sources:
Program data: Data collected by the home visiting program. This may include data self-reported by the adult participant.
Administrative data: Data collected by another agency, organization, or program. Grantees may elect to use administrative data if it applies directly to the item being reported on and can be acquired within reporting deadlines.
Please note that the data sources selected should be consistent over time for each data element.
Section B includes Program Staff Demographics and requests information on demographics of program staff.
Individuals and families reported on the DSUR must be served by a trained home visitor who receives 25% or more of his/her personnel costs (salary/wages including benefits) paid for with Tribal MIECHV funding.
After each sub-section, the form includes a "Notes" field. Grantees should aim to submit reports that reflect the highest level of data quality possible. However, there is often unavoidable missing data due to challenges connecting with families, staff turnover, etc. Grantees should use this field to explain the reasons for missing data if more than 10 percent of data are missing (i.e., Unknown/Did not report) for a particular data element, and to provide any other contextual information that may be helpful to ACF in understanding the data reported. For each explanation of missing data, include the Table number for ease of reference.
Summary of Tables in the DSUR
Table |
Title |
Variable |
A.1: Participant Demographics during Reporting Period (Newly Enrolled and Continuing) |
||
Table 1 |
Unduplicated Count of Adult Participants and Index Children Served by Tribal MIECHV Home Visitors during Reporting Period (Newly Enrolled and Continuing) |
Adult participants Index children
|
Table 1a |
Pregnant Participants Who Remained in the Program After Giving Birth During the Reporting Period |
Pregnant participants |
Table 2 |
Unduplicated Count of Households Served by Tribal MIECHV Home Visitors (Newly Enrolled and Continuing) |
Households |
Table 3 |
Adult Participants by Current Educational Status (Newly Enrolled and Continuing) |
Educational Status |
Table 4 |
Adult Participants by Employment Status (Newly Enrolled and Continuing) |
Employment Status |
Table 5 |
Household Income in Relation to Federal Poverty Guidelines (Newly Enrolled and Continuing) |
Federal Poverty Income |
Table 6 |
Index Children by Age (Newly Enrolled and Continuing) |
Age |
Table 7 |
Adult Participants by Housing Status (Newly Enrolled and Continuing) |
Housing Status |
Table 8 |
Adult Participants and Index Children by Type of Health Insurance Coverage (Newly Enrolled and Continuing) |
Health Insurance |
A.2: Participant Demographics during Reporting Period (Newly Enrolled Only) |
||
Table 9 |
Adult Participants by Age (Newly Enrolled) |
Age |
Table 10 |
Participants by Ethnicity (Newly Enrolled) |
Ethnicity |
Table 11 |
Participants by Race (Newly Enrolled) |
Race |
Table 12 |
Adult Participants by Marital Status (Newly Enrolled) |
Marital Status |
Table 13 |
Adult Participants by Educational Attainment (Newly Enrolled) |
Educational Attainment |
Table 14 |
Primary Language Spoken at Home of Index Children (Newly Enrolled) |
Primary Language |
Table 15 |
Secondary Language Spoken at Home of Index Children (Newly Enrolled) |
Secondary Language |
Table 16 |
Priority Population Household Characteristics (Newly Enrolled) |
8 Household characteristics |
A.3 Participant Service Utilization during Reporting Period |
||
Table 17 |
Unduplicated Count of Households by Evidence-Based Home Visiting Model and Total Number of Home Visits |
Home Visits/Households |
Table 18 |
Families Receiving In-Person Home Visits |
Receipt of Services |
Table 19 |
Family Engagement by Household (Newly Enrolled and Continuing) |
Household |
Table 20 |
Place-based Services |
Community (zip code) |
Section B: Program Staff Demographics |
||
Table 21 |
Program Staff by Age |
Age |
Table 22 |
Program Staff by Gender |
Gender |
Table 23 |
Program Staff by Ethnicity |
Ethnicity |
Table 24 |
Program Staff by Race |
Race |
Table 25 |
Program Staff by Educational Attainment |
Educational Attainment |
Table 26 |
Unduplicated Count of Home Visiting Staff Full Time Equivalents |
FTE Equivalents |
Plan for DSUR Data Collection and Entry
In table 61, please describe your data collection process for Tribal MIECHV DSUR. Please consider model requirements and forms. The table includes the following columns:
Table Number with Title – Insert the Table number with the Title of the Table. If the same collection form and approach are used for multiple tables, you can list the tables in the same row. You do not need a separate row for each table, but all tables should be accounted for in this section.
Data collection form – Identify the form staff will use to collect the information. Please specify the form name.
Mode and method of data collection – Identify how the data will be collected for the form. For example – paper form, electronic tablet, etc. Please specify whether data are gathered by participant self-report or home visitor observation, and/or whether the participant completes the form or the home visitor completes it as an interview.
Person responsible – Identify who on your home visiting team will be collecting the data for the table (e.g., home visitor, evaluator).
Data collection timepoint – Specify when the data are collected from the data source. For example – the intake form is completed during the first home visit.
Table 61
Table Number with Title |
Data Collection Form |
Mode and Method of Data Collection |
Person Responsible for Data Collection |
Data Collection Time point |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In table 62, please describe your data entry process for Tribal MIECHV DSUR. Include all data collection forms identified in table 56 above. Please consider model requirements. The table includes the following columns:
Data Collection Form – Create a row for each data collection form identified in table 61 above.
Person Responsible – Identify who on your home visiting team will be entering the data for the form (e.g., home visitor, program coordinator, evaluator).
Data System Used – Identify the data system(s) where these data are entered.
Data Entry Turnaround/Timeframe – Identify when the person responsible will be required to enter the data into the system once its collected. For example, some grantees require home visitors enter data into the system within 48 hours of data collection.
Table 62
Data Collection Form |
Person Responsible for Data Entry |
Data System Used |
Data Entry Turnaround/Timeframe |
|
|
|
|
|
|
|
|
Data Quality Monitoring Plan for the DSUR
In Table 63, please describe how your team will regularly verify the validity, accuracy, completeness, consistency, and uniformity of data collected for the DSUR. Include the frequency of data quality checks. Data collection quality issues may include missed assessment questions, not collecting forms during required timeframe, etc.
Table 63
Process for verifying the validity, accuracy, completeness, consistency, and uniformity of data collection |
|
|
|
In Table 64, please describe how your team will regularly verify the validity, accuracy, completeness, consistency, and uniformity of DSUR data entry. Include the frequency of data quality checks. Data entry quality issues may include inputting data into incorrect fields in the data system, missing entry fields, lags between data collection and entry, etc.
Table 64
Process for verifying the validity, accuracy, completeness, consistency, and uniformity of data entry |
|
|
|
Overview of the Performance Measurement Data Report (PMR)
Grantees under the Tribal MIECHV program must collect, analyze, use, and report data on program implementation and improvements for eligible families participating in the program in the legislatively-mandated benchmark areas of: I) improved maternal, newborn, and child health; II) prevention of child injuries, child abuse, neglect, or maltreatment, and reduction of emergency room visits; III) improvements in school readiness and child academic achievement; IV) reductions in crime or domestic violence; V) improvements in family economic self-sufficiency; and VI) improvements in the coordination and referrals for other community resources and supports. These data are reported in the Tribal MIECHV PMR. Grantees submit this report on an annual basis. The draft submission of the PMR is due by October 31st. The grantee then works with their TEI liaison and FPO to revise and finalize their report, and the final submission of the PMR is due on December 31st.
The PMR includes 12 Core Measures and 11 Flex Measures. Grantees must report on all 12 Core Measures. Grantees must select 3 Flex Measures that are ideally aligned with their selected home visiting model goals and target population. Flex Measures are described on page 29. Measures have been operationally standardized across grantees. There may be no validated tool in AIAN communities, but grantees are asked to select and use the most appropriate tools for their programs and communities.
Core Measures
Implementation Measures
Receipt of home visits
Home visit implementation observation
Reflective supervision
Maternal and Newborn Health
Depression screening
Substance abuse screening
Well-child visit
Child Maltreatment, Injuries, and ED Visits
Child injury prevention
School Readiness and Achievement
Parent-child interaction
Developmental screening
Crime or Domestic Violence
IPV screening
Family Economic Self-Sufficiency
Screening for economic strain
Coordination and Referrals
Completed developmental referral
Plan for PMR Data Collection and Entry
CORE MEASURES
CORE MEASURE 1: Receipt of Home Visits
Construct Information (not modifiable) |
Benchmark Area: Implementation Construct: Receipt of home visits |
Type of Measure: Process |
Indicator: Percentage of recommended home visits received by families enrolled in the home visiting program during the reporting period |
Numerator: Number of home visits received by families during the reporting period Denominator: Number of home visits families should receive according to model developer fidelity requirements during the reporting period |
Target Population: Families |
Unit of Analysis: Incidents |
Data Source: Home visitor observation |
Performance Measurement Plan |
Definitions of Key Terms |
Number of home visits families should receive:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where home visit information is documented:
|
Form or tool to keep track of whether a family is receiving the recommended number of home visits:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 2: Home Visit Implementation Observation
Construct Information (not modifiable) |
Benchmark Area: Implementation Construct: Home visit implementation observation |
Type of Measure: Process |
Indicator: Percentage of recommended home visits where home visitors are observed for implementation quality and receive feedback from their supervisors during the reporting period |
Numerator: Number of home visits where home visitors are observed for implementation quality and receive feedback from their supervisors during the reporting period Denominator: Number of home visits where home visitors should receive an observation according to model developer guidelines during the reporting period |
Target Population: Home visitors |
Unit of Analysis: Incidents |
Data Source: Supervisor observation |
Performance Measurement Plan |
Definitions of Key Terms |
Number of home visits families should receive:
|
Number of home visits where a home visitor should be observed by a supervisor per family served:
|
Supervisor feedback on observed home visits:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 3: Reflective Supervision
Construct Information (not modifiable) |
Benchmark Area: Implementation Construct: Reflective supervision |
Type of Measure: Process |
Indicator: Percentage of recommended individual reflective supervision sessions received by home visitors during the reporting period |
Numerator: Number of individual reflective supervision sessions received by home visitors during the reporting period Denominator: Number of individual reflective supervision sessions that home visitors should receive during the reporting period |
Target Population: Home visitors |
Unit of Analysis: Incidents |
Data Source: Home visitor/supervisor self-report |
Performance Measurement Plan |
Definitions of Key Terms |
Reflective supervision session (individual):
|
Number of reflective supervision sessions home visitors should receive:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 4: Depression Screening
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Depression screening |
Type of Measure: Process |
Indicator: Percentage of primary caregivers enrolled in HV who are screened for depression using a validated tool within six months of enrollment (for those not enrolled prenatally) or by three months post-delivery (for those enrolled prenatally) and at least annually thereafter |
Numerator: For those not enrolled prenatally, number of primary caregivers enrolled in HV who are screened for depression within the first six months since enrollment; for those enrolled prenatally, the number of primary caregivers screened for depression by three months post-delivery; or annually after the first year of enrollment Denominator: Number of primary caregivers not enrolled prenatally who are enrolled in HV for at least six months; the number of mothers enrolled prenatally who have reached 3 months post-delivery |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Name of validated tool used to screen for depression:
|
Procedure for screening individuals who are already receiving mental health services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Form used to document when a depression screening has been conducted:
|
Process for obtaining depression screening data from other agencies (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 5: Substance Abuse Screening
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Substance abuse screening |
Type of Measure: Process |
Indicator: Percentage of primary caregivers enrolled in HV who are screened for substance abuse using a validated tool within six months of enrollment and at least annually thereafter |
Numerator: Number of primary caregivers enrolled in HV who are screened for substance abuse using a validated tool within six months of enrollment and annually after the first year of enrollment Denominator: Number of primary caregivers enrolled in HV for at least six months |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Substance abuse (please identify which substances your screening tool assesses; note: if your definition only includes a limited set of substances, please provide a justification for this decision):
|
Name of validated tool used to screen for substance abuse:
|
Procedure for screening individuals who are already receiving substance abuse services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Program form or tool to keep track of whether caregivers receive the screening within 6 months of enrollment and annually thereafter:
|
Process for obtaining substance abuse screening data from other agencies (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 6: Well Child Visit
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Well child visit |
Type of Measure: Outcome |
Indicator: Percentage of AAP-recommended number of well-child visits received by children enrolled in home visiting during the reporting period |
Numerator: Number of AAP-recommended well-child visits received by children (index child) enrolled in home visiting during the reporting period Denominator: Number of AAP-recommended well-child visits children (index child) enrolled in home visiting should receive during the reporting period |
Target Population: Index children |
Unit of Analysis: Incidents |
Data Source: Primary caregiver self-report or medical records |
Performance Measurement Plan |
Definitions of Key Terms |
AAP-recommended well-child visits (including window of time allowed for recommended visits, e.g., 2-week window before and after recommended time):
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where AAP-recommended well-child visit information is documented:
|
Form/field in the medical records that will be used to collect the information needed for this construct (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 7: Child Injury Prevention
Construct Information (not modifiable) |
Benchmark Area: Child Maltreatment, Injuries, and ED Visits Construct: Child injury prevention |
Type of Measure: Process |
Indicator: Percentage of primary caregivers enrolled in home visiting who are provided with an annual training on prevention of infant and child injuries |
Numerator: Number of primary caregivers enrolled in home visiting who are provided with training on prevention of infant and child injuries during the reporting period Denominator: Number of primary caregivers enrolled in home visiting |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Training on prevention of infant and child injuries (including which topics and trainings count towards indicator):
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model home visit summary form or log where home visitors should document delivered curriculum:
|
Program form or log home visitors use to document delivered curriculum (if needed):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included (please note whether all trainings in definition are required to be completed to count in numerator, or just particular ones):
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 8: Parent-Child Interaction
Construct Information (not modifiable) |
Benchmark Area: School Readiness and Achievement Construct: Parent-child interaction |
Type of Measure: Process |
Indicator: Percentage of primary caregivers enrolled in HV who receive an annual observation of caregiver-child interaction by the home visitor using a validated tool |
Numerator: Number of primary caregivers enrolled in HV who receive an observation of caregiver-child interaction by the home visitor using a validated tool during the reporting period Denominator: Number of primary caregivers enrolled in HV with children in the age range specified by the tool |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Observation of caregiver-child interaction:
|
Age range (note: all children who fall within the age range approved by the tool must be assessed):
|
Name of validated tool:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 9: Developmental Screening
Construct Information (not modifiable) |
Benchmark Area: School Readiness and Achievement Construct: Developmental screening |
Type of Measure: Process |
Indicator: Percentage of children enrolled in HV screened at least annually for developmental delays using a validated parent-completed tool |
Numerator: Number of children (index child) enrolled in HV with at least one documented developmental screening during the reporting period Denominator: Number of children (index child) enrolled in HV during the reporting period that required a screening |
Target Population: Index children |
Unit of Analysis: Participants |
Data Source: Parent self-report |
Performance Measurement Plan |
Definitions of Key Terms |
Required a screening:
|
Name of the validated tool used to screen for developmental delays:
|
Procedure for screening individuals who are already receiving early intervention services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where developmental screenings are documented:
|
Program form or tool to keep track of whether an index child receives the developmental screening(s) at the required time points:
|
Process for obtaining developmental screening data from other agencies (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 10: IPV Screening
Construct Information (not modifiable) |
Benchmark Area: Crime or Domestic Violence Construct: IPV screening |
Type of Measure: Process |
Indicator: Percentage of primary caregivers enrolled in HV who are screened for intimate partner violence using a validated tool within six months of enrollment and at least annually thereafter |
Numerator: Number of primary caregivers enrolled in HV who are screened for IPV using a validated tool within six months of enrollment and annually after the first year of enrollment Denominator: Number of primary caregivers enrolled in HV for at least six months |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Name of validated tool used to screen for IPV:
|
Procedure for screening individuals who are already receiving IPV services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where home visitors document when an IPV screening has been conducted:
|
Program form or tool to keep track of whether the primary caregiver receives the IPV screening within 6 months of enrollment and annually thereafter:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 11: Screening for Economic Strain
Construct Information (not modifiable) |
Benchmark Area: Family Economic Self-Sufficiency Construct: Screening for economic strain |
Type of Measure: Process |
Indicator: Percentage of primary caregivers who are screened for unmet basic needs (poverty, food insecurity, housing insecurity, etc.) within six months of enrollment and at least annually thereafter |
Numerator: Number of primary caregivers who are screened for unmet basic needs within six months of enrollment and annually after the first year of enrollment Denominator: Number of primary caregivers enrolled in home visiting for at least six months |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Unmet basic need:
|
Name of screener:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where unmet basic need screening information is documented:
|
Program tool or form to keep track of whether a primary caregiver receives the screening within 6 months of enrollment or annually thereafter:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
CORE MEASURE 12: Completed Developmental Referrals
Construct Information (not modifiable) |
Benchmark Area: Coordination and Referrals Construct: Completed developmental referrals |
Type of Measure: Process |
Indicator: Percentage of children enrolled in home visiting with positive screens for developmental delays (measured using a validated tool) who receive a referral for services and individualized developmental support from their home visitor |
Numerator: Number of children (index child) enrolled in HV who were referred to other community services and received individualized developmental support from a home visitor during the reporting period Denominator: Number of children (index child) enrolled in HV during the reporting period with positive screens for developmental delays (measured using a validated tool) |
Target Population: Index children |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Referred to other community services:
|
Received individualized developmental support from a home visitor:
|
Procedure for tracking referrals for individuals who are already receiving early intervention services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURES
Grantees must select 3 Flex Measures that are ideally aligned with their selected home visiting model goals and target population. Two measures must be selected from items 1 to 7, and one measure must be selected from items 8 to 11. Please check which three measures you have decided to report on below. Measures have been operationally standardized across grantees. There may be no validated tool in AIAN communities, but grantees are asked to select and use the most appropriate tools for their programs and communities.
Maternal and Newborn Health
☐1. Breastfeeding
☐2. Postpartum care
☐3. Immunizations
Child Maltreatment, Injuries, and ED Visits
☐4. Screening for parenting stress
☐5. Safe sleep
☐6. Child injury
School Readiness and Achievement
☐7. Early language and literacy activities
Coordination and Referrals
☐8. Completed IPV referrals
☐9. Completed depression or parenting stress referrals
☐10. Completed substance abuse referrals
☐11. Completed economic strain referrals
FLEX MEASURE 1: Breastfeeding
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Breastfeeding |
Type of Measure: Outcome |
Indicator: Percentage of participants enrolled prior to child’s birth who initiate breastfeeding |
Numerator: Number of participants who enrolled prenatally who initiate breastfeeding Denominator: Number of participants enrolled prenatally who gave birth within the reporting period |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Participant self-report |
Performance Measurement Plan |
Definitions of Key Terms |
Breastfeeding initiation:
|
Medically unable to breastfeed:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model or program form where breastfeeding initiation is documented:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
Flex Measure 2: Postpartum Care
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Postpartum care |
Type of Measure: Outcome |
Indicator: Percentage of participants enrolled in HV prenatally or within 30 days after delivery who received a postpartum visit with a health care provider within eight weeks (56 days) of delivery |
Numerator: Number of participants enrolled in HV prenatally or within 30 days after delivery who received a postpartum visit with a health care provider within eight weeks (56 days) of delivery Denominator: Number of participants who enrolled in HV prenatally or within 30 days after delivery and remained enrolled for at least eight weeks (56 days) after delivery |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Primary caregiver self-report or medical records |
Performance Measurement Plan |
Definitions of Key Terms |
Postpartum visit:
|
Health care provider:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Space on the tool/form to identify whether the postpartum visit occurred within 8 weeks of delivery:
|
Medical record form/field that will be used to collect the information needed for this construct (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
Construct Information (not modifiable) |
Benchmark Area: Maternal and Newborn Health Construct: Immunizations |
Type of Measure: Outcome |
Indicator: Percentage of children enrolled in home visiting who receive all AAP-recommended immunizations during the reporting period |
Numerator: Number of children (index child) enrolled in HV who receive all AAP-recommended immunizations during the reporting period Denominator: Number of children (index child) enrolled in HV during the reporting period |
Target Population: Enrolled children |
Unit of Analysis: Participants |
Data Source: Primary caregiver self-report, medical record, or State immunization registry |
Performance Measurement Plan |
Definitions of Key Terms |
AAP-recommended immunizations:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where immunization information is documented:
|
Process for staff to enter medical record or state immunization registry data into the client’s file or for data to be automatically entered into the data system (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 4: Screening for Parenting Stress
Construct Information (not modifiable) |
Benchmark Area: Child Maltreatment, Injuries, and ED Visits Construct: Screening for parenting stress |
Type of Measure: Process |
Indicator: Percentage of primary caregivers who are screened for parenting stress using a validated tool within six months of enrollment (for those not enrolled prenatally) or by three months post-delivery (for those enrolled prenatally), and at least annually thereafter |
Numerator: For those not enrolled prenatally, number of primary caregivers enrolled in HV who are screened for parenting stress within the first six months since enrollment; for those enrolled prenatally, the number of primary caregivers screened for parenting stress by three months post-delivery; or annually after the first year of enrollment Denominator: Number of primary caregivers not enrolled prenatally who are enrolled in HV for at least six months; the number of mothers enrolled prenatally who have reached three months post-delivery |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Parenting stress:
|
Name of validated tool used to screen for parenting stress:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where screening for parenting stress information is documented:
|
Program form or tool used to keep track of whether a primary caregiver receives the screening within 6 months of enrollment, within 3 months of delivery (if enrolled prenatally), or annually thereafter:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
Construct Information (not modifiable) |
Benchmark Area: Child Maltreatment, Injuries, and ED Visits Construct: Safe sleep |
Type of Measure: Process |
Indicator: Percentage of primary caregivers educated about the importance of putting infants to sleep on their backs, without bed-sharing and soft-bedding |
Numerator: Number of primary caregivers educated about the importance of putting infants to sleep on their backs, and without bed-sharing and soft-bedding during the reporting period Denominator: Number of primary caregivers enrolled in home visiting during the reporting period who are either pregnant or have a child under 12 months of age and have not already received safe sleep education in the previous reporting period |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
The lesson content on safe sleep that addresses “the importance of putting infants to sleep on their backs, without bed-sharing and soft-bedding” (including which topics and trainings count towards indicator):
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where safe sleep education provided is documented:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included (note whether all trainings are required to be completed to count towards the numerator, or just specific ones):
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
Construct Information (not modifiable) |
Benchmark Area: Child Maltreatment, Injuries, and ED Visits Construct: Child injury |
Type of Measure: Outcome |
Indicator: Rate of injury-related visits to the Emergency Department (ED) or urgent care since enrollment among children enrolled in HV |
Numerator: Number of parent-reported nonfatal injury-related visits to the Emergency Department (ED) or urgent care since enrollment among children (index child) enrolled in HV Denominator: Number of children (index child) enrolled in HV during the reporting period |
Target Population: Index children |
Unit of Analysis: Incidents |
Data Source: Parent self-report, medical record |
Performance Measurement Plan |
Definitions of Key Terms |
Emergency Department or Urgent care:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where nonfatal injury-related visits for children to the ED or urgent care are documented:
|
Form/field within the medical records that will be used to collect the information needed for this construct (if applicable):
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 7: Early Language and Literacy Activities
Construct Information (not modifiable) |
Benchmark Area: School Readiness and Achievement Construct: Early language and literacy activities |
Type of Measure: Outcome |
Indicator: Percentage of children enrolled in HV with a caregiver who reported that during a typical week the caregiver or family member read, told stories, and/or sang songs with their child every day |
Numerator: Number of children (index child) enrolled in HV with a caregiver who reported that during a typical week the caregiver or a family member read, told stories, and/or sang songs with their child every day Denominator: Number of children (index child) enrolled in HV during the reporting period |
Target Population: Index children |
Unit of Analysis: Participants |
Data Source: Caregiver self-report |
Performance Measurement Plan |
Definitions of Key Terms |
Family member:
|
Typical week:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Model form where information on whether a caregiver or family member read/told stories/sang songs with a child every day in a typical week is documented:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 8: Completed IPV Referrals
Construct Information (not modifiable) |
Benchmark Area: Coordination and Referrals Construct: Completed IPV referrals |
Type of Measure: Process |
Indicator: Percentage of primary caregivers screening positive for intimate partner violence using a validated tool who receive a referral for services |
Numerator: Number of primary caregivers enrolled in HV who received referral information to appropriate services (and met the conditions specified in the denominator) Denominator: Number of primary caregivers enrolled in HV with positive screens for IPV (measured using a validated tool) |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Received referral information:
|
Procedure for tracking referrals for individuals who are already receiving IPV services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Form that tracks screening results:
|
Form that tracks referrals:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 9: Completed Depression or Parenting Stress Referrals
Construct Information (not modifiable) |
Benchmark Area: Coordination and Referrals Construct: Completed depression or parenting stress referrals |
Type of Measure: Process |
Indicator: Percentage of primary caregivers screening positive for depression or parenting stress using a validated tool who receive a referral for services |
Numerator: Number of primary caregivers enrolled in HV who received referral information to appropriate services (and met the conditions specified in the denominator) Denominator: Number of primary caregivers enrolled in HV who had a positive screen for depression or parenting stress (measured using a validated tool) |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Screening focus (depression or parent stress):
|
Received referral information:
|
Procedure for tracking referrals for individuals who are already receiving mental health services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Form that tracks screening results:
|
Form that tracks referrals:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 10: Completed Substance Abuse Referrals
Construct Information (not modifiable) |
Benchmark Area: Coordination and Referrals Construct: Substance abuse referrals |
Type of Measure: Process |
Indicator: Percentage of primary caregivers screening positive for substance abuse using a validated tool who receive a referral for services |
Numerator: Number of primary caregivers enrolled in HV who received referral information to appropriate services (and met the conditions specified in the denominator) Denominator: Number of primary caregivers enrolled in HV who had a positive screen for substance abuse (measuring using a validated tool) |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Received referral information:
|
Procedure for tracking referrals for individuals who are already receiving substance abuse services:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Form that tracks screening results:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
FLEX MEASURE 11: Completed Economic Strain Referrals
Construct Information (not modifiable) |
Benchmark Area: Coordination and Referrals Construct: Completed economic strain referrals |
Type of Measure: Process |
Indicator: Percentage of primary caregivers with unmet basic needs who receive a referral for services |
Numerator: Number of primary caregivers enrolled in HV who received referral information to appropriate services (and met the conditions specified in the denominator) Denominator: Number of primary caregivers enrolled in HV who had positive screen for unmet basic needs |
Target Population: Primary caregivers |
Unit of Analysis: Participants |
Data Source: Home visitor documentation |
Performance Measurement Plan |
Definitions of Key Terms |
Received referral information:
|
Data Collection Tool(s) or Form(s) and Question(s) |
Data collection tool or form that will be used to collect the information for this construct:
|
Specific questions on the tool/form that will be used to capture the information needed:
|
Questions designed to determine unmet basic needs:
|
Form that tracks screening results:
|
Data Collection Time Points |
Time point(s) at which the data will be collected:
|
Data Elements for Calculations |
Data elements that will be needed to calculate the numerator and denominator:
|
Numerator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the numerator and what data are not included:
|
Denominator Inclusion and Exclusion Criteria |
Criteria used to determine what data are included in the denominator and what data are not included:
|
Data Quality Monitoring Plan for the PMR
In Table 65, please describe how your team will regularly verify the validity, accuracy, completeness, consistency, and uniformity of data collected for the PMR. Include frequency of data quality checks. Data collection quality issues may include missed assessment questions, not collecting forms during required timeframe, etc.
Table 65
Process for verifying the validity, accuracy, completeness, consistency, and uniformity of data collection |
|
|
|
In Table 66, please describe how your team will regularly verify the validity, accuracy, completeness, consistency, and uniformity of data entered for the PMR. Include frequency of data quality checks. Data entry quality issues may include inputting data into incorrect fields in the data system, missing entry fields, lags between data collection and entry, etc.
Table 66
Process for verifying the validity, accuracy, completeness, consistency, and uniformity of data entry |
|
|
|
Overview of Quarterly Performance Data Report
Grantees are required to submit data related to program capacity, family engagement, and staff vacancies on a quarterly basis.
Quarterly Reporting Periods are defined as:
Quarter 1: October 1 – December 31
Quarter 2: January 1 – March 31
Quarter 3: April 1 – June 30
Quarter 4: July 1 – September 30
Reports are due 30 days after the end of each reporting period.
There are 3 Tables in the QPR:
Table 1: Program Capacity
Table 2: Family Engagement
Table 3: Staff Vacancies
When completing the tables with data, the notes section should be used to provide any information relevant to data quality for that table or any other contextual information relevant to the reported numbers.
Individuals
and families reported on the QPR must be served by a trained home
visitor who receives 25% or more of his/her personnel costs
(salary/wages including benefits) paid for with Tribal MIECHV
funding.
Table 1 Program Capacity
Column A |
Number of New Households Enrolled |
Column B |
Number of Continuing Households |
Column C |
Current Caseload (A + B) |
Column D |
Maximum Service Capacity |
Column E |
Capacity Percentage (C/D) |
Table 2 Family Engagement
Column A |
Number of Households Currently Receiving Services |
Column B |
Number of Households who Completed Program |
Column C |
Number of Households who Stopped Services Before Completion |
Column D |
Number of Households Enrolled but Not Currently Receiving Services/Other |
Column E |
Total (A+B+C+D) |
Column F |
Attrition Rate (C divided by E) |
Table 3 Staff Vacancies
Data will include a head count of all staff positions that were vacant at the end of the reporting period, regardless of when the position became vacant.
Column A |
Number of Vacant MIECHV Home Visitors |
Column B |
Number of Vacant MIECHV Project Directors/Managers/Coordinators |
Column C |
Number of Vacant MIECHV Data/Evaluation Staff |
In table 67, please describe your data reporting process for Tribal MIECHV QPR. The table includes the following columns:
Table Number with Title – Insert the Table number with the Title of the Table and Column. If the same reporting approach is used for multiple tables, you can list the tables in the same row. You do not need a separate row for each table in this report, but all tables should be accounted for in this section.
Data reporting process – Describe how you will report on the pertinent data elements for the table. For example, is there a report you run in your data system? What are the data elements that you run in the report to be able to calculate the values for the table?
Person responsible – Identify who on your home visiting team will be analyzing and reporting the data for the table (e.g., program coordinator, evaluator).
Table 67
Table Number |
Data Reporting Process |
Person Responsible |
Table 1 Program Capacity – New Households enrolled |
Program Coordinator runs report in THV Program MIS – includes count on all households marked as “enrolled” between the dates of the quarter being reported. |
Program Coordinator |
|
|
|
Data Quality Monitoring Plan for the QPR
In Table 68, please describe how your team will regularly verify the validity, accuracy, completeness, consistency, and uniformity of data reported for the QPR. Include frequency of data quality checks. Data quality issues may include inputting data into incorrect fields in the data system, missing entry fields, lags between data collection and entry, etc.
Table 68
Process for verifying the validity, accuracy, completeness, consistency, and uniformity of data reported |
|
|
|
SECTION 5: PLAN FOR USING DATA TO INFORM FIDELITY MONTIORING, PROGRAM MANAGEMENT, AND IMPROVEMENT
The goal of Section 5 is to describe and plan for the ways in which your team will use data to monitor the implementation of your home visiting program. Grantees are encouraged to use data that they are already collecting for required reporting (see Section 4), but your team may decide to collect additional data to enhance your ability to monitor implementation over time.
Quality assurance is a process that occurs on a regular, predetermined schedule and determines current performance in relation to a set of standards. In the context of Tribal Home Visiting, quality assurance helps teams identify whether their program implementation meets model standards. In other words, quality assurance helps you determine whether your program is implemented as intended/planned. In this Section, you will identify which standards your team will assess on a regular basis and the data you will use to examine whether you achieved those standards.
SECTION 5.1: Fidelity Monitoring and Quality Assurance Plan
It is important to monitor the extent to which components of your home visiting model are being implemented as intended. Regular monitoring will allow your team to assess the quality of your service delivery and make changes to strengthen program processes, as needed. ACF would like you to regularly monitor the following service delivery data elements. These data will already be collected for THV reporting purposes, so should not require additional data collection efforts. Checking this data on a regular basis can help strengthen data reporting overall. ACF will check in on grantee progress on quality assurance on a regular basis during monthly calls.
Supervision sessions
Caseload
Number of families served
Completed in person and virtual home visits
Incoming referrals
Completed referrals to supplemental services
Number of families exited, inactive, and that completed the program
Completed home visit observations
Staff vacancies
In the table 66, specify the following:
QA Target — Describe your team’s desired performance for this measure. Developing a target for quality assurance involves assessing the relevant literature, examining past performance, and engaging staff in dialog. Reach out to your model developer as well as PATH and TEI TA providers for support in developing a realistic yet motivating target.
Method of Monitoring — Describe how your team will ensure that relevant staff members see these data on an ongoing basis. Indicate which staff will review this data, and how often. Examples include staff meetings, individual supervision, etc.
Table 69
Performance or activity being monitored |
QA target |
Method of monitoring |
Example: Completed visits |
80% of visits completed each week |
All Home Visitors; Coordinator will review data during weekly staff meeting |
Supervision sessions |
|
|
Caseload |
|
|
Number of families served |
|
|
Completed in person visits |
|
|
Completed virtual visits |
|
|
Incoming referrals |
|
|
Completed referrals to external services |
|
|
Number of families exited |
|
|
Number of families inactive |
|
|
Number of families that completed the program |
|
|
Completed home visit observations |
|
|
Staff vacancies |
|
|
Another way to monitor quality over time is to review your program logic model (see Section 2.3) and identify core components that you could track that would indicate the extent to which your program was being implemented as intended (or with high quality). In this section, you will look back at your logic model and identify activities or outputs to monitor over time. You will then establish appropriate indicators for tracking those activities, using data already being collected for this grant, and outline a plan for regular monitoring. Grantees should be careful not to replicate the service delivery data elements in Table 69 above.
Using your logic model, specify the following in Table 70 below:
Activity – Identify the logic model activity you will be regularly monitoring to assess implementation quality.
Indicator – Define the indicator you will use to assess that activity.
Reported measure – Note whether data on this indicator will be collected for other THV required reporting (e.g., DSUR, PMR, and QPR) and specify which data elements you will use to collect data on the indicator.
QA Target — Describe your team’s desired performance for this measure. Developing a target for quality assurance involves assessing the relevant literature, examining past performance, and engaging staff in dialog. Reach out to your model developer as well as PATH and TEI TA providers for support in developing a realistic yet motivating target.
Plan for monitoring – Describe how your team will ensure that relevant staff members see these data on an ongoing basis. Indicate which staff will review this data, and how often. Examples include staff meetings, individual supervision, etc.
Table 70
-
Activity
Indicator
Reported Measure - Yes or No?
QA Target
Plan for Monitoring
Introduction of cultural strengths handout by 6th foundational visit (part of program’s cultural adaptation)
Number of foundational home visits where cultural strengths handout was introduced by Home Visitor
No
Introduction of cultural handout in 80% of completed 6th home visits
Program Coordinator will check count of home visits where cultural handout was introduced each quarter and discuss with team during staff meeting.
In Table 71, please discuss your approach to engaging staff in quality assurance, including strategies for ensuring staff participation and engagement in QA (for example, using data dashboards for visual monitoring, celebrating successes, etc.).
Table 71
Approach to engaging staff in QA |
|
Note:
Connect with the model to identify the essential elements of the
model and ensure there is a plan in place to track each essential
element.
Table 72
Essential Requirement Measurement Criteria |
Plan for Collecting the Required Indicator |
Plan for Monitoring |
Plan for Staff Participation |
Example At least 60% of enrolled children receive a complete child developmental screening within 90 days of enrollment or birth, and at least 60% of children receive a complete annual child developmental screening during the program year. |
Home visitors enter developmental screenings for each enrolled child in the program database |
Program manager runs a monthly developmental screening report to monitor screening rates |
Collective screening rates are reviewed with home visitors once per month during staff meetings and individual results with each home visitor once per month during administrative supervision. |
Example: The program conducts at least two advisory committee meetings during the program year. |
Program manager keeps electronic records of agendas, attendance, and membership |
Program manager reviews discussion outcomes, attendance and membership with program administrator semi-annually during supervision meeting |
Program manager reviews discussion outcomes with staff once semi-annually during staff meetings |
|
|
|
|
Note:
Connect with the model to understand the model’s specific
approach to fidelity and reporting (e.g., model-specific annual
reporting).
In Table 70, please discuss your approach to working with the model developer to review adherence to fidelity and quality standards, and address challenges, including frequency, and method of review.
Table 72
Approach to working with the model |
|
APPENDIX
Definitions
Adaptation
An adaptation of an evidence-based model or promising approach includes changes to the model that have not been tested with rigorous impact research and are determined by the model developer to alter core components related to program impacts, are aligned with Tribal MIECHV program requirements, and are agreed to by the model developer and ACF in partnership with the recipient.
Administrative supervision
Administrative supervision is the oversight of employees' adherence to tribal, federal, state, and agency regulations and policies (including the rules and policies of the Tribal MIECHV program; program policies and procedures; quality assurance; and safety.) Administrative supervision is aimed at monitoring an employee’s productivity and performance.
At-risk tribal community
An at-risk tribal community can be defined in the following ways, reflecting diverse circumstances of tribal populations:
A tribe or tribes within a discrete geographic region (e.g., on a reservation, Tribal Jurisdictional Service Area, Alaska Native village) could be considered an at-risk tribal community;
Subgroups or communities of a tribe or tribes within a discrete geographic region could be considered an at-risk tribal community; or
Members of a tribe(s) could live scattered throughout a larger, non-tribal geographic area interspersed with non-tribal members (e.g., AI/AN living in an urban environment) and be considered an at-risk tribal community.
Clinical supervision
Clinical supervision is case-focused and supports a practitioner in reviewing, discussing, and evaluating cases, including treatment planning, implementation of intervention strategies, and progress of clients. Clinical supervision may or may not be reflective.
CQI
CQI supports the ongoing use of performance and implementation data to optimize program outcomes, facilitate cultural and contextual adaptations of evidence-based models to meet community and program needs, identify and disseminate best practices, and test new approaches in home visiting that can increase efficiency and enhance effectiveness of programs.
Early childhood system
An early childhood system brings together health, early care and education, and family support program partners, as well as tribal and community leaders, families, and other stakeholders to achieve agreed-upon goals for thriving children and families. An early childhood system aims to: reach all children and families as early as possible with needed services and supports; reflect and respect the strengths, needs, values, languages, cultures and communities of children and families; ensure stability and continuity of services along a continuum from pregnancy to kindergarten entry; genuinely include and effectively accommodate children with special needs; support continuity of services, eliminate duplicative services, ease transitions in order to improve the overall service experience for families and children; value parents and community members as decision makers and leaders; and catalyze and maximize investment and foster innovation.
Eligible family
The MIECHV legislation (section 511(l)(2) of the Social Security Act) states that an eligible family in MIECHV means a woman who is pregnant, and the father of the child if the father is available; or a parent or primary caregiver of a child, including grandparents or other relatives of the child, and foster parents who are serving as the child's primary caregiver from birth to kindergarten entry, and including a non-custodial parent who have an ongoing relationship with, and at times provide physical care for, the child. Section 511(d)(4) of the Act further requires that MIECHV recipients give priority to serving high-risk groups including: eligible families who reside in at-risk tribal communities in need of such services, as identified in the needs assessment; low-income eligible families; eligible families who are pregnant women who have not attained age 21; eligible families that have a history of child abuse or neglect or have had interactions with child welfare services; eligible families that have a history of substance abuse or need substance abuse treatment; eligible families that have users of tobacco products in the home; eligible families that are or have children with low student achievement; eligible families with children with developmental delays or disabilities; and eligible families who, or that include individuals who, are serving or formerly served in the Armed Forces, including such families that have members of the Armed Forces who have had multiple deployments outside of the United States.
Enhancement
An enhancement of an evidence-based model or promising approach includes changes or additions to the model that have not been tested with rigorous impact research but are determined by the model developer not to alter the core components related to program impacts, are aligned with Tribal MIECHV program requirements, and are agreed to by the model developer and ACF in partnership with the recipient.
Evidence-based home visiting model
An evidence-based home visiting model is used to describe both models that meet the HHS criteria for evidence of effectiveness in tribal communities and models that are considered promising approaches. HHS uses Home Visiting Evidence of Effectiveness (HomVEE), http://homvee.acf.hhs.gov/, to conduct a thorough and transparent review of the home visiting research literature and provide an assessment of the evidence of effectiveness for home visiting models that target families with pregnant people and children from birth to kindergarten entry. This also includes a tribal-specific review (Tribal HomVEE). There is currently one model that meets the HHS criteria for evidence of effectiveness in AI/AN communities. All other home visiting models, including those that have been designated by HHS as meeting criteria for evidence of effectiveness for the general population through the HomVEE review, are currently considered promising approaches for use with AI/AN populations.
Fidelity
Fidelity constitutes a program's adherence to model developer requirements for high-quality implementation as well as any affiliation, certification, or accreditation required by the model developer, if applicable. These requirements include all aspects of initiating and implementing a home visiting model, including, but not limited to, recruiting and retaining clients; providing initial and ongoing training, supervision, and professional development for staff; establishing a management information system to track data related to service delivery and model fidelity; and developing an integrated resource and referral network to support client needs. Changes to a model that alter the core components related to program outcomes (otherwise known as drift) could impair fidelity and undermine the program's effectiveness.
Home visiting program
A home visiting program:
Includes home visiting as the primary service delivery strategy (excluding programs with infrequent or supplemental home visiting);
Is offered on a voluntary basis to eligible families in at-risk tribal communities; and
Targets outcomes specified in the MIECHV legislation, including: improved maternal and child health; prevention of child injuries, child abuse, or maltreatment, and reduction of emergency department visits; improvement in school readiness and achievement; reduction in crime or domestic violence; improvements in family economic self-sufficiency; and improvements in the coordination and referrals for other community resources and supports.
Infant and early childhood mental health consultation
Infant and early childhood mental health consultation is a prevention-based service that pairs a mental health consultant with families and adults who work with infants and young children in the different settings where they learn and grow, such as child care, preschool, and their home. The aim is to build adults’ capacity to strengthen and support the healthy social and emotional development of children, early and before intervention is needed. In contrast to direct therapeutic services, infant and early childhood mental health consultation offers an indirect approach to promoting positive social and emotional development among children and families. Mental health consultation can be an important support to home visiting programs by enhancing their capacities to address children’s and families' mental health needs.
Local advisory committee
A local advisory committee is comprised of community members (including families) from the program's service area and provides input on a variety of program and evaluation activities. Local advisory committees are created to ensure that the program is in touch with the community's needs and incorporates the community's perspective in program operations and evaluation activities. Local advisory committees should meet on a regular basis and perform a number of valuable functions, including providing input on needs and readiness assessment, performance measurement, CQI, evaluation, and dissemination activities; defining program vision, goals, objectives, and design; and helping shape many core functions and activities of the program.
MIECHV caseload slot
A MIECHV caseload slot is defined as a family served during the reporting period by a trained home visitor implementing services with fidelity to the model and that is identified as a MIECHV caseload slot at enrollment. All members of one family or household represent a single MIECHV caseload slot. Families are designated as MIECHV caseload slots at enrollment based on the designation of the home visitor they are assigned. Using this methodology, recipients designate all families as MIECHV caseload slots that are served by home visitors for whom at least 25 percent of his/her personnel costs (salary/wages including benefits) are paid for with MIECHV funding. Once designated as a MIECHV caseload slot, the family is tracked for the purposes of data collection through the tenure of service in the program. All members of one family or household represent a single MIECHV caseload slot. The count of MIECHV caseload slots should be distinguished from the cumulative number of enrolled families during the reporting period.
MIECHV Learning Agenda
From its inception, the MIECHV program has incorporated a learning agenda approach. A learning agenda aims to “continually improve program performance by applying existing evidence about what works, generating new knowledge, and using experimentation and innovation to test new approaches to program delivery” (OMB-13-17). MIECHV’s learning agenda involves a combination of continuous quality improvement, performance measurement, rigorous evaluation at the national and local levels, and support for research infrastructure in the field. The evidence generated by each of these activities contributes unique perspectives to HRSA and ACF’s understanding of the program to help improve MIECHV's effectiveness and to build the broader knowledge base regarding home visiting. More information is available at: https://mchb.hrsa.gov/sites/default/files/mchb/programs-impact/miechv-learning-agenda-overview.pdf
Needs and readiness assessment
A needs and readiness assessment has the following two major components: an assessment of community needs and an analysis of community readiness and capacity of organizations and programs to meet these needs.
Qualitative data
Descriptive and conceptual data, often rich in meaning and detail, and analyzed to discover underlying meanings and patterns of relationships.
Quantitative data
Numerical data, that can be aggregated and analyzed using mathematical and statistical methodologies.
Reflective supervision
Reflective supervision is a distinctive form of competency-based professional development that is provided to multidisciplinary early childhood home visitors who are working to support very young children's primary caregiving relationships. It is a practice that acknowledges that infants and toddlers have unique developmental and relational needs and that all early learning occurs in the context of relationships. Reflective supervision is distinct from administrative supervision and clinical supervision due to the shared exploration of the parallel process, that is, attention to all of the relationships is important, including the relationships between home visitor and supervisor, between home visitor and parent, and between parent and infant/toddler. Reflective supervision supports professional and personal development of home visitors by attending to the emotional content of their work and how reactions to the content affect their work. In reflective supervision, there is often greater emphasis on the supervisor's ability to listen and wait, allowing the supervisee to discover solutions, concepts and perceptions on his/her own without interruption from the supervisor.
Supplement
A supplement to an evidence-based model or promising approach is the addition of a supportive or complementary curriculum to an evidence-based home visiting model. The supplement may or may not have been tested with rigorous impact research, but must be determined by the model developer not to alter the core components related to program impacts, aligned with Tribal MIECHV program requirements, and agreed to by the model developer and ACF in partnership with the recipient.
Virtual home visiting
Home visiting conducted solely by use of electronic information and telecommunications technologies.
1 Complete conceptual framework is available: https://mchb.hrsa.gov/sites/default/files/mchb/programs-impact/conceptual-framework-report.pdf and for more information, visit: https://mchb.hrsa.gov/programs-impact/programs/home-visiting/miechv-evaluation-research
2 Inform: Inform or educate stakeholders in one-way communication. Consult: Gain information and feedback from stakeholders to inform decisions made internally through limited two-way communication. Involve: Work directly with stakeholders throughout the process to ensure that issues and concerns are understood and considered through two-way communication where learning takes place on both sides. Collaborate: Partner with stakeholder and/or stakeholder groups for the development of mutually agreed solutions and joint plan of action through two-way communication where learning, negotiation, and decision making occur on both sides. Stakeholders work together to take action (Source: Stakeholder Engagement Framework) .
OMB Control No.:
Expiration Date: XX/XX/XXXX
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Beltangay, Moushumi (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2023-08-31 |
© 2024 OMB.report | Privacy Policy