MIECHV Benchmarks Listening Sessions

MIECHV Reporting Requirements - Discussion Group Topic Guide

OMB #: 0970-0356

EXPIRATION: XX/XX/XXXX

Maternal, Infant and Early Childhood Home Visiting: reporting requirements Listening sessions

TOPIC GUIDE FOR Discussion GroupS

with PERFORMANCE measurement experts and stakeholders

This topic guide will be used for group discussions with Maternal, Infant and Early Childhood Home Visiting (MIECHV) experts and stakeholders who can inform federal considerations about the current performance measurement reporting system and how it might be simplified to reduce burden and more accurately and reliably provide data for program performance, accountability, and management purposes.

1. Warm-up/Introductions

• Purpose

HRSA and ACF are interested in gathering comments and feedback from grantees, measurement, quality improvement and subject matter experts on the current MIECHV performance measurement reporting requirements. As experts in performance measurement, they are interested in hearing from you about important things to consider when attempting to simplify and change performance measurement systems. What criteria should be used to select performance measures? Should some measures be standardized across grantees? How do you implement these changes? This effort will build on the work that the Pew Home Visiting Campaign undertook over the last year to define a simpler and more robust core set of standardized performance measures for home visiting, which I will discuss in more detail shortly.

Please note that participation is voluntary and at any time during the listening session you may decide not to participate. We estimate that this session will last two hours. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it has a currently valid OMB control number. The OMB control number for this collection is 0970-0356

• Ground rules and expectations

Before we get started, I would like to set some ground rules to ensure we have a productive and useful conversation. First, please identify yourself when you speak. Second, please speak one at a time. Finally, please keep the conversation within the group and to not discuss the comments of others outside of the session. All of the information and feedback that you provide will be only be shared with the consultants, contractors working on this project and relevant federal staff. The information shared during the session will not be published but will be used to inform HRSA and ACF. A note taker will be taking detailed notes to make sure we capture your feedback accurately. However, your name will not be connected to your responses. If you have further thoughts or ideas, please contact me or a member of the HRSA/ACF team to follow up via email or phone. Does anyone have questions before we begin?

B. Review of the current performance measurement requirements

The current MIECHV performance measurement guidelines require grantees to measure several indictors under six performance measurement areas. The areas and their corresponding performance measures include: [FILL IN THE PERFORMANCE MEASUREMENT AREA(S) AND PERFORMANCE MEASURES LISTED IN TABLE 1]

Table 1. MIECHV Performance Measurement Areas and Measures

MIECHV Performance Measurement Area	List of Performance Measures
Maternal and newborn health	Prenatal Care Parental use alcohol, tobacco, or illicit drugs Preconception Care Interbirth interval Screening for Maternal depression Breastfeeding Well-child visits Maternal and child health insurance status
Child injuries, child abuse, neglect, or maltreatment, and reduce emergency visits	Visits for children to the emergency department from all causes Visits of mothers to the emergency department from all causes Information provided or training of adult participants on prevention of child injuries including topics such as safe sleeping, shaken baby syndrome or traumatic brain injury, child passenger safety, poisonings, fire safety (including scalds), water safety (e.g., drowning; unsafe levels of lead in tap water), and playground safety Incidence of child injuries requiring medical treatment Reported suspected maltreatment for children in the program (allegations that were screened in by the child protective service agency but not necessarily substantiated) Reported substantiated maltreatment (substantiated/indicated/alternative response victim) for children in the program First-time victims of maltreatment for children in the program
School readiness and achievement	Parent support for children's learning and development (e.g., having appropriate toys available, talking, and reading with their child) Parent knowledge of child development and of their child's developmental progress Parenting behaviors and parent-child relationship (e.g., discipline strategies, play interactions) Parent emotional well-being or parenting stress Child’s communication, language, and emergent literacy Child’s general cognitive skills Child’s positive approaches to learning including attention Child’s social behavior, emotion regulation, and emotional well-being Child’s physical health and development
Crime or domestic violence	Screening for domestic violence Of families identified for the presence of domestic violence, number of referrals made to relevant domestic violence services (e.g., shelters) Of families identified for the presence of domestic violence, number of families for which a safety plan was completed.
Family economic self-sufficiency	Household income (including earnings, cash benefits, and in-kind and non-cash benefits) Employment or education of participating adults Health insurance status of participating adults and children
Coordination and referrals for other community resources and supports	Number of families identified for necessary services Number of families that required services and received a referral to available community resources Number of completed referrals (i.e., the home visiting provider is able to track individual family referrals and assess their completion, e.g., by obtaining a report of the service provided) MOUs: Number of Memoranda of Understanding or other formal agreements with other social service agencies in the community Information sharing: Number of agencies with which the home visiting provider has a clear point of contact in the collaborating community agency that includes regular sharing of information between agencies.

2. Criteria for simplifying the performance measurement reporting requirements

Pew and others have been exploring home visiting reporting broadly and the potential for shared indicators or performance measures across states. In terms of MIECHV, there are many criteria that one could use to reduce the number of measures under each performance measurement area. Pew, through their Home Visiting Data Initiative, has used an adapted version of the Results Based Accountability (RBA) framework to identify key indicators to help examine the shared impact of home visiting, build on existing administrative data, and measure progress toward outcomes:

• Communication power: Can be understood and is meaningful to experts, policy makers, and the general public, has policy value/ salience, and represents consensus among diverse stakeholders.

• Proxy power: Represents a relevant topic, is proximal to work; based on evidence that home visiting can have impact, says something important about the desired outcome, and reflects a set of related issues.

• Data power: Is measurable now with reliable and readily available data, maximizes administrative data, and maximizes data collected by models/ programs.

We would like you to reflect on these criteria during our discussion today. Please consider whether any adaptions need to be made to the criteria to fit the MIECHV context and performance measurement system.

• Do you find these criteria useful for coming up with recommendations for how to reduce the number of measures under each MIECHV performance measurement area?

• How would you modify the criteria to better capture key considerations when selecting performance measures for MIECHV?

  • Are there any criteria that you would add or takeaway?

  • How specifically would applying these criteria increase the quality of the measures selected or improve the process of selecting measures?

• Which (if any) criteria should be weighed more heavily than others? Why?

C. Important Considerations for Selecting Performance Measures

• What are the most important features of a sound performance measurement system for MIECHV? For program management purposes? For accountability purposes? For documenting changes in targeted child and family outcomes over time?

• When selecting performance measures, what other issues should be considered?

• Utility of the performance measurement data for state- or community-level program management and improvement

• Comparability of the performance measurement data across models and grantees and to meet federal reporting requirements.

• Alignment with what is actually happening on the ground and what has the most potential for improvement in a short period of time.

• Difficulty of reporting on certain performance measures and why.

• Alignment of standardized measures with state priorities.

D. Standardized Performance Measures

• What do you see as the utility or drawbacks of standardized measures in a performance measurement system?

• Are there particular performance measures that lend themselves more or less to standardized measurement? Which measures should be standardized across grantees and why?

• What are the biggest challenges to standardizing performance measures, particularly in the context of MIECHV? What issues would grantees need to be prepared for?

• What are the biggest benefits to standardizing the performance measures?

• For the constructs that are not measured in a standardized way, what are the implications for looking across grantees to talk about the program as a whole?

  • Benefits and drawbacks to having a mix of standardized measures and non-standardized measures across grantees.

E. Recommendations for process of making changes to performance measurement reporting requirements

• What are the implications or any proposed changes for grantees and their data systems?

• What do you recommend for the amount of time needed to plan and implement any changes?

• What opportunities exist for shared learning across grantees and models about how to make implement changes?

F. Closing recommendations

• Other recommendations that were not discussed

1