Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Feedback on the HomVEE Criteria for Evidence-Based Early Childhood Home Visiting Models
Formative Data Collections for Program Support
0970 – 0531
Supporting Statement
Part A - Justification
October 2024
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Part A
Executive Summary
Type of Request:
This Information Collection Request is for a generic information collection under the umbrella generic, Formative Data Collections for Program Support (0970-0531).
Description of Request:
The Administration for Children and Families (ACF) and Health Resources and Services Administration (HRSA) at the U.S. Department of Health and Human Services (HHS) collaborate to administer the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program. As part of this collaboration, ACF’s Office of Planning, Research, and Evaluation (OPRE) oversees the Home Visiting Evidence of Effectiveness (HomVEE) review. ACF and HRSA are engaged in on a broad effort to improve the HomVEE’s HHS Criteria for Evidence-based Early Childhood Home Visiting Models (“HHS Criteria”). To inform these efforts, ACF seeks approval to conduct focus groups and a listening session with constituencies with direct interest in and engagement with the MIECHV program. The purpose of these focus groups and listening session is to learn about the strengths and challenges with the current HHS Criteria. The proposed information collection includes semi-structured focus groups (held on a virtual platform) with evidence-based policy experts, home visiting model developers, home visiting administrators, home visiting advocates, and tribal home visiting experts. It also includes a virtual listening session with home visiting researchers. We do not intend for this information to be used as the principal basis for public policy decisions.
Time Sensitivity:
Funding is available to support this effort through June 2025.
A1. Necessity for the Data Collection
The Administration for Children and Families (ACF) and Health Resources and Services Administration (HRSA) at the U.S. Department of Health and Human Services (HHS) collaborate to administer the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program. As part of this collaboration, ACF’s Office of Planning, Research, and Evaluation (OPRE) oversees the Home Visiting Evidence of Effectiveness (HomVEE) review. ACF and HRSA are engaged in on a broad effort to improve the HomVEE’s HHS Criteria for Evidence-based Early Childhood Home Visiting Models (“HHS Criteria”). To inform these efforts, ACF seeks approval to conduct focus groups and a listening session with constituencies with direct interest in and engagement with the MIECHV program. The purpose of these focus groups and listening session is to learn about the strengths and challenges with the current HHS Criteria. The proposed information collection includes semi-structured focus groups (held on a virtual platform) with evidence-based policy experts, home visiting model developers, home visiting administrators, home visiting advocates, and tribal home visiting experts. It also includes a virtual listening session with home visiting researchers. We do not intend for this information to be used as the principal basis for public policy decisions.
The HHS Criteria are established by the U.S. Department of Health and Human Services to determine which home visiting models are considered evidence-based for the purposes of the MIECHV Program. By law, state and jurisdiction awardees must spend the majority of their MIECHV Program grants to implement evidence-based home visiting models, with up to 25 percent of funding available to implement promising approaches that will undergo rigorous evaluation. To be eligible for implementation as an evidence-based model with MIECHV funding, a model must meet statutory requirements for model eligibility, including the evidence requirements in the HHS Criteria.
The current HHS Criteria are:
“To qualify as an evidence-based early childhood home visiting service delivery model, a
model must meet at least one of the following criteria:
At least one high- or moderate-quality impact study of the model finds favorable, statistically significant impacts in two or more of the eight outcome domains.
At least two high- or moderate-quality impact studies of the model using non- overlapping analytic study samples find one or more favorable, statistically significant impacts in the same domain.
In both cases, the impacts must either (1) be found in the full sample or (2) if found for subgroups but not for the full sample, be replicated in the same domain in two or more studies using non- overlapping analytic study samples. Additionally, following the MIECHV-authorizing statute, if the model meets the above criteria based on findings from randomized controlled trial(s) only, then two additional requirements apply. First, one or more favorable, statistically significant impacts must be sustained for at least one year after program enrollment. Second, one or more favorable, statistically significant impacts must be reported in a peer-reviewed journal. These criteria are consistent with the MIECHV statutory requirements: Section 511 (d)(3)(A)(i)(I).”
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Use
The proposed focus groups and information session are designed to engage the voices of different constituencies that have interest in MIECHV. Specifically, the focus groups and listening session will explore the perspectives of researchers, evidence-based policy experts, home visiting administrators, and home visiting advocates regarding the current HHS Criteria. The feedback will be used by ACF and HRSA to inform revisions to the HHS Criteria to better reflect the current state of the field and better support the use of evidence-based home visiting programs in the MIECHV program.
Both the focus group and listening session findings will inform the drafting of updated HHS Criteria, which will be released to the public for comment through a Federal Register Notice in 2025. By undertaking this data collection activity, it is intended that HHS can (among other possibilities):
Improve the HHS Criteria to better meet best practices in evidence-based policymaking.
Improve the effectiveness of the HHS Criteria in identifying evidence-based early childhood home visiting program models.
Reduce any real or perceived barriers to reviewing and evaluating the effectiveness of early childhood home visiting program models.
Increase awareness of the home visiting and evidence communities regarding the updates to the HHS Criteria.
Obtaining feedback about processes and/or practices to inform ACF program development or support.
Processes for Information Collection
ACF will collect information through a series of semi-structured focus groups, with the following groups:
Evidence-based policy experts (1 focus group)
Home visiting model developers (2 focus groups)
Home visiting administrators (1 focus group)
Home visiting advocates (1 focus group)
Tribal home visiting experts (1 focus group)
Focus groups will be held via an online platform such as Zoom. A facilitator will use a semi-structured interview protocol that includes a scripted introduction and consent, an overview of the HHS Criteria and the process for updating them, and a combination of high-level questions and detailed probes to gather input into specific topics. See Instruments 1-5 for focus group materials.
ACF will also collect information through a listening session with home visiting researchers.
The listening session will be held virtually and facilitated using a semi-structured protocol that includes a scripted introduction and consent, an overview of the HHS Criteria and the process for updating them, and a series of activities using technological applications such as GroupMap, JamBoard, or PollEverywhere to gather and discuss responses to a set of questions posed to participants. See Instrument 6 for listening session materials.
A3. Improved Information Technology to Reduce Burden
All information will be gathered electronically. The focus groups will be scheduled and held virtually, and, pending participant consent, will be recorded. The recordings will be transcribed to ensure accuracy and completeness. Similarly, the listening session will be held virtually as well. Virtual technology will be utilized to gather input from the larger group within the listening session through polling, virtual sticky note exercises, or other approaches.
A4. Efforts to Identify Duplication
Feedback from these constituent groups on the HHS Criteria update is not available through any other sources.
We will not engage participants in more than one activity.
A5. Involvement of Small Organizations
To reduce potential burden on all focus group participants, which may include small organizations or businesses, the focus groups will be held at a time during business hours during which the participant indicates they are available. Participants will be informed on multiple occasions that their participation in focus group is voluntary. Similarly, to reduce burden on listening session participants, scheduling will consider the needs of participants located in different time zones.
A6. Consequences of Less Frequent Data Collection
This is a one-time data collection.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection request to extend approval of the umbrella generic with minor changes. The notice was published on January 28, 2022, (87 FR 4603), and provided a sixty-day period for public comment. ACF did not receive any comments on the first notice. A second notice was published, allowing a thirty-day period for public comment, in conjunction with submission of the request to OMB. ACF did not receive any comments on the second notice.
The team has consulted with a few groups to discuss the active engagement plan. These include the contractor for the Home Visiting Evidence of Effectiveness systematic review, the leadership team for the Home Visiting Applied Research Collaborative, and a few federal agencies that manage tiered evidence programs. These groups were always fewer than nine individuals, and the conversations focused on getting input on how best to engage in this information-gathering process.
A9. Tokens of Appreciation for Respondents
No tokens of appreciation for respondents are proposed for this information collection.
A10. Privacy of Respondents
This request includes the collection of basic personally identifiable information (name, job title, organizational/institutional affiliation) for the purpose of 1) contacting participants for recruitment in focus groups, 2) following up with participants if clarification is required for their comments, such as verifying the accuracy of transcripts. Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.
Focus group participants will be asked if they agree to being recorded. Once recorded, focus groups will be transcribed. Transcripts will be reviewed for accuracy, and once verified, recordings will be securely destroyed. Names will be removed from the transcript and replaced with unique identifiers. Transcripts will be maintained until conclusion of the project; at which time they will be securely destroyed. At no point will any reporting of findings identify individuals and their responses or provide information in which respondents could be identified such as through identification of a title or role that few hold.
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. ACF complies with all Federal and Departmental regulations for private information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations.
A11. Sensitive Questions
There are no sensitive questions in this data collection.
A12. Estimation of Information Collection Burden
Burden Estimates
Selection of potential focus group participants will be made with input from HRSA and ACF staff who have knowledge of the availability and experience of potential participants. Invitations will be made to a select number of Program Administrators, Evidence-Based Policy Experts, and Advocates. The Listening Session will be advertised through the Home Visiting Applied Research Center, with registration capped at 25.
We estimated burden for all activities based on commonly accepted practices, including considering the number and types of questions asked, time allocated for effective discussion, and size of group that allows for effective discussion. Each focus group will be capped at 8 participants and focus groups will be 60 minutes in length with 4-5 substantive discussion topics. We estimate the listening session will be 60 minutes in length and include 25 participants. There will be no burden to participants outside their participation in the focus groups or listening session. Estimated burden for participants is reported in Table A12.
Cost Estimates
The cost to respondents was calculated using the Bureau of Labor Statistics (BLS) job codes as follows1:
Focus Group Protocol for Evidence-Based Policy Experts: Social Scientists and Related Workers, All Other [19-3099]
Wage data from May 2023 is $49.14 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $98.28.
Focus Group Protocol for Home Visiting Model Developers: Social Scientists and Related Workers, All Other [19-3099]
Wage data from May 2023 is $49.14 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $98.28.
Focus Group Protocol for Home Visiting Administrators: Social and Community Service Managers [11-9151]
Wage data from May 2023 is $38.13 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $76.26.
Focus Group Protocol for Home Visiting Advocates: Social Scientists and Related Workers, All Other [19-3099]
Wage data from May 2023 is $49.14 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $98.28.
Focus Group Protocol for Tribal Home Visiting Experts: This group will include a combination of researchers, model developers, administrators, and advocates. Therefore, we’ve taking an average wage from the groups described in 2 through 4 above,
The average of $45.47 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $90.94.
Listening Session Protocol for Home Visiting Researchers: Social Scientists and Related Workers, All Other [19-3099]
Wage data from May 2023 is $49.14 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $98.28.
Table A12. Estimated annual and total burden and cost
Instrument |
Total Number of Respondents |
Total Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Focus Group Protocol for Evidence-Based Policy Experts |
8 |
1 |
1 |
8 |
$98.28 |
$786.24 |
Focus Group Protocol for Home Visiting Model Developers |
16 |
1 |
1 |
16 |
$98.28 |
$1,572.48 |
Focus Group Protocol for Home Visiting Administrators |
8 |
1 |
1 |
8 |
$76.26 |
$610.08 |
Focus Group Protocol for Home Visiting Advocates |
8 |
1 |
1 |
8 |
$98.28 |
$786.24 |
Focus Group Protocol for Tribal Home Visiting Experts |
8 |
1 |
1 |
8 |
$90.94 |
$727.52 |
Listening Session Protocol for Home Visiting Researchers |
25 |
1 |
1 |
25 |
$98.28 |
$2,457.00 |
Totals: |
73 |
1 |
1 |
73 |
|
$6,939.56 |
A13. Cost Burden to Respondents or Record Keepers
There are no additional costs to respondents.
A14. Estimate of Cost to the Federal Government
It is anticipated that contract staff will spend 51 hours on this project at an estimated cost of $4,636.92. This estimate includes time required to analyze program data for recruitment, recruit and schedule individual programs for participation, administer focus groups with two co-facilitators, and code and analyze transcripts. The cost for contract staff was calculated using the Bureau of Labor Statistics (BLS) job codes for Social Scientists and Related Workers, All Other [19-3099]. Wage data from May 2023 is $45.46 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $90.92.
A15. Change in Burden
This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
Data collection will begin after OMB approval and is expected to take place over 3 months. Information collected during the focus groups will be used to inform ACF and HRSA’s activities related to the HHS Criteria for Early Childhood Home Visiting Models.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
Attachments
Instrument 1: Focus Group Protocol for Evidence-Based Policy Experts
Instrument 2: Focus Group Protocol for Home Visiting Model Developers
Instrument 3: Focus Group Protocol for Home Visiting Administrator
Instrument 4: Focus Group Protocol for Home Visiting Advocates
Instrument 5: Focus Group Protocol for Tribal MIECHV Experts
Instrument 6: Listening Session Protocol for Home Visiting Researchers
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Created | 2024:11:26 12:50:39Z |