FLASH-V Formative Generic - Supporting Statement B

Family Level Assessment and State of Home Visiting (FLASH-V) - Formative Data Collection

OMB Information Collection Request

0970-0356

Supporting Statement

Part B

January 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Laura Nerenberg

B1. Respondent Universe and Sampling Methods

FLASH-V seeks to gather information from local implementing agencies (LIAs) and centralized intake (CI) agencies associated with the Maternal, Infant, and Early Childhood Home Visiting program (MIECHV). The universe of LIAs funded by state grantees is approximately 761. The exact universe of centralized intake agencies is unknown, but we know that there is a centralized intake system(s) operating in 31 of 47 MIECHV grantees for which we have information. While a handful of centralized intake agencies operate at the state level, most operate at the regional or local level. As such, there are multiple centralized intake agencies in operation throughout many of these grantees’ states.

With the assistance of HRSA Project Officers, the Project Team will reach out to up to 20 state grantees—10 candidate state grantees and 10 alternate state grantees—to ask for information about the LIAs and CIs in their states. The Project Team will first reach out to the 10 candidate state grantees. We expect that between 6 to 8 of the original 10 candidate state grantees will respond to our request, yielding information about 97 LIAs and about six state grantees with CIs (exact number of CIs is unknown, but we anticipate multiple CIs in operation throughout most of the six states).

From this list of LIAs and CIs, the Project Team will select up to 48 LIAs and 20 CIs to contact for additional information about their programs and to assess eligibility. We assume that 85% of the LIAs (n = 40) will be eligible for the study and that all CIs will be eligible (n = 20). We also assume that half of eligible LIAs and CIs will agree to participate. If after this step we still have a large candidate pool, the Project Team will select the final sample of up to 20 eligible LIAs and up to 10 CIs using selection criteria aimed to increase sample diversity. For LIAs this includes geography, national models implemented, availability of MIS data, and overall capacity. CI criteria includes geography, number of national models part of system, number of clients arriving at CI, availability of MIS data, state versus county system, and a broad service intake system versus home visiting programs only.

If we do not identify enough LIAs and/or CIs for the study through the first round of contact with 10 candidate state grantees, the Project Team will begin reaching out to up to 10 alternate state grantees who are also good candidates for participation. In addition, the Project Team aims to have representation from at least six HRSA regions across the country. Therefore, the desired sample size of eligible and interested participants can be attained through the recommendations of no fewer than six grantees.

B2. Procedures for Collection of Information

The Project Team will send up to 20 state grantees an email requesting recommendations for candidate programs (Appendix A – Outreach to State Grantees). Attached to the email will be the project description (Appendix B – Project Description) and an Excel sheet where they can enter information about their recommended programs (1 – Eligibility Assessment Form). The email requests recommendations for two distinct groups of individuals – LIAs and CIs. The Project Team identified these two participant groups because some home visiting programs receive referrals from a centralized intake system, while others receive referrals from community agencies and providers. We want our sample to represent the different ways in which they receive referrals.

Using the information provided by the state grantees, the Project Team will reach out to recommended LIAs and CIs via email to request a phone meeting to discuss the program characteristics, program capacity/referrals, and existing data systems, with the purpose of assessing its eligibility for FLASH-V. Attached to this outreach email will be the project description (Appendix B – Project Description). Based on the information gathered in the call, the Project Team will select up to 20 LIAs and up to 10 CIs for study inclusion. The Project Team will send these programs a second email to schedule the semi-structured interview and request relevant documents about their program’s recruitment, enrollment, and referral processes (note that this request does not impose burden, as the request is only for existing materials in their current format). The communications and protocols for this component of the study are included in 2 - Outreach to LIAs and CIs, Assessing Eligibility, and Scheduling Interviews. There are separate materials for LIAs and CIs.

Finally, one trained interviewer from the Project Team will conduct semi-structured interviews over the phone with the selected LIAs (see 3 – Interview Protocol - LIA) and CIs (see 4 – Interview Protocol – Centralized Intake Agency). The Project Team will go through a consent process with all individuals (Appendix C – Informed Consent) before conducting the interviews. The interviews will take approximately 1 hour to complete. Interview protocols will be tailored to the two participant groups (see 3 – Interview Protocol – LIA, and 4 – Interview Protocol – Centralized Intake Agency). The LIA interview protocol is deigned to solicit information about the program’s incoming referral processes, eligibility assessment practices, and the prioritization of eligible families. The centralized intake agency interview protocol is designed to solicit information about a centralized intake agency’s resources for families, their incoming referral sources, referral assessment practices, and outgoing referral protocols.

In addition, the Project Team will conduct trainings with interviewers to cover general interview guidelines, provide tips on conducting successful interviews, and review the specific interview protocols. The training will also cover the logistics of scheduling the interviews, recording instructions, pre-interview tasks (e.g., reviewing supporting materials and all protocols), and post-interview tasks (e.g. saving recordings to the secured drive).

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

The Project Team assumes that we will need to oversample LIAs and centralized intake agencies to obtain interviews with up to 20 LIAs and up to 10 centralized intake agencies. This is based on the assumption that about 85% of LIAs will be eligible to participate, and that half of LIAs and CIs contacted will agree to participate. In consultation with ACF/OPRE, the Project Team will identify approximately 10 candidate Federal Home Visiting Program grantees that fund LIAs and/or operate centralized intake systems that refer families to funded LIAs. These LIAs and centralized intake agencies may be invited for participation in the study. An LIA and centralized intake agency that operate in the same community are not precluded from both participating in the study.

Dealing with Nonresponse

If we do not identify enough LIAs and CIs for the study through the first round of contact with 10 state grantees, the Project Team will being reaching out to up to 10 alternate state grantees who are also good candidates for participation.

If after working through our initial candidate list and our alternate candidate list the Project Team has not yet found 20 eligible and interested LIAs and 10 eligible and interested CIs, we will place a call for participants via the Home Visiting Applied Research Collaborative (HARC).

Maximizing Response Rates

We anticipate that some candidate LIAs and/or centralized intake agencies will decline participation. As such, the Project Team has taken a number of steps to ensure we can recruit up to 30 eligible participants. First, the Project Team has solicited input from HRSA project officers on grantees that may be in a good position to participate in this study (e.g. grantees that are typically interested in research and are not experiencing any significant program implementation issues that might interfere with their ability to participate). In addition, the Project Team has requested that HRSA project officers send the emails to candidate grantees introducing the project.

B4. Tests of Procedures or Methods to be Undertaken

The Project Team conducted an internal pilot test of both interview protocols to ensure that the length of the protocols is appropriate for one-hour interviews and that the language used is clear. Four pilot interviews were conducted in total, two of the LIA protocol and two of the centralized intake protocol. All variations of the skip patterns were tested during the pilot testing. Each pilot test was well within the one-hour allotment. Following the pilot tests, the Project Team adjusted some of the language in the protocols to make it more conversational. In addition, the Project Team edited the order of some questions to improve the flow of conversation.

The Project Team will also debrief after the initial interviews to discuss any needed adjustments in procedures. If edits to the data collection protocols are deemed necessary, we will provide them to OMB as a nonsubstantive change. Since the Project Team conducted several pilot tests, changes to the protocols are unlikely.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The information for this study is being collected by James Bell Associates and MDRC on behalf of ACF. Principal Investigator Jill Filene, and Co-Project Directors Melanie Estarziau, and Helen Lee led development of the study design plan and data collection protocols, and will oversee collection and analysis of data gathered through interviews.

The agency responsible for receiving and approving contract deliverables is:

The Office of Planning, Research, and Evaluation (OPRE),

Administration for Children and Families (ACF)

U.S. Department of Health and Human Services

The Federal project officer for this project is Laura Nerenberg.

4