SSB - CW-SEED Formative GenIC for Research

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Child Welfare Study to Enhance Equity with Data (CW-SEED)

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

October 2023

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Christine Fortunato

Jenessa Malin

Nicole Denmark

Part B

B1. Objectives

Study Objectives

The Child Welfare Study to Enhance Equity with Data (CW-SEED) aims to understand how and to what extent data are used to explore equity in service delivery and child and family outcomes, to identify barriers or problematic data practices, and to explore efforts by child welfare agencies and their partners to use data to reduce barriers across the continuum of child welfare services. The project has received approval to conduct case studies in up to six sites (OMB control number 0970-0607). The primary objective of this generic information collection (GenIC) request is to gather preliminary information to identify potential sites to invite for participation in those case studies.

Generalizability of Results

This information collected through this GenIC is intended to present an internally valid description of data practices used to explore equity in participating sites, not to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

We will collect information using a topical discussion guide to explore and gather preliminary information about the data practices used to advance equity in up to 20 purposively selected sites. The use of qualitative methods are appropriate given that the objective is to identify potential sites for future case studies. Because this project aims to learn about approaches, processes, challenges, and facilitators to using data practices to advance equity, qualitative methods will address the guiding questions, using a flexible instrument that can be specified based on publicly available local information the study team has reviewed.

Data collected are not intended to be representative. Findings from this GenIC will be descriptive and cannot be used to make assessments of the effectiveness of the data practices being implemented. Written products associated with this GenIC will only be for internal ACF use and will clearly describe key limitations.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The target population is child welfare agency leaders and staff and partners in up to 20 sites that have been identified through prior and ongoing work (see below). We estimate that up to 100 respondents will participate across the sites.

Sampling and Site Selection

The CW-SEED study team has used multiple sources to identify potential sites for outreach for this information collection. This includes (1) recommendations from federal partners; (2) recommendations from experts familiar with child welfare agencies; (3) information gathered through an environmental scan; (4) responses from a general call for information; and, (5) recommendations from the CW-SEED study team.

The CW-SEED team used criteria to select sites for outreach. Examples of selection criteria include: (1) data practices linked to the goal of addressing equity and improving services and outcomes; (2) participatory practices, such as the inclusion of community voices; (3) the child welfare service continuum focus, such as prevention or permanency; (4) the population focus, such Black, Latino, and Indigenous populations or LGBTQ youth; and, (5) jurisdictional characteristics, such as structure and location of the child welfare agency.

Sites participating in these discussions are not intended to represent child welfare agencies nationally. Once OMB approval is received, the CW-SEED study team will outreach to the sites to schedule the informational calls. The preliminary information collected will inform the selection of potential case study sites.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

The CW-SEED study team designed the informational call topic guide to efficiently address the guiding questions, which were informed by a review of existing published documents, existing instruments and expert input.

B4. Collection of Data and Quality Control

The Administration for Children and Families (ACF) has contracted Mathematica and their partners to collect all necessary data. Table 2 in Supporting Statement A summarizes the data collection that will be conducted for under this GenIC.

Members of the study team will conduct semi-structured interviews. The remainder of this section describes the study team’s procedures for outreach and data collection.

Site Outreach

The CW-SEED study team will email the sites requesting participation in an informational call (See Appendix A). In the email and through other email communications, the study team will provide guidance to the site to ensure that interview participants have at least a minimal level of knowledge about the respective data practices of interest at the site. The study team will send up to three emails if the site is not responsive to the initial request.

Data Collection and Quality Control

A two-person team of CW-SEED study team staff will join each informational call. To ensure data quality, all study team members conducting these informational calls will receive training that includes a careful review of the following:

1. The objectives of the interviews and the guiding questions that the interviews will address

2. The informational call topic guide and how to specify it to each site, with an emphasis on the intent of each question to ensure interviewers have a keen sense of how to properly probe, which is critical because of the open-ended nature of interviews

3. Procedures for outreach and scheduling interviews

4. Proper note-taking to ensure that the research team captures this preliminary information consistently

B5. Site Selection

Response Rates

The interviews are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.

NonResponse

Based on previous experience with similar methods and respondents, we do not expect substantial nonresponse on the interviews. As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. Respondent demographics will not be documented and reported in written materials associated with the data collection.

B6. Production of Estimates and Projections

This GenIC seeks to present internally valid descriptions of the sites. The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

To ensure that interview notes are complete and consistently prepared, we will use a standard note template. Each two-person team conducting interviews will be responsible for reviewing their notes for completeness.

Data Analysis

The study team will analyze the qualitative information from the interview notes to identify which sites should be recommended for participation in the future case studies. Because the objective of this data collection is to identify sites for future case studies, the analysis of the information collected will be descriptive.

Data Use

The CW-SEED study team aims to use the information collected to select sites for future case studies.

General limitations to the resulting information collected will be included in the internal materials shared with ACF. Written products, which will only be internal to ACF, will clearly state that this GenIc aims to present a preliminary, high-level description of the data practices select child welfare agencies and their partners are using to advance equity, not to promote statistical generalization to other agencies or service systems.

Data collected through this information collection do not lend themselves to secondary analysis, and no data sets will be shared publicly. We will not generate a document for public use on how to properly interpret, analyze, and evaluate information from this collection because we are not creating a public-use file.

B8. Contact Persons

Mathematica is conducting this project under contract number 140D0421F0729. Mathematica developed the plans for this information collection. Experts in their respective fields from the ACF Office of Planning, Research, and Evaluation and Mathematica helped design the data collection plan and instrument for which we request clearance.

The following people can answer questions about this information collection effort:

CW-SEED study team:

1. Elizabeth Weigensberg, project director and principal investigator (eweigensberg@mathematica-mpr.com)

2. Roseana Bess, deputy project director (rbess@mathematica-mpr.com)

3. Leonard Burton, co-principal investigator (lburton@cssp.org)

Federal project officers:

1. Christine Fortunato (Christine.Fortunato@acf.hhs.gov)

2. Nicole Denmark (Nicole.Denmark@acf.hhs.gov)

3. Jenessa Malin (Jenessa.Malin@acf.hhs.gov)

Staff from Mathematica and the Center for the Study of Social Policy will be responsible for collecting, processing, and analyzing the information for the Office of Planning, Research, and Evaluation.

Attachments

Instrument

Instrument 1: informational call topic guide

Appendix

Appendix A: Outreach email template

7