Supporting Statement B - FaMLE final report

Alternative Supporting Statement Instructions for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Fatherhood and Marriage Local Evaluation and Cross-Site Services Components

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

February 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Katie Pahigiannis

Senior Social Science Research Analyst

Pooja Gupta Curtin

Social Science Research Analyst

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

Part B

B1. Objectives

Study Objectives

An important objective of the ACF Office of Planning, Research, and Evaluation’s (OPRE) Fatherhood and Marriage Local Evaluation (FaMLE) Cross-Site project, contracted to Mathematica, is to strengthen the capacity of grantees, working with their own local evaluators. OPRE supports grantees in conducting evaluations that add to the body of evidence on HMRF program effectiveness, operations, and outcomes. The final report templates (the information collection instruments submitted through this request) will help the federal government and FaMLE Cross-Site Mathematica staff provide research and evaluation support to grantees. Specifically, as part FaMLE Cross-Site project, we are submitting this information collection request for the following:

1. To ask HMRF grantees conducting local impact evaluations to complete final report templates that discuss their analyses and main findings. The information request includes:

   1. Impact Evaluation Final Report Template (Instrument 1)

   2. Impact Evaluation Final Report Table Shells (Instrument 2)

2. To ask HMRF grantees conducting local descriptive evaluations to complete a final report template that discusses their analyses and main findings. The information request includes:

1. Descriptive Evaluation Final Report Template (Instrument 3)

2. Descriptive Evaluation Final Report Table Shells (Instrument 4)

The templates have accompanying instructions to guide the grantees in completing them. We have provided these instructions as Appendices A and B.

Generalizability of Results

The set of descriptive evaluation final reports will describe the service populations, their outcomes, and the implementation of the interventions to add to the evidence base on ways to engage and serve populations involved with HMRF programs. These reports are intended to present internally-valid descriptions of the service population and/or the implementation of an intervention in chosen sites, not to promote statistical generalization to other sites or service populations

The set of impact evaluation final reports will produce estimates of impacts of HMRF programs on a range of outcomes, such as attitudes about relationships and knowledge of conflict resolution strategies, parenting behavior, and employment. They will also provide information about the implementation of the programs. These reports are intended to present internally-valid estimates of the intervention’s causal impact, not to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

The templates for the final reports will provide a structure for presenting information about the methods and findings of the local evaluations. This structure will help ACF and the HMRF field to assess whether the impact evaluations were well-executed and the data were analyzed appropriately so they can be confident about the reported findings. It will also help ACF and the HMRF field to readily identify the key findings from the impact and descriptive evaluations, to better understand lessons and insights for the field about the outcomes, implementation, and impacts of healthy marriage and responsible fatherhood programs implemented through the HMRF grant program since 2015.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The target population for this information collection is 33 HMRF grantees with local evaluations: 16 descriptive evaluations and 18 impact evaluations (one grantee is conducting both an impact and a descriptive local evaluation). These 33 grantees will be asked to respond to the information collection.

Sampling

All grantees will be asked to respond because they are each conducting unique evaluations of their specific programs and will require tailored technical assistance based on their evaluation findings.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

The final report templates, the data collection instruments submitted under this request, were informed by HMRF grantees’ accepted analysis plans and reflect best practices in evaluation reporting.

B4. Collection of Data and Quality Control

ACF will send the final report template and instructions to grantees upon OMB approval. ACF and the contractor’s evaluation team technical assistance liaison are in regular communication with grantees and will monitor completion of the final report template. The liaisons will work closely with grantees and their local evaluators to address any challenges they may face in completing their final reports. Both ACF and the contractor’s evaluation team technical assistance liaison will review each final report and provide feedback. Grantees will submit completed final report templates to ACF via email.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

ACF expects all HMRF grantees to produce a final report.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination. ACF may make final reports available on its website.

B7. Data Handling and Analysis

Data Handling

Grantees will complete the templates using the instructions as guidance, and then ACF and FaMLE Cross-Site Mathematica staff will review them and provide detailed feedback designed to mitigate or correct detectable errors.

Data Analysis

As part of the cross-site FaMLE Cross-Site project, ACF may use impact estimates presented in the final impact reports in a meta analysis of impacts of HMRF programs, as approved in a prior OMB package (OMB control number 0970-0460).

Data Use

ACF may make final reports available on its website. In the data collection guidance, grantees are asked to summarize and present content in a table or figure when possible for clear interpretation. While reviewing final reports, the FaMLE Cross-Site technical assistance liaisons will also provide feedback to grantees about whether data analyses were conducted correctly, findings were interpreted correctly, and the appropriate conclusions are drawn based on the analysis presented in the report, and suggestions for how to correct any inaccurately conveyed information.

B8. Contact Person(s)

ACF staff and its contractor, Mathematica Policy Research, will work with grantees to collect and review the data. No consultation was necessary for either sampling or data collection, as no statistical sampling procedures will be employed.

Table 1. Individuals who will collect, process, and analyze the information collected

Name	Affiliation
Megan Reid	Office of Planning, Research, and Evaluation, Administration for Children and Families Megan.Reid@acf.hhs.gov
Seth Chamberlain	Office of Family Assistance, Administration for Children and Families Seth.chamberlain@acf.hhs.gov
Millicent Crawford	Office of Family Assistance, Administration for Children and Families Millicent.crawford@acf.hhs.gov
Phyllis Carroll	Office of Family Assistance, Administration for Children and Families Phyllis.carroll@acf.hhs.gov
Tanya Howell	Office of Family Assistance, Administration for Children and Families Tanya.howell@acf.hhs.gov
Meghan Heffron	Office of Family Assistance, Administration for Children and Families Meghan.heffron@acf.hhs.gov
Toya Joyner	Office of Family Assistance, Administration for Children and Families Toya.joyner@acf.hhs.gov
Barbara Spoor	Office of Family Assistance, Administration for Children and Families Barbara.spoor@acf.hhs.gov
John Allen	Office of Family Assistance, Administration for Children and Families John.Allen@acf.hhs.gov
Jacqueline Proctor	Office of Family Assistance, Administration for Children and Families Jacqueline.proctor@acf.hhs.gov
Christine Ross	Mathematica CRoss@mathematica-mpr.com
Elizabeth Brown	Mathematica EBrown@Mathematica-Mpr.com
Julieta Lugo-Gil	Mathematica JLugo-Gil@mathematica-mpr.com
Sarah Avellar	Mathematica SAvellar@Mathematica-Mpr.com
Adam Dunn	Mathematica ADunn@mathematica-mpr.com
Angela D’Angelo	Mathematica ADAngelo@mathematica-mpr.com
Derekh Cornwell	Mathematica DCornwell@mathematica-mpr.com
Jane Choi	Mathematica JChoi@mathematica-mpr.com
Armando Yanez	Mathematica AYanez@mathematica-mpr.com
Miranda Kharsa	Mathematica AYanez@mathematica-mpr.com

Attachments

• Instrument 1: Final Impact Evaluation Report Text Template for Healthy Marriage and Responsible Fatherhood Grantees

• Instrument 2: Final Impact Evaluation Report Tables Template for Healthy Marriage and Responsible Fatherhood Grantees

• Instrument 3: Final Descriptive Evaluation Report Text Template for Healthy Marriage and Responsible Fatherhood Grantees

• Instrument 4: Final Descriptive Evaluation Report Tables Template for Healthy Marriage and Responsible Fatherhood Grantees

• Appendix A: Instructions for the Final Impact Evaluation Report for Healthy Marriage and Responsible Fatherhood Grantees

• Appendix B: Instructions for the Final Descriptive Evaluation Report for Healthy Marriage and Responsible Fatherhood Grantees

8