Attachment A - Formative Usage Report 0970-0356

Attachment A: use of Formative generic CLEARANCE (0970-0356) From 2015-2018

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

Mary Switzer Building

330 C St., SW

Washington, D.C. 20201

In March 2015, OMB approved ACF’s request to renew the generic information collection for formative data collections (0970-0356). This report describes the use of the generic IC over the three years of approval, including the number of hours used, as well as the nature and results of the activities completed under this generic clearance.

The renewal of the generic IC was approved for three years, during which time ACF requested 18 generic ICs for formative data collection. The use of the formative generic IC has been beneficial to the development and improvement of ACF program and demonstration evaluations. By August 2016, project use of this generic clearance met the original estimate approved in March 2015 and we were approved at that time to increase the burden level from 1600 to 2000 hours. In February and December 2017, we again reached the burden ceiling and again increased the burden level. The last revision to burden under the formative generic clearance set the burden hours to 3600 hours over three years. The increased use is indicative of how useful this formative information collection process has been to informing our research and evaluation projects.

	Date	Project	Request Type	Annual # Responses	Annual # Burden Hours
1	12/30/141	MIECHV Benchmarks Listening Sessions	Gen IC	66	132
2	6/29/15	Strengthening Relationship Education and Marriage Services (STREAMS)	Gen IC	150	270
3	4/22/15; 6/12/15; 7/30/15	Building Bridges and Bonds (B3)	Gen IC, IC Chgs	150	630
4	5/17/16	Early Childhood Training and Technical Assistance (T/TA) Cross-System Evaluation Project	Gen IC	184	276
5	5/18/16	Phase II Evaluation Activities for Implementing a Next Generation Evaluation Agenda for the Chafee Foster Care Independence Program	Gen IC	120	120
-	8/16/16	Change to Increase Burden	Increased burden hours from 1600 to 2000
6	9/9/16	Planning Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)	Gen IC	87	435
-	2/9/17	Change to Increase Burden	Increased burden hours from 2000 to 3000
7	2/25/17	Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education	Gen IC	140	210
8	3/31/17	Program Implementation Design Information from Personal Responsibility Education Program (PREP) Grantees	Gen IC	240	240
9	3/31/17	Formative Data Collections for Culture of Continuous Learning Project: A Breakthrough Series Collaborative for Improving Child Care and Head Start Quality	Gen IC	40	40
10	3/31/17	Supporting and Learning from Child Care and Development Block Grant (CCDBG) Implementation Research and Evaluation: Understanding the Two-Phase Grant Structure to Inform Future Research	Gen IC	64	80
11	9/13/17	Using Concept Mapping to Develop a Theory to Describe the Work of the National Domestic Violence Hotline- – Formative Data Collection	Gen IC	90	105
12	10/27/17; 11/29/17	State and Local Human Services Customer Feedback	Gen IC, IC Chg	193	187
13	11/5/17	Change to Increase Burden	Increased burden hours from 3000 to 3600
14	1/19/18	The Building Evidence on Employment Strategies for Low-Income Families Project (BEES)	Gen IC	55	100
15	1/19/18	Family Level Assessment and State of Home Visiting (FLASH-V)	Gen IC	118	57
16	2/6/18	PREP Studies of Performance Measures and Adult Preparation Subjects (PMAPS)	Gen IC	75	19
17	2/23/18	Assessing Options to Evaluate Long-term Outcomes Using Administrative Data: Targets of Opportunity	Gen IC	20	40
18	3/26/18	Fathers and Continuous Learning in Child Welfare Project	Gen IC	65	135
	Total over 3 Years			1792	3076

Example uses of the Formative Generic Clearance

Building Bridges and Bonds (B3) study

The Building Bridges and Bonds (B3) study is a rigorous evaluation that is designed to test innovative, evidence-informed programming for fathers, with the goal of building practical evidence that can be used to improve services. The evaluation includes both a process and an impact study.

The B3 study used the formative generic clearance to gather information from staff at responsible fatherhood programs on existing services to aid in intervention design. Due to limited information available about the core components of fatherhood programs or what approaches are most effective in improving fathers’ outcomes, the study team gathered information about programs’ current practices and asked for their input on potential research questions to test in B3.

The B3 study used the information collected under the formative generic to inform the design of the process and impact studies, which received OMB approval in September 2016 (0970-0485).

The project benefitted greatly from the generic OMB clearance because it allowed us to get into the field quickly and begin gathering information about the state of the field to inform the larger study. Because of the generic clearance, we were able to speak with practitioners in the field and gain valuable feedback on study design, including what types of interventions programs were currently using and what types might be appropriate to test in our evaluation. This type of information was not readily available via written materials. This information was critical to creating the implementation and impact studies, which were later approved under a full OMB clearance package.

In addition, this study coincided with a new cohort of grant awards, and because of the short turn-around time for approval of a generic package, we were able to keep the evaluation well-aligned with the timing of the grant schedule. A full OMB package would have put us months behind the new grants starting.

Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education

The Variations in Implementation of Quality Interventions (VIQI): Examining the Quality-Child Outcomes Relationship in Child Care and Early Education project will inform policymakers, practitioners, and stakeholders about effective ways to support the quality and effectiveness of early care and education (ECE) centers for promoting young children’s learning and development. The project aims to 1) identify dimensions of quality within ECE settings that are key levers for promoting children’s outcomes; 2) inform what levels of quality are necessary to successfully support children’s developmental gains; 3) identify drivers that facilitate and inhibit successful implementation of interventions aimed at strengthening quality; and 4) understand how these relations vary across different ECE settings, staff, and children.

The VIQI study used the formative generic clearance to collect preliminary information about the landscape of child care and early childhood education (CCEE) programs and policy context at national, regional, state, and local levels. The goal was to develop a detailed understanding of the current CCEE landscape, as well as identify potential feasibility and practical challenges and considerations for carrying out VIQI. The information collected will inform the final design of the pilot and full-scale phases of VIQI, including its research design, recruitment and sampling strategies, and selection of interventions to be tested. An information collection request for the pilot and full-scale phases was submitted November 17, 2017.

The initial phases of data collection using the formative generic clearance yielded important information about possible sites to implement the pilot and full scale studies. Data from landscaping activities revealed important differences across potential study sites that could affect both planned recruitment efforts as well as the likelihood of implementing the two evidence-based interventions to fidelity in the targeted CCEE programs (i.e., Head Start centers and community based child care centers serving children 3- to 5- years, not yet in kindergarten, in mixed-aged classroom). For example, in some of the sites that were explored, conversations with local- and state-level stakeholders revealed that many centers separated 3- and 4-year olds in the assigned classrooms to facilitate the implementation of curricula with those age groups. This practice limits the number of centers that would be eligible to participate in the study because the majority would not be providing CCEE in mixed-aged classrooms. Other information that surfaced from these landscaping activities involved information about existing initiatives to provide professional development to teachers working directly with preschool-aged children, and that involved coaching. These efforts would compete with the professional development model to be implemented in the two groups implementing the study’s interventions and the “business as usual” condition for the control group would involve this competing PD model. Landscaping activities at the local levels in the sites considered also confirmed that center-based CCEE programs differ significantly in the resources available to install the study interventions, highlighting the need for additional funding to ensure appropriate resources are present prior to installation. Overall, the project has benefitted greatly from the opportunity to collect this information and has been able to use it to make important decisions about recruitment strategies and to better understand the contexts that will shape study implementation.

1 Although approved prior to March 2015, burden carried over under the renewed generic since data collection was still in process.