Supporting Statement A - Formative Generic - Next Generation of Enhanced Employment Strategies Project

Next Generation of Enhanced Employment Strategies Project

OMB Information Collection Request

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part A

February 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer: Hilary Bruck

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for a series of tailored, semi-structured discussions under ACF’s existing formative generic OMB clearance (OMB #0970-0356) to inform the selection of interventions and programs for the Next Generation of Enhanced Employment Services (NextGen) Project. The project, which is sponsored by the Office of Planning, Research, and Evaluation (OPRE), aims to identify and evaluate innovative interventions designed to promote employment and economic security among people with multiple, complex barriers to employment. The NextGen Project could include up to 10 rigorous evaluations.

This generic information collection request (Gen IC) includes initial activities to identify promising interventions, and programs that offer the interventions, to recommend for rigorous evaluation. It is expected to begin upon OMB approval and continue for up to about 12 months. This submission seeks OMB approval for two data collection instruments:

1. Discussion guide for semi-structured conversations with stakeholders about innovative employment interventions and programs

2. Discussion guide for interviews with administrators of programs that are candidates for inclusion in the project

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections: (1) inform the development of ACF research and (2) maintain a research agenda that is rigorous and relevant.

ACF will submit additional information collection requests (ICRs) as part of the NextGen Project. This will include generic information collection requests (Gen ICs) for in-person visits to gather information about potential programs and for formative evaluations of interventions; and full ICRs, for baseline, follow-up, implementation, and cost data collection in support of evaluations and case studies of selected interventions.

Study Background

Many Americans have trouble fulfilling their goal of gainful employment and economic independence because they face complex challenges. These challenges may be physical and mental health conditions, substance misuse, a criminal history, or limited work skills and experience. Still, with the right work opportunities and supports, people facing these challenges may be able to find and keep good jobs.

OPRE has spent decades studying strategies to help low-income people find and keep jobs. To further build this evidence, OPRE contracted with Mathematica Policy Research to conduct the NextGen Project. This project will identify and test up to 10 innovative, promising employment interventions designed to help people facing complex challenges secure a pathway toward economic independence. Additionally, the project is working closely with the Social Security Administration (SSA) to incorporate a focus on employment-related early interventions for individuals with current or foreseeable disabilities who have limited work history and are potential applicants for Supplemental Security Income (SSI).

Complex problems often require diverse solutions. These may include work opportunities, treatments for health conditions, personal and workplace supports, and opportunities to build hard and soft skills. Interventions of interest may be delivered by public-private partnerships, interagency collaborations, government initiatives, nonprofit agencies, or social enterprises (organizations that hire workers who face complex challenges to produce goods or services for commercial sale).

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The purpose of this Gen IC is to help ACF identify promising interventions and programs for rigorous evaluation in the NextGen Project. This request includes two discussion guides that will be used in telephone or in-person interviews with stakeholders and program administrators.

Research Questions

This Gen IC aims to address the following research questions:

• What innovative employment interventions are promising for serving people with multiple, complex barriers to employment?

• Which promising programs might meet the conditions for a rigorous random assignment evaluation?

Study Design and Universe of Data Collection Efforts

The NextGen Project aims to identify interventions and programs that will advance the field’s knowledge of what employment interventions work for people experiencing multiple, complex barriers to employment. It will include up to 10 rigorous evaluations of interventions of interest. Each evaluation will include research on the implementation of the intervention, including how the services were designed and implemented; what contextual, organizational, and other factors impeded or facilitated implementation; characteristics of study participants and the services they received; and the role of employers. Each evaluation will include a rigorous randomized controlled trial (RCT) to determine the intervention’s effects on participants’ employment and other outcomes, based on administrative data and surveys. The project will also include a benefit-cost analysis. A future full ICR will include details about the full study design.

This Gen IC request focuses on collecting information to inform the selection of interventions for the evaluations conducted for the NextGen Project. The information gathering conducted under this Gen IC will inform the development of a full ICR. Under this GenIC, we expect to identify interventions and programs that implement them that are potential candidates for rigorous testing immediately and others that would require some additional developmental work before being rigorously evaluated.

The evaluation team will conduct formative data collection through discussions with select informants. Our approach has three stages, each with its own data collection instrument. We will use an outreach email for initial engagement with potential informants (Attachment C)

1. During Stage 1, we will cast a broad net, having informal, semi-structured, short conversations with a range of stakeholders (Attachment A). These conversations will focus on eliciting stakeholders’ ideas and suggestions for promising interventions to evaluate and promising programs currently implementing those interventions. Key stakeholders include policy makers; program administrators and managers; knowledge brokers and intermediaries; employer and social enterprise representatives; program developers and training and technical assistance providers; advocates for low-income populations and people with disabilities; and researchers. The project team will identify informants for Stage 1 through a purposeful, snowball sampling process that draws from recommendations from experts at Mathematica and OPRE, referrals from other informants, and Internet searches of employment interventions. The project team plans to interview Stage 1 informants through a combination of voluntary phone and in-person discussions conducted one-on-one or in small groups. In identifying and selecting informants, we will contact individuals who will add new information, based on our extant knowledge base and gaps where we hope to gain further insights, to minimize potential burden on Gen IC participants. The project team will engage in further knowledge development activities using publicly available resources for those interventions that seem promising.

2. Based on the knowledge gathered in Stage 1, during Stage 2 we will conduct phone or in-person discussions (Attachment B) with administrators of programs implementing the interventions that were identified in Stage 1. The purpose of these discussions is to learn more about the program’s goals, services provided, and populations served, and to begin to assess the feasibility of an evaluation.

3. For programs that seem promising and evaluable in Stage 2, during Stage 3 we will conduct in-person visits to further explore the quality of the program’s implementation and whether it is a good candidate for a rigorous evaluation. The discussion guides and other materials for Stage 3 visits will be informed by the activities in this current request and will be submitted in a later Gen IC request.

The discussion guides are similar to guides that have been approved and used successfully in previous OPRE studies, including the Job Search Assistance Strategies Evaluation (OMB #0970-0440), Subsidized and Transitional Employment Demonstration (OMB #0970-0413), and Building Evidence on Employment Strategies for Low-Income Families Project (OMB #0970-0356).

A3. Improved Information Technology to Reduce Burden

The project team plans to use improved information technology wherever possible. We will initially reach out to potential informants via email, providing information about the NextGen Project and the purpose of the call. The email is included in this Gen IC request as a supplemental document (Attachment C).

A4. Efforts to Identify Duplication

The project team will begin by reviewing the existing literature before talking to stakeholders to determine which interventions and populations will add to the knowledge base, rather than duplicating what is already known. By reviewing the literature and talking to stakeholders before asking in-depth information of programs, we will ensure that we will not spend informants’ time discussing topics that are already well known or not ultimately informative to advancing knowledge in the field.

A related project sponsored by ACF, called Building Evidence on Employment Strategies for Low-Income Families (BEES), is also conducting program identification and recruitment activities. OPRE will coordinate stakeholder and program outreach across the two projects to leverage information already collected and to avoid duplication. Additionally, the two projects will coordinate to include a set of common instruments and/or questions in their data collection activities. The full ICRs that ACF will submit as part of the NextGen Project will clearly note these areas of overlap.

A5. Involvement of Small Organizations

We expect involvement of small organizations for program identification and selection to be limited. For the most part, the project team will consult with individual stakeholders and program administrators. If the project team reaches out to small organizations, the burden will be minimized for respondents by conducting telephone discussions at times convenient for the respondents, and requiring no record-keeping or written responses on the part of respondents.

A6. Consequences of Less Frequent Data Collection

Information will be collected only once. If formative data collection is necessary to inform future, additional evaluations, we will submit additional Gen ICs specific to those evaluations.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

Formative Generic

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Experts Outside of the Study

The evaluation team will consult select experts in federal agencies before we begin the information collection. These include select agency staff from the Social Security Administration, Department of Health and Human Service, and Department of Labor. We will also consult the BEES project team.

A9. Incentives for Respondents

No incentives for respondents are proposed for this information collection.

A10. Privacy of Respondents

For the semi-structured discussions that are part of the Gen IC, no personal identifying information beyond name and professional affiliation (e.g., name of the academic/research institution, name of the State) will be sought. Discussants will be told that their conversations will be kept private to the fullest extent of the law and that it is expected that their name and affiliation will only be included in summary information provided to ACF. ACF staff may participate in discussions.

A11. Sensitive Questions

There are no sensitive questions in this data collection.

A12. Estimation of Information Collection Burden

The total burden requested under this Gen IC is 69 hours (see table below). The project team expects to conduct semi-structured interviews with 350 stakeholders and 30 program administrators. The discussions with stakeholders are expected to last an average of 0.5 hours; the discussions with program administrators are expected to last an average of 1 hour. The total burden has been annualized over the three-year approval period for the overarching umbrella formative generic clearance.

Total Burden Requested Under this Information Collection

Instrument	Total Number of Respondents	Annual Number of Respondents1	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage	Total Annual Cost
NextGen Discussion Guide for Semi-structured Stakeholder Interviews	350	117	1	0.5	59	$60.47	$3,567.73
NextGen Discussion Guide for Program Administrator Interviews	30	10	1	1.0	10	$49.12	$491.20
Estimated Annual Burden Total					69		$4,058.93

Total Annual Cost

The annualized cost burden to respondents is based on the estimated burden hours and the assumed hourly wage rate for respondents. The assumed wage rate is based on the May 2017 employment and wages from Occupational Employment Statistics survey from the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_stru.htm). The rate used for stakeholders, who include researchers and policy experts, is $60.47; this is equivalent to management, scientific, and technical consulting services under SOC code 19-3011. The rate used for program administrators, $49.12, is equivalent to the local government managers under SOC code 11-1021. The estimated annualized cost is $4,058.93.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $707,277. Annual costs to the Federal government will be $235,759 for the proposed data collection. This has been annualized over the three-year period of approval of the umbrella formative generic clearance.

A15. Change in Burden

This is a new individual information collection under the umbrella generic clearance.

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

The information collected will be used to inform the research design and subsequent data collection efforts for up to 10 evaluations included in the NextGen Project. For this request, discussions with informants will take place upon OMB approval, and are expected to continue for about 12 months. Consistent with the described potential ways we may share information from GenICs under 0970-0356, the data collected under this IC may be shared to inform research design documents or reports; research or technical assistance plans; background materials for technical workgroups; concept maps, process maps, or conceptual frameworks; contextualization of research findings from a follow-up data collection that has full PRA approval; or informational reports to TA providers. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy. Plans for use of later data collected during the study will be further explained in a subsequent package.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Annualized over the three year period of the umbrella generic.

8