Supporting Statement B - BEES Formative Generic (Updated July 2018)

The Building Evidence on Employment Strategies for Low-Income Families Project (BEES)

OMB Information Collection Request

0970-0356

Supporting Statement

Part B

July 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Tiffany McCormack

Carli Wulff

The purpose of the current Generic Information Collection (Gen IC) is to conduct a scan focused on identifying high priority interventions and sites that could participate in rigorous evaluation. Information will be gathered via tailored, semi-structured discussions with national, regional, state, and local informants about employment-based interventions for TANF recipients and other disadvantaged groups, including those facing substance/opioid use disorder and mental health challenges. The scan will inform the research design, recruitment, and sampling strategies used for the first two phases of a research and evaluation study sponsored by the Office of Planning, Research, and Evaluation (OPRE) called The Building Evidence on Employment Strategies for Low-Income Families Project (BEES).

B1. Informant Universe and Sampling Methods

This Gen IC for the BEES project seeks to gather information from key informants in the fields of workforce development, substance abuse, and mental health. First, the evaluation team will develop a list of possible informants based on recommendations from the Contracting Officer’s Representatives (CORs) and other staff in OPRE, referrals from other informants, and Internet searches of employment interventions. Informants in this Gen IC will include:

• Researchers and policy experts in the fields of workforce development, substance abuse, and mental health (25 respondents)

• State and local administrators of systems such as Temporary Assistance for Needy Families (TANF), Supplemental Nutrition Assistance Program (SNAP), Workforce Innovation and Opportunity Act (WIOA), behavioral health, and criminal justice (70 respondents)

• Program staff of employment interventions for TANF recipients and other disadvantaged groups, including those facing substance abuse and mental health challenges (100 respondents).

Next, the evaluation team will use a purposeful, snowball sampling strategy to determine which informants to engage and when. The informants will be selected to participate in an iterative fashion based on their expertise and the study’s need for information from different sources and localities. The team will return consistently to the list of potential informants to determine which individuals with select expertise should be engaged next given the evaluation team’s remaining gaps in knowledge. Overall, we expect that 95 percent of the individuals we contact will participate in a discussion.

B2. Procedures for Collection of Information

Informal, semi-structured discussions with the field assessment participants will be conducted by telephone and in-person. The evaluation team will use the approved discussion guides to help ensure that appropriate topics are covered, given the type of stakeholder being interviewed. Using semi-structured discussions to collect data will ensure that each discussion covers the key topics, while allowing the interviewer to tailor the conversation as needed to efficiently elicit information from each individual respondent. Discussions will be led by senior members of the evaluation team. The evaluation team will obtain relevant written materials that are readily available, and prepare written summaries of each discussion for submission to ACF. These written materials will be for internal purposes.

Discussions under this request will be conducted commencing with the receipt of OMB clearance and will occur from August 2018 to June 2019. The discussions will require up to 415 burden hours total from those being interviewed.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

Participation in this Gen IC is voluntary. In our prior efforts to gather insights and information from key informants in the early design phases of studies, we have found that informants are eager to participate in these types of discussions. We therefore expect to obtain a very high response rate (95 percent) among stakeholders who are contacted for an interview.

Several factors will help ensure a high rate of cooperation among informants. First, senior members of the evaluation team who are familiar with the issues will conduct the discussions. Second, it is expected that staff conducting the discussions will personally know many of the informants through their professional connections. Finally, many states and localities are interested in improving efforts to support employment for low-income families and are actively seeking ideas and evidence to respond to the opioid epidemic, and, therefore, we expect that they will be interested in identifying interventions to be tested in the study.

Dealing with Nonresponse

If a potential informant does not respond to our initial email request (see Attachment D) for a meeting, the evaluation team will send a second, similar email. If we do not receive a response to our second email, the team may follow up with a phone call. After three attempts, the evaluation team will not continue to pursue a response.

Maximizing Response Rates

The evaluation team will be accommodating of informants’ schedules and will be flexible about the meeting dates and times. The informants will be contacted by a senior member of the evaluation team. These senior members have had significant experience in working closely with programs and their stakeholders on previous evaluations and have had a high level of success in engaging programs in preliminary discussions such as these.

B4. Tests of Procedures or Methods to be Undertaken

The discussion guides were approved as part of the original BEES Gen IC and are similar to guides that have been approved and used successfully in prior OPRE studies, including the Job Search Assistance Strategies Evaluation (OMB #0970-0440), Subsidized and Transitional Employment Demonstration (OMB #0970-0413) and the Assets for Independence Evaluation (OMB #0970-0414).

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

This data collection effort will be designed and carried out by senior members of the evaluation team, on behalf of ACF, with ACF oversight and approval. The analysis of the interview data will be qualitative, and statistical methods will not be employed. The key individuals involved in this effort follow:

• Dan Bloom, Policy Area Director, MDRC

• Sue Scrivener, Senior Research Associate, MDRC

• Peter Baird, Senior Research Associate, MDRC

• Megan Millenky, Research Associate, MDRC

• Karin Martinson, Principal Associate, Abt Associates

• Doug McDonald, Principal Associate, Abt Associates

• Mike Fishman, President, MEF Associates

• Mary Farrell, Executive Vice President, MEF Associates

• Tiffany McCormack, Senior Social Science Research Analyst, ACF

• Carli Wulff, Social Science Research Analyst, ACF

4