Jobs Plus OMB_Part B FINAL 6_13_16

Jobs Plus Pilot Evaluation Study

Supporting Statement for Paperwork Reduction Act Submission, Part B

Submitted to:

Office of Management & Budget (OMB)

Submitted by:

U.S. Department of Housing & Urban Development (HUD)

June 13, 2016

B. Collections of Information Employing Statistical Methods

1. Respondent Universe, Sampling Selection, and Expected Response Rates

This Information Collection Request (ICR) includes four data collection instruments: (a) Site Visit Interview Guide Round 1 to be administered to program staff and stakeholders in nine focal developments, including program directors, housing authority senior managers, property management staff, selected frontline staff, resident leaders, community stakeholders, and service partners, (b) Focus Group Discussion Guide Round 1 to be administered to public housing residents in nine target developments, (c) Site Visit Interview Guide Round 2 to be administered to program staff and stakeholders in nine sites, including program directors, housing authority senior managers, property management staff, selected frontline staff, resident leaders, community stakeholders, and service partners, and (d) Focus Group Discussion Guide Round 2 to be administered to public housing residents in nine target developments.

This study seeks to document the start-up of the Jobs Plus Pilot Program, its early outcomes and its costs through a comprehensive process study that compares experiences across nine housing developments. As such, the information collection for this study uses qualitative methods including semi-structured interviewing and focus groups. Semi-structured interviewing is a method used to elicit depth of information of particular processes and/or events. Focus groups are commonly used to gather information from users of a product or service about service delivery, marketing themes or product/service characteristics.

The rational for visiting all nine sites is that we expect the implementation of the Jobs Plus program and its associated outcomes to differ greatly across sites, in ways that make sampling of a limited number unable to develop valid conclusions about the role of program context or service delivery strategy. The program model requires that the grantee implement the three primary components of the Jobs Plus program (employment services, JPEID, and community supports for work), but also gives the grantee a fair amount of latitude in exactly how these components are implemented, allowing the PHA to tailor the program to suit the identified needs of the residents and leverage the partnerships that each community has leveraged for their particular program. In addition to the varying implementation approaches and different community contexts, the PHAs themselves present different environments in which to implement the Jobs Plus program. For example, across the nine sites in the first cohort, there are public housing developments that are also part of the Rental Assistance Demonstration (RAD), Choice Neighborhood planning grant recipients, Sustainable Community designees, and Promise Zone designees. It will be important for HUD to understand the interactions of these different programs, and to understand the extent to which these various place-based programs can be successfully implemented simultaneously.

Because of these various, interacting factors, even grantees with similar “profiles” in one dimension of context – such as having similar types of service partners – may achieve very different outcomes. While in-depth case studies will be helpful to understand implementation processes in more detail, these opportunities for additional data collection will need to be paired with a broader scan across the nine to avoid over-generalization about the role of any one factor in Jobs Plus implementation. That is, by looking at only three sites, the risk is that invalid inferences may be drawn about the role of implementation strategy, service partnerships, or concurrent HUD programs, when it might be the interaction of these factors, or when outcomes may be driven by idiosyncratic reasons such as a high-performing staff member at one site.

HUD sees value in an effort to document the range and interaction of program implementation practices for several other reasons. First, HUD has already awarded funding to a second cohort of grantees, and a third cohort of grantees will be announced in the next several months. Observations from the full set of the first 9 grantees in the first cohort will be valuable in identifying with greater validity those areas where grantees are struggling with or finding success in program implementation, thus enabling HUD to develop more useful technical assistance efforts for subsequent grantees. Second, HUD will be supporting a quasi-experimental outcomes evaluation of the program that will be measuring changes in earned income and work effort among residents of Jobs Plus developments and comparing those changes to a matched comparison group of non-Jobs Plus developments. Observations from the full cohort of Jobs Plus grantees will provide important context for interpreting the findings from the full outcomes study.

Criteria for selection of staff and stakeholders for semi-structured in-person interviews

Participants interviewed during site visits in Round 1 and 2 will be intentionally selected based on their role and contribution to the implementation of the Jobs Plus Pilot program. Potential interviewees include program directors, housing authority senior managers, property management staff, frontline staff, resident leaders, community stakeholders, relevant HUD staff, and service partners.

Criteria for selection of focus groups participants

Focus group participants for Rounds 1 and 2 will be drawn from the pool of program participants in each of the nine developments where the Jobs Plus Pilot is implemented. The potential respondent universe will vary by development (See Table 1). Participants will be selected using each program’s management information system. The sample will be random and outreach will be stratified by groups of interest, including: 1) residents working or not at program enrollment, 2) residents with and without young children, and 3) residents with different levels of engagement in the program. However, given the nature of focus group dynamics and the objectives for data to be collected from this type of qualitative effort, no formal statistical tests will be employed to assess findings or response/non-response biases; instead, this effort is meant to ensure a more balanced range of views to be elicited as part of this data collection method.

There has been no previous data collection for the Jobs Plus Pilot Program.

Expected response rates

As all data on the sample will be derived from in-person interviews and focus groups, we expect high response rates.

2. Procedures for the collection of information

In-person interviews with staff and key stakeholders

The research team will coordinate site visits with senior program staff at each of the developments and with the program managers at HUD. In collaboration with senior staff, the research team will identify program staff that are closely involved in various aspects of program implementation, as well as key stakeholders outside the program, such as service delivery partners or housing authority administrators. The team will contact staff and stakeholders in advance to request an in-person interview during site visits.

Focus groups with program participants

Once the research team draws a sample of potential participants based on the criteria described above, we will contact potential participants via postal mail to invite them to participate in a focus group, specifying time, place, compensation, and confidentiality of participation. (The initial contact will be followed by a phone call to confirm participation. In coordination with program staff, the research team will hold focus groups at a place that is convenient to most participants, most likely within the development.

3. Methods to maximize response rates

During individual interviews with program staff and key stakeholders, as well as during focus groups, participants will be encouraged to share their experiences, informed that there are no consequences for not answering any particular question, and told that their name will not be associated with any information that they provide. Researchers will obtain verbal consent from participants (See Appendix B for Written Statement to Obtain Oral Consent). Based on MDRC’s experience conducting interview and focus groups, individuals who participate in these research activities are interested in sharing their experiences, making non-response less of an issue.

4. Pre-Testing

The protocols are designed to serve as semi-structured interview and focus group guides for discussion with program staff and stakeholders, as well as program participants. The development of these guides have been informed by MDRC and its partners’ (NIMC and CURS) experience conducting qualitative work around Jobs Plus, and their knowledge of Jobs Plus and similar initiatives that focus on public housing developments. Once in the field, we expect researchers to make minor adjustments to the protocols informed by specific contextual factors.

5. Individuals consulted on statistical aspects and individuals collecting and/or analyzing data

MDRC and its subcontractors (NIMC and CURS) are collecting the information for the Jobs Plus Pilot study. With HUD oversight, MDRC and its subcontractors are responsible for developing the study documents included in this submission. Key Staff include:

• David M. Greenberg, MDRC, (202) 340-8812

• Mark Joseph, Case Western Reserve University, (216) 368-5359

• William Rohe, University of North Carolina, (919) 962-3077

Page 3 – Part B Supporting Statement