FUP Part A Support Statement_082718 Clean

Evaluation of the Family Unification Program

OMB Information Collection Request

New Collection

Supporting Statement

Part A

June 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C St, SW

Washington, DC 20201

Project Officer:

Kathleen Dwyer, Ph.D.

Overview

Status of study: This is a new information collection as part of the Evaluation of the Family Unification Program

What is being evaluated (program and context) and measured: The Family Unification Program (FUP) provides Housing Choice Vouchers to homeless or unstably housed families involved in the child welfare system to prevent removing children into out-of-home care or to increase reunification for children in out-of-home care. Public housing authorities collaborate with public child welfare agencies and the local continuum of care to identify eligible families and provide them with vouchers and additional services. In May 2017, the Department of Housing and Urban Development (HUD) received new funding ($10 million), for the first time since 2010, for incremental voucher assistance through FUP, and in March 2018, HUD received additional new funding ($20 million) for FUP. In April 2018, HUD published the “Family Unification Program (FUP) Notice of Funding Availability for Fiscal Years 2017 and 2018.” While existing FUP vouchers do not turn over frequently enough to support the sample size required for an evaluation, the proposed evaluation takes advantage of the release of a large number of new vouchers at one time. The evaluation will examine how the program is implemented in a sample of 5-10 sites and measure impacts on child welfare outcomes of removals into out-of-home care, reunification of children with their parents, and new reports of abuse and neglect.

Type of study: Impact, measured through a randomized controlled trial (RCT), and implementation.

Utility of the information collection:

• Homelessness and unstable housing is associated with increased child welfare involvement. Lack of stable housing can lead to children being removed from their parents or can be a barrier to reunifying children in out-of-home care with their parents. By providing housing subsidies to families, FUP aims to prevent children’s placement in out-of-home care, promote family reunification for children placed in out-of-home care, and decrease new reports of abuse and neglect. However, only limited evidence exists on whether the program is effective at keeping families together. This evaluation will address this question.

• In the area of child welfare policy and practice, the evidence base is limited, creating a situation in which agencies, at all levels of government, are constrained in their ability to implement evidence-based programs and practices. Only 31 of the 441 programs (7%) catalogued in the California Evidence-Based Clearinghouse for Child Welfare (CEBC) meet the criterion of “well-supported by research,” the highest standard of evidence in the CEBC, and only five of those are rated as having “high” relevance to child welfare systems. This evaluation will contribute to the evidence base for child welfare programs and practices.

• The evaluation will also contribute to HUD’s understanding of how housing can serve as a platform for improving quality of life.

• The findings from this study will support policymakers’ decisions on whether to maintain, expand, or eliminate the program. Furthermore, the implementation study can inform policymakers and local agencies on the elements that contribute to program success.

A. Justification

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for data collection necessary to conduct a rigorous evaluation of the Family Unification Program.

Although considerable research over the past several decades documents the overlap between child welfare involvement and homelessness, very little is known about the effectiveness of housing vouchers aimed at improving child welfare outcomes and reducing housing instability. The Family Unification Program (FUP), a program of the U.S. Department of Housing and Urban Development (HUD), provides child welfare–involved families with permanent Housing Choice Vouchers. The U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research, and Evaluation (OPRE) plans to conduct a rigorous evaluation of FUP to build the evidence on the effectiveness of housing vouchers for child welfare involved families.

We are seeking OMB approval to collect data for the rigorous evaluation. These activities include interviews with agency heads, program leaders, leaders at partner organizations, and parents; focus groups with front-line staff; and collection of program and administrative data. This evaluation is part of a larger project to help ACF build the evidence base in child welfare through rigorous evaluation of programs, practices, and policies. It will also contribute to HUD’s understanding of how housing can serve as a platform for improving quality of life.

Study Background:

The Family Unification Program (FUP), which began in 1990 and is funded by the U.S. Department of Housing and Urban Development (HUD), provides child welfare–involved families with permanent Housing Choice Vouchers. The program provides vouchers to families for whom the lack of adequate housing is a primary factor in (a) the imminent placement of the family’s child, or children, in out-of-home care or (b) the delay in the discharge of the child, or children, to the family from out-of-home-care. The program aims to prevent children’s placement in out-of-home care, promote family reunification for children placed in out-of-home care, and decrease new reports of abuse and neglect. Vouchers may also be provided to youth transitioning from foster care who do not have adequate housing, although this population is not the focus of this evaluation. FUP is administered by local public housing authorities (PHA) in partnership with local public child welfare agencies (PCWA) and local Continuums of Care (CoC). FUP is currently available in 44 states, the District of Columbia, and Puerto Rico.

To date, there is limited evidence on whether or not FUP is effective, including one randomized controlled trial (RCT) and one quasi-experimental design study. In the RCT (Fowler and Chavira 2014), sixty-five families in Chicago that were at risk of family separation because of living circumstances were referred to the Illinois child welfare agency’s housing and cash assistance program. Referred families were randomly assigned to receive FUP vouchers (31 families) or to receive services as usual (34 families). Results suggested the program decreased the likelihood of out-of-home placement from 29 percent to 13 percent over a 10-month follow-up period.

The study that used a quasi-experimental design examined two sites, Portland, Oregon and San Diego, California using a waitlist comparison approach in both sites and a propensity score matching (PSM) approach in Portland (Pergamit, Cunningham, and Hanson, 2017). The waitlist analyses included 326 children in Portland and 502 children in San Diego. The sample in both sites included children living with their parents and children in out of home care. In Portland, the waitlist analyses indicated that children referred to FUP were significantly more likely to have their case closed within 18 months and their cases were closed significantly faster, than children placed on a waiting list. However, no statistically significant differences in removal rates, reunification rates, or new reports of abuse and neglect were found. In San Diego, the waitlist analyses indicated that FUP decreased the time to case closure for preservation families, but not reunification families. The results also indicated that FUP reduced the likelihood of a new report of abuse and neglect for all families. The PSM analyses in Portland only included children in out of home care (N = 1602). The results indicated that FUP increased the probability of reunification for children in out-of-home care relative to the children in the matched comparison group.

To build the evidence for FUP, an RCT evaluation in a location other than Chicago (the location of the only other RCT) is required. In April 2018, HUD released the first Notice of Funding Availability (NOFA) for FUP since FY2010. Because existing FUP vouchers do not turn over frequently enough to support the sample size an evaluation would require, the additional funding presents a unique opportunity for an evaluation of the program through the release of a large number of new vouchers at one time. The proposed study will take advantage of this opportunity to implement an RCT impact study and an implementation study in up to 10 sites across the United States. This study will further ACF’s goal of building the evidence base for programs for child welfare involved families.

Legal or Administrative Requirements that Necessitate the Collection:

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose of Data Collection Procedures

Overview of Purpose and Approach

The purpose of this study is to test, through a rigorous evaluation, the impact of participation in the FUP program on preventing children’s placement in out-of-home care and promoting family reunification for children placed in out-of-home care. The study will have two main components: an impact study to assess the effects of program participation on outcomes of interest and an implementation study to describe and document how the FUP program is implemented in the participating grantee sites. Only administrative data will be collected for the impact study. The implementation study data collection will consist of interviews with parents, interviews and focus groups with staff, and program data collected using forms filled out by staff from organizations involved in implementing FUP.

The data collection instruments contained in this request are central to the successful execution of both components of the study. The data collection for this study will occur in the 36 months after a site receives its FUP vouchers through the 2018 NOFA. Reporting will occur 18 months after data collection is complete. According to the 2018 FUP NOFA, the awards are expected to be made by September 24, 2018 and implementation is expected to begin by October 24, 2018. Below, we describe the research questions, study design, information to be collected, burden estimate, estimation methods, and timeframe.

Research Questions

The impact study seeks to replicate the effects found in the previous RCT (Fowler and Chavira 2014) to build the evidence base on FUP. The proposed study will build on the prior research evidence by examining additional child welfare outcomes. In particular, we will examine the use of FUP for families with children in out of home care, a population not studied in the prior RCT. Further, we will examine a broader set of outcomes including new reports of abuse and neglect and time spent involved in the child welfare system. This additional data will provide a better understanding of FUP’s impact on child welfare involved families. These outcomes were examined in the quasi-experimental study (Pergamit, Cunningham, and Hanson, 2017), but the results were inconsistent across the two sites. The proposed study will reexamine these outcomes without the limitations of the quasi-experimental design.

The core research questions for the impact evaluation include:

• Do FUP vouchers improve child welfare outcomes?

  • Does FUP reduce the probability that a child is removed and placed into out-of-home care (removal)?

  • Does FUP increase the probability that a child in out-of-home care is reunified with the child’s family? Does FUP decrease the time to reunification?

  • Does FUP reduce the number of new reports of child maltreatment?

The program may have impacts on other outcomes beyond preservation and reunification including mediating outcomes. Providing FUP vouchers should reduce homelessness and stabilize housing. These mediating outcomes provide stability for families to engage in the activities they need to undertake to keep their family together or reunify with their children who are in out of home care. Further, through higher and faster rates of reunification and reduction of removals, families may spend less time in the child welfare system. For instance, once a family has reunified they are likely to have their child welfare case closed. Therefore, we are also interested in supplemental impact study research questions:

• Does FUP increase the probability that a child welfare case will be closed?

• Does FUP decrease the amount of time a child welfare case is open?

• Does FUP reduce emergency homeless shelter stays?

The implementation study research questions focus on determining the design and execution of the model at each site. Previous studies of FUP (Cunningham, Pergamit, et al. 2014) have found that there can be substantial variability in how FUP is implemented across sites, with different locations focusing on different target populations; providing different services to supplement FUP; and with varying levels of coordination between the PHA, the PCWA, and other organizations participating in the implementation of FUP. Documenting this variability provides context for understanding the impact study findings as well as helping identify core components of the FUP model.

Our core implementation study questions include:

• Which families are targeted by the public child welfare agency for FUP?

• How is the public child welfare agency identifying eligible families?

• What types of services are provided along with the FUP housing subsidy?

• Which agency provides these services?

• What is the nature and frequency of the services?

• What data are the public housing authority and public child welfare agency collecting as part of the FUP program?

• How is the partnership between the PHA, the PCWA, and the CoC structured?

• What are the major implementation challenges and key facilitators to successful implementation of the model?

• What share of families who receive FUP vouchers sign a lease and maintain their housing?

• What are the barriers and facilitators to a family signing a lease and to maintaining their housing?

• What are the relevant aspects of the local demographic, housing, economic, and service environment?

  • How do these relevant aspects shape the FUP program in each site?

• How do families experience FUP?

• Which families benefit most from the program and under what conditions?

• How do differences across sites in each aspect of their FUP models (target population, identification process, partnerships, housing assistance, case management, support services, and local context) relate to possible outcome differences across sites?

Study Design

The study design involves two concurrent components: an impact study and an implementation study. As noted before, the implementation study provides important context for understanding the impact study results. The two studies will be analyzed simultaneously. As discussed in more detail in part B, we will randomly select up to 10 sites from the PHAs who receive 50 or 100 FUP vouchers through the 2018 NOFA. The results of this study will be generalizable to these sites, though the small number of sites may limit generalizability. Generalizability may be limited further if some sites decide not to allocate enough vouchers to families or are unwilling to participate in the evaluation. Furthermore, these 10 sites may not be representative of smaller PHAs (those receiving fewer than 50 vouchers) or PHAs without sufficient number of eligible families to support a control group. However, this study will provide greater generalizability than the previous RCT conducted in only one site.

The impact study is designed to determine the impacts of FUP on the primary outcomes of family preservation and reunification through a randomized controlled trial (RCT). Randomized controlled trials are widely considered to be the gold standard in measuring the effectiveness of a policy or intervention, therefore the quality of the information obtained by employing this design will be high. In addition, the larger sample size proposed in this RCT than in the previous RCT will lead to more precise impact estimates. Randomization is expected to occur for up to 12 months, when sites will have awarded all of the vouchers received in 2018.

As discussed in more detail in Supporting Statement B (B1) the evaluation team will estimate impacts using both Intent-to-Treat (ITT) and Treatment-on-the-Treated (TOT) analyses. The ITT estimate is defined as the difference between the average outcomes for the treatment group, and the control group, adjusting for pre-randomization covariates. The TOT estimates the impact of the program per family in the treatment group that are referred and receive the intervention. For this evaluation, we define the treated group as those who obtain a lease using a FUP voucher. ITT estimates are of interest to policymakers who want to know whether offering an intervention is effective at addressing the problem it was chosen to solve. TOT estimates are of interest to program and practice stakeholders who want to know how the program impacted those who actually received the services. The impact study will exclusively use administrative data in its analysis, which will come from three sources: the PCWA, the PHA, and the CoC.

The implementation study will capture the differences in each sites context, allowing for interpretation of findings regarding outcomes measured in the impact study. Special attention will be paid in this analysis to determining the differences in FUP implementation across sites and the extent to which sites are implementing the same program model. The implementation study will also focus on “falsifying” the logic model (Epstein and Klerman 2012). This involves mapping information collected to the site’s FUP logic model (see attachment 8 for the general FUP logic model), and then verifying that in practice the program is enrolling the target population, providing the specified services, and achieving the outputs specified in the model. For example, we will check whether families are completing voucher applications, obtaining housing, and staying housed, thus achieving the output of housing stability. This process will allow us to confirm the effective functioning of implementation supports as part of the necessary pathways toward expected outcomes. The implementation study data collection will consist of interviews with parents, interviews and focus groups with staff, and forms collecting program data to be filled out by staff from organizations involved in implementing FUP.

Universe of data collection efforts

Data collection will include phone interviews, three site visits, program data collection, and administrative data collection. A crosswalk of the data to be collected and the research questions is presented in table A1 below. Table A1 also outlines the data collection group, timing, respondents, and type of data collection for each instrument. The implementation study guides were developed based on a past implementation study of FUP (Cunningham et al 2015) and a past implementation study and in-depth parent interviews from an evaluation of a supportive housing program for child welfare involved families (Cunningham et al 2014).

Data will be collected from staff at the primary organizations involved in FUP: the PCWA, the local PHA, the local CoC, and any partnering organizations. Generally, the PCWA is the primary source of referrals and services for FUP, however it is possible that the CoC or another partnering organization that interacts with FUP eligible families could provide referrals and case management for FUP families. We expect some sites to have one or more referring partner organizations while many will have none. For planning purposes, we expect the CoC will be a referring partner in all sites. Based on Cunningham et al. (2015), we expect that 40 percent of sites will have one additional referring partner organization leading to an expected average of 1.4 referring partners per site. We expect many sites will also have one or more partner organizations who provide services. For planning purposes, we expect the CoC will be a service partner in all sites. Based on Cunningham et al. (2015), we expect 50 percent of sites will have zero other service providing partners, 25 percent will have one service partner and 25 percent will have two service partners, for an expected average of about 1.8 service providing partners per site.

Preliminary calls (Appendices A-C):

In the first two weeks after awards are made through the 2018 NOFA, we will conduct phone interviews to collect information relevant for site selection and recruitment (appendix A) and evaluation plan (appendix B and appendix C). Per the 2018 NOFA, awards are expected to be made by September 24, 2018. The phone interviews will focus on understanding how an evaluation can be integrated into the site’s FUP model. These protocols were developed based on past evaluability assessment protocols used in an evaluation of supportive housing for child welfare involved families (Cunningham et al 2014).

First site visit (Appendix D): The first site visit will occur in the two weeks before implementation begins, expected to be October 24, 2018 per the 2018 NOFA. During this site visit, the evaluation team will set up the randomization process and train staff on program data collection. While on site, the evaluation team will interview the PCWA management to gather basic information about the structure of the FUP partnership.

Second site visit (Appendices E-L):

The second site visit will occur 6-9 months after implementation begins. This site visit will focus on fidelity to the referral process and the process of getting families housed.

Program data collection at referral (Appendices O-Q): The evaluation team will use a variety of program data to understand each site’s program model. We propose to have caseworkers across all sites complete two forms as part of program operations.

• The housing status form (appendix O) will collect information on a family’s current housing situation for the families on the caseworkers’ caseloads at the beginning of the evaluation or entering the child welfare system while FUP vouchers are available. This form will be collected on an ongoing basis during the first year of implementation. The information collected by the housing status form is not regularly or systematically collected by public child welfare agencies, and the housing status form is the simplest way to collect these data. This data collection activity is also consistent with the NOFA’s requirement that “…the PCWA’s active caseload is reviewed at least once a month (when the PHA has FUP vouchers available) to identify FUP-eligible families…” This form allows us to know, for each site, the pool of eligible families, what aspect of housing made them eligible, and, in conjunction with the referral form (described below), whether all eligible families are referred to FUP. One of the reasons human services programs can fail to show impacts is lack of fidelity to identifying and/or referring the eligible population (Courtney et al. 2014). By knowing the full pool of eligible families and which families are referred, we can assess fidelity to the targeting criteria. Furthermore, by knowing the aspects of housing that made families eligible, we can assess how sites differ in their eligible populations, essential for assessing external validity. We note, however, that we are proposing that caseworkers fill this form out only once for each family on their caseload, either soon after the project start date or when a new family comes onto their caseload. As a result, we could miss changes in housing status that lead to a family becoming eligible for FUP.

• The referral form (appendix P) collects information on the family being referred including a household roster, housing status, and child welfare status. Housing status may be copied directly from the housing status form. While all FUP programs use some sort of referral form, for some FUP sites, this standard form may include items they would not have otherwise collected. Child welfare agencies can replace their referral form with this form so as not to collect the same information twice. The referral form will only be collected for families referred to FUP.

The housing status form combined with the referral form, allows the evaluation team to assess fidelity to the target population. The housing status form and referral form were developed based on referral forms used by FUP sites studied by Cunningham et al. (2015) and a past implementation study of supportive housing for child welfare involved families (Cunningham et al. 2014). This form will be collected on an ongoing basis during the first year of implementation.

• The randomization tool (appendix Q) is an online system we will have the PCWA’s staff use when a family is referred to FUP. The randomization tool collects the family’s identification number, whether they are referred to prevent removal of a child (preservation) or to facilitate reunification of a child in out-of-home care (reunification), and provides a place for the PCWA to upload the family’s referral form. It randomizes families to either be referred to FUP or receive services as usual and relays this information to the site. This randomization tool was developed based on a past random assignment study of supportive housing for child welfare involved families (Cunningham et al. 2016). The information entered into the randomization tool will be collected on an ongoing basis as families are referred to the program during the first year after program implementation begins.

Third site visit (Appendices K-N): The third site visit will occur 18-21 months after randomization has begun and will focus on the services provided to families while in housing and families’ ability to maintain their housing.

Program data collection after referral (Appendices R-T)

• The dashboard (appendix T) collects information on how the family moves through the referral and leasing process including key dates, such as referral date, voucher issuance date, and lease signing date. The dashboard will allow us to track how families that are randomized to FUP move through the referral pathway and into housing. This will allow us to check that all families randomized to treatment were referred to housing and that no families randomized to control were referred. It will also allow us to monitor whether families are getting housed and how long it is taking them to get housed. The dashboard was developed based on a dashboard used for a past impact study of supportive housing for child welfare involved families (Pergamit et al 2016) and a past evaluation of supportive housing (Cunningham et al. 2016). This dashboard will be collected on a twice monthly basis during the first two years of implementation.

• The housing assistance questionnaire (appendix R) and ongoing services questionnaire (appendix S) collect information on which services the family is receiving through FUP. These data sources allow us to compare the services families receive with the description of services in the site’s logic model. We will also be able to compare service constellations across sites. The housing assistance questionnaire and on-going services questionnaire are newly developed instruments based on the application evaluation criteria in the 2018 FUP NOFA and services observed in FUP sites in Cunningham et al. (2015). These questionnaires have not been tested. The housing assistance questionnaire will be filled out by the PCWA caseworker or other service provider once for each family immediately after a family signs a lease or upon voucher denial. We expect the housing assistance questionnaire to be collected on an ongoing basis as families sign leases, for approximately 18 months after the beginning of program implementation. The on-going services questionnaire will be collected once for each family 6 months after they sign a lease or upon exiting housing if the family exits before 6 months in housing. We expect it to be collected on an ongoing basis as families sign leases and move into housing in the first 24 months after program implementation.

Administrative data (Appendix U)

We will collect administrative data from three sources: PCWA’s administrative data system, the PHA’s administrative data system, and the CoC’s Homelessness Management Information System (HMIS). The list of data elements to be collected from each agency is outlined in the administrative data list (appendix U). This list was developed based on past impact studies of FUP (Pergamit et al. 2017), supportive housing for child welfare involved families (Cunningham et al. 2016), and supportive housing (Cunningham et al. 2016).

The impact analysis will be based on outcomes measured primarily from child welfare administrative data. Baseline data will come from child welfare administrative data on case records, placement histories, and reports of abuse and neglect.

PHA data will indicate whether the family completed a voucher application, whether it was approved (and if denied, the reason for denial), whether the family signed a lease, for how long the family remained housed with their voucher, and the dates for each of these.

HMIS data provide dates of shelter entry for homeless adults and families. HMIS data are generally available from the CoC, which will be a partner agency to implementing FUP along with the PCWA and the PHA.

We will collect administrative data at two points in time: at one year and two years after the last family is randomized (or approximately 24 months and 36 months after implementation). The first data collection provides the baseline and one-year outcome data, prepares the agencies for data file construction for the final round of data collection, allows us to look at outcomes at one year after randomization, and provides an early check of data quality and completeness. The second data collection will be used to measure final program impacts at two years post randomization.

Table A1

Data Collection and Research Question Crosswalk

Data Collection Group	Timing	Appendix	Instrument (Type of Data Collection)	Respondents	Topic	Research Question(s)
Preliminary Phone Calls	2 weeks after grants are awarded	A	Guide for Recruitment with PHA and PCWA Administrators (phone Interview)	PCWA and PHA FUP Management	Recruitment	Determines if site meets selection criteria. Identifies if site has the ability and willingness to participate in an RCT.
		B	Guide to Develop an Evaluation Plan for PCWA FUP Management (phone Interview)	PCWA FUP Management	Evaluation Design	Which families are targeted by the public child welfare agency for FUP? How is the child welfare agency identifying eligible families? What types of services are provided along with FUP housing subsidy? What data is the public child welfare agency collecting as part of the FUP program?
		C	Guide to Develop an Evaluation Plan for PHA FUP Management (phone Interview)	PHA FUP Management	Evaluation Design	What data is the public housing authority collecting as part of the FUP program?
First Site Visit	1 week prior to program implementation	D	Guide for Implementation Study for PCWA Management (in-person interview)	PCWA Management	Program Structure	How is the partnership between the PHA, the PCWA and CoC structured?
Second Site Visit	6-9 months after program implementation	E	Guide for Implementation Study for PHA Management (in-person interview)	PHA Management	Program Structure	How is the partnership between the PHA, the PCWA and CoC structured?
		F	Guide for Implementation Study for CoC Management (in-person interview)	CoC Management	Program Structure	How is the partnership between the PHA, the PCWA and the CoC structured?
		G	Guide for Implementation Study for Referral Provider Administrators (in-person interview)	Referral Provider Management	Referral and Leasing Process	Which families are targeted by the public child welfare agency for FUP? How is the public child welfare agency identifying eligible families?
		H	Guide for Implementation Study with PCWA FUP Management (in-person interview)	PCWA FUP Management	Referral and Leasing Process	How is the partnership between the PHA, the PCWA and the CoC structured? What data is the public child welfare agency collecting as part of the FUP program? Which families are targeted by the public child welfare agency for FUP? How is the public child welfare agency identifying eligible families? What are the major implementation challenges and key facilitators to success?
		I	Guide for Implementation Study for PHA FUP Management (Second) (in-person interview)	PHA FUP Management	Referral and Leasing Process	How is the partnership between the PHA, the PCWA, and the CoC structured? What are the barriers and facilitators to signing a lease and to maintaining their housing?
		J	Guide for Implementation Study Focus Groups for PHA Frontline Workers (focus groups)	PHA Frontline Workers	Referral and Leasing Process	What are the barriers and facilitators to signing a lease and to maintaining the family’s housing?
		K	Guide for Implementation Study for Parents (in-person interview)	Up to 6 parents per site who have signed a lease with FUP	Referral and Leasing Process	How do families experience the program? What types of services are provided along with the FUP housing subsidy? What is the nature and frequency of the services? Which families benefit most from the program and under what conditions?
		L	Guide for Implementation Study Focus Groups with Frontline Workers (focus group)	PCWA Caseworkers and Partner Frontline Staff	Referral and Leasing Process	Which families is the public child welfare agency targeting for FUP? How is the public child welfare agency identifying eligible families? What are the major implementation challenges and key facilitators to successful implementation of the model?
Program data collection after referral Form Form (web)	Ongoing (1-12 months after program implementation)	O	Housing Status Form (form)	PCWA case workers	Eligibility	Which families are targeted by the public child welfare agency for FUP? How is the public child welfare agency identifying eligible families?
		P	Referral Form (form)	PCWA case workers	Eligibility	Which families are targeted by the public child welfare agency for FUP?
		Q	Randomization Tool (form)	PCWA FUP Manager	Randomize eligible families into the treatment or control group
Third Site Visit	18-21 months after program implementation	K	Guide for Implementation Study for Parents (in-person interview)	Up to 6 parents per site who have signed a lease with FUP	Housing Stability and Services	How do families experience the program? What types of services are provided along with the FUP housing subsidy? What is the nature and frequency of the services?
		L	Guide for Implementation Study Focus Groups with Frontline Workers (focus group)	PCWA Caseworkers and Partner Frontline Staff	Housing Stability and Services	What types of services are provided along with the FUP housing subsidy? What is the nature and frequency of the services? What are the major implementation challenges and key facilitators to successful implementation of the model?
		M	Guide for Implementation Study for PCWA FUP Management (Third) (in-person interview)	PCWA FUP Management	Housing Stability and Services	What types of services are provided along with the FUP housing subsidy? Which agency provides these services? What is the nature and frequency of the services? What are the major implementation challenges and key facilitators to successful implementation of the model? What are the barriers and facilitators to a family signing a lease, and maintaining their housing?
		N	Guide for Implementation Study for Service Provider Management (in-person interview)	Service Provider Management	Housing Stability and Services	What types of services are provided along with the FUP housing subsidy? Which agency provides these services? What is the nature and frequency the services?
Program data collection after referral	Ongoing (1-18 months after program implementation)	R	Housing Assistance Questionnaire (form)	PCWA Caseworker or Services Provider Frontline Workers	Housing Assistance Services	What types of services (e.g. financial assistance, housing search) are provided along with the FUP housing subsidy? Which agency provides these services? What is the nature and frequency of the services?
	Ongoing (1-24 months after program implementation)	S	Ongoing Services Questionnaire (form)	PCWA Caseworker or Services Provider Frontline Workers	Ongoing Services	What types of services (e.g. employment, case management, behavioral health) are provided along with the FUP housing subsidy? Which agency provides these services? What is the nature and frequency of the services?
	Ongoing (1-24 months after program implementation)	T	Dashboard (form)	PCWA and PHA FUP Management	Referral, Lease Signing, and Housing Stability	What share of families who receive FUP vouchers sign a lease and maintain their housing?
First administrative data pull	24 months after program implementation	U	Administrative Data List (data pull)	PCWA, PHA and HMIS data administrators	Program, Baseline, and Outcomes Data	Do FUP vouchers improve child welfare outcomes at 1-year post randomization? Do FUP vouchers decrease use of emergency shelters at 1-year post randomization?
Second administrative data pull	36 months after program implementation	U	Administrative Data List (data pull)	PCWA, PHA and HMIS data administrators	Program, Baseline, and Outcomes Data	Do FUP vouchers improve child welfare outcomes at 2-years post randomization? Do FUP vouchers decrease use of emergency shelters at 2-years post randomization?

A3. Improved Information Technology to Reduce Burden

Sites will be able to log into the randomization tool (appendix Q) website and enter the information necessary for randomization. After sites enter a family’s information they will get immediate feedback with the group to which the family was randomly assigned. We will make all other forms, the referral form (appendix P), the dashboard (appendix T), the housing assistance questionnaire (appendix R) and the ongoing services questionnaire (appendix S), available in both an electronic and paper format. Sites will be able to use whichever format they prefer.

With respondents’ permission, the evaluation team will audio record the interviews and focus groups to minimize time needed for potential follow-up to clarify notes.

A4. Efforts to Identify Duplication

The administrative data being collected from the child welfare agencies does not duplicate any information accessible to ACF. The study will make use of data currently reported by states to ACF as part of the Adoption and Foster Care Analysis and Reporting System (AFCARS): OMB Control # 0980-0267 and the National Child Abuse and Neglect Data System (NCANDS): OMB Control # 0980-0229. The information request includes the burden associated with preparing or delivering specific information from these existing administrative data to the evaluation team.

Where the proposed data collection overlaps with information already collected by the child welfare agency, notably the housing status form (appendix O) and the referral form (appendix P), we have designed these forms to substitute for any locally used forms. If the agency prefers to use its own form, it can submit that form and only use the evaluation form for items not covered in their form. Our plan is to work with each site to identify the best way to collect the needed information with the least burden on staff.

For information collected from the same respondents at different points in the evaluation, we have avoided asking questions to collect the same information more than once. When the same information is needed from the same respondent for two different purposes, such as from the referral form and the dashboard, we will provide the means for one form to be used and we will transfer the data as needed. We have designed the data collection instruments so that no two instruments collect the same information, even when addressing the same research question. We note, however, that different respondents may be asked the same questions in order to capture different knowledge and different perspectives. This provides a more robust description of the program model and provides qualitative measures of model fidelity.

A5. Involvement of Small Organizations

It is possible that some of the sites will have partner organizations that are small organizations. The team will minimize the burden on program staff by keeping the interviews and focus groups as short as possible, scheduling the interviews and focus groups at a time most convenient for respondents, holding the interviews and focus groups at the organization’s location, and by not requesting written responses.

A6. Consequences of Less Frequent Data Collection

Instruments such as the referral form, housing assistance questionnaire, the on-going services questionnaire, and dashboard, provide information on individual families and are only collected once per family. Although we plan to conduct multiple interviews or focus groups with some key staff, specifically the PCWA FUP manager, the PHA FUP manager, and the PCWA case workers, each of these interviews is designed to collect different information. The interviews take place at different points in time to obtain the most recent information and relate to the primary activities occurring at different points since the program was implemented. Because agencies can have frequent turnover or practices can change over time, it is important to collect this information in a timely manner. For instance, the second site visit will cover the eligibility and referral process and will take place while families are being referred. If we waited to ask these questions until the third site visit, staff involved in determining eligibility and referrals may have left the agency, leading to less accurate information.

The only data collection at multiple points that will be the same is for child welfare administrative data, public housing authority data, and homeless management information system data which are being collected twice, at one year and two years after the last family is randomized. These data collections serve two different purposes. To eliminate the first of these data collections would risk not catching programmatic and/or data issues in a timely manner.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on Wednesday, January 3, 2018, Volume 83, Number 2, pages 281-382, and provided a sixty-day period for public comment. A copy of this notice is attached as Attachment 2. No comments were received during the notice and comment period.

Consultation with Experts Outside of the Study

To design the study, the evaluation team based the materials on a previous study of FUP (Cunningham, Pergamit, et al. 2014 & Pergamit, Cunningham, and Hanson, 2017). The previous study was a mixed methods study with both an implementation study and a quasi-experimental impact study. The team consulted with one of the previous study’s principal investigators, Mary Cunningham, on the design of this study. The previous study had an advisory committee which was consulted in developing the materials for that study, which included the individuals listed in table A2 below.

Table A2

Advisory Committee

Name	Affiliation	Expertise
Mark Courtney	University of Chicago	Child welfare, also a principal investigator on this current study
Dennis Culhane	University of Pennsylvania	Housing and homelessness
Patrick Fowler	Washington University (St. Louis)	Evaluator of the FUP RCT (Fowler and Chavira 2014)
Rob Geen	Annie E. Casey Foundation	Child welfare
Kirk O’Brien	Casey Family Programs and University of Washington	Child welfare
Susan J. Popkin	Urban Institute	Housing
Debra Rog	Westat	Homelessness; evaluator of the first nonexperimental evaluation of FUP (Rog, Gilbert-Mongelli and Lundy 1998)
Brook Spellman	Abt Associates	Housing and homelessness
Matthew Stagner	Mathematica	Child welfare
Ruth White	National Center on Housing and Child Welfare	Housing and child welfare

A9. Incentives for Respondents

Recall that the proposed study has two main components: an impact study and an implementation study. The impact study relies solely on administrative data, which will be reported to the study team by the public child welfare agency (PCWA), the public housing authority (PHA), and the continuum of care (CoC). No incentives will be offered for this portion of the data collection. All three organizations are required to execute a memorandum of understanding as part of the PHA’s application for funding, and participation in a HUD-sponsored program evaluation, including an evaluation that is financially sponsored and or directed by another agency, is a requirement of the grant program.

The implementation study involves program site visits, including interviews and focus groups with agency heads, program leaders, and staff. We do not propose incentives for these institutional participants, as we do not anticipate systematic non-response that would affect the quality of data we collect. As discussed in Supporting Statement B, we anticipate that once management of a program agree to participate in the evaluation, they will encourage program staff to participate in all study activities.

During the second and third site visits, the implementation study also includes in-depth interviews with parents in the FUP treatment group, in order to understand their experiences with the program, their descriptions of the services provided alongside the FUP housing subsidy, and the nature and frequency of services they receive. These data are not intended to be representative in a statistical sense, in that they will not be used to make statements about the prevalence of experiences in the population. Nonetheless, they may provide valuable anecdotal information that can better help us to understand parent challenges in relation to the FUP program and that cannot be obtained through administrative data or focus group with other potential respondents. We aim to vary parent characteristics as much as possible, to capture a range of possible experiences with program services. The in-depth interviews will take place in person, at a location determined by the family. We estimate interviews will last an average of 1.5 hours, and we propose to offer participating parents a $35 gift card at the conclusion of the interview. The $35 incentive is intended to offset expenses that might prevent some in our target population from participating. We anticipate that $35 will serve as a reasonable amount that is high enough to support participation, but is not so high as to appear coercive for potential participants.

A10. Privacy of Respondents

The information we collect will be kept private to the extent permitted by law. As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access. The contract with the Urban Institute also explicitly requires a data security plan that outlines how the project will store, transfer, and destroy sensitive information as well as the precautions to be taken during each of those activities to ensure the security of those data. Urban will obtain Institutional Review Board (IRB) approval for all data collection under this contract. An Urban Institute developed and IRB approved confidentiality pledge, agreeing to adhere to the data security procedures laid out in the approved IRB submission, will be read and signed during the project training process by all researchers working with the data.

For in person interviews, phone interviews, and focus groups with program staff, Urban will use the informed consent document (attachment 4) to obtain consent for participation in the study from program staff. This form details the risks and benefits of participating and the level of expected privacy for each participant. Program leaders, staff, program partners, and other stakeholders are categories of respondents not designated as vulnerable populations, and the information the evaluation team will collect is not highly sensitive. The team will ask respondents for factual information about their programs (e.g., what the programs do, the number of people they serve, who is eligible, the outreach and referral process). Because some study participants will be local agency or organization leaders, administrators or staff members, and because the team will name the sites in our reports, individuals reading the reports may be able to attribute particular information or comments to that respondent. The evaluation team will tell respondents about this potential risk.

Information collected from all interviews and focus groups will not be maintained in a paper or electronic form that includes the individual’s personal identifier. To achieve this, we will develop a project ID and all notes will use this deidentified ID. We will maintain a crosswalk of project IDs with individual identifiers. The crosswalk will be the only document with the individual’s identifiers and will be maintained separately from any documents that include either personal identifiers or project IDs.

For in-depth interviews with parents, Urban will use the informed consent for parents (attachment 3). This consent statement details the risks and benefits of participating and the level of expected privacy for each participant. Although there are some sensitive questions that will be asked, the questions primarily revolve around the parent’s experience with FUP. Parents will be informed that they may choose not to answer any and all questions during the interview. Information collected during the in-depth interviews with parents will be handled in the same way as with agency personnel, using the project-specific ID created during the referral process.

Our intent is for the information we will collect on FUP participants to be de-identified. We will work with each site to develop a method to assign each family member a project-specific ID as part of the referral process; the ID will not be related to anything identifiable outside the FUP agencies. Personal identifiers appear on the referral form, but we will ask the sites to redact this information before transmitting it as part of the randomization process. The randomization tool, dashboard, and the housing assistance and ongoing services questionnaires will also include only the project-specific ID and not include any identifying information such as name, social security number, address, or phone number. Furthermore, we will request that the project-specific IDs be attached to administrative data and personal identifiers be removed.

Information will not be maintained in a paper or electronic system from which they are actually or directly retrieved by an individuals’ personal identifier.

A11. Sensitive Questions

There are no sensitive questions that will be asked of program staff. The only sensitive questions that will be asked as a part of the data collection are in the guide for the implementation study for parents (appendix K). The goal of these in-depth interviews with parents is to understand how they have experienced the FUP program including what services they received and how FUP has affected their lives. All sensitive questions are asked purely in the context of how the families experience the program. These questions will be used to describe the program from the family’s perspective. The sensitive topics include:

• Mental Health and Substance Abuse Service Receipt. There are questions in the guide that ask whether they received counseling or substance abuse treatment. The guide does not ask about mental health or substance abuse beyond receiving services.

• Child Welfare Involvement. All families in the study will have been child welfare involved. In the guide, we ask questions about how the program has impacted their child welfare case.

• Material Hardship. The guide also covers how the program has impacted the family’s ability to pay for things they need.

Before starting the in-depth interviews, all respondents will be informed that their identities will be kept private and that they do not have to answer any question that makes them uncomfortable. Although such questions may be sensitive for many respondents, they have been successfully asked of similar respondents in other data collection efforts, such as in the in-depth parent interviews conducted for the supportive housing study of child welfare involved families (Cunningham et al. 2014).

A12. Estimation of Information Collection Burden

Burden Hours

Data collection activities will span 3 years.

Implementation study data collection will occur at three points in time: (1) prior to the implementation (“first site visit”), (2) 6–9 months into the implementation (“second site visit”), and (3) 18–21 months into implementation (“third site visit”). Semi-structured interviews will be conducted with agency/organization management (first and second site visits) and FUP management (second and third site visits), and focus groups will be conducted with front-line staff (second and third site visits). In addition, semi-structured interviews will be conducted with parents (second and third site visits). Program data, collected using a housing status form, a referral form and questionnaires about housing assistance and other services, will be completed by frontline staff. FUP management staff will complete an online randomization tool and a form (“dashboard”) to facilitate monitoring of the evaluation.

Table A3

Burden Estimates

Instrument	Total number of respondents	Annual number of respondents	Number of responses per respondent	Average burden hour per response	Annual burden hours	Hourly wage rate	Annual Respondent Cost
Appendix A: Guide for Recruitment with PHA and PCWA Administrators	30	10	1	1.00	10	$33.91	$339.10
Appendix B: Guide to Develop an Evaluation Plan for PCWA FUP Management	10	4	1	1.00	4	$33.91	$135.64
Appendix C: Guide to Develop an Evaluation Plan for PHA FUP Management	10	4	1	1.00	4	$33.91	$135.64
Appendix D: Guide for Implementation Study for PCWA Management	10	4	1	1.00	4	$33.91	$135.64
Appendix E: Guide for Implementation Study for PHA Management	10	4	1	1.00	4	$33.91	$135.64
Appendix F: Guide for Implementation Study for CoC Management	10	4	1	1.00	4	$33.91	$135.64
Appendix G: Guide for Implementation Study for Referral Provider Administrators	14	5	1	1.00	5	$33.91	$169.55
Appendix H: Guide for Implementation Study with PCWA FUP Management (Second)	10	4	1	1.00	4	$33.91	$135.64
Appendix I: Guide for Implementation Study for PHA FUP Management	10	4	1	1.00	4	$33.91	$135.64
Appendix J: Guide for Implementation Study Focus Groups with PHA Frontline Workers	30	10	1	1.50	15	$23.28	$349.20
Appendix K: Guide for Implementation Study for Parents (Second, Third)	120	40	1	1.50	60	$7.25	$435.00
Appendix L: Guide for Implementation Study Focus Groups with Frontline Workers	440	147	1	1.50	221	$23.28	$5,144.88
Appendix M: Guide for Implementation Study for PCWA FUP Management (Third)	10	4	1	1.00	4	$33.91	$135.64
Appendix N: Guide for Implementation Study for Service Provider Management	18	6	1	1.00	6	$33.91	$203.46
Appendix O: Housing Status Form	616	206	31	0.04	255	$23.28	$5,936.40
Appendix P: Referral Form	200	67	6	0.17	68	$23.28	$1,583.04
Appendix Q: Randomization Tool	10	4	106	0.02	9	$33.91	$305.19
Appendix R: Housing Assistance Questionnaire	200	67	3	0.09	18	$23.28	$419.04
Appendix S: Ongoing Services Questionnaire	200	67	3	0.09	18	$23.28	$419.04
Appendix T: Dashboard	20	7	27	0.17	32	$33.38	$1,068.16
Appendix U: Administrative Data List	30	10	2	5.00	100	$23.60	$2,360.00
Totals					849		$19,817.18

Total Annual Cost

The total annual cost burden to respondents is approximately $19,817.18, as shown in table A3. For program leaders and program partners the figure ($33.91/hr) is based on the mean wages for “Social and Community Services Managers;” as reported in the 2017 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates. For program front-line staff the figure ($23.28 /hr) is based on the mean wages for “Child, Family and School Social Workers,” as reported in the 2017 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates. For the compilation and submission of data files, the figure $23.60 /hr) is based on the mean wages for “Statistical Assistants,” as reported in the 2017 U.S. Department of Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates.

Wage data for focus group participants is based on the federal minimum wage of $7.25/hr as set by the U.S. Department of Labor.

A13. Cost Burden to Respondents or Record Keepers

There are no additional costs to respondents.

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request are estimated to be $1,351,037 for an annualized amount of $450,346. The estimate includes the costs of project staff time to draft the discussion guides, collect the information, analyze the responses, and write up the results.

A15. Change in Burden

This is a new data collection.

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

The evaluation of the Family Unification Program (FUP) will consist of an impact study and an implementation study in up to 10 sites. The impact study will employ an RCT. Families will be randomly assigned to be referred either to receive a FUP voucher or to receive services as usual. Data collected for the impact study will come from existing administrative data. The data will be analyzed to determine the impact of the program on child welfare outcomes. In addition, we will explore the effects of the program on the mediating outcome of housing stability as measured by emergency shelter stays. The implementation study data collection will consist of interviews, focus groups, and forms collecting program data to be filled out by staff from organizations involved in implementing FUP as well as existing public housing authority and child welfare administrative data. In addition, data collection will include in-depth interviews with parents. Implementation study data will be analyzed using qualitative and quantitative methods to understand how the program is implemented across sites. See Supporting Statement B for additional information.

Time Schedule and Publication

Table A1 above provides a detailed data collection schedule. According to the 2018 NOFA, grant awards are expected by September 24, 2018 and program implementation is expected to begin October 24, 2018. Pending OMB approval, preliminary phone calls related to recruitment and evaluation plan development will occur within 2 weeks of grant award. The first site visit, which will consist of one interview with program staff and setting up processes necessary for implementing the evaluation, will occur 1 week prior to program implementation. The second site visit, which will consist of interviews with program staff, focus groups with front line workers and interviews with parents, will occur 6-9 months after program implementation. Program data collection prior to referral will occur on an on-going basis from program implementation to 12 months after program implementation. The third site visit, which will again consist of interviews with program staff, focus groups with frontline workers and interviews with parents, will occur 18-21 months after program implementation. Program data collection after referral will occur on an on-going basis from program implementation to 24 months after program implementation. Finally, administrative data collection will occur at 24 months and 36 months after program implementation.

Implementation study analysis will be completed at two points: qualitative data analysis will be completed after the third site visit and quantitative analysis will be completed after the receipt of all program and administrative data. Impact analysis will be completed after the receipt of all child welfare administrative data.

The evaluation will result in three publications:

1. Technical Report describing the study design, implementation findings, and impact findings. Approximately 18 months after data collection is complete.

2. Practitioner-Focused Brief describing evaluation findings of relevance to practitioners and in a manner consistent with a non-technical audience.

Approximately 18 months after data collection is complete.

3. Journal Article that presents the evaluation findings with the information required to be rated by the California Evidence Based Clearinghouse on Child Welfare.

Timing of submission to a journal to be determined after data collection is completed.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

References

Angrist, Joshua, Guido W. Imbens and Donald Rubin. “Identification of Causal Effects Using Instrumental Variables.” Journal of the American Statistical Association. 91.434 (1996): 444-455.

Berlin, Martha, et al. "An experiment in monetary incentives." Proceedings of the Survey Research Methods Section of the American Statistical Association. 1992.

Cunningham, Mary, Michael Pergamit, Maeve Gearing, Simone Zhang, Brent Howell. 2014. Supportive Housing for High-Need Families in the Child Welfare System. Urban Institute. https://www.urban.org/research/publication/supportive-housing-high-need-families-child-welfare-system

Cunningham, Mary, Michael Pergamit, Abigail Baum, Jessica Luna. 2015. Helping Families Involved in the Child Welfare System Achieve Housing Stability: Implementation of the Family Unification Program in Eight Sites. Urban Institute. https://www.urban.org/sites/default/files/publication/41621/2000105-Helping-Families-Involved-in-the-Child-Welfare-System-Achieve-Housing-Stability.pdf

Cunningham, Mary, Michael Pergamit, Sarah Gillespie, Devlin Hanson, Shiva Kooragayala. 2016. Denver Supportive Housing Social Impact Bond Initiative: Evaluation and Research Design. Urban Institute. https://www.urban.org/sites/default/files/publication/79041/2000690-Denver-Supportive-Housing-Social-Impact-Bond-Initiative-Evaluation-and-Research-Design.pdf

Epstein, Diana and Jacob Alex Klerman. “When Is a Program Ready for Rigorous Impact Evaluation? The Role of a Falsifiable Logic Model.” Evaluation Review 60.36 (2012): 375-401.

Fowler, Patrick and Dina Chavira. “Family Unification Program: Housing Services for Homeless Child Welfare-Involved Families.” Housing Policy Debate 24.4 (2014): 802-814.

Fredrickson, Doren D, Teresa L. Jones, Craig A. Molgaard, Claudia G. Carman, Jay Schukman, S. Edwards Dismuke, and Elizabeth Ablah. "Optimal Design Features for Surveying Low-Income Populations." Journal of Health Care for the Poor and Underserved, 16, 4 (2005): 677-690.

James, Jeannine and Richard Bolstein. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” 54.3 (1990): 346-361.

Pergamit, Michael, Mary Cunningham, Julia Gelatt, Devlin Hanson. 2016. Analysis Plan for Interim Impact Study: Supportive Housing for Child Welfare Families Research Partnership. Urban Institute.

https://www.urban.org/sites/default/files/publication/80986/2000802-Analysis-Plan-for-Interim-Impact-Study-Supportive-Housing-for-Child-Welfare-Families-Research-Partnership.pdf

Pergamit, Michael, Mary Cunningham, and Devlin Hanson. "The impact of family unification housing vouchers on child welfare outcomes." American Journal of Community Psychology 60.1-2 (2017): 103-113.

8