TANF-RCA Supporting Statement_Part B_8-7-15

In this document, we discuss the statistical methods to be used in the initial data collection activities for the Understanding the Intersection Between TANF and Refugee Cash Assistance Services Study (hereafter, TANF-RCA study). This study is sponsored by the Office of Planning, Research and Evaluation (OPRE) in the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS). This submission seeks OMB approval for six data collection instruments that will be used as part of the site selection process and field assessment:

Survey of State Refugee Coordinators and Wilson-Fish Program Coordinators
Site Visit Interview Guide for Public Agency Temporary Assistance for Needy Families Managers and Staff
Site Visit Interview Guide for Public Agency Refugee Cash Assistance Managers and Staff
Site Visit Interview Guide for Voluntary Agency Staff
Site Visit Interview Guide for Other Community-Based Organization Staff
Focus Group Guide for Service Recipients

This justification provides supporting statements for each of the five points outlined in Part B of the OMB guidelines.

B.1 Respondent Universe and Sampling Methods

The respondent universe for this study includes: (1) State Refugee Coordinators and Wilson-Fish Program Coordinators, (2) staff and managers at agencies and organizations providing services to refugees, and (3) individuals who receive refugee services.

B.1.1 State Refugee Coordinators/Wilson-Fish Program Coordinators

The survey of State Refugee Coordinators and Wilson-Fish Program Coordinators will be administered to 51 State Refugee Coordinators representing 49 of the 50 states as well as Washington, DC and San Diego, CA.¹ There are a small number of survey respondents and each state’s policies and practices are unique, so a sample cannot provide a representative description of programs. The survey will be the only data collection effort that has full representativeness. In addition, the survey will be used to inform site selection by helping to identify unique or informative program characteristics. Therefore it is crucial to survey coordinators from each state and San Diego rather than a sample of coordinators. The online survey is included in Appendix A.

B.1.2 Managers and staff at agencies and organizations providing services to refugees

The TANF-RCA study team will purposefully select eight local communities to visit to address the objectives of this study.

Site selection will be a multi-stage process. The study team will start by considering readily-available data from all fifty states and the District of Columbia, to make a preliminary determination of states meeting minimum thresholds for consideration, largely based on the size of the populations of refugees and related groups in the state.

States meeting those minimum criteria will then be the starting point for a “site selection matrix.” The study team will use that matrix to summarize additional information reviewed and gathered during the site selection process. It will focus on a set of site selection criteria (discussed below). The matrix will primarily include qualitative descriptions relevant to each of the site selection criteria.

Rather than selecting sites that provide full representativeness of the characteristics of refugee programs or of refugees served, the research team will select sites that, as a group, will inform ACF’s Office of Refugee Resettlement (ORR) and other stakeholders about a variety of ways state programs operate, and have the potential to provide useful lessons that ORR and states can draw on to improve programs or guide future policy. Towards this end, the selection criteria will focus on ensuring that the programs: a) serve a large enough refugee population to provide lessons applicable to other programs and b) contain variety on a number of dimensions (described below). The research team will also include sites identified as having unique practices or characteristics for serving resettled refugees.

The list of proposed criteria that will be used to select the study sites includes:

Service provision model: Study sites will encompass RCA’s three program arrangements: state-administered, Wilson-Fish, and Public-Private Partnerships. In state-administered programs, state agencies provide cash assistance, medical assistance, and social services; and Voluntary Agency (Volag) affiliates (local organizations that have agreements with the national Volags) provide reception and placement services. In these states, refugees who have children and are income-eligible apply at the local public assistance agency for TANF and Medicaid benefits; refugees without children apply for RCA and Refugee Medical Assistance (RMA). Wilson-Fish programs use private agencies to serve both TANF-eligible and non-TANF eligible refugees. Public-Private Partnership states operate the RCA programs for non-TANF eligible refugees and the TANF program is available for TANF-eligible recipients. These different models are thought to have different implications for how services are delivered, coordinated, or integrated. For example, a Volag providing cash assistance under a Wilson-Fish model may be better able to provide culturally competent services, given their other refugee-focused responsibilities, than a TANF agency serving the general public. It is important for the study to examine programs implemented under the three arrangements to document similarities and variations in service delivery.
Size of the refugee population: The research team plans to include sites with large enough refugee populations and refugee programs to generate lessons relevant for other refugee programs. We will determine this based on the number of “ORR-eligible” individuals in a county (or similar locality); that is, the number of refugees, asylees, and other populations eligible for RCA, TANF, and other related ORR programs on the basis of their status. While the research team would exclude the states with the smallest ORR-eligible populations, they may include one or more states with moderate-sized ORR-eligible populations (and sufficient concentration in one or more counties) instead of just focusing on where the largest ORR-eligible populations are, in the interest of including programs with a variety of characteristics.

Characteristics of the refugee population served: The research team will aim to include sites in which the skill level of the refugee population varies. Examples of other useful dimensions in which to seek variety are the type of ORR-eligible populations (refugee, asyslee, etc.) and English language ability.

Stability of refugee population: States where arrivals of refugees, asylees, and other related populations come from similar national origins from year to year may be able to rely on existing infrastructures of community organizations to help them serve new arrivals. In order to learn about how such programs prepare for new populations and serve them differently from programs where there is the absence of preexisting infrastructure in the community, the research team plans to include states where the national origins of refugees served vary from year to year, in addition to states where there may be more established infrastructure for serving particular groups.
Economic context: The research team plans to include a variety of states with challenging labor markets, as well as places exhibiting stable or better labor market conditions.
Benefit levels: The research team plans to include states and localities with varying cash assistance benefit levels offered through TANF and RCA.
Implementation of innovative or promising strategies: The research team will identify states and localities that are using interesting or innovative strategies to serve refugees and related populations.
Willingness to participate: The research team will work with sites interested in participating in the study. For each of the eight sites, the research team will interview managers and staff at selected organizations and agencies providing services to refugees. Selected organizations will include local TANF agencies; resettlement agencies, such as local affiliates of a Volag; Mutual Assistance Associations or other community-based organizations that serve a recently resettled refugee groups; organizations that provide cash assistance to refugees; and organizations that provide employability and/or social adjustment and cultural orientation services. Interviews with managers and staff at these agencies will provide information about how services are provided, the respective roles of the various agencies and organizations in serving participants, how agencies and organizations integrate services internally and/or collaborate with other organizations, approaches to addressing barriers refugees face, promising practices and strategies in assisting refugees, and gaps in services.

B.1.3 Individuals who receive refugee services

As described above, the study will focus on selecting sites that, as a group, will inform ORR and other states about a variety of ways programs operate, and have the potential to provide useful lessons that ORR and states can draw on to improve programs or guide future policy. Focus groups with program participants will promote discussion among refugees where they can provide their perspective and impressions of the services they received, how those services were delivered, their experiences attempting to achieve self-sufficiency within a rapid timeframe, and the challenges they faced. Individuals selected for focus groups will be recruited from local organizations and agencies selected for the study, as described above. Focus groups will include six to nine participants per site.

Participants will consist of refugees who have recently received cash assistance and associated services through TANF and/or RCA. Focus groups will include individuals with a mix of experiences accessing cash assistance and related services. For example, they will include recently resettled refugees who are still eligible for RCA, and those who were resettled less recently and therefore are no longer eligible for RCA and/or had enough time to potentially transition off cash assistance. As another example, they will include a mix of families eligible for TANF, and adults without children who would need to rely on RCA or an alternative. We will only include people who have received some services in the recent past. We presume they understand why they are eligible or not eligible for certain services.

B.1.4 Target Response Rates

This study uses a qualitative approach to document state refugee policies and programs. As described above, the research team will select sites that, as a group, will inform ORR and others about the variety of ways programs operate and will provide useful lessons for ORR and state and local policy-makers to draw on to improve programs or guide future policy. We expect response rates to be sufficiently high for the survey of State Refugee Coordinators and among the sites selected to provide an in-depth understanding of a range of ways in which states provide services to refugees, and how TANF and RCA services intersect.

We expect the following response rates:

State Refugee Coordinators: The study team has achieved very high completion rates (nearing 100 percent) using similar strategies (described in B.2) on other studies. For example, members of the research team obtained a 100 percent completion rate in a study of the federal 60-month limit on TANF receipt that surveyed state welfare directors. Similarly, members of the proposed study team conducted a survey of Colorado’s TANF county administrators and achieved a 97 percent response rate.

Staff and Managers at agencies and organizations that provide services to refugees: Because the research team will be working with staff and managers who have expressed their willingness to participate, response rates will be close to 100 percent. In the event of scheduling conflicts, the research team will conduct follow-up phone interviews at a time convenient to the respondent.
Individuals who receive refugee services: Again, the research team will seek focus group participants who are interested in participating, and will offer tokens of appreciation for participation. Additionally, we will plan to recruit 8-10 individuals per site to yield focus groups of 6-9 individuals yielding response rates of about 75-90 percent.

B.2 Procedures for Collection of Information

B.2.1 Survey of State Refugee Coordinators

The research team plans to conduct a short online survey of State Refugee Coordinators and Wilson-Fish Program Coordinators, to document the diversity of state policies and practices across the country, and inform site selection. The survey will be administered to all State Refugee Coordinators. The survey (included as Appendix A) will focus on state-level characteristics of refugee programs and on obtaining input from coordinators about how programs are implemented locally in their states. The survey will be programmed by the contractor using an online survey software package (FluidSurvey). ACF will send an email to all State Refugee Coordinators explaining the purpose of the survey and its importance. The research team will then send an email to all coordinators with the URL address and instructions for completing the survey. The research team will track who has begun and completed the survey and send reminders to those who have not completed it. All survey-related correspondence is included in Appendix A.

B 2.2 Staff and Managers at agencies and organizations that provide services to refugees

Upon finalizing the list of potential study sites, the study team will have individualized conversations with each selected state’s State Refugee Coordinator reminding them of the study and its goals, and will emphasizing its importance to ORR and ACF. The team will inform them that they are considering selecting a county or local community within their state to serve as a study site, and will discuss their potential participation and ask for their cooperation. These calls will provide an opportunity for the study team to provide more information to coordinators about the study and what the roles and responsibilities of the participating sites will be, as well as to confirm a site within their state for the purpose of site selection. This will include recommendations for local contacts at the site with whom we would need to work to arrange the site visit.

Once states are selected and local agencies and organizations are identified and dates of the visits are arranged based on the availability of local staff, the study team will interview staff at several agencies and organizations providing services to refugees. Many of the interviews will be in groups. The interviews will be semi-structured, and the research team will customize the guides before the site visits based on the information provided by State Refugee Coordinators and Wilson-Fish Program Coordinators in advance. The guides are included in Appendices B-E.

B 2.3 Individuals who Receive Refugee Services

The research team plans to conduct one focus group per site with six to nine individuals of the same national origin in each group. The study will include refugees who have been in the country between six and twelve months, and who received cash assistance during that period. The study team will work with the local organizations to identify a member of the community to lead the focus group (in the native language spoken by the group) and help recruit focus group participants. The researchers will train the focus group leader and will be on-site to observe the group discussion and provide support as needed. The group discussions will last 60 to 90 minutes. The focus group leader will provide the research team with notes from the group discussion translated into English. The focus group discussion guide is included in Appendix F.

B.2.4 Procedures with Special Populations

Focus groups will consist of refugees who have recently received services, including cash assistance. As described above, given language barriers and issues related to cultural sensitivity, the research team will recruit a member of the community to lead the focus group, and prepare that person for this role. The focus group leader will also provide the research team with notes from the group in English.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

B.3.1 Expected Response Rates

As noted above, we expect response rates close to 100 percent for the online survey of State Refugee Coordinators and for the site visits, and response rates of about 75-90 percent for the focus groups.

B.3.2 Dealing with Nonresponse

As described above, ACF will send an email to all State Refugee Coordinators and Wilson-Fish Coordinators explaining the purpose of the survey and its importance. The research team will carefully monitor response rates and contact non-respondents. Similarly, the research team will reach out to the coordinators to encourage the participation of the agencies and organizations that provide services to refugees at the state and local level.

B.3.3 Maximizing Response Rates

Survey of State Refugee Coordinators

The research team will carefully monitor response rates and data quality on an ongoing basis. The research team will be responsible for contacting non-respondents if a survey has not been opened within the first two weeks, via an email reminder and/or telephone call, and then more frequently as the survey deadline approaches. In addition to providing a reminder, this contact also presents an opportunity for the respondent to complete the instrument over the phone, if desired. The survey website will display the phone number and email address of a member of the research team who can assist respondents in completing the survey.

Tokens of Appreciation

Offering appropriate monetary tokens of appreciation to study participants in appreciation for their time can help ensure a high response rate. Focus group participants will be provided with a $30 gift card after participating in the focus group.

B.4 Tests of Procedures or Methods to be Undertaken

The research team will conduct pilot testing of the online survey with no more than nine State Refugee Coordinators.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The individuals listed in Exhibit B-1 below made a contribution to the design of the evaluation.

Exhibit B-1: Individuals Consulted

Individual	Affiliation	Role in Study
Mary Farrell	MEF Associates	Principal Investigator
Robin Koralek	Abt Associates	Project Director
Sam Elkin	MEF Associates	Instrument Development Task Lead
Karen Gardiner	Abt Associates	Project Quality Advisor
Charles Shipman	Arizona State Refugee Coordinator	Member of Expert Panel
Donna Magnuson	International Rescue Committee (Phoenix)	Member of Expert Panel
Mette Brogden	Lutheran Immigration and Refugee Service	Member of Expert Panel
Vince Kilduff	Maryland Family Investment Administration	Member of Expert Panel
Tom Medina	Washington State Refugee Coordinator	Member of Expert Panel
Liz Schott	Center on Budget and Policy Priorities	Member of Expert Panel
Randy Capps	Migration Policy Institute	Member of Expert Panel

The individuals listed below will be involved in data collection and analyzing data:

Individual	Affiliation	Role in Study
Mary Farrell	MEF Associates	Principal Investigator
Robin Koralek	Abt Associates	Project Director
Sam Elkin	MEF Associates	Field Work Task Lead
Bethany Boland	Abt Associates	Field Work Team Member
Hannah Engle	Abt Associates	Field Work Team Member
Claire Ma	MEF Associates	Field Work Team Member
Deena Schwartz	Abt Associates	Field Work Team Member

1 San Diego operates a Wilson-Fish program separate from the rest of California. Wyoming has no refugee program.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Tiffany McCormack
File Modified	0000-00-00
File Created	2021-01-24

TANF-RCA Supporting Statement_Part B_8-7-15