A DESCRIPTIVE STUDY OF TRIBAL TEMPORARY ASSISTANCE FOR NEEDY FAMILIES (TANF) PROGRAMS
SUPPORTING STATEMENT B:
Collections of Information Employing Statistical Methods
REQUEST FOR CLEARANCE
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
7th Floor, West Aerospace Building
370 L’Enfant Promenade, SW
Washington, D.C. 20447
October 2012
Respondent Universe.
Given the unique nature of Tribal TANF programs and the level of resources available to the study, it is impractical to select a random sample representative of all 66 Tribal TANF programs currently operating. Using geographic diversity and caseload characteristics, ACF identified 9 Tribal TANF programs to potentially participate in the study. The selection of these 9 programs was based on location, size, years of operation, and information about program operations gleaned from reports generated by the Tribal TANF programs. Discussions will be held with the 9 Tribal TANF Program Directors in addition to ACF Regional TANF Program Managers. These discussions will provide insight on the 9 Tribal TANF programs’ administrative capacity, history of addressing administrative and programmatic challenges, history of innovative approaches, and an overview of issues the contractor could encounter, if any, when contacting the local Tribal TANF programs.
From the list of 9 potential Tribal TANF programs, a mixed purposeful sample of 4 will be selected for case studies. These 4 programs will be selected to ensure that, together, they provide as much diversity as possible. Site selection will be purposive with the goal of selecting for the study Tribal TANF programs that provide information that will allow assessment of the programs’ effects and attributes including problems encountered, lessons learned, and promising approaches, as well as consideration of the replicability of those practices elsewhere.
Working in collaboration with ACF, the research team developed 14 selection dimensions/criteria to be used in selecting the sample of 4 programs. It is impossible for the 4 sites to incorporate variation across all 14 criteria. Rather, the criteria were prioritized and were designed to ensure that, together, the 4 sites provide as much diversity as possible. The primary feature the contractor will look for in a program is evidence of innovative administrative and/or programmatic practices. The contractor will also take into consideration program staff and tribal leadership’s interest in and availability to participate in the study. The 14 selection dimensions/criteria are:
Ability to provide data to the research team as well as depth and breadth of available data. Each of the original nine tribes will be contacted to determine 1) the extent to which they will be willing and able to participate in the study (e.g., IRB and other review and approval, availability for scheduled site visit), and 2) the depth and breadth of program and related data. It is critical that all four of the programs selected be ready, willing, and able to effectively participate in the study.
Size of population served. It is desirable to have variation in this characteristic/attribute so that neither all nor none of the four programs have similar (large, medium, or small) populations served. It is important that the sample include at least one of the two largest programs.
Program maturity (years of existence). It is desirable to have variation in this characteristic/attribute so that neither all nor none of the four programs have similar (high, medium, or low) numbers of years of existence.
Size of service area. It is desirable to have variation in this characteristic/attribute so that neither all nor none of the four programs have service areas of a similar size (large, medium, or small). It is important that the sample include at least one program with a large service area as such a service area represents special challenges.
Level of resources available to the program. Existing data suggest there is variation across programs in resources that are available from the tribe, state(s), and other sources. It is anticipated that the level of resources available may have a dramatic effect on program performance and outcomes. Thus, it is desirable to have variation in the level of resources available so that neither all nor none of the four programs have similar (e.g., high, medium, or low) levels of available resources.
Incorporation of TANF in 477 Programs. It is desirable to have variation in this attribute/characteristic so that neither all nor none of the four programs are incorporated in the 477 program. In some ways, this attribute stands as a proxy for level of integration of TANF with other tribal programs.
Nature of tribal-state relations. It is desirable to have variation in this characteristic so that there is diversity across the four participating programs’ relations (e.g., relatively good or relatively bad) with the state and relevant state programs.
Degree to which the TANF program incorporates tribal culture, values, and institutions. It is desirable to have variation in this attribute/characteristic so that there is diversity across the four participating programs’ levels of cultural incorporation (e.g., high, medium, or low).
Unemployment rate in and near service area. It is desirable to have variation in this attribute/characteristic so that there is diversity across the four participating programs’ unemployment rates (e.g., high, medium, or low).
Region. It is desirable to have variation in this attribute/characteristic so that so that there is diversity in the ACF regions of the four participating programs; preferably each of the four programs will be located in a different ACF region.
Type of service provider (contractor, intertribal consortia, state, other). It is desirable to have variation in this attribute/characteristic so that not all four programs have a single type of service provider.
Type of service area (rural, suburban, urban). It is desirable to have variation in this attribute/characteristic so that not all four programs have the same type of service area.
Program work participation requirements. Under PRWORA, subject to ACF approval, tribes may define acceptable work activities and can set their own adult work participation rates. It is desirable to have variation in this attribute/characteristic so that there is diversity across the four programs participating.
Participation time limits. Under PRWORA, the months of assistance received by an adult residing in Indian country are not counted toward the 60-month limit when the not-employed rate exceeds 50%, in accordance with 42 U.S.C. Y08 (7) (D). It is desirable to have variation in this attribute/characteristic so that some or all of the programs in the study have different participation time limits.
Process for Applying Selection Criteria.
The 14 selection criteria were placed in a matrix (rows) with the 9 potential Tribal TANF programs (columns) (copy in Appendix 4). Information obtained from review of program-related documents, secondary data, and the telephone interviews was applied to the 14 selection criteria. The first screen in the selection process was a program’s availability and capacity to participate in the study. Having passed that screen, site selection involved was an iterative process as the advantages and disadvantages of each site were determined in relation to other potential selections. Consequently, the selection process was represented by a decision tree – if more than one program included in the list of candidates met a key criterion, selection of one of these programs was affected by the relative distribution of other program attributes. For example, one Tribal TANF program may rank higher than another on the selection scale because it has either relatively high or relatively low unemployment--whichever is needed to balance the sample. This approach was applied iteratively to generate one or more balanced samples.
Expected Response Rates
In the on-site, in person interviews with staff of Tribal TANF and other related programs in the mixed purposeful sample of 4 Tribal TANF programs, the contractor anticipates a response rate of 95%. The contractor anticipates this relatively high response rate because the selection criteria include willingness and capacity to participate in the study. Furthermore, the scheduling of site visits is flexible and can, to a considerable degree, be scheduled in accordance with the needs and circumstances of potential respondents. Finally, if necessary, it will often be possible for a coworker to stand in for a person unexpectedly unavailable for the interview.
Focus group participants will be recruited in coordination with Tribal TANF staff. Ten to twelve individuals will be invited to participate in each focus group. The contractor anticipates a 50% response rate which is the norm for this population, yielding focus groups of 5 individuals in each of the 4 sites, for a total of 20 focus group participants.
This section explains the procedures for data collection. The contractor will work closely with the ACF/OPRE staff and Tribal Advisory Group in arranging the data collection activities.
As part of the site selection process, the study team contacted the TANF program director in each of the 9 potential sites. Appendix 5 includes the initial correspondence with the 9 program directors. One of the key purposes of this process was to determine the willingness of the program to participate in the study. Of the 9 tribal TANF programs contacted, only 5 agreed to participate even in preliminary discussions about the study. Based on the overall selection criteria, 4 of the 5 sites were selected to be included in the study and the 5th site was identified as an alternate, in case one of the 4 is not able to participate after all, due to scheduling or other reasons.
For each of the 4 selected sites, the study team is engaging in discussion with the tribal leaders and/or tribal TANF program staff to respect their administrative and cultural needs in the structure of the site visits and to determine the most appropriate individuals to participate in interviews and focus groups.
Interviews with Staff in Tribal TANF and Related Programs as well as focus groups with current and former Tribal TANF recipients. This section describes the plan for conducting in-person interviews in 4 sample sites, including focus groups with recipients, site visit scheduling, and on-site activities.
Respondents will vary in each site depending upon the tribal organization and administration of the Tribal TANF program. For purposes of tailoring discussion guides and planning the visits, the contractor will group respondents into the following general categories:
Tribal TANF officials and other tribal officials responsible for making organizational, policy and major implementation decisions related to TANF, including business/economic development and/or identification of employment opportunities for Tribal TANF clients;
Service delivery managers and staff responsible for implementing Tribal TANF policies and for providing TANF and supportive services to clients;
Managers and staff in related programs as appropriate (e.g., workforce development, education/post-secondary education, social services, child welfare, youth services, child support enforcement);
Recipients of Tribal TANF;
Informed Consent. Verbal consent will be requested. See each instrument for the verbal informed consent language that will be used in the interviews and the interview guides.
Scheduling Site Visits.
The Tribal TANF managers of the 4 programs selected for site visits will be contacted by telephone to discuss the logistical details of the visit. In addition to arranging the logistical details of the visit, the initial contact, conducted with fewer than 10 respondents, will serve two other important functions. First, it will provide an opportunity to answer questions that tribal respondents may have prior to our visit, and second, it will provide a chance for the study team to learn more about the tribal organization. A confirmation of the visit and the schedule of interviews will be provided to the site in advance. The purpose of the outreach effort is to build relationships and effective communication with the sampled tribes early in the study. This is crucial to gaining tribal cooperation for all elements of the data collection so that the project team conducts the fieldwork successfully and in a manner that respects tribal research requirements. Informal Preliminary efforts are already under way to engage the tribes with a known probability of being included in the study sample.
On-Site Activities. Fieldwork will include two and a half day visits to each of the 4 sites in the sample. The primary mode of data and information collection will be a series of semi-structured interviews complemented with focus groups and compilation of administrative data. The site visit teams are composed of experienced staff from the UI and SSI. Each site will be visited by a 2-person research team, one senior and one junior level staff, who will conduct interviews, focus groups, and observations over the course of the two and a half days. The contractor plans to conduct 4-5 interviews per day. The contractor plans to have both site visitors present at each interview whenever possible. The senior staff will lead the interviews and focus groups. The junior staff will assist with documentation and data collection, ensuring that all topics and subtopics are covered. This approach assures more thorough interviews and notes, allowing one interviewer to pursue additional clarification and follow-up questions while the other is taking notes. Having two people listening and recalling the interview is helpful in analysis and interpretation as well.
Quantitative Data Collection Methods. The contractor will gather quantitative data available in Tribal TANF plans and program documents from the four selected sites and some data for all 66 current Tribal TANF programs. All tribes operating TANF programs are required to submit monthly reports to ACF that include a wealth of quantitative data on participation, household characteristics, and administrative expenditures. The contractor will initiate the collection of these reports from the 4 sample tribes by first reaching out to state and regional ACF offices that collect the reports from the Tribal TANF programs.
Additional economic and program information will be collected from sources other than tribe reports, such as state labor market information units in employment security agencies, as well as data from the Department of Health and Human Services, Decennial Census, American Community Survey, County Business Patterns, the Bureau of Labor Statistics, and the Bureau of Indian Affairs. The Urban Institute is uniquely positioned to use the Welfare Rules Database (developed and maintained by the Urban Institute for HHS/ACF) to compare the Tribal TANF policies in the selected study sites with states’ TANF policy choices. The contractor will begin gathering economic and program data from state and federal sources in April 2012.
Outreach and Scheduling. Methods to maximize response rates include: intensive outreach; use of tribal interviewers; extensive training for interviewers, including scheduling procedures and site visit training and incentives for focus group participants. The contractor has extensive experience conducting research in Indian country, they are respectful of tribal sovereignty, the scarce resources and time pressures often experienced by program staff as well as being sensitive to the need to develop relationships, build rapport, and establish effective communication protocols among tribal program staff and clients and members of the study team. The contractor is aware that some tribes may be wary about federally-sponsored research initiatives and may be reluctant to participate in such research including the current effort (Caldwell et al. 2005). The contractor will communicate by phone and in writing with the four selected sites prior to site visits to share information about the study, to schedule the visits, to seek from the selected sites tribal protocols and etiquette to observe while visiting and to address any questions raised by the selected sites. These activities will build on the communication and outreach that will have occurred during the site selection process. Additional input will be welcome from the Tribal Advisory Group with respect to developing and implementing this strategy to ensure effective communication and cooperation.
Time will be needed for internal communication and decision-making with each tribe. Therefore, it is anticipated that outreach will be an iterative process involving engagement, communication and, sometimes, renegotiation.
The contractor will identify and follow tribal IRB procedures. Experience has shown that obtaining approval from tribal Institutional Review Boards (IRBs) can represent a practical barrier to completing research in Indian country within the allowed period of performance and level of resources available. A key task the contractor will accomplish early during the telephone outreach is to understand the requirements of each tribal IRB, if any. If so, telephone interviews will include the head of the tribal IRB to determine the feasibility of securing approval with the time and resources available to the study.
In our experience, the support of the program and other stakeholders can have a dramatic impact on the IRB approval process. For example, if the tribe’s executive branch, TANF/social service program, and/or other related programs support the study, the IRB review and approval process tends to be expedited. Even then, with such support, the IRB process can be lengthy and/or difficult. The UI team is sensitive to the increasingly active role played by tribes in challenging, as well as generating, research and program evaluation. Consistent with tribal sovereignty, self-determination, and self-governance, tribes seek opportunities to influence research agencies and to exercise the power to reject unwanted research on their lands and with their people. The emergence of research requirements by IRBs has supported the influence of tribes on research conducted in their communities. Tribes are concerned not only with informed consent and the need to present information in a participant’s primary language, but also with promotion of collaborative, participatory research models (Caldwell et al. 2005; one of the members of the UI team is a co-author in the Caldwell study).
In some cases it may be necessary for the contractor to go on site to meet with tribal officials and engage in face-to-face discussion on the topics noted above. They may also be asked to make a presentation in person before a Tribal Council, the tribal IRB, a research review committee, or the community. The contractor intends to do so in a manner most conducive to the concerns of each tribe. If a tribal IRB does not approve participation in this study, the contractor will select an alternative site from among the initial 9 potential sites and will seek their participation in the study.
A number of tribes have issued guidelines on conducting research in Indian country, such as providing suggestions to sensitize researchers to local culture, traditions, lifestyles, and research priorities, as well as detailing researchers’ responsibility and accountability to the tribe and its people (e.g., communication and coordination with tribal leaders, negotiation to participate, sharing results, protecting participant and tribal identity, etc.).
Respondent incentives. Focus group participants will be provided a small remuneration of $25 to offset the burden of participation in the focus group. The incentive payment will be provided at the time of the focus group after careful explanation of the procedures for the focus group. Any individual who chooses not to participate after receiving the detailed explanation will be provided the incentive payment so that the payment is not a coercion to participate.
Tribal cooperation or approval rescinded. It has to be recognized that a tribe may change its mind about participating mid-stream. In such cases we (1) gather details on the nature of the tribe‘s concerns, (2) renegotiate with the Tribal Council (through the Principal Investigator), and (3) address concerns as directed by the Tribal Council. In the event that a resolution cannot be reached, then we will replace the site with an alternate site.
Community events. The contractor is mindful that data collection needs to be scheduled around community events and religious ceremonies. It is equally important to recognize that a tragedy, death or crisis in the family or in the community may affect data collection efforts. In such cases we will confer with the tribe about how to proceed in a respectful and responsive manner (e.g., suspending data collection for a limited period of time).
The contractor pre-tested the in-person interview guides to minimize burden and improve utility.
Pretesting the On-Site In-Person Interview Guides. The interview guides were formally pre-tested over the telephone with two Tribal TANF programs (two tribal TANF program directors and one tribal TANF caseworker). These two programs were not among the 4 Tribal TANF programs selected for the study. They came from the remaining Tribal TANF programs. Because the pretests were conducted with fewer than 10 individuals, they were not included in the estimates of burden. The pre-test allowed us to determine if the in-person interview guides would work as planned. The interview protocols were revised as needed based on the pre-test findings.
The changes made as a result of the pre-test are highlighted in the attached versions of the interview guides. Changes included re-ordering questions, clarifying the language of the questions, and adding questions on the following topics at the suggestion of the pre-test respondents.
The relationship of the Tribal TANF program with the ACF regional office:
Communication with other Tribal TANF programs;
The availability of support services;
Coordination with Indian Child Welfare; and
How their community would be different without the Tribal TANF program.
The information for this study is being collected by the Urban Institute and its subcontractor Support Services International, Inc. on behalf of ACF. With ACF oversight, the Urban Institute and Support Services International, Inc. are responsible for developing the pre-tests, telephone interview guides, and in-person interview guides.
The agency responsible for receiving and approving contract deliverables is:
The Office of Planning, Research, and Evaluation (OPRE), Administration for Children and Families (ACF)
U.S. Department of Health and Human Services
ACF/OPRE Contact:
Girley Wright
(202) 401-5070
girley.wright@acf.hhs.gov
The organization responsible for administering the telephone discussions, on-site interviews, and focus groups is:
The Urban Institute
2100 M St. NW
Washington, DC 20037
UI Principal Investigator:
Heather Hahn
(202) 261-5736
Appendix 1: Conceptual Framework
Appendix 2: 60-Day Notice for Federal Register
Appendix 3: Detailed Quantitative Analysis Plan
Appendix 4: Site Selection Matrix and Program Selection Matrix
Appendix 5: Invitation Letter to 9 Tribal TANF Project Directors
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Department of Health and Human Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |