OMB-report.revised.2-19-07

Contract No.: 233-02-0086/HHSP233200600007T

MPR Reference No.: 6292-310

Supporting Justification
for OMB Clearance
of Data Collection
Instruments for the
Identifying Promising
Temporary Assistance for
Needy Families Diversion
Practices Study

February 19, 2007

Linda Rosenberg

Submitted to: Administration for Children and Families Office of Research Planning and Evaluation 370 L’Enfant Promenade Washington, DC 20447 Project Officer: Michael Dubinsky

Submitted by: Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Linda Rosenberg

CONTENTS

Chapter Page

I JUSTIFICATION 1

A. CIRCUMSTANCES NECESSITATING DATA COLLECTION 1

1. Mail Survey 3

3. Site Visits 5

4. Possible Study Limitation 7

B. HOW, BY WHOM, AND FOR WHAT PURPOSE INFORMATION IS
TO BE USED 8

c. use of automated, electronic, mechanical, and other technological collection techniques 8

d. efforts to avoid duplication of effort 8

e. sensitivity to burden of small entities 9

f. consequences to federal program or policy activities if the collection is not conducted or is conducted less frequently than proposed 10

g. special circumstances 10

h. federal register announcement and consultation 10

1. Federal Register Announcement 10

2. Consultation 10

I. payments of gifts to respondents 11

J. CONFIDENTIALITY OF THE DATA 11

K. ADDITIONAL JUSTIFICATION FOR SENSITIVE QUESTIONS 12

l. estimates of hour burden of the collection of information 12

m. estimate of total annual costs and burden to respondents or record keepers 13

CONTENTS (continued)

Chapter Page

n. estimates of annualized costs to the federal government 13

o. reasons for program changes or adjustments 13

p. plans for tabulation and publication and schedule for the project 14

1. Publication Plans 14

2. Tabulation Plans 14

Q. APPROVAL NOT TO DISPLAY THE EXPIRATION DATA FOR OMB APPROVAL 15

r. exception to the certification statement 15

II collection of information employing statistical
methods 16

a. respondent universe and sampling methods 16

1. Survey and Telephone Interviews 16

2. Site Visits 16

b. statistical methods for sample selection and degree of accuracy needed 17

c. methods to maximize response rate and to deal with nonresponse 17

1. Mail Survey and Telephone Interview 17

2. Site Visits 18

d. test of procedures and methods to be undertaken 19

1. Survey and Telephone Interviews 19

2. Site Visits 19

e. individuals consulted on the statistical aspects
of the design 20

REFERENCES 21

CONTENTS (continued)

Chapter Page

Appendix A: Advance Letter and Questionnaire 22

Appendix B: State and Local Telephone Interview Guides 29

Appendix C: Site Visit Interview Guides 49

Appendix D: Public Comments and Responses 79

TABLES

Table Page

I.1 ESTIMATED RESPONSE BURDEN FOR RESPONDENTS FOR
THE IDENTIFYING PROMISING TANF DIVERSION
PRACTICES STUDY 13

I. JUSTIFICATION

A. CIRCUMSTANCES NECESSITATING DATA COLLECTION

Changes in welfare over the past ten years have led states to implement policies and programs that encourage welfare applicants to participate in work-related activities or find an alternative to welfare. The passage of the 1996 Personal Responsibility and Work Opportunities Reconciliation Act (PRWORA) changed the welfare system in the United States by replacing the Aid to Families with Dependent Children with the Temporary Assistance for Needy Families (TANF) program. PRWORA gave states increased flexibility in designing their welfare programs but also placed a limit of 60 months on recipients’ lifetime receipt of federal welfare benefits and required an increasing percentage of the TANF caseload to be engaged in a work-related activity. By redefining the calculation of the caseload reduction credit that states can receive for reducing the number of families receiving cash assistance, the Deficit Reduction Act of 2005 (DRA) effectively increased the proportion of states’ caseloads required to participate in work-related activities. This change has caused states to reevaluate their welfare programs and to identify new strategies for meeting the needs of poor families seeking cash assistance.

To increase work participation rates, decrease the welfare rolls, and preserve families’ lifetime welfare limit, states have implemented programs designed to divert TANF applicants from the welfare program. The most common type of diversion program implemented in the wake of PRWORA was an offer of a lump-sum payment in lieu of TANF cash benefits to applicants meeting certain criteria, most often those who were work-ready or already employed. Typically, the lump sum is equivalent to several months of welfare receipt, and lump-sum recipients often face a period of ineligibility for TANF. As of 2005, approximately 30 states had a lump-sum program (The Urban Institute 2006).

Another common policy to discourage reliance on welfare is placing requirements on TANF applicants before they are certified for cash assistance. For example, about 18 states require all or certain types of applicants to participate in a job search activity before their TANF applications can be approved (The Urban Institute 2006). States have also implemented additional work-related application requirements to assist applicants in finding work, such as the development of an employability plan or attendance at a work-related orientation session. Through these activities, applicants understand the importance placed on work in the welfare program and may have the opportunity to find employment before their TANF application is approved. Another approach to diverting applicants from TANF is to require applicants to explore alternative community and other resources, ensuring that the TANF program is a program of last resort.

With the passage of the DRA, some states are creating applicant programs with the goal of engaging applicants in work activities before they begin receiving TANF while also providing short term financial assistance. Through these programs states expect to prepare applicants for the work participation requirements before they become eligible for TANF assistance. Since assistance is short term and is considered “non-assistance,” applicants’ time in these programs is not counted toward their lifetime TANF limit and these applicants are not included in the calculation of the state’s work participation rates.

Other states are implementing or considering solely state-funded programs for families that are unlikely to meet the work participation requirements required for those families who receive cash assistance funded with federal TANF or state maintenance-of-effort funds. These state programs may have the benefit of providing targeted families with needed services that are not countable federal work activities and removing these families from the calculation of the state’s work participation rate. In some states, applicants for the TANF program may be referred to these state programs.

In September 2006, the U.S. Department of Health and Human Services (HHS), Administration for Children and Families (ACF) contracted with Mathematica Policy Research, Inc., to document the types of diversion and upfront applicant strategies that states have implemented and their prevalence across the country. The project will provide valuable information on how state and local offices are using strategies to divert TANF applicants or engage them in work activities to meet the work participation requirements under DRA.

The Identifying Promising TANF Diversion Practices study will collect details on these programs, identify promising practices, and learn more about the types of data state and local offices maintain on their programs through the following activities:

• A mail survey of the 50 states and the District of Columbia to document the types of applicant programs and requirements in place

• Interviews with state and local administrators about their strategies regarding applicant diversion and employment-related programs

• Site visits to two local offices in each of three states to gather details on implementation

This Office of Management and Budget (OMB) submission requests approval to conduct these study components.

1. Mail Survey

To document the types of diversion and TANF applicant activities in place across the country, the study team will administer a mail questionnaire to the TANF directors in the 50 states and the District of Columbia (see Appendix A). This survey will enable the project team to provide ACF with an up-to-date count of the states that utilize each type of diversion and applicant program. However, we will not request details of these activities through the survey. Through follow-up telephone interviews, the study team will acquire the specific information needed.

Upon OMB approval, we will mail out the short questionnaire to the TANF directors. We will follow up by telephone as necessary to ensure a 100 percent completion rate. Responses from the 51 questionnaires will be entered into an Excel spreadsheet.

2. State and Local Telephone Interviews

The research team will conduct a follow-up telephone interview with each of the state TANF directors and selected local TANF directors. Through the former, we will learn more about each state’s approach to TANF diversion and applicant programs; through the latter, we will learn more about how implementation of diversion efforts occurs at the local level. The interviews have four objectives:

1. Clarify Key Aspects of Applicant Diversion and Employment Strategies. The interviews will explore the nature of practices by gathering information on the details of key dimensions and by determining whether, how, and why these policies may have changed over the course of the state’s implementation of the TANF program and since passage of DRA.

2. Ascertain the Degree of Standardization or Variation in Practices Within a State. The interviews will seek to understand the extent of discretion granted to local TANF administrators and how widely diversion policies and practices vary across a state.

3. Gather Detailed Information on Implementation of Policies. The interview will explore the typical client flow for a particular program or requirement in order to understand how a client experiences the program, what staff is involved in determining eligibility and delivering services, and whether lessons may be learned from state and local implementation experiences.

4. Determine the Availability and Accessibility of Data on Applicant Diversion and Employability Programs. The interview will investigate whether states and/or localities collect data on client participation and outcomes, what specific data elements are tracked, how the data are collected, whether the data are accessible for analysis, and whether any analysis or reports are available.

Depending on a state’s diversion strategies, the state-level interview will take from 10 to 60 minutes. Based on preliminary information on the implementation of these policies across states, we anticipate that 16 of the interviews will be approximately 20 minutes each, while 35 will average 60 minutes each. Since the local interviews will be targeted to local offices with applicant requirements and/or programs, we estimate that these interviews will last about 60 minutes.

To accommodate the different diversion strategies the state and local offices have implemented, the interview guide consists of several modules (see Appendix B). Each module contains a set of interview questions for the particular type of diversion strategy employed. Thus we will be able to tailor each interview to the types of applicant diversion and employment strategies implemented in the state or local office.

We will schedule each state’s telephone interview after we have received its completed state mail questionnaire so that we can tailor the interview appropriately. Based on information we obtain from the state-level telephone interviews on the extent of variation in diversion practices and the types of activities being implemented across their local offices, we will purposively select approximately 30 local offices (3 from each of 10 states) for interviews. We will select local offices in states that have a diverse range of diversion practices or that offer local office discretion in selecting and implementing diversion strategies.

3. Site Visits

Site visits will offer an opportunity for deeper exploration of the implementation details of diversion programs and the identification of promising practices. We will interview key staff at the local level in three states with innovative applicant diversion and employment programs. Key topics we plan to cover during site visits include the following:

• Development of Innovative Practices. We will ask local administrators about their motivation for adopting a particular practice and the timeline for implementation. We will also investigate how the policies and procedures were developed and changed over time.

• Implementation and Service Delivery. Our interviews will explore in detail the process for implementing and delivering the applicant strategy, including the types of staff members involved, the training provided to staff, and typical experiences among clients.

• Major Challenges and Lessons Learned. To provide a full understanding of promising practices, we will gather information on program administrators’ impressions of the challenges and successes encountered in implementing the practices. We will ask administrators to suggest lessons for other states or localities that may consider adopting similar practices.

• Program Administrative Data. We will make every effort to obtain readily accessible data files, system reports, or analyses that can help answer questions about the level of program participation and the characteristics and outcomes of TANF applicants.

We plan to conduct site visits to three states, visiting two localities in a single state when possible. Our main criterion for selecting local sites will be whether they currently employ an applicant diversion or employment approach that is likely to be of interest to other states and localities. The program should have the potential to serve as a promising practice for possible replication and should have been operating long enough to have resolved initial implementation issues. We also will aim to include a mix of approaches among the local sites visited. We will develop a list of possible locations for site visits by using information gathered through a research review, from interviews with representatives of national organizations and noted welfare researchers. In addition, we will use the pretests of the survey instrument as an opportunity to learn more about states and localities that appear to be particularly strong candidates for site visits. Based on these criteria, we expect that states selected for site visits will have work-related application requirements and/or a pre-TANF job search program.

The site visits will include semi-structured interviews with a variety of agency staff members, including directors, mid-level managers responsible for implementing diversion practices, and line staff who screen applicants for participation in applicant diversion or employment programs and/or deliver services to applicants. Site visitors will tailor the site visit interview guide for each interview to ensure the questions reflect the experience and responsibilities of the person being interviewed (see Appendix C). If partner organizations are involved in the practice, we will seek to interview representatives of those organizations as well. Finally, we will observe the delivery of services related to applicant diversion and employment efforts—such as individual intake (or screening) meetings, group orientations, or required group job search sessions or workshops—in order to gain as much insight as possible into how the diversion approach functions in practice and is experienced by potential TANF applicants.

Two project team members, the project director and a research analyst, will jointly conduct each of the visits to the two local offices within the three selected states. In a typical day, site visitors will aim to conduct a total of four to five interviews plus observations of program activities. In general, both staff members will be present at any given interview. The two-person approach increases the effectiveness of the probing during the interview and allows visitors to compare notes and impressions following the interviews to ensure that all information is captured accurately. At the same time, a pair of staff members on site allows flexibility in the site visit schedule, should it be necessary.

4. Possible Study Limitation

The study’s main limitation is that it will not evaluate the effectiveness of state and local offices’ applicant diversion and employment programs. While the study will be able to identify and describe innovative and potentially promising practices, it will not determine which approaches are most effective at diverting applicants or preparing them for TANF’s work-related requirements. In addition, although the study will attempt to collect administrative data on these programs, it will not be able to report the outcomes for participants of these programs.

B. HOW, BY WHOM, AND FOR WHAT PURPOSE INFORMATION IS TO BE USED

This study will inform ACF and states of the types of applicant diversion and employability programs in place across the country. Specifically, the study aims to describe states and local offices’ approaches to and experiences with diverting applicants from TANF or requiring their participation in work-related activities. This information may inform these offices’ decisions about policies and implementation guidelines for their own applicant programs.

c. use of automated, electronic, mechanical, and other technological collection techniques

The study will not use any technological techniques to collect data from the programs.

d. efforts to avoid duplication of effort

This study will enhance existing efforts to document state and local offices’ use of TANF applicant diversion and employment strategies. Through the Welfare Rules Database (WRD), the Urban Institute has documented state rules with regard to particular TANF policies. While the database contains information on states’ TANF diversion strategies and applicant job search requirements, it does not describe states’ overall approach to diverting applicants and engaging them in work activities. Additionally, the WRD does not capture all types of diversion activities such as applicant job search programs that provide short-term cash assistance or requirements that applicants complete an employability plan or attend a work-related orientation. This study will cover a broader range of diversion strategies and provide more analysis of the states’ goals for their programs, whether the strategies are meeting intended goals, and other approaches they are considering in the wake of DRA.

To avoid duplicating the work of the Urban Institute, wherever possible, we will use the data it has collected on these programs so that we do not ask states or localities for information already collected. For example, in interviewing an administrator in a state with a known lump-sum payment program, we will summarize the information in the WRD database and ask the respondent to confirm the information, rather than asking the respondent to relate all the policy details.

e. sensitivity to burden of small entities

The information requested is the minimum required to meet the study objectives. The burden on state and local offices has been minimized as much as possible by designing an approach to data collection that intrudes only minimally. First, state TANF directors will be asked to complete a short, non-labor intensive questionnaire to help frame the rest of the study. Then, based on the results, we will tailor subsequent telephone interviews to ask each state director only those questions that are appropriate for his or her state context. We will schedule the telephone interviews for a time most convenient for the respondent.

Before each round of site visits, the research team member who will be leading the visit will contact the local TANF director to explain the purpose of and review possible dates for the visit. We will offer directors alternative dates and allow them to select the dates most convenient for program staff. Following this initial contact, we will send the director a letter that details what we hope to accomplish during the visit, who we need to interview, the approximate amount of time needed for each interview, and the amount of time needed for the discussions.

f. consequences to federal program or policy activities if the collection is not conducted or is conducted less frequently than proposed

The data collected in this study will provide important information to understand how states use TANF diversion and applicant strategies, especially in response to the new requirements contained in the DRA. If these data were not collected, ACF would not have a comprehensive understanding of states’ strategies to divert TANF applicants from public assistance or to prepare them for the TANF work-participation requirements.

g. special circumstances

There are no special circumstances.

h. federal register announcement and consultation

1. Federal Register Announcement

The initial Federal Register announcement was printed on November 16, 2006 in Volume 71, No. 221, p. 66786. The second notice was printed on February 13, 2007 in Volume 72, No. 29, pp. 6736-6737. For additional information, see the Office of the Secretary Certification Statement.

2. Consultation

A number of individuals beyond the study team have been consulted on the design of the study and the development of the data collection instruments. Their names, organizational affiliations, and telephone numbers are listed below:

• Naomi Goldstein, ACF (202-401-9220)

• Karl Koerper, ACF (202-401-4535)

• Michael Dubinsky, ACF (202-401-3442)

• Peter Germanis, ACF (202-401-6495)

• Gretchen Kirby, MPR (202-484-3470)

• Liz Schott, Center on Budget and Policy Priorities (202-408-1080)

I. payments of gifts to respondents

No payments will be made to respondents for their participation in this study.

J. CONFIDENTIALITY OF THE DATA

This study is being conducted in accordance with all relevant regulations and requirements, including the Privacy Act of 1974 (5USC 552a), the Privacy Act Regulations (34 CFR Part 5b), and the Freedom of Information Act (5 CFR 552) and related regulations (41 CFR Part 1-1, 45 CFR Part 5b, and 40 CFR 44502). In our introductions to each telephone or in-person interview, we will assure respondents that (1) their participation in the study is voluntary; (2) their name will never be used in reporting the results, and answers will not be released in any manner that would enable someone to identify them; (3) the information they provide will be held in strict confidence and used only for the study; and (4) the information they provide will not be used for monitoring or accountability purposes and will in no way affect their employment. To further put respondents at ease, we will assure them that our study is not evaluating state or local TANF offices but is collecting information to understand how diversion programs are being used and implemented. In past research studies, this approach has helped elicit useful information from respondents about their perceptions and opinions on key topics.

Project staff members responsible for the data collection will be fully informed of MPR’s policies and procedures regarding confidentiality of data. They will be trained to remind survey and site visit respondents of these procedures as necessary. MPR routinely employs the following safeguards to ensure confidentiality:

• All employees at MPR sign a confidentiality pledge that emphasizes the importance of confidentiality and describes their obligations.

• Access to personal identifying information with regard to the respondent universe for a survey and to completed hardcopy survey instruments is strictly limited to MPR staff directly involved in the survey. Documents are stored in locked files and cabinets. Discarded material is shredded.

• Computer files are protected with passwords, and access is limited to specific users. All identifying respondent information is stripped from electronic data analysis files. In addition, data are maintained on removable storage devices that are kept physically secure in when not in use.

• Data confidentiality and security at MPR’s survey operation center is further ensured by code access to the center, lockable storage areas for sensitive documents, and controlled access to computerized files and systems.

K. ADDITIONAL JUSTIFICATION FOR SENSITIVE QUESTIONS

We are not collecting any sensitive data. We will ask state and local respondents about the details of their applicant programs and requirements. We also will ask about their opinions of these policies, how they believe they affect applicants’ decision to enroll or not enroll in welfare, and the successes and challenges of these policies.

l. estimates of hour burden of the collection of information

We estimate the total respondent burden for the Identifying Promising TANF Diversion Practices study to be 137.63 hours in 2007—16.83 hours for the mail survey, 69.8 hours for telephone interviews of state and local TANF administrators, and 51 hours for the in-person site visits (Table I.1). Based on the pretest of the mail questionnaire of state TANF directors, we are estimating that the questionnaire will take an average of 20 minutes to complete. We assume that the telephone interview will average 60 minutes for about 70 percent of states and 18 minutes for the other 30 percent. Interviews with approximately 30 local offices will take no more than 60 minutes to complete. For the in-person site visits we are assuming that there will be an average of 8 respondents per site (with a maximum of 6 sites) and that most interviews will take 60 minutes to complete. All data collection will occur in 2007.

TABLE I.1

estimated response burden for respondents for the identifying
promising tanf diversion practices study

	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (Hours)	Annual Burden (Hours)
Mail Questionnaire	51	1	.33	17.00
State Director Interview	51	1	.80	39.80
Local State Director Interview	30	1	1.00	30.00
Site Visit
Administrator Interview	6	1	1.50	9.00
Supervisor Interview	12	1	1.00	12.00
Line Staff	18	1	1.00	18.00
Partner Organization	12	1	1.00	12.00
Total for 2007				137.80

m. estimate of total annual costs and burden to respondents or record keepers

State and local TANF offices will not incur any financial costs for participating in evaluation activities.

n. estimates of annualized costs to the federal government

The estimated cost to the federal government through September 2007 of the Identifying Promising TANF Diversion Strategies study—including designing and administering the data collection instruments; collecting, processing, and analyzing the data; and preparing reports summarizing the results—is $288,028. This estimate is based on MPR’s previous experience managing data collection efforts of these types.

o. reasons for program changes or adjustments

This is a new data collection effort.

p. plans for tabulation and publication and schedule for the project

1. Publication Plans

As part of this data collection, we will produce one final report. The final study report, expected to be delivered in September 2007, will summarize findings from all of the data collection activities—the mail survey, state and local TANF office interviews, and local TANF office site visits. To supplement dissemination of these reports, team members will also seek to present their research at several professional conferences.

2. Tabulation Plans

a. Survey and Telephone Interviews

Based on survey and interview data with state and local TANF administrators, we will develop spreadsheets with important features of the state and local programs. Using these spreadsheets, we will be able to summarize information about these programs across state and local TANF offices.

b. Site Visits

Soon after each site visit is completed, one member of the site visit team will organize and consolidate his or her notes, consulting with the other team member as necessary. During the consolidation process, the designated staff member can identify any ambiguities or missing data elements for resolution through brief follow-up telephone calls. We will compile and synthesize all the collected information into a site-specific, internal memorandum that will be used for the analysis and writing of the final report. Each memorandum will follow a uniform outline based on the key study topics and will draw on information obtained from interviews; any program observations; and documents obtained before, during, and after the site visits. The memorandum will highlight points of disagreement and close agreement among data sources.

To facilitate cross-site comparisons, we will construct tables that summarize key findings for each site. In qualitative research, the identification of themes or categories is a standard method of data analysis (Yin 1994). In addition, the project staff members will meet after each site visit to debrief and consider what we have learned with respect to the key research questions. The meetings will provide an opportunity to discern common themes emerging from the site visits and generate ideas for information to be highlighted in the final report.

Q. APPROVAL NOT TO DISPLAY THE EXPIRATION DATA FOR OMB APPROVAL

All study materials will display the OMB expiration date.

r. exception to the certification statement

No exceptions to the certification statement are requested.

II. collection of information employing statistical methods

a. respondent universe and sampling methods

1. Survey and Telephone Interviews

We plan to administer the mail questionnaire to all 50 states and the District of Columbia. Similarly, all states and the District of Columbia will participate in the follow-up telephone interview. However, the length of the interview will depend on the number and types of TANF applicant diversion and employment programs implemented.

We will purposively select the 30 local TANF offices to interview based on the state interviews. First, through the state interviews, we will identify which states allow local offices to determine (1) the types of applicant programs to implement or (2) how to implement programs that have been authorized by the state. In our interviews with states that do give local offices latitude, we will ask states for their recommendations of local offices to include in the telephone interviews and the later site visits. Then, based on this information, we will purposively select local offices to interview. Our expectation is that our state interviews will indicate that about 10 states grant broad local discretion, and we will purposefully select 3 local offices in each that represent different approaches.

2. Site Visits

We will select three states to include in the site visits purposively based on the state interviews, a review of available public documents and data, and conversations with and recommendations of colleagues in the welfare policy and research fields. We will include in the study those states that allow local offices discretion in how or what kinds of programs they implement and states that represent different applicant diversion and employment strategies. We will also identify state and local offices that have recently changed their programs and policies, especially if they have changed them in response to the DRA. We will then contact the TANF administrator in each of the three selected states to inform him or her that two local offices have been selected for the site visit component of the study and enlist his or her support in eliciting the participation of the local offices.

Once the local offices agree to participate, we will arrange convenient times to conduct the visits. Site visit respondents include local office program administrators, case managers, eligibility workers, other front-line staff who have responsibilities in the applicant programs and requirements, and administrators and front-line staff at contracted service providers and community-based organizations with responsibilities for these programs. We will ask each site to designate one contact to serve as site visit coordinator with whom we can discuss the appropriate number and combination of staff members to include in each site visit interview.

b. statistical methods for sample selection and degree of accuracy needed

There will be no statistical methods used to select the sample of survey and telephone interview respondents or case study sites.

c. methods to maximize response rate and to deal with nonresponse

1. Mail Survey and Telephone Interview

Achieving a high response rate in the mail survey and telephone interview is essential in order to capture a complete picture of states’ TANF applicant diversion and employment strategies, and we will take special steps to assure success. First, MPR staff will send an advance letter to state TANF directors to inform them about the study and the survey stages (see Appendix A). The letter will explain the study’s purpose and sponsors, describe what the study will entail, and assure the confidentiality of respondents. Administrators of selected local TANF offices will receive a similar letter. In addition, as we identify the local welfare offices for the telephone interview, we will inform both the state TANF director and the local offices of the selection.

Second, telephone interviews will be conducted by MPR researchers who have experience conducting interviews and are familiar with TANF programs. They are particularly attuned to addressing respondent concerns about the study and encouraging participation without alienating potential respondents. For TANF administrators who are reluctant to participate because of lack of time, interviewers will stress that the survey may be done at their convenience and over several appointment times. For those concerned about confidentiality, interviewers will explain MPR’s confidentiality procedures.

2. Site Visits

We will take several steps to maximize response rates during the site visits. First, we will attempt to increase response and cooperation by guaranteeing confidentiality to all potential respondents. Before each interview, we will assure respondents that (1) their participation in the study is voluntary, (2) their name will never be used in reporting the results and answers will not be released in any manner that would enable someone to identify them, (3) the information they provide will be held in strict confidence and used only for the study, and (4) the information they provide will not be used for monitoring or accountability purposes and will in no way affect their employment.

Second, we will attempt to reduce nonresponse due to scheduling conflicts or lack of time through advance planning. All site visits will be scheduled at least one month in advance, allowing time to identify the important respondents, contact them individually, and gain assurance of their availability. We will give each site visit coordinator a detailed site visit schedule and request that he or she (1) share copies of the schedule with each respondent well in advance of the visit and (2) alert MPR staff to any problems in individuals’ schedules before the visit so that alternate respondents can be identified and scheduled.

Finally, we will attempt to reduce nonresponse for other reasons by distributing to each respondent, in advance of each site visit, hardcopy materials that explain the study and its sponsors and encourage participation.

d. test of procedures and methods to be undertaken

1. Survey and Telephone Interviews

Prior to this OMB submission, we conducted a pretest of the mail questionnaire and telephone interviews with seven states. Following receipt of the seven mail questionnaires, we discussed the questionnaire with each respondent for their feedback. Based on this feedback we modified the questionnaire. For the pretest telephone interviews, MPR project staff conducting each interview considered whether responses to questions were consistent with the intent of the questions and whether respondents understood each question appropriately. Based on the pretest, we modified a draft version of the protocol to improve the clarity of question wording and the sequencing and flow of questions.

2. Site Visits

The site visit protocols draw heavily on MPR’s experience developing protocols for other studies, such as the Implementation of TANF Sanctions study and the Assessing Medicaid and Food Stamp Program Access and Participation study. We do not believe that further testing is necessary since the protocols are intended to serve as general guides rather than instruments to be followed precisely; in this kind of data collection, early experience in the field will lead to ongoing feedback about which topics merit more or less emphasis and what avenues are most fruitful for follow-up probing to explore issues. We expect that senior staff from MPR will adapt and tailor lines of questioning in each site based on the particular circumstances in the site.

e. individuals consulted on the statistical aspects of the design

No individuals beyond the study team were consulted on the statistical aspects of the design.

REFERENCES

The Urban Institute. “Welfare Rules Database.” Accessed October 9, 2006 from http://anfdata.urban.org/WRD/WRDWelcome.CFM.

Yin, Robert. Case Study Research: Design and Methods, Second Edition. Sage Publications: Thousand Oaks, CA, 1994.