0960-0666 supporting statement A (2009)

Ticket to Work Program Evaluation Survey (National Beneficiary Survey)

OMB No. 0960-0666

Supporting Statement Part A

A. Justification

1. Circumstances Necessitating the Data Collection/Legal Justification for Collecting Information

Overview

The 1999 Ticket to Work and Work Incentives Improvement Act, Public Law 106-170, established the Ticket to Work program (TTW) to create additional access to services for Social Security beneficiaries through a new system of public and private Employment Network (EN) providers. Along with establishing the TTW program, the legislation mandated an evaluation of the program.

In February 2003, the Social Security Administration (SSA) began a multi-phase evaluation of this program. Although we had originally planned to complete the final data collection wave by 2009, significant changes to the TTW program in 2008 (such as changes to the way state vocational rehabilitation agencies can provide services) compelled us to push off the final evaluation to 2010.

In this ICR, we are seeking clearance for round 4 of the National Beneficiary Survey and two associated experiments (all three activities will use the same data). The respondents are Social Security beneficiaries and TTW enrollees. As with the previous three phases of this project, a contractor will conduct this study for SSA.

Background

The TTW program provides eligible SSDI and SSI disability beneficiaries with a Ticket, which one uses to obtain vocational rehabilitation (VR) or employment services through participating providers, called Employment Networks (ENs). The TTW program replaced the SSA/VR program, which had existed since 1982. This SSA/VR program financed services for SSA beneficiaries exclusively through SVRAs.¹ SVRAs evaluated the beneficiary and determined whether to accept them for services. Services purchased or provided by VR to those accepted for services were reimbursed by SSA only if they led to a successful rehabilitation. We defined a successful rehabilitation as the beneficiary working at Substantial Gainful Activity (SGA) level² for a continuous period of at least nine months. Under TTW, one could obtain services from VRs under this older reimbursement mechanism (traditional reimbursement). However, TTW created additional access to services through a new system of public and private Employment Network (EN) providers.

The underlying rationale for the program was that some beneficiaries currently lacked the resources necessary to return to work at a level above SGA, either because they did not have easy access to such services, or because they lacked the incentive to invest resources in returning to work activities because of a variety of factors affecting the decision to work. While the traditional SSA/VR program was sufficient for some beneficiaries, TTW provided a means to access employment services in a less restrictive manner than under the traditional program. We expected that the manner in which we implemented the program would increase beneficiary demand for employment-related services and activities. We also expected to increase the number and diversity of providers, relative to the traditional SSA/VR reimbursement program, in response to the less restrictive participation requirements and increased consumer demand for services. ³

After approximately 5 years of implementation, in July 2008, we revised the TTW program to improve its overall effectiveness and to maximize the economic self-sufficiency of beneficiaries through work opportunities. Some of the major changes included creating a three-phase return to work process and changes to the following areas: 1) the eligibility requirements for beneficiaries, 2) the EN payment systems and 3) the provision of greater financial incentives and flexibility for ENs, 4) progress review requirements for Continuing Disability Reviews (CDR), and 5) the ways that SVRA provides services. Under the revised regulations, a new “in use-SVRA” status provides CDR protection for beneficiaries receiving services from an SVRA providing services under the traditional Cost Reimbursement method. Beneficiaries cannot assign their Ticket to an EN while receiving services from the SVRA, but as part of the new “Partnership Plus” model, they can use their Ticket after the SVRA has closed their case to receive services from another provider to receive job retention services or ongoing support services.

SSA’s Office of Disability and Income Security Program has conducted three of the four rounds planned for the NBS as part of the TTW evaluation; one each in 2004, 2005, and 2006. The surveys collected data from a national sample of SSA disability beneficiaries (hereafter referred to as the Representative Beneficiary Sample) and a sample of TTW participants (hereafter referred to as the Ticket Participant Sample). We collected the Representative Beneficiary Sample data collected cross-sectionally. The Ticket Participant Sample contained cross-sectional and longitudinal components. We collected data using computer-assisted telephone interviewing (CATI) with computer-assisted personal interviewing (CAPI) follow-ups of CATI non-respondents and those who preferred or needed an in-person interview to accommodate their disabilities.

Due to changes in the TTW program outlined above, we postponed the fourth and final round of the NBS until now. In this Information Collection Request (ICR), SSA is seeking OMB clearance to continue the NBS data collection activities, specifically to conduct the fourth and final round. We also plan to conduct two experiments, explained in detail below. These experiments will not involve collecting any further data from respondents; rather, we will perform a series of statistical analyses and comparisons on the data we obtained.

1.2. Legal Authority

Section 101(a) of Public Law 106-170 establishes the Ticket to Work and Self-Sufficiency Program. Section 101(d)(4)(A) requires the Commissioner of SSA to conduct independent evaluations of the program to assess cost-effectiveness and the program’s effects on work outcomes of program participants.

2. How, by Whom, and for What Purpose Information SSA Will Use the Information

The primary users of the information collected for the round 4 NBS will be the same as for the earlier rounds: evaluation contractors, researchers and policy makers in the federal government (especially in SSA, the Department of Health and Human Services, and the Department of Education), and SVRAs and EN private providers. In addition, SSA is making available a public use data set from the surveys, with all personal identifiers removed, that may be used by other interested researchers to address issues related to the health and employment-related activities of SSI and SSDI beneficiaries.

The primary purpose of the evaluation is to provide policy makers information about the net outcomes of TTW (compared to outcomes under the current system), total and net costs of the program, characteristics of ENs and beneficiaries who do and do not participate, factors that affect return to work, employment outcomes for participants, and beneficiary satisfaction with the program.

The survey data in particular will allow an assessment of who participates and who doesn’t, the relationship between participation and both beneficiary and area characteristics, and reasons for non-participation. In conjunction with administrative data, the survey data will be used to estimate the impacts of TTW on employment, earnings, program participation, benefits, and net program costs.

See below for a detailed description of the components of this project and how we will collect the data.

2.1. NBS Data Collection Activities Requiring OMB Clearance

2.1.1. Round 4 NBS

Under the original design, the round 4 surveys would have concentrated largely on following participants interviewed in earlier rounds and interviewing new Ticket Participants in Phase 3 states.⁴ We planned the cross-sectional sample of Representative Beneficiaries in round 4 to be substantially smaller than the cross-sections in earlier rounds. As Table 1 shows, we only allocated 1,500 (29 percent) of the 5,056 round 4 interviews to the national cross-section of Representative Beneficiaries. We allocated just over half of the interviews to following Ticket Participants interviewed in earlier rounds: 784 with Ticket Participants who enrolled in a Phase 1 state (selected for the sample in 2002), 850 with Ticket Participants who enrolled in a Phase 2 state (selected in 2003), and 922 with Ticket Participants who enrolled in a Phase 3 state (selected in 2004). We allocated the remaining 20 percent of the sample to approximately 1,000 new Phase 3 Ticket Participants who enrolled in 2005.

TABLE 1

NATIONAL BENEFICIARY AND TTW PARTICIPANT SAMPLE SIZES

Samplea		Round 1	Round 2	Round 3	Round 4	All Roundsc
National Beneficiary Samples		7,200	4,800	2,400	1,500	15,900
Longitudinal TTW Participant Samples	Phase 1 Cohorts (1)b	1,000	922	850	784	3,556
	(2)		1,000			1,000
	Phase 2 Cohorts (1)		1,000	922	850	2,772
	(2)			1,000		1,000
	Phase 3 Cohorts (1)			1,000	922	1,922
	(2)				1,000	1,000
	Total	1,000	2,922	3,772	3,556	11,250
Total Sample Size		8,200	7,722	6,172	5,056	27,150

Source: NBS Sample Design Report (Bethel and Stapleton 2002).

^aSample sizes refer to number of completed interviews

^b(1)=TTW participant longitudinal sample and (2)=TTW participant cross-sectional supplement

^cThe All Years column is a tabulation of the number of interviews, not the number of sample members. Longitudinal cases may be included up to three times in these counts, depending upon the number of completed interviews for the sample member in question.

The first three rounds of the NBS data (2004, 2005, and 2006) provide valuable information on current beneficiaries, their perceptions and participation in the TTW program, and changes in participants’ outcomes over time. However, 2008 changes in the federal regulations which substantially altered the TTW program mean that it is no longer meaningful to track long-term experiences of beneficiaries who participated in the program under the old regulations. As a result, SSA considers it necessary to change the sample design and instrumentation for the round 4 survey. Including a representative cross-sectional participant sample will provide information on employment, service use, and attitudes of all current participants. We can compare this information with the findings from earlier survey rounds to study the extent to which changes in the TTW program and the general environment have led to changes in the characteristics, experiences, service use, and employment outcomes of TTW participants. The information may also help SSA and ENs in thinking about how to best structure service delivery to TTW participants. While the longitudinal follow-up samples in round 4 would have provided important information on the longer-term affects of TTW, the longitudinal data collected through round 3 will adequately address these issues.

The originally approved sample design stratified the TTW Participant Sample by payment type (traditional, milestone-outcome, and outcome-only) rather than provider type SVRA vs. ENs. As a result, participants with SVRAs were overrepresented in the sample because many of the milestone-outcome and outcomes-only participants assigned their tickets to SVRAs. Thus, the original stratification did not yield enough information about TTW Participants who assigned their Tickets to ENs. The original design also stratified on phase, which was important initially because of rollout, but is of more limited value now that the program is up and running in all areas. To overcome this, the new the round 4 design will stratify based on provider type and will over-sample participants who assigned their ticket to an EN.

Finally, a larger cross-section sample of beneficiaries would allow cross round comparisons of changes in overall service use, employment, work attitudes, ticket awareness, and benefits. Given the new regulations which change the structure of the TTW program, primarily by offering more generous payments to providers and new outreach to beneficiaries, boosting the Representative Beneficiary Sample would provide information about how the new regulations affect interest in the TTW program, attitudes/interest in employment, the nature of current barriers, work activity; job characteristics, and the prevalence and nature of service use.

Therefore, SSA plans to increase the target number of completes from the Nationally Representative Beneficiary Sample to 2,400 for round 4. This sample will be divided into the following age groups, 18-24, 25-39, 40-54, and 55-64, which will be used as the sampling strata. The target number of completed cases for the TTW Participant sample will remain 3,000 but will be stratified by provider type (1,000 TTW Participants with tickets assigned to SVRAs, and 2,000 Ticket Participants with tickets assigned to ENs). See A15 for estimated changes in burden.

As was the case for the prior rounds, we will conduct the interviews primarily by telephone (using CATI). We will conduct personal interviews for those who cannot be contacted by phone or who, due to a physical or mental impairment, find it difficult to respond by phone.

All sample members will receive an advance letter informing them of the study prior to the initial telephone contact (see attached file). The survey will collect information on socio-demographic characteristics, income, program participation, health/disability status, use of employment services, employment obstacles and outcomes, and awareness of and experience with the TTW program. The questionnaire was modified as necessary to capture the experiences of Representative Beneficiaries and TTW Participants following the implementation of the new regulations. The survey instrument is provided as a separate file. The instrument will remain approximately 45 minutes long. As in prior rounds, if the sample person is unable to complete either a telephone or in-person interview due to a cognitive impairment, a proxy respondent will be sought. An innovative “mini-cognitive test” designed expressly for the survey, will be used to identify when proxy respondents are needed.⁵ To promote response among Hispanic populations, the questionnaire will be available in Spanish. A number of additional accommodations will also be made available for those with hearing and/or speech impairments including teletypewriter (TTY), Telecommunications Relay Service (TRS), amplifiers, and instant messaging technology. We plan to begin data collection for the fourth round approximately 20 months after the re-launch of the TTW program, in February 2010.

2.1.2. Experiment

As part of continuing efforts to verify the reliability and validity of NBS data and inform methodological improvements in the way that data is collected from persons with disabilities, SSA plans to include an experiment in the round 4 NBS. Funding to design, conduct, and analyze findings from the experiment, which will compare data collected by CATI and by CAPI, has been received as part of a five year grant from the National Institute on Disability and Rehabilitation Research (NIDRR) in the U.S. Department of Education (ED) to support a Rehabilitation Research and Training Center on Disability Demographics and Statistics (StatsRRTC). The StatsRRTC is a collaborative effort led by investigators at Hunter College, MPR, New Editions Consulting, Inc. and the Center for Essential Management Services (CEMS). The experiment has received IRB approval through Hunter College.

The experiment will examine how the use of CATI compared to CAPI affects responses from subjects with disabilities. This study is important to assess whether there are differences in data quality that are due to the data collection mode. For this experiment, a subset of cases (n=360) will be randomly selected as the treatment group to receive CAPI only. The remaining subjects, assigned to CATI/CAPI, will be treated as CATI only. These cases will not receive a CATI attempt, but will go directly to in-person interviewing. All other cases will receive the standard NBS treatment (CATI with CAPI follow-up if needed). Cases that do not complete in CATI will still be sent to CAPI per the study design, but will be considered non-respondents for the purposes of the experiment. The results from this experiment will inform whether data collected by CATI and CAPI are comparable.

For the CATI/CAPI experiment, we will examine data comparability across modes including item non-response, number of options checked for check-all-that-apply items, non-differentiation among items in a series, proportion of agree/yes responses, length of responses to open-ended items, and distribution of responses or means for sensitive items. In addition, we will examine response effects by type of disability (cognitive or physical).

3. Use of Automated, Electronic, Mechanical or Other Technological Collection Techniques

Due to the nature of the collection (i.e., an in-depth survey requiring guidance to complete), SS did not create an electronic version of this collection under the agency’s Government Paperwork Elimination Act plan. However, SSA’s contractor will collect this data using CATI software with CAPI follow-ups, so respondents will not need to complete the survey by hand.

4. Efforts to Avoid Duplication of Effort

The nature of the information SSA is collecting and the manner in which we are collecting it precludes duplication. SSA does not use another collection instrument to obtain similar data.

5. Sensitivity to Burden on Small Entities

This collection will not affect small businesses or other small entities.

6. Consequences to Federal Programs or Policies if Data Collection is Not Conducted

If we did not conduct the round 4 NBS data collection, we would have means of addressing the impact of the TTW changes on beneficiaries, beneficiary satisfaction with services, job quality, efficacy of the program, etc. This would place the agency in violation of the provisions of P.L. 106-170. Since we will only collect the information once, we cannot conduct the information collection less frequently.

There are no technical or legal obstacles to burden reduction.

7. Special Circumstances

There are no special circumstances that would cause SSA to conduct this information collection in a manner inconsistent with 5 CFR 1320.5.

8. Federal Register Notices/Solicitation of Outside Opinion

8.1. Federal Register Announcement

SSA published the 60-day advance Federal Register Notice for this collection on August 19, 2009, at 74 FR 41959, and we did not receive any public comments. SSA published the 30-day Federal Register Notice on October 26, 2009, at 74 FR 55080, and we did not receive any public comments in response to this Notice.

8.2. Consultations Outside the Agency

SSA’s contractors consulted a number of outside sources to develop this survey. The initial evaluation design and survey instrument for the NBS evolved over many months. The Lewin Group, Inc. and their subcontractors, Cornell University and Westat, developed a design for the evaluation in consultation with SSA staff and a panel of experts, the Technical Evaluation Support Group (TESG), convened for the project. TESG members reviewed drafts of evaluation design and met on three occasions to discuss design and implementation issues.

Besides their participation in the group meetings, selected TESG members were consulted on an ad hoc basis when specific issues related to their expertise were being addressed in the development of the evaluation design. The Lewin Group, Cornell University, Westat, and MPR staff also provided expert input in the development of the design. The individuals who contributed to development of the initial evaluation design and associated data collection instruments are listed in Table 2, with their areas of expertise. Consultants contributing to revisions for round 4 include Eric Grau, Gina Livermore, Frank Potter, David Stapleton, and Debra Wright at Mathematica Policy Research.

9. Respondent Payments

The evaluation contractor will pay beneficiary survey respondents a small amount to encourage response, facilitate cooperation, and demonstrate appreciation to participants for their time and effort. We will offer payments of $10 per interview to participants once they have met all appropriate criteria for participation in the interview. SSA believes that some compensation is particularly important to engender a positive attitude about the study. OMB has approved a post-paid incentive of $10 to participants in all prior rounds. Research shows that payments increase response rates without compromising data quality (Singer 2000) and that they are effective at increasing response rates for people with lower educational levels (Berlin et al. 1992) and low-income and nonwhite populations (James and Bolstein 1990).

Furthermore, the contractor will send pre-paid checks for $10 to non-respondents in the final 4 weeks of the field period to encourage timely response. This decision is based on an experiment conducted as part of the round 1 NBS (the results of which we previously reported to OMB).⁶

Table 2

Consultants Outside the Agency

Name	Affiliation	Area(s) of Expertise
Gina Livermore	Mathematica Policy Research Washington, DC	Evaluation methods, survey design, disability programs, SSA data
David Stapleton		Evaluation methods, statistics, survey design, disability programs, SSA data
Craig Thornton		Evaluation methods, statistics, survey design, disability and employment programs, SSA data
Ellen Bouchery	The Lewin Group Falls Church, VA	Evaluation methods, statistics, survey design, disability programs, SSA data
James Bethel	Westat Rockville, MD	Sampling methodology
William Frey		Survey design and methods, disability policy and programs, disability measurement
Graham Kalton		Sampling methodology
Burt Barnow	Johns Hopkins University, Institute for Policy Studies Baltimore, MD	Evaluation methods, statistics, survey design, disability and employment programs
John Kregel	Virginia Commonwealth University Richmond, VA	Evaluation methods, disability and employment programs, service provider issues, RSA data
Technical Evaluation Support Group Members
Joshua Angrist	Mass. Institute of Technology, Dept. of Economics Cambridge, MA	Evaluation and statistical methods
Monroe Berkowitz	Rutgers University New Brunswick, NJ	Disability and employment policy and programs
Patricia Doyle	U.S. Census Bureau, Demographic Surveys Division Suitland, MD	Survey design, demography
Jack Feldman	Center for Health Affairs/Project HOPE Bethesda, MD	Survey design, health measurement
Lex Frieden	Institute for Rehabilitation & Research Houston, TX	Disability and employment policy and programs
Joseph Hight	U.S. Department of Labor Washington, DC	Employment policy and programs
Allan Hunt	W.E. Upjohn Institute for Employment Research Kalamazoo, MI	Disability and employment policy and programs
Harold Kay	U.S. Department of Education, OSERS/RSA Washington, DC	VR programs, provider issues, disability and employment programs, RSA data
Arthur Meltzer	Centers for Medicare and Medicaid Services Baltimore, MD	Evaluation methods, disability programs
Larry Orr	Abt Associates Bethesda, MD	Evaluation methods, statistics, survey design, disability and employment policy and programs
David Salkever	Johns Hopkins University, School of Public Health Baltimore, MD	Evaluation methods, statistics, disability and employment policy and programs
Mark Shroder	U.S. Department of Housing & Urban Development Washington, DC	Evaluation methods, statistics
Jeffrey Smith	University of Maryland, Department of Economics College Park, MD	Evaluation methods, statistics, survey design, employment programs
Ernst Stromsdorfer	Washington State University, Dept. of Economics Pullman, WA	Evaluation methods, statistics, employment programs

While a comparison of the overall response rates between waves 1 (control group) and waves 2-5 (treatment group) showed there was no statistically significant improvement in completion rate for the treatment group, there were significant gains in the time it took to complete cases in the treatment group, suggesting that the pre-pay encouraged call-ins and reduced follow-up effort for unresolved cases.

10. Confidentiality Assurances

SSA will protect and hold confidential the information it is requesting in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular No. A-130. To ensure this is the case, we will take the following measures to ensure confidentiality:

• Before a respondent’s initial interview, we will notify the beneficiary will by mail of the purpose of the survey and SSA’s desire to interview them. The letter will provide assurance that all information collected will remain confidential and will not be used in any way that would affect their program eligibility or payments. It will also indicate the toll-free telephone number to call if they have questions about the study (provided as a separate file). Respondents selected to participate in the CATI/CAPI experiment will receive a slightly modified version of this letter (rather than indicating that “an interviewer from Mathematica will call you by telephone to conduct an interview”, the letter will state that “an interview will contact you to conduct an interview in your home”). When survey sample members are subsequently contacted to conduct the interviews, they will again be advised of the nature and purpose of the survey, and provided assurances of confidentiality.

• In upholding the confidentiality of beneficiary information, SSA’s contractor will operate within the guidelines established by the Privacy Act to protect respondents’ privacy and the confidentiality of all data collected. The Privacy Act states that “microdata files prepared for purposes of research and analysis are purged of personal identifiers and are subject to procedural safeguards to assure anonymity.” The contractor has a great deal of experience handling sensitive data and has routine procedures in place to ensure the confidentiality of computerized and paper records, including the use of passwords and encrypted identifiers to prevent direct or indirect disclosures of information. Furthermore, the information management systems of these contractors will fully comply with the government’s ADP system requirements.

• Sample selection and survey data are stored on an encrypted network drive. Access to sample selection data will be limited to those who have direct responsibility for providing the sample and maintaining sample locating information. Staff access to data storage and files will be limited to authorized personnel who have passwords. At the conclusion of the research these data are destroyed. Audits will be conducted on an ongoing basis that will compare the list of cleared staff to those charging time and with access to restricted folders to verify that all appropriate staff have clearance.

• The workstations of all NBS project staff have a password protected screen saver which is automatically activated when they are away from their work area.

• Identifying information and survey responses are maintained in separate files which are linkable only by sample identification number. Access to the file linking sample identification numbers with the respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information.

• Access to the hard copy documents is strictly limited. Documents are labeled with a subject identification number and stored in a locked file cabinet in a secure facility. Discarded material is shredded.

• Laptops used for CAPI data collection are password protected and all survey data that are collected encrypted. Confidential case information (name, address, and telephone number) is transmitted to field interviewers over a secure connection. SSN is never be transmitted to field interviewers.

• Data are transferred to SSA using a secure transfer file site or by email using WinZip 9.0 password encryption (256-bit AES encryption) to protect the file.

11. Additional Justification for Sensitive Questions

Although some questions in this survey may be viewed as being potentially sensitive, they are necessary to conduct a thorough survey of the NBS. Moreover, SSA will ensure the information remains confidential, as per the measures described in question #10.

12. Estimates of Hour Burden

Burden data for this survey are in the chart below.

Table 3

ANNUAL BURDEN ESTIMATES

Instrument	Annual Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Estimated Annual Burden Hours
National Beneficiary Sample	2,400	1	.750	1,800
Ticket Participant Sample	3,000	1	.917	2,750
Grand Total – Burden for NBS
Grand Total for All	5,400	-	-	4,550

The total burden for this collection is 4,550 hours. This figure represents burden hours, and we did not calculate a separate cost burden.

13. Estimate of Total Annual Cost Burden to Respondents or Record-Keepers

There is no known cost burden to the respondents.

14. Estimates of Annualized Cost to the Federal Government

The estimated annual cost to the federal government for this ICR is $1,952,137. This estimate includes MPR’s cost to revise the sample design and questionnaire, administer the data collection, and prepare the data files and documentation.

15. Reasons for Program Changes or Adjustments

Since the round 4 survey is a new one (necessitated by change in the TTW program), and this ICR is a re-clearance of a previously discontinued OMB number, the burden is new as well.

16. Plan for Tabulation and Publication and Schedule for Project

16.1. Tabulation Plans

SSA provided evaluation reports for the previous three rounds of NBS surveys to Congress. The NBS was critical in providing adequate responses to Congressional directives and questions. Moreover, P.L. 106-170 requires SSA to evaluate the efficacy of the TTW program. For these reasons, SSA will publish and provide a report to Congress as well for the round 4 NBS.

Analyses based on the round 4 NBS will focus on changes in beneficiary use of TTW and outcomes following the implementation of the new TTW regulations. Specifically, analyses based on round 4 survey data will encompass the following: TTW awareness and participation, use of services, employment outcomes and program exits, changes in experiences and outcomes since the implementation of the new TTW regulations, and differences across beneficiary subgroups (for example, differences based on provider type, Adequacy of Incentives (AOI) status, and program Title). The contractor will draw on previous NBS analyses for the purposes of making comparisons.

Finally, SSA will use the survey data for process and outcome analyses designed to document how well we are implementing TTW under the new regulations. We will link the round 4 NBS data to SSA administrative data for analyses of examining how various beneficiary characteristics correlate with duration of benefit receipt, benefit amounts, and program exits, and to add administrative information on the use of work incentives and program exits due to work. The analyses will focus on changes in beneficiary use of TTW and outcomes following the implementation of the new TTW regulations.

16.2. Time Schedule

We provide the full timeline for the round 4 data collection evaluation in Figure 1. The timeline calls for major design activities to begin in May 2009 (completed), for data collection to begin in February 2010 (dependent on OMB approval), and for data file preparation activities to begin in October 2010.

Figure 1
Project Timeline

	2009										2010												2011
Task	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	March	April	May
Instrument Revisions and CAI programming
Revise Sample Design
Implement Sample Design
Train Interviewers
Data Collection
Prepare Data Files and Documentation

Note: Under a separate contract, data analysis and report writing will begin in March of 2010 and continue through December 2011.

17. Approval Not to Display the Expiration Date for OMB Approval

SSA is not seeking an exemption from the requirement to display the OMB number.

18. Exception to the Certification Statement

SSA is not requesting an exception to the certification requirements at 5 CFR 1320.9 and related provisions at 5 CFR 1320.8(b)(3).

Please see next page for references

references

Bethel, J. and D. Stapleton. “Evaluation Design For The Ticket To Work Program: Final Survey Sample Design.” Prepared for the Social Security Administration. Washington, DC 2002.

Berlin, Martha et al. “An Experiment in Monetary Incentives.” In Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association, 1992.

James, J., and R. Bolstein. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly, vol. 54, 1990.

Singer, E., and R.A. Kulka. “Paying Respondents for Survey Participation.” Ann Arbor, MI: The University of Michigan, Institute for Social Research, Survey Research Center, 2000.

1 SSA also adopted the alternate participant (AP) program in 1994 to allow private sector firms to compete with VR in serving SSA disable beneficiaries. However, this program was of minor consequence since only a handful (less than 40) of SSA beneficiaries enrolled with APs in the six years of its existence.

2 SGA is currently defined as $980 per month for non-blind beneficiaries and $1640 per month for blind beneficiaries.

3 In addition to providing increased opportunity for beneficiaries to obtain employment services, the program was intended to generate savings for the federal and state governments. For example, the Federal Register noted (based on a U.S. General Accounting Office study) if just one-half of one percent of current SSDI and SSI recipients were to cease receiving benefits as a result of engaging in self-supporting employment, savings in cash assistance would total $3.5 billion over the work lives of those individuals. SSA anticipated that not only would there be an increase in the number of beneficiaries leaving the SSDI and SSI disability rolls due to work or earnings, some individuals would secure work with employers who offered group health coverage, thereby reducing Medicaid and Medicare expenses. The increased earned income of beneficiaries should have also yielded tax receipts while reducing expenses in SSDI and disabled or blind SSI benefits, food stamps, housing subsidies, and Veterans benefits. Overall, the expectation was that TTW would increase the likelihood of self-sustaining employment for SSDI and SSI beneficiaries, resulting in decreased government expenditures on benefits, increased tax revenues, and a general strengthening of communities and the workforce.

4 The TTW program was implemented in three phases. In Phase 1, which began in February 2002, the program was rolled out in 13 states across the country. In Phase 2, which began in November 2002, the program was extended to an additional 20 states plus the District of Columbia. Phase 3, which began in November 2003, implemented TTW in the remaining 17 states and U.S. territories.

5 Westat designed the test as part of the design of the Ticket to Work evaluation; MPR modified it after pretesting. MPR will conduct an assessment of interviewer reliability and accuracy in interpreting and coding responses both during interviewer training and throughout data collection.

6 The treatment group for this experiment included 2,553 unresolved cases--those with a valid address who were mailed the prepayment incentive letter (n=1,711) and those without a valid address for whom a check could not be mailed (n=842). The control group consisted of 1,427 cases in wave 1, 480 of which were unresolved at the time of the experiment. The control group continued to receive the post-pay $10 incentive once an interview was completed. All eligible unresolved cases in the treatment group received a pre-payment letter with a $10 check before additional calls were attempted. The experiment was designed to be able to detect an improvement of as little as 3.5 percent. 

0960-0666 (Supporting Statement A)

2/3/2021

14