Supporting Statement 0960-NEW (NBS) - Part A

SUPPORTING STATEMENT FOR

THE NATIONAL BENEFICIARY SURVEY—

GENERAL WAVES AND SEMI-STRUCTURED INTERVIEWS

OMB No. 0960-NEW

Supporting Statement: Part A

CONTENTS

Part A: Justification 1

A.1. Introduction/Authoring Laws and Regulations 1

A.2. Description of Collection 3

A.3. Use of Information Technology to Collect the Information 5

A.4. Why We Cannot Use Duplicate Information 6

A.5. Minimizing Burden on Small Respondents 6

A.6. Consequence of Not Collecting Information or Collecting it Less Frequently 7

A.7. Special Circumstances 7

A.8. Solicitation of Public Comment and Other Consultations with the Public 7

A.9. Payment or Gifts to Respondents 8

A.10. Assurances of Confidentiality 11

A.11. Justification for Sensitive Questions 12

A.12. Estimates of Public Reporting Burden 13

A.13. Annual Cost to the Respondents (Other) 15

A.14. Annual Cost to Federal Government 15

A.15. Program Changes or Adjustments to the Information Collection Request 15

A.16. Plans for Publication Information Collection Results 15

A.17. Displaying the OMB Approval Expiration Date 18

A.18. Exception to Certification Statement 18

Part A References 19

Part A: Tables

A.1 NBS–General Waves Sample Sizes and Semi-Structured Interviews per Round 4

A.2 Consultants Outside the Agency 8

..A.3 Incentive Conditions 10

..A.4 Annual Burden Estimates 14

Part A: Figures

A.1 Project Timeline 17

Part B: Collection of Information Employing Statistical Methods (see separate document)…………………………………………………………………………..…..1

B.1. Statistical Methodology………………………………………………………..……..1

B.2. Procedures for Collecting the Information........……………………………….…..4

B.3. Methods to Maximize Response Rates.……………………………………….....11

B.4. Tests of Procedures……………………………………………………………..….14

B.5. Statistical Agency Contact for Statistical Information …………..……………….15

Part B References…………………………………………………………………….…16

Part B Tables

B.1 NBS–General Waves Sample Sizes by Strata 3

B.2 Projected Minimal Detectable Differences Between Groups In

Representative Beneficiary Sample 8

B.3..Projected Minimal Detectable Differences Between Successful

Worker Groups 9

B.4..Incentive Conditions 10

B.5..Individuals Consulted on Technical and Statistical Issues 15

Appendices

Appendix A: NBS-General Waves Instrument

Appendix B: Respondent Correspondence

Appendix C: Semi-Structured Interview Recruitment Script and Guide

Appendix D: Respondent Correspondence for Incentive Experiment

Appendix E: Confidentiality Pledge

SUPPORTING STATEMENT FOR

THE NATIONAL BENEFICIARY SURVEY GENERAL WAVES

AND

SEMI-STRUCTURED INTERVIEWS

OMB No. 0960-NEW

Part A: Justification

A.1. Introduction/Authoring Laws and Regulations

A.1.1. Overview

The Social Security Administration (SSA) is proposing to undertake the National Beneficiary Survey (NBS), a survey intended to gather data from Supplemental Security Income (SSI) recipients and Social Security Disability Insurance (SSDI) beneficiaries about their characteristics, their well-being, and other factors that promote or hinder employment. In particular, the survey seeks to uncover important information about the factors that promote beneficiary self-sufficiency and, conversely, factors that impede beneficiary efforts to maintain employment. We will use this data to improve the administration and effectiveness of the SSDI and SSI programs. These results will be valuable as SSA and other policymakers continue efforts to improve programs and services that help SSDI beneficiaries and SSI recipients become more self-sufficient.

SSDI and SSI programs provide a crucial and necessary safety net for working-age people with disabilities. By improving employment outcomes for SSDI beneficiaries and SSI recipients, SSA supports the effort to reduce the reliance of people with disabilities on these programs. SSA conducted the prior NBS in 2004, 2005, 2006, and 2010, and was an important first step in understanding the work interest and experiences of SSI recipients and SSDI beneficiaries, and in gaining information about their impairments, health, living arrangements, family structure, pre-disability occupation, and use of non-SSA programs (e.g., SNAP). The prior NBS data is available to researchers and the public.

In this new information collection request (ICR), SSA is seeking Office of Management and Budget (OMB) approval to conduct 1) an additional three rounds of survey data collection under the new NBS–General Waves; and 2) semi-structured, qualitative interviews with SSI recipients and DI beneficiaries who have achieved high earnings and were, or were not, able to sustain them. In this new survey effort, we will no longer include a focus on evaluating the Ticket to Work (TTW) program; however, survey data will continue to serve as a major source about the well-being and factors that promote or hinder employment among a nationally representative sample of active SSI recipients and DI beneficiaries. As with the previous three phases of this project, an SSA-funded private contractor will conduct this study.

A.1.2. Background

For the new NBS-General Waves, SSA originally planned to collect three rounds of data from a national sample of SSA disability beneficiaries and a sample of beneficiaries whose benefits had been suspended due to high earnings or successful work beginning in 2014. We planned to complete approximately 4,000 interviews with active¹ beneficiaries in each of the three rounds while the sample sizes for those whose benefits SSA had suspended would vary across rounds². In addition, some beneficiaries we identified as individuals in suspense status at the time of the round 1 interview, we would follow longitudinally in rounds 2 and 3.

Due to difficulties associated with developing a sample design that would provide sufficient numbers of beneficiaries who earned enough to have had their benefits suspended in the recent past, SSA postponed the start of the survey to 2015 and we will not include a sample of successful workers as part of the round 1 NBS-General Waves. In lieu of including a sample of successful workers in the survey data collection, we plan to conduct semi-structured interviews with this group. This will give us more time to settle on an adequate design for this sample so that we can include them at rounds 2 and 3 while still allowing us to collect important information about factors that aid or inhibit beneficiaries in their efforts to obtain and retain employment on the general sample. We will also use the semi-structured interviews to inform the sample and survey design of rounds 2 and 3 for the successful earners.

Because we will need to revise the survey at round 2 to accommodate the successful worker sample and incorporate what we learned from the semi-structured interviews, we anticipate submitting a request for non-substantive or nonmaterial changes (i.e., change request) to OMB which would include an updated survey for rounds 2 and 3.

A.1.3. Legal Authority

Section 1110 of the Social Security Act (Act) [42 U.S.C. 1310] authorizes SSA to conduct research and evaluation projects. This includes making grants to states, public and other organizations and agencies for paying part of the cost of research or demonstration projects such as those relating to the prevention and reduction of dependency; or which will aid in the coordination of planning between private and public welfare agencies; or will help improve the administration and effectiveness of programs carried on or assisted under the Act. We intend to gather data from SSI recipients and SSDI beneficiaries about their well-being and factors that promote or hinder employment. SSA will use this data to improve the administration and effectiveness of its programs.

A.2. Description of Collection

The primary purpose of the NBS-General Waves and semi-structured, qualitative interviews is program planning, management, and research. The NBS-General Waves and qualitative interviews will provide SSA, Congress, and other policymakers with information about key factors that contribute to DI beneficiaries’ and SSI recipients’ successful or unsuccessful employment efforts. In particular, the study will generate important information about the factors that promote DI beneficiary and SSI recipient self-sufficiency and, conversely, factors that impede their efforts to maintain employment. These results will be valuable in our continuing efforts to improve programs and services so that DI beneficiaries and SSI recipients can become self-sufficient.

NBS-General Waves – Via an SSA-funded private contractor, SSA will field round 1 of the NBS-General Waves beginning in 2015. We will conduct two subsequent rounds of the survey in 2017 (round 2) and 2019 (round 3). To moderate costs, SSA will field the new NBS every two years, rather than annually, as previously conducted. The survey will collect data not available from SSA administrative data or other sources. The survey will address five major questions as follows:

1. What are the work-related goals and activities of SSI recipients and DI beneficiaries, particularly as they relate to long-term employment?

2. What are the short-term and long-term employment outcomes for SSI recipients and DI beneficiaries who work?

3. What support mechanisms help them in finding and keeping jobs, and what barriers to work do they encounter?

4. What are the characteristics and experiences of SSI recipients and DI beneficiaries who work?

5. What factors and circumstances hinder or promote their employment and self-sufficiency?

We will collect survey data from a nationally representative sample of active SSI recipients and DI beneficiaries. In each round, we plan to complete approximately 4,000 interviews with a cross-sectional sample of active beneficiaries or recipients in each of the three rounds (SSI and SSDI). We will add a cross-sectional successful worker sample in rounds 2 and 3 yielding 4,500 and 3,000 completed interviews respectively. In round 3, we will re-interview successful workers who complete the round 2 interview (n=2,250). In Table A.1 we provide an overview of the sample sizes for each round.

We will conduct the survey interviews primarily by telephone, but for those who cannot complete the interview by telephone because of a physical or mental impairment or for whom we cannot locate a valid telephone number, we will interview in person. The survey will collect information on socio-demographic characteristics, income, program participation, health and disability status, use of employment services, and employment supports, obstacles and outcomes. We will send all sample members an advance letter informing them of their selection into the study prior to the initial telephone contact. We will use a mini-cognitive test, designed expressly for the survey, to identify when we need a proxy respondent. To promote response among Hispanic sample members, we will translate the survey into Spanish. We will employ a number of additional accommodations for those with hearing or speech impairments including teletypewriter (TTY), Telecommunications Relay Service (TRS), amplifiers, and instant messaging. We have scheduled the data collection to last approximately eight months for each round.

As mentioned previously, the respondents to the NBS-General Waves are active SSI recipients and DI beneficiaries. Participation is voluntary and the decision to participate or not has no impact on current or future receipt of benefits or payments.

The primary users of the information collected for the NBS-General Waves will be researchers and policymakers in the Federal Government (especially SSA, the Departments of Health and Human Services, and Education), as well as state vocational rehabilitation agencies (SVRAs). SSA will make available a public use data set from each round of the NBS-General Waves, with all personal identifiers removed. Other interested researches may use this public use file (PUF) to address issues related to the health and employment-related activities of SSI recipients and SSDI beneficiaries.

Semi-Structured Interviews - In addition to the survey data collection, in round 1, the SSA-funded contractor will conduct semi-structured, qualitative interviews with three groups of respondents who have had periods of successful work: 1) those identified as having sustained high earnings from employment; 2) those identified as having high earnings from employment but who did not sustain high earnings; and 3) employed respondents under the age of 30. We will conduct 20-30 interviews with each of the three groups for maximum of 90 interviews (see Table A.1). We will conduct these interviews in 2015 only. The recruitment and discussion guides for the qualitative interviews are included in Attachment C. The interviews will focus on the following:

1. How does SSA, employers, community-based organizations (CBOs), or SSI recipients and DI beneficiaries themselves aid or inhibit efforts to obtain employment?

2. How does SSA, employers, CBOs, or beneficiaries themselves aid or inhibit efforts to retain employment and advance in the workplace?

3. How do employment experiences of SSI recipients and DI beneficiaries with higher earnings differ from those with lower earnings and among those in suspense status?

Table A.1. NBS–General Waves Sample Sizes and Semi-Structured Interviews per Round

		Completed Interviews Per Year (Round)
Samplea		2015 (R1)	2017 (R2)	2019 (R3)	All Years
Representative Beneficiary Sample		4,000	4,000	4,000	12,000
Successful Worker Semi-Structured Interviews		90			90
Successful Workers	Cross-sectional		4,500	3,000	7,500
	Longitudinal			2,250b	2,250
	Subtotal Successful Workers		4,500	5,250	9,750
Total Interviews		4,090	8,500	9,250	21,840

^a The sample sizes refer to the number of completed interviews. With a target response rate of 80 percent, the number of cases actually sampled for the survey would be inflated by 1.25.

^b In round 3 (2019), we will attempt follow-up interviews with successful workers at the time of the round 2 interview. We estimate that approximately one-half of successful workers at the time of sample selection will remain successful at the time of the round w interview and that the other half will have experienced reduced earnings and increased reliance on benefits.

We will conduct the qualitative, semi-structured interviews by telephone. If it is not possible for the respondent to participate in a phone interview, we may attempt to conduct the interview at the respondent’s home or at a mutually convenient location. We will select interview participants through telephone recruitment efforts targeting SSI recipients and DI beneficiaries SSA identifies (using administrative data) as having had high earnings within the past 12 months. SSA may have stopped or suspended cash payments of some high earners at the time of sample selection if their earnings exceeded the maximum threshold for receiving cash payments. In addition, we will attempt to recruit employed beneficiaries and recipients under the age of 30; beneficiaries and recipients in this age group have the highest employment success rates. Once recruited, we will schedule an interview. As with the survey, we will provide accommodations for those with hearing or speech impairments. We will conduct these qualitative interviews in both English and Spanish.

As mentioned previously, the respondents to the semi-structured interviews are SSI recipients or DI beneficiaries 1) who have sustained high earnings from employment (suspense status); 2) who had high earnings from employment but who did not sustain high earnings; and 3) employed under the age of 30. Participation is voluntary and the decision to participate or not has no impact on current or future receipt of benefits.

Researchers and policymakers in the Federal Government (especially in SSA, the Departments of Health and Human Services, and Education) will be the primary users of the information collected through the semi-structured interviews. While the survey captures more general information on DI beneficiaries and SSI recipients, including their disabilities, interest in work, use of services, and employment, the semi-structured interviews will provide in-depth information about factors that support or impede employment. SSA and other policymakers will use the information from this study in their future efforts to improve SSA programs that encourage work.

A.3. Use of Information Technology to Collect the Information

NBS-General Waves - SSA’s contractor will collect the NBS-General Wave data using computer-assisted telephone interviewing (CATI) and computer-assisted personal interviewing (CAPI) software in circumstances when telephone interviews are not feasible. We will use identical survey instruments in each mode.

Semi-Structured Interviews - While we will conduct the qualitative interviews by telephone whenever possible, SSA did not create an electronic version of this interview under the agency’s Government Paperwork Elimination Act (GPEA) plan because only 90 respondents will complete this interview. This is less than the GPEA cut-off of 50,000. Due to their semi-structured nature, we will record then transcribe the interviews.

A.4. Why We Cannot Use Duplicate Information

The nature of the information SSA is collecting and the manner in which we are collecting it preclude duplication. SSA does not use another collection instrument to obtain similar data representative of all our DI beneficiaries and SSI recipients.

NBS-General Waves - The NBS-General Waves will collect data not available from SSA administrative data or other sources. SSA data are limited in part because we collect it for administrative purposes, not research. The NBS-General Waves addresses the most significant of SSA’s administrative record limitations, including the following:

• There is no useful information on current job characteristics other than annual earnings for DI beneficiaries. Job characteristics information is also unavailable for SSI recipients, but self-reports of monthly earnings are available. This survey will collect detailed information about job quality, especially wages, occupation, hours worked, and fringe benefits.

• Information on impairments, health, living arrangements, marital status, family members, pre-disability occupation, other personal income, and education are each limited in very significant ways. We include these topics in the survey because they are often determinants of employment success.

• There is no useful information on use of employment services, family income, use of non-SSA programs (e.g., Food Stamps), recipient knowledge of DI and SSI work incentive programs, obstacles to return to work, factors facilitating attaining and keeping a job, and beneficiary interest and motivation to return to work. We cannot address any of these issues with SSA administrative data.

Semi-Structured Interviews - SSA has not conducted in-depth interviews on this scale to learn about employment successes and challenges and what distinguishes successful and unsuccessful work attempts.

A.5. Minimizing Burden on Small Respondents

This collection does not significantly affect small businesses or other small entities

NBS-General Waves – We will administer the vast majority of the NBS-General Waves’ surveys to DI beneficiaries and SSI recipients residing in the general population. We will attempt to interview individuals institutionalized or living in group settings who are able to participate. We expect this number to be very small (less than 1 percent of all completed surveys). In these instances, we will contact institution staff in advance and explain the purpose of the study before we attempt an interview. We expect that the involvement of small entities will be negligible.

Semi-Structured Interviews - All individuals participating in the qualitative interviews will be living in a non-institutionalized, community-based setting.

A.6. Consequence of Not Collecting Information or Collecting it Less Frequently

NBS-General Waves - If the NBS-General Waves were not conducted, SSA would be unable to address important issues regarding DI beneficiaries’ and SSI recipients’ success in finding, maintaining, and advancing in employment. SSA would lack information about the impairments, health, living arrangements, family structure, pre-disability occupation, use of non-SSA programs (e.g., Food Stamps), knowledge of DI and SSI work incentive programs, obstacles to return to work, and interest and motivation to return to work; all factors that can positively or negatively impact employment success.

Longitudinally tracking successful workers will allow SSA to better understand how beneficiaries and recipients retain their employment and advance in their work.

Semi-Structured Interviews - The semi-structured interviews will add important and rich detail on the employment success and failures of beneficiaries and recipients that cannot be fully examined with administrative data alone. If these issues are not addressed, policymakers will be unable to determine the underlying factors that lead to the success or failure of work activities for disability beneficiaries. This information will assist policymakers in refining programs designed to encourage sustained employment with the goal of reducing the number of active disability beneficiaries. Semi-structured qualitative interviews will occur in 2015 only.

There are no technical or legal obstacles to burden reduction.

A.7. Special Circumstances

There are no special circumstances that would cause SSA to conduct this information collection in a manner inconsistent with 5 CFR 1320.5.

A.8. Solicitation of Public Comment and Other Consultations with the Public

A.8.1. Federal Register

SSA published the 60-day advance Federal Register Notice on April 16, 2014, at 79 FR 21499, and we received no public comments. We published the 30-day FRN on July 2, 2014 at 79 FR 37828. If we receive any comments in response to this Notice, we will forward them to OMB.

A.8.2. Consultation with the Public

NBS-General Waves – We are basing this survey on the prior NBS which was designed under a prior SSA-approved contract. The Lewin Group, Cornell University, Westat, and Mathematica Policy Research staff provided input in the development of the original design. SSA consulted with its contractor, Mathematica Policy Research, and a technical support group (TSG) that convened in January 2013 to discuss revising the NBS to suit the survey’s new goals. In addition to their participation in the group meetings, selected TSG members were consulted on an ad hoc basis when we were addressing specific issues in the development of survey instrument design.

Semi-Structured Interviews – We developed the qualitative, semi-structured interview guide in consultation with experts at Mathematica Policy Research. The individuals who contributed to development of the revised General Waves and semi-structured interview design are listed in Table A.2, with their areas of expertise.

Table A.2. Consultants Outside the Agency

Name	Affiliation	Area(s) of Expertise
Gina Livermore	Mathematica Policy Research Washington, DC	Evaluation methods, survey design, disability programs, SSA data
David Stapleton		Evaluation methods, statistics, survey design, disability programs, SSA data
Debra Wright		Questionnaire design, interviewing persons with disabilities
Kirsten Barrett		Questionnaire design, interviewing persons with disabilities

A.9. Payment or Gifts to Respondents

SSA believes that some compensation is important to engender a positive attitude about the study and reduce attrition in follow-up interviews. Research shows that incentives increase response rates without compromising data quality (Singer and Kulka 2000), are effective at increasing response rates for people with lower educational levels (Berlin et al. 1992), and low-income and nonwhite populations (James and Bolstein 1990) as well.

NBS-General Waves - The contractor will pay survey respondents a modest sum to encourage response, facilitate cooperation, and demonstrate appreciation to participants for their time and effort. We propose to offer gift cards of $20 per interview to participants once they have met all appropriate criteria for participation. We used a $10 incentive in all prior rounds of the NBS. However, our experience at round 4, where we received significantly lower response rates than in the 3 prior rounds, suggests that a higher incentive may be necessary to achieve the targeted 80 percent response rate.

Several studies, including a meta-analysis of 39 experiments by Singer et al. (1999) suggest the effect of incentives on response rates is linear; the greater the incentive, the greater the difference in response rates between the lowest and highest incentive conditions (Church 1993; Yu and Cooper 1983). Therefore, we propose raising the incentive amount to $20 to help offset the general trend in declining response rates. We plan to offer a $20 gift card to participants to complete an interview once they have met all appropriate criteria for participation. We will use this approach in all three rounds.

In addition to the post-pay incentive, the contractor will send a $5 pre-paid gift card to nonrespondents within the final three months of each field period to encourage timely response. Once the survey is completed, respondents will then receive an additional $15 gift card. We base this decision on an experiment conducted as part of the round 1 NBS (the results of which were previously reported to OMB)³.

While a comparison of the overall response rates between wave 1 (control group) and waves 2-5 (treatment group) showed there was no statistically significant improvement in the completion rate for the treatment group, there was a significant decrease in the time it took to complete cases in the treatment group, suggesting that the prepay encouraged call-ins and reduced follow-up for unresolved cases.

Incentive Experiment - Due to declining response rates observed in the prior round 4 NBS, we propose embedding an experiment in the round 1 NBS-General Waves to determine whether an alternative incentive approach would maximize response at a lower overall cost. Evidence suggests that offering a higher incentive to sample members to respond early can improve response and reduce the overall costs associated with administering a survey (LeClere et. al 2012; Markesich and Kovac, 2003).

For this experiment, we will randomly assign two groups of respondents to one of two experimental conditions (as seen in Table A.3)-both offering higher incentives for early responses. Sample members in the “early differential incentive” group would have the opportunity to earn a $30 gift card if they complete the survey early in the interview period (for example, within the first two to four weeks) of the data collection period, or $20 for completing after that. Those in the “late differential incentive” group will have the opportunity to earn a $30 gift card if they complete the survey in the two to four weeks’ time period just prior to in-person interviewing attempts (approximately 12 weeks into the data collection period when a reminder letter is sent) or $20 after that. A third group, those receiving the standard $20 gift card upon survey completion, will serve as the comparison group. We will target 450 completes in both experimental groups; the remaining 3,100 completed interviews will come from the comparison group.⁴

Table A.3. Incentive Conditions

Group	Completes	Incentive Condition
Early differential incentive	450	$30 gift card for call-ins during the first two weeksa $20 gift card for remainder of data collection period
Late differential incentive	450	$20 gift card up until two weeks prior to CAPI $30 gift card for call-ins during two weeks preceding CAPIa $20 gift card for remainder of data collection period
Standard incentive	3,100	$20 gift card throughout data collection period

^a We may consider lengthening the call-in period by an additional one to two weeks, if early call-in productivity warrants.

We will send all three groups an advance letter. In the advance letter for the early differential group, we will include reference to the higher incentive offered for completing the survey in the first few weeks of data collection; the late differential and standard incentive group’s letter will not. Sample members will also receive a reminder postcard; the incentive amount referenced will differ based on group assignment. After the initial early call in period and up to the mailing of the reminder letter, we will treat all groups the same with regard to contact efforts and the incentive amount. We will send the reminder letter to all non-responders approximately 12 weeks into the field period. We will include in the late differential group’s letter a reference to the late call-in period; the early differential and standard incentive group’s letter will not. All groups receive a field locating letter approximately two- to four-weeks before the start of CAPI (all correspondence related to the experiment is included in Attachment D).

Our analysis of data from the experiment will allow us to determine if the differential incentive significantly increases call-in completions or significantly reduces interviewing and locating efforts for those cases. The knowledge gained from the NBS-General Wave incentive experiment will inform subsequent rounds of data collection. If the experiment demonstrates success in increasing call-in completes during the prescribed time periods and reducing overall cost, we plan to expand the offering to all sample members in subsequent rounds. Further, the results of this experiment could inform other studies involving SSA beneficiaries.

Semi-Structured Interviews - The contractor will provide a $20 gift card to respondents who participate in the semi-structured interviews at the completion of the interview to encourage participation and thank them for their participation. It is important to offer a reasonably high incentive to participants to ensure timely recruitment and completion of the interviews within the desired time frame.

A.10. Assurances of Confidentiality

NBS-General Waves - Before the NBS-General Waves interviews, we will notify SSI recipients and DI beneficiaries by mail of their selection into the survey, the survey’s purpose, and SSA’s desire to interview them. The letter will provide assurance that all information collected will be confidential, unless required by law, and will not be used in any way that would affect their program eligibility or payments. It will also indicate the toll-free telephone number to call if they have questions about the study, as well as links to the contractor and SSA websites (see Attachment B). When we subsequently contact survey sample members for an interview, we will again advise them of the purpose of the survey and reassure them of confidentiality.

Semi-Structured Interviews - When we recruit respondents for the qualitative, semi-structured interviews via phone, we will assure participants that the information will be kept confidential, unless required by law, and not used in any way that would affect their program eligibility or payments, if applicable. At the time of the interview, we will again advise participants of the purpose of the survey and reassurance of confidentiality. At this time, we will also provide the participants with a toll-free telephone number to call if they have questions about the study.

SSA protects and holds confidential the information we collect in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular No. A-130. We conducted a privacy threshold analysis examining the procedural safeguards and policies for the new rounds of the NBS in March 2013. The SSA-approved contractor has a great deal of experience handling sensitive data and has standard operating procedures in place to ensure the confidentiality of computerized and paper records, including the use of passwords and encrypted identifiers, to prevent direct or indirect disclosures of information. Furthermore, the contractors information management systems fully complies with the Government’s information systems requirements.

The contractor will employ the following safeguards to carry out privacy assurances:

• We will store sample selection, survey data, and qualitative interview materials on an encrypted network drive. Access to data is limited to those who have direct responsibility for providing the sample and maintaining sample locating information. Staff access to data storage and files is limited to authorized personnel who have passwords. At the conclusion of the research, we will destroy these data. We will conduct audits on an ongoing basis to compare the list of cleared staff to those charging time and with access to restricted folders verifying all appropriate staff has clearance.

• We will employ a password protected screen saver that is automatically activated when NBS project staff are away from their work area.

• We will maintain, in separate files that can be linked only by sample identification number, all identifying information, survey responses, and interview materials. Access to the file linking sample identification numbers with a survey respondent’s identification and contact information is limited to a small number of individuals who have a need to know.

• We will strictly limit any access to hard copy documents. We will label documents with a subject identification number and store such information in a locked file cabinet in a secure facility. We will shred discarded material.

• We will protect via password all electronic tablets used for CAPI data collection, and will encrypt all audio files created during qualitative interviews and all other data collected. For CAPI, we will transmit only over secure connection any confidential case information (name, address, and telephone number) to field interviewers. We will never transmit Social Security numbers to field interviewers.

• The contractor will only transfer data to SSA using a secure transfer file site or by email using WinZip 9.0 password encryption (256-bit AES encryption) to protect the file.

• All employees will sign and adhere to a confidentiality pledge (provided in Attachment E).

A.11. Justification for Sensitive Questions

Although some questions in this survey may be potentially sensitive, they are necessary to conduct a thorough survey. Moreover, SSA will ensure the information remains confidential, as per the measures described in Section A.10.

NBS-General Waves - The NBS-General Waves contains questions on demographic and household information, education, health status, functioning, employment and employment service use, use of public programs, personal attitudes, income, and health insurance coverage. A number of questions are potentially sensitive, particularly given the fact that sample members, by definition, may have a disability. All health status and functioning questions are potentially sensitive. A variety of attitudinal questions (such as those relating to the respondent’s willingness to re-enter the labor force) and behavioral questions (such as those relating to drug and alcohol use) may also be viewed as sensitive in nature. Finally, as in most surveys, questions on earnings and income are often viewed as sensitive. This will be the case in this study, given that SSA is sponsoring the study and earnings are pertinent to eligibility for disability benefits and SSI program eligibility.

We will use the survey data collected to provide critical information about the supports and challenges that DI beneficiaries and SSI recipients face going back into the labor force. We will collect only information not available from other sources and considered critical to the evaluation of important outcomes. Research has shown that health status and functioning variables are critical determinants of employment outcomes, and treatment or accommodation of health conditions or impairments can be a key component of rehabilitation. The only administrative data on this topic is the impairment designation SSA examiners use when deciding an individual’s SSI/SSDI eligibility. Drug or alcohol abuse is also a key determinant of employment outcomes, and treatment of abuse could be a key determinant to the program’s success. SSA administrative data have some information on substance abuse at the time of the disability determination, but it is incomplete and there is no information on post-award behaviors. Income and benefits from other sources, including family members, can be a key determinant of success, and might also be affected by return to work (e.g., reduced or lost due to higher earnings).

Whenever possible, questions were adopted from or modeled after questions on existing collection instruments that we have used in previous studies of the general population or studies of DI beneficiaries and SSI recipients. In particular, questions are modeled from the following instruments:

• Alcohol use and abuse from the CAGE.⁵

• General health status from the short form (SF)-8 (8-Item Short-Form Health Survey), which is a derivation of the longer SF-36, an instrument that has been widely used in studies to rate overall health status.⁶

• Activities of daily living limitations and functioning adapted from the Survey of Income and Program Participation (SIPP), and the main survey instrument being used in SSA’s National Study of Health and Activity (NSHA).

• Sources of income and employment status adapted from questions in the Current Population Survey (CPS) NSHA, and SIPP.

During the consent process and at the start of each interview, we will inform respondents that the information they provide is confidential; used only for research purposes. We will also tell them they are at liberty to decline answering questions they find too sensitive. Interviewers will be trained on how to administer the instruments and probe on sensitive issues.

Semi-Structured Interviews - The qualitative, semi-structured interview questions are not sensitive in nature. Respondents will determine what they are comfortable sharing during the interview. The interviewer may provide probes or ask follow-up questions to gain more in-depth information, but respondents can opt to not provide that information if they are uncomfortable doing so.

A.12. Estimates of Public Reporting Burden

NBS-General Waves - SSA will obtain the NBS-General Waves survey data from sampled DI beneficiaries and SSI recipients. Table A.4 shows the expected number of survey respondents, the frequency with which we will interview them, and the expected completion time by round. To estimate the annualized hour burden on survey respondents and the qualitative interview participants, we have made the following assumptions for each group in Table A.4:

• We will administer in each year-2015, 2017, and 2019-the NBS-General Waves survey to 4,000 active SSI recipients and DI beneficiaries. Based on prior survey administrations, the average time for active respondents to complete the survey is estimated at 45 minutes each round.⁷

• We will administer in 2017 and 2019 the NBS-General Waves survey to 4,500 and 3,000 successful workers. In 2019, we will re-interview 2,250 successful workers from round 2. Based on prior survey administrations, we estimate the average time for successful workers to complete the survey at 55 minutes (due to additional items specific to this group).

Semi-Structured Interviews - We will conduct the qualitative, semi-structured interviews in 2015 only. We estimate the average time for respondents to complete a semi-structured interview at approximately 1 hour (see Table A.4).

Table A.4. Annual Burden Estimates

Administration Year	Number of Respondents	Number of Responses per Respondent	Average Burden Per Response (in hours)	Total Annualized Burden Hours
2015
Cross-Sectional Samples
Representative Beneficiary Sample	4,000	1	.75	3,000
Successful Worker Qualitative Interviews	90	1	1.00	90
			Subtotal	3,090
2017
Cross-Sectional Samples
Representative Beneficiary Sample	4,000	1	.75	3,000
Successful Workers	4,500	1	.92	4,140
			Subtotal	7,140
2019
Cross-Sectional Samples
Representative Beneficiary Sample	4,000	1	.75	3,000
Successful Workers	3,000	1	.92	2,760
Longitudinal Samples
Successful Workers	2,250	1	.75	1,688
			Subtotal	7,448
Total Burden	21,840			17,678

The total burden for this ICR is 17,678 hours across years. This figure represents burden hours, and we did not calculate a separate cost burden.

A.13. Annual Cost to the Respondents (Other)

This collection does not impose a known cost burden on the respondents.

A.14. Annual Cost to Federal Government

The estimated cost to the Federal Government for the NBS-General Waves is $12,758,932. This estimate includes the contractor’s cost to revise the sample design and questionnaire, administer the qualitative and quantitative data collection, and prepare data files and documentation for each round. The average annual cost is $1,822,705. We base these estimates on the contractor’s experience with prior rounds of the NBS.

A.15. Program Changes or Adjustments to the Information Collection Request

This is a new data collection that increases the public reporting burden. See question #12 for updated burden figures.

A.16. Plans for Publication Information Collection Results

A.16.1. Tabulation Plans

NBS-General Waves - In the NBS-General Waves, we will focus our analyses on factors leading to successful and unsuccessful work attempts. Specifically, our analyses based on this survey data will encompass the following: use of services, employment outcomes and program exits, factors leading to successful and unsuccessful work attempts, and differences across respondent subgroups (for example, differences based on program title). SSA will draw on previous NBS analyses for the purposes of making comparisons.

We will link the NBS-General Waves data to our administrative data for analyses, examining how various respondent characteristics correlated with duration of benefit or payment receipt, benefit and payment amounts, and program exits; we will add administrative information on the use of work incentives and program exits due to work.

Semi-Structured Interviews - We will review qualitative data and develop a coding scheme. We will determine the frequency of codes and identify recurring themes. We will describe potential theories to explain how personal, environmental, and employment factors relate to the respondents’ ability to obtain and retain employment and advance in the workplace.

A.16.2. Publication Plans

NBS-General Waves - We will make the findings from the survey available in a report containing tabulations, graphics, and narrative that will provide a comprehensive understanding of DI beneficiaries and SSI recipients and their service use, employment outcomes, and degree of self-sufficiency.

The prior rounds of the NBS were critical in providing adequate responses to Congressional directives and questions. SSA will continue to use the NBS-General Waves to address these questions and provide the information publicly through our responses to quarterly financial reports. We will focus further analyses on better understanding DI beneficiary and SSI recipient work behavior and how to develop and support increased employment and earnings. The documentation and public use data files will be available on both SSA’s and DATA.gov’s websites.

Semi-Structured Interviews – We will also make the findings from the qualitative interviews available in a report containing narrative, tabulations, and figures of themes and relationships that emerged through the qualitative inquiry. We will post the reports on the SSA website.

A.16.3. Time Schedule

The timeline for the NBS-General Waves and semi-structured interviews is shown in Figure A.1. It calls for three waves of data collection, spaced two years apart. We expect data collection for the first round to begin in February 2015 and continue through October 2015. We will make public-use data files for this round available in August 2016. Subsequent survey rounds follow a similar schedule, with the final combined data and documentation, including all three survey rounds, completed in the fall of 2020. The timing of the qualitative interviews is shown under the round 1 section in Figure A.1.

Figure A.1. Project Timeline

Round 1

	2014										2015												2016
Task	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	March	April	May	June	July	August
Instrument Revisions and Computer Assisted Interviewing (CAI) programming
Revise Sample Design
Implement Sample Design
Train Interviewers
Data Collection (Survey)
Recruitment for Interviews
Conduct Interviews
Prepare Data Files and Documentation

Round 2

	2016										2017												2018
Task	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	March	April	May	June	July	August
Implement Sample Design
Train Interviewers
Data Collection
Prepare Data Files and Documentation

Round 3

	2018										2019												2020
Task	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	Mar	April	May	June	July	Aug	Sept	Oct	Nov	Dec	Jan	Feb	March	April	May	June	July	August
Implement Sample Design
Train Interviewers
Data Collection
Prepare Data Files and Documentation

A.17. Displaying the OMB Approval Expiration Date

SSA is not requesting an exception to the requirement to display the OMB approval expiration date.

A.18. Exception to Certification Statement

SSA is not requesting an exception to the certification requirements at 5 CFR 1320.9 and related provisions at 5 CFR 1320.8(b)(3).

Part A References

Berlin, M., L. Mohadjer, J. Waksberg, A. Kolstad, I. Kirsch, D. Rock, and K. Yamamoto. “An Experiment in Monetary Incentives.” In Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association, 1992.

Church, Allen H. “Estimating the Effects of Incentives on Mail Response Rates: A Meta-Analysis.” Public Opinion Quarterly, vol. 57, 1993, pp. 62-79.

James, J., and R. Bolstein. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly, vol. 54, no. 3, autumn, 1990, pp. 346–361.

LeClere, Felicia, Sheldonn Plummer1, Jennifer Vanicek, Ashley Amaya, and Kari Carris. Household Early Bird Incentives: Leveraging Family Influence to Improve Household Response Rates. Paper presentation, 2012 Joint Statistical Meeting, San Diego.

Livermore, Gina, David C. Stapleton, and Meghan O’Toole. “Health Care Costs Are a Key Driver of Growth in Federal and State Assistance to Working-Age People with Disabilities.” Health Affairs, vol. 30, no. 9, September 2011a, pp. 1664–1674.

Livermore, Gina A. “Social Security Disability Beneficiaries with Work-Related Goals and Expectations.” Social Security Bulletin, vol. 71, no. 3, August 2011, pp. 61–82.

Markesich, Jason and Kovac, Martha. “The Effects of Differential Incentives on Completion Rates: A Telephone Survey Experiment with Low-Income Respondents.” Paper presentation, 2003 American Association of Public Opinion Research Annual Meeting, Nashville.

Mann, David R., and David C. Stapleton. “Fiscal Austerity and the Transition to Twenty-First Century Disability Policy: A Road Map.” Princeton, NJ: Mathematica Policy Research, November 2011.

Singer, E., and R. A. Kulka. “Paying Respondents for Survey Participation.” Ann Arbor, MI: The University of Michigan, Institute for Social Research, Survey Research Center, 2000.

Singer, E., R.M. Groves, and A.D. Corning. “Differential Incentives: Beliefs about Practices, Perceptions of Equity, and Effects on Survey Participation,” Public Opinion Quarterly, vol. 63, 1999, pp. 251-260.

Wright, Debra, Gina Livermore, Denise Hoffman, Eric Grau, and Maura Bardos. “2010 National Beneficiary Survey: Methodology and Descriptive Statistics.” Washington, DC: Mathematica Policy Research, 2011.

Heaviside, Sheila, Donsig Jang, Kirsten Barrett, Geraldine Mooney, and Kelly Kang. “Impact of Monetary Incentives and Web Survey Options in the 2008 National Survey of Recent College Graduates (NSRCG) on Increasing Response Rates in Historically Low Responding Groups.” Paper presentation, 2010 American Association of Public Opinion Research Annual Meeting, Chicago.

Yu, J., and H. Cooper. “A Quantitative Review of Research Design Effect on Response Rates to Questionnaires.” Journal of Marketing Research, vol. 20, 1983, pp. 36-44.

1 Active status includes beneficiaries who are currently receiving cash benefits as well as those whose benefits have been temporarily suspended for work or other reasons. It does not include beneficiaries whose benefits have been terminated.

2 At round 1, we planned to interview approximately 4,500 beneficiaries whose benefits have been suspended due to work. In rounds 2 and 3, we planned to complete approximately 3,000 interviews with suspended beneficiaries that have been selected for the cross-sectional samples. In addition, 2,500 beneficiaries in suspense at round 1 would be followed longitudinally in rounds 2 and 3.

3 The treatment group for this experiment included 2,553 unresolved cases—those with a valid address who were mailed the prepayment incentive letter (n=1,711) and those without a valid address for whom a check could not be mailed (n=842). The control group consisted of 1,427 cases in wave 1, 480 of which were unresolved at the time of the experiment. The control group received the post-pay $10 incentive once an interview was completed. All eligible unresolved cases in the treatment group received a prepayment letter with a $10 check before additional calls were attempted. The experiment was designed to be able to detect an improvement of as little as 3.5 percent.

4 With 450 completes in each group, we could detect a 7.5 percentage point difference in completion rates between the two groups at 80 percent power. For example, a significant difference in the rate of completes within the first 2 weeks would be found if 17.5 percent of sample members in the differential group called in and completed the survey during that time period as compared to 10 percent of sample members in the standard incentive group.

5 The CAGE is a brief four-item screen for alcohol abuse. The letters in the acronym reflect the four alcohol-related concepts queried: cut-down; annoyed; guilty, and eye-opener.

6 Various SF health surveys are used as measures of health status and quality of life. The SF-8 is an empirically derived subset of items from the 36-item SF-36, which many considered to be the “gold standard” in the assessment of health status. The SF-36 was designed for use in clinical research, health policy evaluations, and general population surveys. The SF-12 and SF-8 were derived with the purpose of reducing respondent burden. The SF-8, SF-12, and SF-36 assess 8 health concepts—limitations in physical activities due to a health problem, limitations in social activities due to a health problem, limitations in usual role activities due to a physical health problem, limitations in usual role activities due to an emotional problem, pain, general mental health, vitality, and general health perceptions.

7 Time estimates are based on interview lengths in prior waves for active beneficiaries. The estimates reflect expected percentages of the sample who will skip selected questions and therefore have shorter interview times (for example nonworking beneficiaries and those who have not used employment services).