DSI Report Card Survey Description

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

TITLE OF INFORMATION COLLECTION: Disability Service Improvement (DSI) Report Card

SSA SUB-NUMBER: B-05

DESCRIPTION OF ACTIVITY (give purpose of activity, provide specific information; i.e., date(s) of survey, number of focus groups, locations, etc.):

BACKGROUND

SSA is currently implementing improvements in its disability application process. To assess beneficiary satisfaction with these improvements, SSA plans to conduct a three-phase survey of initial disability applicants. Three groups of disability applicants will be surveyed: 1) mid-process applicants, who have just recently filed an application but have not yet received a decision; 2) post-adjudicative applicants who were awarded benefits; and 3) post-adjudicative applicants whose claims were denied.

SURVEY

Description of Survey

The survey will utilize two questionnaires developed by SSA, one for mid-process applicants and one for post-adjudicative applicants. Based on SSA’s experience conducting previous surveys with this population, we have identified some key service elements that have an impact on satisfaction with the disability application process. The mid-process survey addresses initial intake in greater detail, as mid-process respondents will have greater recall of that experience; the post-adjudicative survey deals with such end-of-process issues as rating processing time and decision notice clarity.

The Mid-Process Survey includes the following questions:

• How respondents filed their disability application;

• If anyone besides an SSA employee helped them with their application;

• How they would rate multiple satisfaction assessments related to when they decided to file, the explanations SSA gave them about the process, providing information to SSA, how SSA employees did their job, and their overall experience.

The Post-Adjudicative Application includes the following questions:

• How respondents filed their disability application;

• If anyone besides an SSA employee helped them with their application;

• How the respondents would rate multiple satisfaction factors related to providing medical information, finding out what happened to their application, how SSA employees performed their job, getting the decision on their application, and their overall experience.

Attached are copies of both surveys.

Statistical Information

Sample Selection

The DSI Report Card Survey will be conducted in three phases between November 2006 and March 2007. Each phase will measure the satisfaction of a different group of disability applicants randomly selected in the timeframes shown:

Stratum	Sample	Date Selected
Mid-process - shortly after filing but before a decision is made	26,000 (500 cases per 50 states, D.C., and Puerto Rico)	November 2006
Post-adjudicative Awards - just after they receive the decision	26,000 (500 cases per 50 states, D.C., and Puerto Rico)	January 2007
Post-adjudicative Denials - just after they receive the decision	26,000 (500 cases per 50 states, D.C., and Puerto Rico)	February 2007

As part of SSA’s overall evaluation of the new DSI process, data on factors such as productivity, timeliness, accuracy, and cost are being captured for each of the 50 States, the District of Columbia and Puerto Rico. In order to include applicant perceptions of the process at a comparable level, samples in all three phases of the survey will be stratified to yield statistically reliable results for each of the 52 state-level entities. We plan to select 500 cases per State in each phase (26,000 total per phase), for an overall sample size of 78,000.

In each phase of the survey, in order to derive a satisfaction rating for all strata combined, results will be weighted to represent the universe of disability applications processed by SSA in fiscal year (FY) 2007. Anticipated volumes for each of SSA’s ten regions are displayed in the table below.

Projected Annual Initial Disability Applications Processed – Title II and Title XVI Combined
	Cases Processed Annually	Percent Cases Awarded	Percent Cases Denied
National	2,616,000	35.7	64.3
Boston includes CT, ME, MA, NH, RI, VT	104,000	44.4	55.6
New York includes NJ, NY, PR	229,000	42.6	57.4
Philadelphia includes DE, DC, MD, PA, VA, WV	261,000	40.2	59.8
Atlanta includes AL, FL, GA, KY, MS, NC, SC, TN	608,000	29.0	71.0
Chicago includes IL, IN, MI, MN, OH, WI	463,000	31.6	68.4
Dallas includes AR, LA, NM, OK, TX	361,000	37.6	62.4
Kansas City includes IA, KS, MO, NE	119,000	30.0	70.0
Denver includes CO, MT, ND, SD, UT, WY	57,000	30.3	69.7
San Francisco includes AZ, CA, HI, NV	321,000	43.7	56.3
Seattle includes AK, ID, OR, WA	93,000	36.3	63.7

The key variable for this survey will be the percent of disability benefit applicants who are satisfied with SSA’s overall service (i.e., those who rate service excellent, very good or good). In a survey of disabled applicants conducted by telephone in 2005, the overall satisfaction rating from awarded respondents was 95 percent and from denied respondents, 70 percent. (Mid-process applicants were not included in that sample.) We expect satisfaction levels to be similar in the current survey; mid-process applicants should have a satisfaction level close to that of awarded applicants because their opinions have not been influenced by any adverse decision.

Methodology

All sampled individuals will be mailed a brief (one page front and back) questionnaire to complete. The questionnaire will be accompanied by a letter explaining the purpose of the survey and encouraging their participation. The letter will also offer participants the option of completing the survey via the Internet. The questions on the Internet version will be identical to those on the paper survey, except that they will be presented one at a time to the respondent. The OMB expiration date will be displayed online. Both the Paperwork Reduction Act and the Privacy Act explanations will be included in the cover letter. Two weeks after the initial survey package is mailed, a follow-up letter and another copy of the survey form will be sent to all sampled individuals. Attached are copies of the surveys, and the cover and follow-up letters.

Response Rate

Using a methodology similar to this one (i.e., a two part mailing of a one page survey form), OQP’s annual Office Visitor Survey (OVS) has consistently achieved an overall response rate of about 50 percent. Most of the transactions included in the OVS sample are less complex and less significant than filing an application for disability benefits. Because becoming too disabled to work is a life-changing event, we believe the DSI Report Card Survey will have considerable salience for sampled individuals. We therefore anticipate that a response rate of at least 60 percent will be attainable in this survey using an initial mailing and one follow-up mailing. We will attempt to increase the response rate by offering sampled individuals the choice of completing the survey over the Internet (an option not available in OVS).

Other steps we will take to maximize the response rates include:
1) Enclosing a cover letter signed by SSA’s Chief Quality Officer;

2) Sending a follow-up letter and questionnaire to all individuals within two weeks of the initial mailing;

3) Designing a form that is designed to be easily read and completed by a disabled population;

4) Sending a survey in Spanish where SSA records indicate that is the preferred language, and

5) Providing an 800 number for individuals to call with questions about the survey.

Sampling Variability

For this key variable, assuming the 60 percent response rate discussed below (300 respondents per stratum), the sampling variability at the 95 percent confidence level will range from +/‑ 2.5 percent (95 percent satisfied) to +/- 5.2 percent (70 percent satisfied). For the overall rating for all strata combined (15,600 respondents), the sampling variability for both the 95 percent and 70 percent satisfaction rates will be less than +/- 1 percent. These are acceptable ranges of variability for the intended purpose of the survey.

All sampling and data analysis will be performed by the Office of Quality Performance (OQP). Statistical support is provided by Rich Lehigh, Director, Division of Statistics, Office of Statistics and Special Area Studies, OQP. He can be reached at (410) 965-2788.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT (No more than $25 can be authorized under OMB rules):

N/A

USE OF SURVEY RESULTS:

SSA’s overall evaluation of the new DSI initiative includes an assessment of many different aspects of the process, such as productivity, timeliness, accuracy, and cost. Applicant perceptions of their experience filing for benefits will provide another important element in measuring the impact of these process changes. Moreover, the survey will allow us to compare satisfaction across strata and provide insight about each group for future planning. Ultimately, the initial year’s DSI Report Card surveys will serve as the baseline for establishing specific numeric goals related to service satisfaction with the new process.

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time

(/60) = annual burden hours):

	Number of Respondents	Participation Time	Burden
Mid-process applicants	26,000	5 minutes	2,167 hours
Post-adjudicative awarded applicants	26,000	5 minutes	2,167 hours
Post-adjudicative denied applicants	26,000	5 minutes	2,167 hours
Totals	78,000	-	6,501 hours

NAME OF CONTACT PERSON: Faye I. Lipsky

TELEPHONE NUMBER: 410-965-8783

MAILING LOCATION: Social Security Administration

Attn: Faye I. Lipsky

1340 Annex

6401 Security Blvd.

Baltimore, MD 21235

MAJOR OFFICE, OFFICE, DIVISION, BRANCH: DCBFM/OPLM/CPM/RCT

DSI Report Card Survey

September 2006

5