Supporting Statement for
Accelerated Benefits Demonstration Project
OMB No. 0960-NEW
Part B
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Respondent Universe and Sampling Methods
The AB demonstration project will be limited to new SSDI beneficiaries who meet the following criteria: (1) they are uninsured, (2) they are not currently receiving Supplemental Security Income (and thus eligible for Medicaid), (3) they are not currently receiving Medicare (for example, some new SSDI beneficiaries are exempted from the waiting period for entitlement to Medicare because they suffer from end-stage renal disease), (4) they live in certain metropolitan areas, and (5) for those new SSDI beneficiaries who are in the waiting period, the onset of their disability occurred recently enough that they have at least 18 months before they will be eligible for Medicare.
Regarding the fourth criterion – living in certain metropolitan areas – we are still working with SSA to determine which metropolitan areas will be included in the demonstration project. Based on our preliminary calculations, we believe at least 10 large metropolitan areas and as many as 150 metropolitan areas will have to be included in the study to reach our target sample size.1 These projections are based on the assumption that we will be able to locate and recruit 40 percent of eligible beneficiaries. The greater number of metropolitan areas will have to be used if only 13 percent of new beneficiaries are uninsured (based on calculations from the National Beneficiary Survey) while the smaller number will be used if 40 percent are uninsured.2 During the first three months of recruitment, we will gather additional information to assess those assumptions and the information will be used to update our plans for where the sample will be recruited.
Information on each of the criteria can be obtained from SSA administrative records except for whether an individual is uninsured. The first step in choosing the sample will therefore be to use SSA administrative records to obtain a list of all new SSDI-only beneficiaries who meet all other criteria and whose disability onset was no more than nine months prior. Because new SSDI beneficiaries are eligible for Medicare 29 months after disability onset, the last restriction would allow us two months to locate and screen the beneficiary and obtain consent, and still have 18 months of eligibility for AB remaining.
Recruitment will take place in two phases. During the first phase, we will attempt to recruit 20 participants per month in July through September, 2007. The purpose of this first phase is to learn more about our ability to recruit participants and to have a gradual roll out of the intervention. Recruitment during this phase will be limited to four large metropolitan areas. To choose the sample, monthly samples of potentially eligible new SSDI-only beneficiaries from the four MSAs will be received from SSA. Several replicates of 10-20 beneficiaries would be randomly chosen from this list and released to interviewers. Enough replicates would be released to keep interviewers working for four weeks. Based on the performance of the first replicates, we would decide how many more replicates to release in the coming weeks. Recruitment in a month will stop when this process produces 20 eligible beneficiaries who consent to be in the study and to be randomly assigned.
During the second phase, we will attempt to contact all individuals chosen through the process described above. That is, for the enrollment period we will begin with the universe of individuals who meet these criteria rather than sampling from that universe3. Those who are contacted will first be screened to determine whether they are uninsured. If they are uninsured, they will be asked to consent to be in the study. The goal is to obtain consent to be in the study from 2,000 individuals. Assuming that 13 percent of new beneficiaries will be uninsured when we screen for study eligibility and that 40 percent provide consent, we would begin with a sample of 85,000 beneficiaries, of whom we expect to obtain telephone numbers for 65 percent or 55,250. Of those sample members who can be reached by telephone, we expect to screen 70 percent or 35,812 cases for insurance status. Since recruitment is schedule to take place over a fifteen-month period, we would screen about 2,500 new beneficiaries each month.
Except for the first phase of recruitment, which is described above, neither stratification nor sampling will be used in conducting the baseline survey or the 6-month early-use survey. We will attempt to contact all individuals who meet study criteria during the projected twelve months of the second phase of enrollment, will screen all who can be contacted for eligibility for the study (i.e., whether they currently have health insurance), and will conduct the full baseline with all of those who provide consent. For the 6-month survey, we will attempt to contact all individuals who enter the study in the first six months of enrollment.
a. Information Collected During First Phase of Recruitment
During the first phase of recruitment for the study, we plan to add 20 individuals per month to the study sample in July, August, and September 2007. Recruitment during this period will also provide information on the recruitment process, including, the proportion who can be reached by phone, the proportion who agree to answer questions to determine whether they have insurance, the proportion who have insurance, and the proportion who will agree to be in the study and to be randomly assigned.
The precision of the information collected during this phase will depend on how easy it is to recruit individuals since that will determine the size of the starting sample. Since the team has conducted surveys in the past with similar samples, we are fairly confident in our assumptions about what proportion we will be able to reach and what proportion will agree to answer the screening questions. We are less certain about what proportion will be uninsured and what proportion will agree to be in the study. We therefore discuss two potential scenarios based on pessimistic and optimistic assumptions of insurance status and the willingness of eligible individuals to agree to be in the study.
Our first scenario, which is based on a much larger screener effort, assumes that 13 percent of beneficiaries will lack health insurance and that 40 percent of those will agree to be in the study. Under this scenario, we would begin with a sample of 2,535 beneficiaries, of whom we expect to obtain telephone numbers for 65 percent or 1,648 beneficiaries. Of that group, we expect to screen 70 percent or 1,153 cases to complete the screener for insurance status. Of this group, 150 will lack health insurance (13 percent of 1,153) and 40 percent of the uninsured will consent to be in the study, or 60 over the three-month period.
Our second scenario assumes that 40 percent will lack health insurance and that 60 percent of those will agree to be in the study. If these assumptions are correct, we would try to contact 550 new SSDI-only beneficiaries, we would expect to reach 357 of them and conduct the screening questions with 250. Of that group, 100 would not have health insurance and 60 would be randomly assigned.
The following exhibit shows the standard errors of our estimates of the four main quantities, under both scenarios.
Table 3
Standard errors of information collected during first phase of recruitment
Variable |
Scenario 1: Pessimistic assumptions for insurance status and consent rate |
Scenario 2: optimistic assumptions for insurance status and consent rate |
Proportion contacted by phone |
0.9 |
2.0 |
Proportion completing screen |
1.1 |
2.4 |
Proportion without insurance |
1.0 |
3.1 |
Proportion who consent to be in the study |
4.0 |
4.9 |
The table suggests that our information will be relatively precise, particularly under scenario 1. For example, if the rate of accurate phone numbers is 65 percent, the standard error of our estimate will be .9-2.0 percentage points. Thus, the 95 percent confidence interval will cover a range of 3.6 percentage points (for example, 63.2-66.8 percent) to 8 percentage points (for example, 61-69).
Our estimate of the uninsurance rate will also be precise if only 13 percent are uninsured, e.g., 11-15 percentage points. Even if 40 percent are uninsured, as under scenario 2, the 95 percent confidence interval would be plus or minus 6.1 percentage points, e.g., 33.9-36.1 percentage points. We will certainly be able to rule out one extreme or the other.
a. Random Assignment Demonstration Study
The main analytical strategy in the impact analysis will be to compare outcomes among those randomly assigned to the three research groups (AB Plus, AB Basic, and the control group). To increase statistical precision, comparisons will be regression-adjusted for baseline characteristics such as primary impairment, time since disability onset, self-reported health status, and prior work history.
As described earlier, the 2,000 sample members will be assigned so that 800 are in the AB Plus group, 400 are in the AB Basic group, and 800 are in the control group. This will allow us to have an 80 percent chance of generating statistically significant findings of the effects of AB Plus if the true impact on an outcome is .124 standard deviations using administrative records and .139 using survey measures, assuming an 80 percent response rate.4 In comparing AB Basic to AB Plus or the control group, the minimum detectable effects are .134 standard deviations for the full sample (that is, using administrative records) and .150 standard deviations for survey-based impacts. For example, if 4 percent of the control group earns above Significant Gainful Activity (SGA), the study would have an 80 percent chance of generating a statistically significant impact estimate of AB Plus if the true impact is 2.4 percentage points. It would likewise have an 80 percent chance of generating a statistically significant impact estimate of AB Basic if the true impact of AB Basic is 3.0 percentage points.
We will rely on our experience conducting disability surveys to develop procedures that will maximize the response rates obtained. These procedures will include:
Multiple methods for tracking and locating beneficiaries, including the use of extracts from SSA administrative data to capture address updates during the course of the survey, the use of an independent vendor providing commercially available contact information, combined with MPR’s internal respondent tracking efforts to locate beneficiaries.
The availability of additional flexible response options, including TTY, TRS and instant messaging to facilitate the participation of respondents with hearing impairments.
Interviewer training that stresses the importance of respondent cooperation, and develops skills for averting and converting refusals.
Interviewer training on when and how to select an appropriate proxy to conduct the interview.
A bilingual module to help bilingual interviewers assess whether to conduct an interview in Spanish or English, and to cover differences in dialects.
Protocols for breaking off and then resuming interviews to accommodate beneficiaries who may become fatigued during the interview.
4. Tests of Procedures
a. Pretest
The draft Baseline Survey instrument will be pretested to assess the data collection process, evaluate the clarity of the questions asked, identify possible modifications to either question wording or question order that could improve the quality of the baseline data, and estimate respondent burden. In addition to monitoring the pretest interviews to identify questions that are problematic for interviewers or respondents, we will conduct an interviewer debriefing upon completion of the pretest. New questions (not used in previous, similar surveys) will be scrutinized especially closely during the pretest.
Howard Bloom, MDRC
Eric Grau, MPR
Frank Potter, MPR
Howard Rolston, Consultant to SSA
These consultants may be reached through SSA if OMB wishes to discuss statistical issues with them.
NOTE: Please see the following page for references.
Committee on Finance, U.S. Senate. 1972. Social Security Amendments of 1972: Report to Accompany HR 1 (92nd Congress 2nd session), S. Rept. No. 92-1230.
Gold, Marsha, and Beth Stevens. “Medicare’s Less Visible Population: Disabled Beneficiaries Under Age 65.” Monitoring Medicare+Choice: Operational Insights, no. 2, May 2001. Washington, DC: Mathematica Policy Research, Inc..
Hall, Jean P., and Michael H. Fox. “What Providers and Medicaid Policymakers Need to Know About Barriers to Employment for People with Disabilities.” Journal of Health & Social Policy, vol. 19, no. 3, 2004.
Thornton, Craig, Thomas Fraker, Gina Livermore, David Stapleton, Bonnie O’Day, Tim Silva, Emily Sama Martin, John Kregel, and Debra Wright. 2005. Evaluation of the Ticket to Work Program: Second Evaluation Report, Mathematica Policy Research, Washington DC (Draft Report).
1The 15 largest metropolitan areas in 2000 were New York City, Los Angeles, Chicago, Dallas-Fort Worth, Washington, Houston, Boston, Atlanta, Philadelphia, Riverside, Phoenix, Minneapolis-St. Paul, San Diego, St. Louis, and Baltimore.
2 Our estimates of an uninsurance rate of 13 percent is based on information on new SSDI-only beneficiaries in the National Beneficiary Survey. While this is our best estimate of insurance status, there are reasons more individuals in our sample will lack insurance. For example, individuals who are eligible for AB must have at least 18 months until they would be eligible for Medicare, while new beneficiaries in the National Beneficiary Survey might have much less time before becoming eligible for Medicare. In addition, the estimate from the National Beneficiary Survey is just an estimate, and it is possible the true uninsurance rate is higher. As an upper bound on the uninsurance rate, we are using the national uninsurance rate of 40 percent.
3 Strictly speaking, this is true only during the second phase of enrollment, during which we plan to enroll 1,940 of the 2,000 study participants. During the three month initial phase of recruitment, we would …
4These minimum detectable effects are for a two-tailed t-test at the 10 percent significance level and have not been regression adjusted for baseline characteristics or for having multiple comparisons (multiple outcomes and comparisons across three research groups).
AB Project Supporting Statement Part B
| File Type | application/msword |
| File Title | Supporting Statement for |
| Author | Faye |
| Last Modified By | Faye |
| File Modified | 2007-05-07 |
| File Created | 2007-05-07 |