Supporting Statement Part B

SUPPORTING STATEMENT

Part B

MEPS Alternative Sampling Design Pilot Test

Revised May 2022

Agency for Healthcare Research and Quality (AHRQ)

Table of Contents

B. STATISTICAL METHODS 3

1. Potential Respondent Universe and Sample Selection Method 3

2. Information Collection Procedures 4

3. Methods to Maximize Response Rate 4

4. Tests of Procedures 5

5. Statistical Consultation 6

B. STATISTICAL METHODS

1. Potential Respondent Universe and Sample Selection Method

The MEPS is a nationally representative sample of households within the United States, and currently samples households from those completing the National Health Interview Survey (NHIS). One of the primary goals of MEPS is to produce reliable estimates of medical expenditure totals, the measures of which show an uneven distribution of expenditures, with most household having relatively low expenditures and a very small proportion having very large expenditures. This pilot study aims to increase the efficiency of the MEPS sample with respect to high expenditure households so that more precise national estimates, and possibly even state-level estimates, can be obtained. The screener will also aim to identify households with Hispanic, Black, or Asian persons so that the sample sizes and precision of estimates for these domains can be increased/improved.

The proposed pilot study will include selecting an address-based sample for a screener to collect demographics and measures that may help identify race/ethnicity minority households and to predict high medical expenditures. We expect to select 11,600 sample members through a two-phase address-based sampling (ABS) design for the pilot study, with a 29% anticipated response rate to collect 3,393 completed screeners through the web and paper modes.

The screener sample will be selected in two phases to take advantage of the Master Beneficiary Summary File (MBSF), maintained by the Centers for Medicare and Medicaid Services (CMS), to identify potential high expenditure cases. Phase 1: A large first-phase sample will be selected from the USPS computerized delivery sequence file. This will be a nationally representative simple random sample and will contain standardized address fields. The cases in the first-phase sample that are also in the MBSF (which includes the entire Medicare population) will be identified through address matching. The MBSF contains rich auxiliary information for these matched individuals (e.g., chronic conditions) to help predict high medical expenditures. Phase 2: The first-phase sample will be sub-stratified with the MBSF information, and then used as the frame to select the second-phase (i.e., final) sample of 11,600 addresses for the MEPS pilot study data collection. Those predicted to have high medical expenditures will be sampled at a relatively higher rate. Finally, the 11,600 addresses will be randomly assigned to the four incentive treatment groups as described in Section 3.

The sample size of 11,600 for the pilot study was based on the power analysis result, the estimated design effect associated with differential sampling rates for selecting the second-phase sample, and the number of incentive treatment groups. First, to detect a difference of three percentage points in response rates, the effective sample size (calculated as the nominal sample size divided by design effect) required for each incentive treatment group is 2,320 (alpha = 0.05, power = 0.80, one-sided test). Second, the estimated design effect due to oversampling potential high medical expenditure cases using the MBSF information is 1.25. Third, there are four incentive treatment groups. Therefore, the overall sample size required for the pilot study data collection is 2,320 * 1.25 * 4 = 11,600.

2. Information Collection Procedures

For the pilot screener data collection, the questionnaire modes are presented sequentially where Web is the only mode available initially, and then the paper questionnaire is introduced in nonresponse follow-up contacts. This sequential multimode approach is referred to as a “Push-to-Web” and has been demonstrated to be effective in maximizing both total response and proportion of response by Web. We propose a total of five contact attempts:

• Advance postcard: All sampled addresses will be mailed an advance postcard to inform them of the pending mail initial invitation to complete the Web screener. The advance postcard also will inform recipients in the prepaid incentive groups that the initial invitation mailing will include a small cash token of appreciation, thus encouraging the household to open the initial invitation mailing.

• Initial invitation: All sampled addresses will be mailed an initial invitation that introduces the study and provides the website URL with a unique access PIN for the household. Based on social exchange theory, the initial invitation to those in the prepaid incentive groups will include a token cash incentive to motivate response.

• Reminder postcard: All sampled addresses will be mailed a sealed reminder postcard, reminding them to complete the screener by Web, or if completed, thanking them for their participation. The postcard will also include the website URL and unique access PIN for the household.

• First Nonresponse Follow-up: Sampled addresses from which no response has been received will then be mailed a nonresponse study package. The package will include a letter with information for accessing the Web screener and a paper questionnaire that can be completed and mailed back.

• Second Nonresponse Follow-up: Sampled addresses from which no response has been received after the first nonresponse follow-up will receive another replacement study package and information for accessing the Web screener.

The timing between contacts is shown in Figure 1. We anticipate a period of 60 days will be required for the full series of contacts and time for paper screeners to be returned following the last nonresponse contact.

Figure 1. Contact timing and sequence

3. Methods to Maximize Response Rate

In addition to testing whether we can recruit members of underrepresented groups through the proposed screening design, we also propose to test different incentive treatments to determine which combination of prepaid and promised incentives maximizes the response rate to the screener. Exhibit 1 provides a review of the four proposed treatments.

Exhibit 1. Proposed Pilot Study Incentive Treatments

Incentive treatment group	Prepaid incentive	Promised incentive	Sample size for treatment group
1	$0	$0	2,900
2	$2	$10	2,900
3	$5	$0	2,900
4	$5	$10	2,900
Total			11,600

In addition to the incentive regimen, the pilot study design includes multiple contacts to alert sample members to complete the web screener, as described in Section 2 above. Sample members not responding to the web screener will receive two nonresponse mailings that include hard copy screener questionnaires.

4. Tests of Procedures

We plan to test and evaluate the following components of the MEPS alternative design through the pilot study.

• The feasibility of accessing CMS data: This includes developing an agreement with CMS and, identifying the steps for obtaining the CMS data, and matching the initial larger sample with the MBSF to form the sampling strata for selecting the final screener sample. There is some uncertainty about whether and how the CMS data will be made available for the sample selection procedure described in Section B.1, so it is important to explore these details through the pilot study.

• Screening questionnaire development: The screener will aim to identify two population subgroups: (1) households with high expenditures or potential high expenditures; and (2) households with Hispanic, Black, or Asian persons. Both health-related questions and race/ethnicity questions will be included in the screener. The goal is to make the screener short enough to limit response burden yet include key questions that will help identify the population subgroups of interest.

• Screener response rate: We will estimate what screener response rate would be achievable and evaluate the impact of incentives on response rates.

• Effectiveness of identifying Hispanic, Black, and Asian households through the screener: We will compare the weighted race/ethnicity distribution among households responding to the screener to external benchmarks (e.g., estimates from the American Community Survey).

• Effectiveness of identifying high expenditure households through the CMS data and the screener: We will compare the MBSF chronic condition variables, respondent-provided information in the screener, and CMS claims data to evaluate the effectiveness of using the CMS data and the screener to reach high expenditure households. For this evaluation work, the claims files are expected to have more recent data on expenditures than the MBSF. Therefore, we will request the most recent claims data for the address matched records identified during the sampling phase, to evaluate the respondent-provided information.

5. Statistical Consultation

The following are responsible for statistical aspects of the MEPS Study:

Hebert Wong, Ph.D.

Director, Div. of Statistical Research and Methods

Center for Financing, Access and Cost Trends

AHRQ

(301) 427-1405

Sadeq Chowdhury, Ph.D.

Division of Statistical Research and Methods

Center for Financing, Access and Cost Trends

AHRQ

(301) 427-1666

7