Response Rates in the Consumer Expenditure Survey

Response Rates in the Consumer Expenditure Survey

by Sharon Krieger and David Swanson

Response rates are one of the most important indicators of survey quality. It is important to monitor response rates because nonresponse can introduce bias into any survey’s published estimates when its responders and nonresponders differ in terms of the characteristic being measured. Most surveys have procedures to adjust for the effects of nonresponse, but any imperfection in the procedures is magnified when response rates decline. Therefore it is important to monitor response rates and to keep them as high as possible.

This article describes how the Bureau of Labor Statistics (BLS) categorizes the outcomes of visits to survey households in terms of response and nonresponse and explains how the BLS calculates response rates for the Consumer Expenditure Survey (CE).

Selection of households

The CE selects a representative sample of households from across the nation to find out how Americans spend their money. Specific households are drawn from the Census Bureau’s Master Address File, which is basically a list of all residential addresses identified in the 2010 census and is updated twice per year with information from the U.S. Postal Service. Data for the Diary and Interview surveys are collected by field representatives who make personal visits to the households in the survey’s sample. The CE emphasizes the importance of gaining cooperation at each household since each completed interview represents many American households.

Nonresponse types and outcome codes

Like any survey, the CE does not get responses from every household it visits. At most addresses, a household member participates in the survey and gives a “completed interview.” However at some addresses, the survey’s field representative finds an occupied housing unit but either is unable to contact an eligible household member or is unable to convince a reluctant household member to participate in the survey. Still, at other addresses the housing unit is unoccupied or is not eligible to participate in the survey. Such households are called “nonresponders.”

Field representatives assign outcome codes to each address they visit. Multiple visits to a housing unit may be required before a final outcome code can be assigned. Sometimes multiple visits are made in an attempt to convince a reluctant household to participate in the survey, and other times it takes more than one visit before a field representative actually contacts an eligible household member. When field representatives make multiple visits and are still unable to find anyone home, they may obtain information from neighbors to verify the eligibility status of the sampled housing unit. A separate outcome code is assigned each quarter for sample households in the Interview Survey and each week for sample households in the Diary Survey. Although field representatives assign an outcome code to every household in the sample, the BLS occasionally changes it later based on criteria which measure the completeness of respondents’ reporting of household income and expenses.

There are more than 30 different outcome codes from which field representatives select to describe the nonresponse situation they find at an address. These outcome codes are categorized as Types A, B, and C nonresponses. Some common outcome codes for Type A nonresponses are: No one home (unable to contact); Refused—hostile respondent; Refused—time-related excuses; and Refused—language problems. Some common outcome codes for Type B nonresponses are: Occupied by persons whose usual residence is elsewhere; Vacant for rent; Vacant for sale; and Unit under construction. Some common outcome codes for Type C nonresponses are: Dwelling demolished; Dwelling condemned; House or trailer moved; and Located on military base or post.

Two important concepts in classifying sample addresses in terms of response and nonresponse are “in-scope” and “eligible.” “In-scope” refers to addresses that have residential housing units, and “out-of-scope” refers to addresses that are non-residential or non-existent. Out-of-scope addresses are generally the result of imperfect information on the sampling frame. Completed interviews and Type A and Type B nonresponders are all “in-scope” while Type C nonresponders are “out-of-scope” for the CE. “Eligible” refers to residential addresses that are occupied by people who use the housing unit as their primary residence. Ineligible addresses are nonresidential, vacant, or occupied by people whose usual residence is elsewhere. Completed interviews and Type A nonresponders are eligible, but Type B and Type C nonresponders are ineligible. Only addresses that are both in-scope and eligible are used in the response rate calculations.

In summary, results of visits to the sample addresses are divided into four main groups:

 Completed Interviews (eligible, in-scope cases that are interviewed)

 Type A nonresponses (eligible, in-scope cases that are not interviewed)

 Type B nonresponses (ineligible, in-scope cases that are not interviewed)

 Type C nonresponses (ineligible, out-of-scope cases that are not interviewed)

Response rates

The CE program defines its response rate as the percent of in-scope eligible housing units that participate in the survey. In other words, it is the number of completed interviews divided by the number of completed interviews plus Type A nonresponders, and it is expressed as a percent. The Type B and Type C nonresponders are not included in the calculation because they are not eligible for the survey.

Response rates are reported independently for the Interview and Diary components of the CE. When calculating a response rate for the Diary Survey, the BLS uses outcome information from the two diary weeks independently. When calculating a response rate for the Interview Survey, the BLS uses outcome information from all four interviews independently.

Prior to 2015 the Interview Survey had five interviews, and response rates were calculated based on the second through fifth interviews only. Information from the first interview was excluded because it was used only for “bounding” purposes, which addressed a problem in which survey respondents tended to report expenditures more recently than they were actually made. Recent research showed that such “telescoping” of expenditures by respondents is no longer a problem, so the bounding interview was removed from the CE survey in 2015. Thus before 2015 the Interview Survey had five interviews and response rates were generally reported only for the four non-bounding interviews, but starting in 2015 the Interview Survey has only four interviews and response rates are reported for all four interviews. The table below shows results for 2013 and 2014, and since that is before 2015 it uses information only from the four non-bounding interviews.

The sample size and response status for housing units “designated” for the Interview Survey in 2013 and 2014 are shown in Table 1. In 2014 there were 47,529 housing units designated for the four non-bounding interviews in the survey. Field representatives occasionally find more than one housing unit at the addresses they visit, so the number of housing units designated is slightly more than the number of addresses selected for the sample. Those 47,529 housing units designated for the survey contained 39,003 eligible units from which 25,908 completed interviews were collected, yielding a response rate of 66.4 percent (25,908 / 39,003 = 0.664). Field representatives visited 8,526 ineligible addresses which were categorized as Type B or Type C nonresponders.

Table 1. Analysis of response in the CE Interview Survey, 2013 and 2014

Sample unit	2013	2014
Housing units designated for the survey	47,524	47,529
Less: Type B or C nonresponses	8,382	8,526
Equals: Eligible units	39,142	39,003
Less: Type A nonresponses	13,034	13,095
Equals: Interview units	26,108	25,908
Percent of eligible units interviewed	66.7	66.4

The sample size and response status for housing units designated for the Diary Survey in 2013 and 2014 are shown in Table 2. In 2014 there were 25,606 housing units designated for the Diary Survey (both diary weeks treated independently). Again, the number of housing units “designated” is slightly more than the number of addresses selected for the sample due to multi-unit addresses. From these designated units, 20,476 housing units were found to be eligible to participate in the survey. Subsequently, 13,306 completed interviews were collected from these households, yielding a response rate of 65.0 percent (13,306 / 20,476 = 0.650). Field representatives visited 5,130 ineligible addresses which were categorized as Type B or Type C nonresponders.

Table 2. Analysis of response in the CE Diary Survey, 2013 and 2014

Sample unit	2013	2014
Housing units designated for the survey	25,362	25,606
Less: Type B or C nonresponses	5,066	5,130
Equals: Eligible units	20,296	20,476
Less: Type A nonresponses	7,961	7,170
Equals: Interview units	12,335	13,306
Percent of eligible units interviewed	60.8	65.0

Conclusion

The CE collects expenditure data from a representative sample of American households. Like any survey, the CE does not get responses from every household it visits. In this article we described how the CE categorizes the outcomes of visits to survey households, and how it calculates response rates for the survey. It is important to monitor response rates and to keep them as high as possible in order to minimize the amount of bias that can be introduced into the survey estimates.

3