CEQ Measurement Issues Study

CEQ Measurement Issues project
I.

Study Overview

This study is the first in a comprehensive and ongoing effort to examine alternative data collection strategies
for the CEQ that would improve data quality, maintain or increase response rates, and reduce data collection
costs. In particular, this study will assess the effects of administering a shorter CEQ questionnaire on
respondent burden, data quality, and nonresponse error. A separate condition in this study will examine the
extent to which using a 1-month (versus a 3-month) reference period affects underreporting due to recall
errors. The study design will enable BLS to perform data quality analyses using both direct measures (e.g.,
number of expenditure reports, expenditure amounts) and indirect measures (e.g., response rates, measures of
perceived burden, item nonresponse, etc.), and to estimate nonresponse bias by comparing response rates,
sample composition, and expenditure estimates across treatment conditions. The results from this study will
be used to inform future CEQ research activities and decisions about how to redesign the production survey.

II.

Study Design

General:
This work shall investigate the aforementioned issues using a truncated CEQ interview and a restricted panel
design. There will be three test conditions in this study (see Table 1, below). In the Control (CG) condition,
sample units will complete a bounding interview in Wave 1. This interview will use a 1-month recall period
and consist of items taken from nine sections of the CEQ plus a ‘core’ set of questions (e.g., demographic
items) that will be commonly administered across all study conditions. Those same CG sample units will be
contacted again three and six months later to complete two additional quarterly interviews using the same
‘Core + 9 sections’ questionnaire and a 3-month recall period. The CG condition parallels the existing CEQ
survey procedures and will serve as the basis of comparison for the other experimental condition.
In the Split Questionnaire (SQ) condition, sample units will complete a full bounding interview in Wave
1, and then will be randomly assigned to one of two subsamples that will be administered shorter
questionnaires in Waves 2 and 3. Subsample (a) will be administered sections 1 through 4 from the full
instrument, the ‘core’ questions, and a small number of global expenditure questions for sections 5
through 9; subsample (b) will be administered sections 5 through 9, the ‘core’ questions, and a small
number of global expenditure items from sections 1 through 4.
In the Recall Period (RP) condition, sample units will be administered the same “full” questionnaire that
is administered to the Wave 1 CG and SQ respondents. They will then receive three consecutive
monthly interviews using the same "full" questionnaire with a 1-month reference period (rather than the
3-month reference period used in the quarterly interviews). Data collection for the RP condition will
stop after four consecutive monthly interviews; data collection for sample units in the Control and SQ
conditions will continue through the 3rd quarterly interview.

Table 1. Test Conditions
Condition

Wave 1

Wave 2

Wave 3

2nd Interview
(3-month recall)
“FULL” Interview
(Core + 9 sections)

3rd Interview
(3-month recall)
“FULL” Interview
(Core + 9 sections)

2nd Interview
(3-month recall)

3rd Interview
(3-month recall)

(a) Core + sections 1 – 4

(a) Core + sections 1 - 4

(b) Core + sections 5 - 9

(b) Core + sections 5 - 9

Bounding
Interview (1month recall)
Control (CG)

Split
Questionnaire
(SQ)

“FULL”
Interview
(Core + 9
sections)

Bounding
Interview (1month recall)
“FULL”
Interview
(Core + 9
sections)

Respondents randomly assigned to
one of two sub-samples – (a) or (b),
each administered a different set of
CEQ sections (plus a ‘core’ set of
questions common to both subsamples)

4 Consecutive 1-month Interviews

Recall Period
(RP)

Bounding
Interview (1month recall)
“FULL”
Interview
(Core + 9
sections)

2nd Interview
(1-month recall)

3rd Interview
(1-month recall)

“FULL”
Interview
(Core + 9
sections)

“FULL”
Interview
(Core + 9
sections)

4th Interview
(1-month recall)
“FULL” Interview
(Core + 9 sections)

Length of Interview:
BLS estimates that the Wave 1 “full” interview will take an average of 30 minutes to complete in each
condition. We estimate that the 2nd and 3rd CG interviews will take 27 minutes each, and that the 2nd – 4th RP
interviews will take 25 minutes each. The 2nd and 3rd SQ interviews (i.e., the shorter, split questionnaire
interviews) are estimated to take an average of 18 minutes to complete. All time estimates include the time
needed to administer the core items, expenditure questions from the CEQ sections selected for this study,
post-survey assessment questions about respondents’ perceptions of burden and degree of motivation, and
case management items (e.g., information on the best time to call/visit for subsequent interviews).
Mode:
All data in this study will be collected by computer-assisted telephone interviewing (CATI) from a centralized
Census CATI center.

Sampling Frame:
BLS is exploring two alternative sample frame possibilities for use in this study – either a list-assisted,
random digit dialing (RDD) sampling frame of landline telephone numbers that are matched to known
addresses, or the address-based, CE research sample sampling frame that are matched to known telephone
numbers. The match rate is a critical element of our test because we need both addresses and telephone
numbers for as many of the sample units as possible so that advance materials can be mailed prior to CATI
contact. BLS and Census will negotiate to jointly determine which sampling frame best fits the study’s
objectives and budget. Census will provide the sampling frame, conduct the address-phone match, purge the
frame of known nonresidential units and nonworking numbers, and draw the sample.
Fielding Period:
The overall fielding period for this study will be nine months. It will cover three waves of data collection for
the CG and SQ conditions and four waves of data collection for the RP condition. The fielding period for each
wave of the survey will be a calendar month, beginning on the first business day of the month. We anticipate
that seventy percent of CATI cases must be closed out within twenty-one days of release; the remaining thirty
percent must be closed by the last day of the month.
We propose a staggered sample release, such that one-third of the cases assigned to each condition will
be interviewed in month n, one-third will be interviewed in month n + 1, and one-third will be
interviewed in month n + 2. This staggered schedule will be carried forward throughout all subsequent
interviews, based on when the case was originally released and the appropriate reference period for the
condition.
Prior to the start of each wave, Census will mail advance materials to sample members with an address match
(see Table 2). The schedule for mailings will be determined jointly by BLS and Census.

Table 2. Advance Materials Distribution
1st Interview
1st Mailing
2nd Mailing

2nd - nth Interview

Advance letter (modified
Advance letter (modified Form CEForm CE-303-L1);
Modified Information
303-L2 ); Modified Information
“Tracking Your Spending Booklet (CE-305(C))
Booklet (CE-305(C))
Behavior” brochure.

Sample Size:
The final sample size requirement is 8,103 completed cases across conditions and interview waves. The key
analytic comparisons in this study are between (1) the CG condition and the SQ subsamples in Waves 2 and 3,
and (2) the quarterly-based estimates from the Wave 2 CG condition and the aggregate data from the three
monthly interviews in the RP condition (i.e., the 2nd, 3rd, and 4th interviews).
Table 3 outlines the number of complete cases that the BLS estimates would be required in preceding waves
to obtain the target number of completed interviews in subsequent waves of the survey.
Table 3. Target Number of Complete Cases by Condition and Wave
Condition
Control (CG)

Split
Questionnaire
(SQ)

Recall Period
(RP)

Wave 1

Wave 2

Wave 3

750

525

450

Total

(n = 1060)

(n = 900)

Subsample A

530

450

Subsample B

530

450

1515

Wave 1

Wave 2

Wave 3

Wave 4

1036

725

617

525

Respondents:
Any adult member of the sampled household can serve as a respondent in this study. If no adult is available,
basic information about the household (e.g., demographic variables) can be collected from any household
members who are at least 16 years old. Information about household spending, however, should be collected

from knowledgeable adult household members. Changes in respondents between survey waves are
acceptable, but the survey instrument must be able to identify the respondent and track when such changes
occur
III.

Burden Hour Estimate

BLS estimates that this study will require 3,456 burden hours. We base this estimate upon the following
assumptions:
Estimated length of full, 1st wave interview:
Estimated length of full interview, CG 2nd and 3rd waves:
Estimated length of full interview, RP 2nd – 4th waves:
Estimated length of SQ interview, 2nd and 3rd waves:
# of full, 1st wave interview cases:
# of full interview cases, CG 2nd and 3rd waves:
# of full interview cases, RP 2nd – 4th waves:
# of SQ interview cases, 2nd and 3rd waves:

30 minutes
27 minutes
25 minutes
18 minutes
3,301
925
1,867
1960

BURDEN ESTIMATE:
(3,301 x 30)/60 + (975 x 27)/60 + (1,867 x 25)/60 + (1,960 x 18)/60 = 3,455.17 hours