Attachment G - Measurement Issues Study

1. Study Overview

This study is the first in a comprehensive and ongoing effort to examine alternative data collection strategies for the CEQ that would improve data quality, maintain or increase response rates, and reduce data collection costs. In particular, this study will assess the effects of administering a shorter CEQ questionnaire on respondent burden, data quality, and nonresponse error. A separate condition in this study will examine the extent to which using a 1-month (versus a 3-month) reference period affects underreporting due to recall errors. The study design will enable BLS to perform data quality analyses using both direct measures (e.g., number of expenditure reports, expenditure amounts) and indirect measures (e.g., response rates, measures of perceived burden, item nonresponse, etc.), and to estimate nonresponse bias by comparing response rates, sample composition, and expenditure estimates across treatment conditions. The results from this study will be used to inform future CEQ research activities and decisions about how to redesign the production survey.

2. Study Design

General:

This work shall investigate the aforementioned issues using a truncated CEQ interview and a restricted panel design. There will be three test conditions in this study (see Table 1, below). In the Control (CG) condition, sample units will complete a bounding interview in Wave 1. This interview will use a 1-month recall period and consist of items taken from nine sections of the CEQ plus a ‘core’ set of questions (e.g., demographic items) that will be commonly administered across all study conditions. Those same CG sample units will be contacted again three and six months later to complete two additional quarterly interviews using the same ‘Core + 9 sections’ questionnaire and a 3-month recall period. The CG condition parallels the existing CEQ survey procedures and will serve as the basis of comparison for the other experimental condition.

In the Split Questionnaire (SQ) condition, sample units will complete a full bounding interview in Wave 1, and then will be randomly assigned to one of two subsamples that will be administered shorter questionnaires in Waves 2 and 3. Subsample (a) will be administered sections 1 through 4 from the full instrument, the ‘core’ questions, and a small number of global expenditure questions for sections 5 through 9; subsample (b) will be administered sections 5 through 9, the ‘core’ questions, and a small number of global expenditure items from sections 1 through 4.

In the Recall Period (RP) condition, sample units will be administered the same “full” questionnaire that is administered to the Wave 1 CG and SQ respondents. They will then receive three consecutive monthly interviews using the same "full" questionnaire with a 1-month reference period (rather than the 3-month reference period used in the quarterly interviews). Data collection for the RP condition will stop after four consecutive monthly interviews; data collection for sample units in the Control and SQ conditions will continue through the 3^rd quarterly interview.

Table 1. Test Conditions

Condition	Wave 1			Wave 2	Wave 3
Control (CG)	Bounding Interview (1-month recall) “FULL” Interview (Core + 9 sections)			2nd Interview (3-month recall) “FULL” Interview (Core + 9 sections)	3rd Interview (3-month recall) “FULL” Interview (Core + 9 sections)
Split Questionnaire (SQ)	Bounding Interview (1-month recall) “FULL” Interview (Core + 9 sections)	R espondents randomly assigned to one of two sub-samples – (a) or (b), each administered a different set of CEQ sections (plus a ‘core’ set of questions common to both sub-samples)		2nd Interview (3-month recall)	3rd Interview (3-month recall)
				(a) Core + sections 1 – 4	(a) Core + sections 1 - 4
				(b) Core + sections 5 - 9	(b) Core + sections 5 - 9
Recall Period (RP)	4 Consecutive 1-month Interviews
	Bounding Interview (1-month recall) “FULL” Interview (Core + 9 sections)	2nd Interview (1-month recall) “FULL” Interview (Core + 9 sections)	3rd Interview (1-month recall) “FULL” Interview (Core + 9 sections)	4th Interview (1-month recall) “FULL” Interview (Core + 9 sections)

Length of Interview:

BLS estimates that the Wave 1 “full” interview will take an average of 30 minutes to complete in each condition. We estimate that the 2^nd and 3^rd CG interviews will take 27 minutes each, and that the 2^nd – 4^th RP interviews will take 25 minutes each. The 2^nd and 3^rd SQ interviews (i.e., the shorter, split questionnaire interviews) are estimated to take an average of 18 minutes to complete. All time estimates include the time needed to administer the core items, expenditure questions from the CEQ sections selected for this study, post-survey assessment questions about respondents’ perceptions of burden and degree of motivation, and case management items (e.g., information on the best time to call/visit for subsequent interviews).

Mode:

All data in this study will be collected by computer-assisted telephone interviewing (CATI) from a centralized Census CATI center.

Sampling Frame:

Census shall develop a national sampling frame that will result in approximately 8,100 completed interviews over all three treatment conditions and interview waves of this project. The sampling frame will be the address-based, reserve CEQ unit-frame cases that are matched to known telephone numbers. The CEQ reserve cases are a sample of cases separate from the regular CEQ sample, though both contain the same types of cases and have the same national distribution. Unit-frame cases refer to CEQ cases obtained from the 2000 Census that typically have complete addresses which include a house number and street name. The match rate is a critical element of our test because we need both addresses and telephone numbers for as many of the sample units as possible so that advance materials can be mailed prior to CATI contact. BLS expects approximately 50% of the cases initially selected to be matched with a telephone number in the telematch procedure (address-telephone-number matching operation); non-matches will be excluded from the CEQ-MIS sample. Census will provide the sampling frame, conduct the address-phone match, purge the frame of known nonresidential units and nonworking numbers, and draw the sample.

Fielding Period:

The overall fielding period for this study will be nine months. It will cover three waves of data collection for the CG and SQ conditions and four waves of data collection for the RP condition. The fielding period for each wave of the survey will be a calendar month, beginning on the first business day of the month. We anticipate that seventy percent of CATI cases must be closed out within twenty-one days of release; the remaining thirty percent must be closed by the last day of the month.

We propose a staggered sample release, such that one-third of the cases assigned to each condition will be interviewed in month n, one-third will be interviewed in month n + 1, and one-third will be interviewed in month n + 2. This staggered schedule will be carried forward throughout all subsequent interviews, based on when the case was originally released and the appropriate reference period for the condition.

Prior to the start of each wave, Census will mail advance materials to sample members with an address match (see Table 2). The schedule for mailings will be determined jointly by BLS and Census.

Table 2. Advance Materials Distribution

1st Interview		2nd - nth Interview
1st Mailing	2nd Mailing
Advance letter (modified Form CE-303-L1); “Tracking Your Spending Behavior” brochure.	Modified Information Booklet (CE-305(C))	Advance letter (modified Form CE-303-L2 ); Modified Information Booklet (CE-305(C))

Sample Size:

The final sample size requirement is 8,103 completed cases across conditions and interview waves. The key analytic comparisons in this study are between (1) the CG condition and the SQ subsamples in Waves 2 and 3, and (2) the quarterly-based estimates from the Wave 2 CG condition and the aggregate data from the three monthly interviews in the RP condition (i.e., the 2^nd, 3^rd, and 4^th interviews).

Table 3 outlines the number of complete cases that the BLS estimates would be required in preceding waves to obtain the target number of completed interviews in subsequent waves of the survey.

Table 3. Target Number of Complete Cases by Condition and Wave

Condition	Wave 1			Wave 2	Wave 3
Control (CG)	750			525	450
Split Questionnaire (SQ)	1515		Total	(n = 1060)	(n = 900)
			Subsample A	530	450
			Subsample B	530	450
	Wave 1	Wave 2	Wave 3	Wave 4
Recall Period (RP)	1036	725	617	525

Respondents:

Any adult member of the sampled household can serve as a respondent in this study. If no adult is available, basic information about the household (e.g., demographic variables) can be collected from any household members who are at least 16 years old. Information about household spending, however, should be collected from knowledgeable adult household members. Changes in respondents between survey waves are acceptable, but the survey instrument must be able to identify the respondent and track when such changes occur

III. Burden Hour Estimate

BLS estimates that this study will require 3,456 burden hours. We base this estimate upon the following assumptions:

Estimated length of full, 1^st wave interview:             30 minutes

Estimated length of full interview, CG 2^nd and 3^rd waves:             27 minutes

Estimated length of full interview, RP 2^nd – 4^th waves:             25 minutes

Estimated length of SQ interview, 2^nd and 3^rd waves:   18 minutes

# of full, 1^st wave interview cases: 3,301

# of full interview cases, CG 2^nd and 3^rd waves: 975

# of full interview cases, RP 2^nd – 4^th waves: 1,867

# of SQ interview cases, 2^nd and 3^rd waves: 1960

BURDEN ESTIMATE:    

(3,301 x 30)/60 + (975 x 27)/60 + (1,867 x 25)/60 + (1,960 x 18)/60 = 3,455.17 hours