CEQ Records Study CEQ Records Study

1. Study Overview

There is a widespread belief that some CEQ data are underreported. One common suggestion to combat underreporting is to increase the use of respondent records, having respondents refer to external sources such as receipts or other documentation to aid in their response process.

The purpose of this study is to conduct qualitative interviewing and analysis to address the following objectives:

1. The first objective, as described in the RFP, is “to gain an understanding of the availability of respondent records and the extent to which the information in those records aligns with the time frame, level of detail, and categories of expenditures asked about in CEQ.” This is an important goal, and one in which the qualitative contributions of this study are essential – This first step is needed in understanding the range of record types respondents keep, their accessibility and the period for which they’re stored, the level of detail they contain, and how those factors map to the questions asked in CE.

2. Secondly, this qualitative study is exploratory in nature and is a initial step in determining the amount of error inherent in using records (or not). To date, there is no field study that provides evidence of the (in)accuracy of respondents unaided self-reports of their expenditures. The results of this research will inform whether investing time and effort in research on various ways of emphasizing and relying on records is warranted. We will be looking at the overall feasibility of the approach, and examining trends in data quality and measurement error. We believe the sample size is sufficient to meet these analysis goals.

3. Thirdly, this study will help us to gain insight into the respondent's interview experience during the CEQ, including perception of burden, task difficulty, expectations of required accuracy, and reactions to the advance letter and Information Book.

The results from this study will be used to inform future CEQ research activities and decisions about how to redesign the production survey.

2. Study Design

General

For this study, each respondent will participate in two personal-visit interviews, with a debriefing following each interview. After the first interview, respondents will also be instructed to collect records for the three month reference period covered by the first interview. After one week, the respondent will be interviewed a second time. During the second interview, the respondent will work with the interviewer to match the records he/she collected to the data recorded during the first interview and to provide an explanation for discrepancies between records and collected data. The respondent will then be asked a second set of debriefing questions. A modified version of the 2010 CEQ interview for new CUs in interviews 2-5 (Attachment W(b) – Modified 2010 CEQ Interview specs for Records Study), the 2009 CEQ Information Booklet (Attachment W(d) – 2009 CEQ Information Booklet), and a modified Advanced Letter (Attachment W(f) – Records Study Advance Letter) will be used.

First Interview

Prior to the start of the first interview, the respondent will be mailed a modified version of the advanced letter. (See Attachment W(e) – Records Study Advanced Letter.)

The first interview will include the administration of the following 9 sections of the CEQ:

1. Section 2 Rented living quarters, or Section 3: Owned living quarters

2. Section 4: Utilities

3. Section 6: Appliances

4. Section 8: Household furnishings

5. Section 9: Clothing

6. Section 14: Health insurance

7. Section 17: Subscriptions

8. Section 19 Miscellaneous

9. Section 22: Income.

A modified version of the CEQ CAPI instrument will be used. (See Attachment W(b)) During the first interview, the interviewer will provide the participant with the 2009 CEQ Information Booklet and encourage the participant to use it as specified in the 2010 CEQ instrument. The use of records will be encouraged only to the extent that they are currently mentioned in the BLS advance letter and CEQ instrument.

After the first interview is completed, the participant will be asked a series of debriefing questions covering the following topics:

• perceptions of burden,

• expectations of required accuracy,

• reactions to advance letter,

• reactions to the Information Booklet, and

• perceived difficulty of completing the interview.

(See attachment W(c) – First interview debriefing questions)

Following the interview, the respondent will be provided with a list of all the expenditure categories covered during the first interview, examples of relevant records, and a folder or other storage device for gathering records. (See attachment W(i) – Expense list)

The respondent will be instructed to gather all records for the reference period covered by the first interview including bills, receipts, check registers, and any records which may be stored online. The respondent will be instructed to gather these records for all household members.

Before leaving the participant’s home, the interviewer will schedule the second interview with the respondent for not more than one week later.

Second Interview

Within a week after the first interview, at the time agreed upon with the respondent, the same interviewer will return to the participant’s home to conduct the second interview. During this interview, the interviewer will review each record (e.g. receipt, bill, checkbook register) with the respondent. The interviewer will document details of every expense listed on the record including date, description and cost.

If the respondent did not collect any records, the interviewer will request they get out what they can, again providing the list of expenditure categories. If a respondent is unable, or unwilling, to gather records, the interviewer will note the reasons no records were gathered as specified in the second interview questions.

If the interviewer identifies any discrepancy between the record and the data from the first interview, the respondent will be asked about the reason for the discrepancy (e.g. recall, comprehension or proxy issues). Similarly, if the record lists an expense that was not reported during the first interview, the interviewer will attempt to determine the reason for the omission. (See Attachment W(e) Second Interview protocol including debriefing)

Any expenses on a record which could not be matched to an expenditure report given in the first interview will also be reviewed with the respondent. The interview will attempt to determine the reason a match could not be made (e.g., the interviewer and/or respondent could not identify the expense, the information on the record was not detailed enough, respondent could not match expense to CEQ question or category, time frame covered by the record did not align with the CEQ reference period).

After the record review process is completed, the interviewer will review the expenditure categories with the participant. The interviewer will identify those categories for which records were not produced, and probe about the reason records were not available (e.g., lost, did not exist, paid cash, held by other family members).

Fielding Period

The overall fielding period for this study will be approximately 7 weeks, from early April 2011 to late May 2011. The contractor will conduct interviews in the Raleigh-Durham, North Carolina, and the Washington, District of Columbia, metropolitan areas. Two-thirds of the interviews will be conducted in North Carolina, and the remaining third will take place in the DC-metro area.

Sampling, Recruitment, and Participant Incentives

The contractor will be responsible for drawing a purposefully-selected sample of approximately 100 respondents. The composition of respondent characteristics will reflect the below stated groups, with each group considered independently.

• Household size: single person households, 2-3 person households, 3 or more person households

• Respondent Education: Less than high school, High school diploma/some college, College Degree or higher

• Annual Household Income: Under $30,000, $30,000 to $60,000, Over $60,000

• Respondent Employment Status: Unemployed (including looking and not looking for work), Employed part time, Employed full time

• Respondent Age: 18 to 25, 25 to 60, Over 60

Participants will receive an incentive of $40 for the first interview and a larger incentive of $60 for the second. This amount was determined to be necessary due to the following reasoning below:

1. In addition to the regular burden of participating in a survey and answering debriefing questions (i.e., the first interview is expected to last as long as 90 minutes), participants are being asked to spend time gathering and organizing records for all purchases over the last three months. Some participants may also choose to print records that are stored electronically, leading to some material cost. Participants will then be asked to engage in a records matching process in the second interview (this process and the debriefing are expected to last as long as 120 minutes). We believe that the combined burden of 3.5 hours of interview time plus time spent gathering record plus any material cost to participants warrants an incentive of $100.
   NOTE: The timing in this paragraph are based on a pre-test by RTI. [The Pretest was conducted below the threshold for OMB clearance as per The ‘nine or fewer’ rule is set by the Paperwork Reduction Act, 5 C.F.R. § 1320.3(c). ]

2. Completing both interviews and ensuring respondents comply with the request to gather as many records as possible is vital in order to meet the objectives of for this project. Without the second interview and a substantial number of collected records estimates of error are not possible. Additionally, without data from both interviews, qualitative data on the feasibility of the records matching process cannot be collected.

3. One of the main constraints of this project is that the data collection period is very short (7 weeks). Specifically, there are currently 7 interviewers handling 100 cases; each case consisting of two interviews spaced 4-7 days apart. In order to ensure 100 full cases are collected during the period allotted $100 was determined to be a sufficient incentive. Failure to meet recruiting and sample size goals could result in substantial risk in accomplishing the objectives of the study and to the investment made by BLS in this study. The results of this pioneering qualitative study and its usefulness in improving survey data quality and respondent burden far outweigh the amount that we are allotting to our respondents for their valuable information.

All interviewing will take place at the participants' homes. Prior to the interview, the Contractor will administer a consent form before each interview (See Attachment W(g) – Interview 1 Consent Form and Attachment W(h) – Interview 2 Consent Form).

Mode

The first interview will be conducted by a modified computer-assisted personal interviewing (CAPI), using an existing Blaise CEQ instrument. (See Attachment W(b) Modified CEQ Interview Specification.)

The second interview will be conducted by a Blaise instrument created by the contractor, RTI. (See Attachment W(e) – Second Interview protocol including debriefing)

Quality Assurance

The contractor will employ computer audio-recorded interviewing (CARI) as a quality assurance technique. If the respondent agrees to the interview being audio recorded, the CAPI instrument will record select questions and their responses to ensure that the interview read the question correctly and that the response was accurately recorded. (See attachment W(g) – Interview 1 Consent Form and attachment W(h) – Interview 2 Consent Form)

2. Burden Hour Estimate

BLS estimates that the first interview, including debriefing and scheduling of the second interview, will take an average of 90 minutes.

It is estimated that the second interview will take an average of 120 minutes.

BLS estimates that this study will require 350 total burden hours. We base this estimate upon the following assumptions:

Estimated length of first interview:             90 minutes

Estimated length of second interview : 120 minutes

# of first interview cases: 100

# of second interview cases: 100

BURDEN ESTIMATE:    

(90 min x l00) + (120 min x 100) /60 = 350 total burden hours