TPOPS Report

TPOPS Advance Letter Experiment

Brian J. Meekins

Office of Survey Methods Research, Bureau of Labor Statistics

Executive Summary

• The Telephone Point-of-Purchase Survey (TPOPS) is a rotating panel survey conducted quarter by the Census Bureau for the Bureau of Labor Statistics.

• The survey collects outlet information to be used in the Consumer Price Index’s pricing survey.

• Recently the TPOPS has experienced a steady decline in response and cooperation.

• This experiment examines the efficacy of using an advance sent to members of the entry wave RDD sample for reducing nonresponse.

• In addition the experiment examines the effect of an abbreviated and informal introductory confidentiality statement on the cooperation of the entry wave RDD sample.

• The advance letter produces modest gains (7-8%) in cooperation and response rates with little apparent bias in the RDD sample that is able to be matched to an address.

• However, the overall response rate for the entire sample for an average survey implementation over this time would only achieve a modest increase of approximately 2.5%.

• The abbreviated confidentiality statement decreased the interview length by approximately 30 seconds, but had a small negative effect on survey response.

• It is recommended that the advance letter be sent to all RDD sample records that are able to be matched to an address.

• It is also recommended that the abbreviated confidentiality statement be re-worded to emphasize more persuasive arguments for survey participation.

Acknowledgements: The author thanks Robie Sangster, Charlie Mason, Cassandra Yocum, and Madeleine Saxton of the Bureau of Labor Statistics, as well as Deborah Kinnaman, Claudia West, Therese Bachmann, and Steve Ash of the U.S. Census Bureau for their contribution to the advance letter experiment.

I. Purpose of the Study

The Telephone Point-of-Purchase Survey (TPOPS) is conducted by Bureau of the Census under the direction of the Bureau of Labor Statistics (BLS). This survey’s main purpose is to generate the list of outlets and establishments where Consumer Units (CUs) purchase various goods. This list of outlets is later used for the CPI’s pricing survey. The TPOPS is collected quarterly by means of telephone interviews. Approximately 43,000 sample records are attempted each quarter. Of the 43,000 approximately 50 percent (22,000) are new list assisted RDD records while the remaining 50 percent are returning panel members from previous quarters. Each CU may be interviewed as many as four times¹.

Prior to this study all households were “cold-called.” No attempt was made to contact a sample unit prior to the initial telephone request for the interview. Past research in other surveys shows pre-notification, in general, and use of an advance letter, in particular, produces modest gains in response by increasing cooperation. Although most of these studies examine pre-notification in surveys utilizing some manner of listed samples (where addresses are known for all sampling units), a few studies using samples of numbers generated by list assisted random digit dialing (RDD) show modest increases in response. However, results for RDD samples are less consistent (Goldstein and Jennings 2002; Camburn et al. 1995; Traugaut, Groves and Lepkowski 1987; Dillman, Gellegos, and Frey 1976). Because the TPOPS utilizes an RDD sample extracted using Genesys®, phone numbers of businesses and other non-households, as well as unlisted and listed households are included in the sample. Any reverse match-up to an address file will necessarily include only listed households.

In recent years, response rates in RDD telephone surveys have declined. Two factors have largely contributed to this decline: a modest but consistent decrease in cooperation, and a more dramatic increase in the number of phone numbers unable to be contacted (Curtin, Presser, & Singer 2005). The dramatic increase in phone numbers that are not able to be contacted is largely due to the recent proliferation of sparsely populated 100 banks, leading to a decline in the proportion of actual household numbers in a given RDD sample (Tucker, Lepkowski, & Piekarski 2002). The latter problem is unavoidable under the TPOPS sampling design, while the former may be partially remedied by advanced notification. Figure 1 shows the response rate (AAPOR RR2), the cooperation rate (AAPOR COOP4), and the relative size of the current RDD sample to that of the first quarter of 2004. The chart covers the first wave sample (RDD sample) from the first quarter of 2001 (Q011) to the first quarter of 2004 (Q041). Over these survey implementations we see a steady decline in both the cooperation and response rates, and a drastic increase in the amount of RDD sample introduced in each quarter.

Figure 1 Response Rate, Cooperation Rate, and Relative

Sample Size for TPOPS RDD Sample

This experiment examines the effect of using an advance letter on respondent cooperation. It was hypothesized that the combined effect of “warming-up” the potential respondents with an advanced letter and making salient key points shown in previous literature to be effective in reducing refusal rates (such as sponsorship) would increase cooperation. An added benefit occurs because confidentiality and other introductory statements are included in the advance letter or addressed on the back of the advance letter in the form of a list of “Frequently Asked Questions.” For those potential respondents who mention seeing the letter, some of the more burdensome statements from the beginning of the telephone interview can be dropped. Implementation of this shortened confidentiality statement was expected to increase cooperation by decreasing respondent burden and interview time. A secondary experiment was conducted to examine the effect of a shortened confidentiality statement on cooperation, data quality, and interview time for those remembering the receipt of advance letters.

II. Methodology

Design

The advance letter experiment was conducted from the second quarter of 2004 (Q042) to the third quarter of 2005 (Q053). For each quarter the entire new sample of telephone numbers, the non-returning, RDD portion of the TPOPS sample was “reverse” matched to an address frame. This was conducted by First Data Resources. The match rate by quarter is given in Table 1. Overall, approximately 25% of the new sample (12% of total sample) was able to be matched to an address. This varied greatly by quarter, where in Q042 First Data Resources, was able to match just over 20% of the new sample records, while in Q051 they were able to match nearly 28%. The reason for this difference is unknown. More on matching will be discussed in the Results section.

Table 1: Address Match Rate by Interview Quarter*
Quarter	Rate
Q042	20.5
Q043	20.6
Q044	28.3
Q051	27.9
Q052	25.9
Q053	25.8
Overall	25.1
*Wave 1, RDD, sample, excludes supplemental sample

The matched sample was then randomly assigned to one of three experimental groups (unmatched sample is labeled Group 1). The first experimental group was the control group (Group 2). This group was not sent an advance letter. The second and third experimental groups were sent a letter, but the second group was given the standard, long form on the confidentiality statement approved by the Office of Management and Budget (OMB statement) in all cases (this is labeled as Group 3), while experimental Group 4 was given a shorter version of the confidentiality statement (Short statement) only if they remembered receiving the advance letter.² If they did not remember, they received the OMB statement. Because of this extra requirement of remembering the letter a larger proportion of the sample was assigned to the experimental Groups 3 & 4. The proportion of the matched sample assigned to experimental Groups 2 and 3 was 37.5%, leaving 25% for Group 1.³

Table 2. Total Sample and Completed Interviews by Treatment Group and Quarter
	Group1: Unmatched	Matched to Address			Total
		Group 2: No Letter	Letter
			Group 3: OMB Statement	Group 4: Short Statement*
Sample
Q042	16,300	915	1,098	1,648	19,961
Q043	16,824	1,014	1,521	1,522	20,881
Q044	16,669	1,518	2,277	2,278	22,742
Q051	17,365	1,545	2,319	2,319	23,548
Q052	18,299	1,493	2,241	2,241	24,274
Q053	17,116	1,486	2,229	2,230	23,061
Total	102,573	7,971	11,685	12,238	134,467
Competed Interviews
Q042	2,038	341	453	703	3,535
Q043	2,403	385	664	693	4,145
Q044	1,910	608	1,034	1,010	4,562
Q051	1,925	630	1,101	1,088	4,744
Q052	1,902	631	1,145	1,091	4,769
Q053	1,638	596	1,035	1,049	4,318
Total	1,1816	3,191	5,432	5,634	26,073
*If respondent remembers receiving the letter, otherwise OMB statement is read.

Table 2, shows the size of the sample allocated to each treatment group, as well as the number of completed interviews for each group. Though much of the analysis is conducted on the overall combined sample across all quarters a significant part of the analysis will be conduced by individual quarter due to some fairly large differences in response and cooperation between quarters.

About 10 days prior to the initial calling we sent out the advance letters to Treatment Groups 3 and 4. The letters should have arrived via first class mail 4 to 7 days prior to an attempted phone contact/request for interview.

Treatments

There are two primary treatments in this nested experiment, the advance letter and the shortened confidentiality statement. These are compared with control groups that were not sent an advance letter or were given the standard OMD confidentiality statement.

The first draft of the advance letter was very formal in language. Drawing heavily on the OMB confidentiality statement, the letter contained all the same legal statements. After further edits and some consultation with some experts in the field, it wa believed that the letter would be more persuasive if the language was informal. To satisfy the requirement that all CUs are briefed in the formal statements a FAQ list was located on the back of the letter and referenced on the front. This allowed what we believed would be more persuasive test in the body of the letter. A copy of the advance letter is located in Appendix A.

The body of the advance letter notifies the potential respondent of the upcoming call from the U.S. Census Bureau, emphasizes the importance of the study, and briefly explains confidentiality. The reverse of the containing the list of “Frequently Asked Questions” addresses purpose, sponsorship, and confidentiality in detail. So as to be consistent with the household member selection on the TPOPS instrument (where the first willing/available adult from the CU is selected for participation), the letters were not addressed to a particular individual in the listing but rather his or her household (e.g. J. Smith is addressed to: The Smith household). In the case of an OSLO or abbreviated name, the letter is addressed “Dear Householder:”

If a participant in Treatment Group 4 remembers seeing the letter then they are not read the standard, cumbersome OMB statement of confidentiality typically used in the interview. Instead, these participants are read INTRO_LET in the place of this introduction (INTRO_1ST). The exact wordings of both of these introductions are provided in Attachment B. Note that while INTRO_LET is much shorter, it does mention the purpose of the study, that it’s administered quarterly, the approximate length of the interview, the voluntary nature of the interview and a brief assurance of confidentiality. All other treatment groups received the standard OMB introduction.

The primary comparison groups for the advance letter are those that were Consumer Units (CUs) that are matched to an address and not sent a letter. Further analysis is conducted on those that remember the letter compared to those that don’t remember the letter. For the confidentiality statement the primary comparison groups are those that were sent a letter and remember receiving a letter who received the OMB confidentiality statement compared to those who were sent a letter, remember receiving the letter, and were given the short confidentiality statement.

Measures

A number of estimates are produced to assess the impact of the advance letter and revised confidentiality statement. Key rates, including response rates, refusal rates, and cooperation rates are calculated. While response rates and refusal rates give an excellent representation of sample productivity and illustrate the extent of possible nonresponse bias, these rates are often driven by noncontacts, especially in RDD samples. Therefore cooperation rates and refusal rates are calculated in order to assess the impact of the treatments on respondent cooperation. These key rates are calculated in accordance to the American Association for Public Opinion Research Standard Definitions (AAPOR 2006) where possible. Response rates calculated previously for TPOPS by Statistical Methods Division (SMD) of BLS have not been AAPOR compliant. Thus, for some of the analysis two sets of two response rates, and two refusal rates are presented, an SMDRR2, RR2, SMDRR4 and RR4, as well as SMDREF and REF (SMD cooperation rates are AAPOR compliant). The calculation of these rates is reported in Appendix C.

In addition to these rates, other measures include data quality estimates, such as the number of outlets reported in the survey by the CU and the amount of missingness on demographics. For any number of purchases a CU can list the same outlet or store as a source of that purchase. Therefore, a single outlet can be mentioned many times in an interview. The total number of times and outlet is mentioned regardless of duplication is referred to as the gross number of outlets. Eliminating mentions of the same outlet from this tally produces the number of unique outlets mentioned, or the total number of different stores or other establishments where the CU purchased the goods in question.

Two measures, the number of contacts, and the number of call attempts are used to assess effort. Finally, for the impact of the confidentiality statement on the length of the interview and interview segments is estimated.

The panel design of the TPOPS allows us to examine the effect of the advance letter on subsequent wave response and panel attrition. In addition, other measures, such as the completeness of demographic information, are examined for each subsequent wave.

A simple list assisted RDD sample produces no design effect. However, the TPOPS uses a number of RDD samples selected using two-stage cluster design in order to keep reported outlets geographically proximate. Taylor Series Linearization is used to estimate the standard errors under this design. Statistical significance tests for all tables use these standard errors.⁴

Weighting was not used in the analysis because base weights (the inverse of the probability of selection) are produced only for completed interviews. Therefore, in the analysis of nonresponse weights could not be used.

The second quarter of 2004 is excluded from most of the overall scores that combine quarters. This is because the response rate for this quarter is much smaller than other quarters. This is the result of implementing new technology a BLAISE instrument within new Windows-based call management software. The hardware was not up to the task and consequently a number of interviews were dropped. In subsequent quarters this was rectified.

III. Results

Advance Letter: Address Match Rate

As mentioned earlier the overall address match rate (the proportion of the new RDD sample able to be matched to addresses) was approximately 25%. This varied by quarter of implementation for unknown reasons, but also varied by Census Region and Primary Sampling Unit (PSU). Table 3 shows the match rate by Census Region. As we would expect the lowest match rate was in the West. The highest match rate was in the South.

Table 3: Address Match Rate by Census Region
Region	Rate
Northeast	25.5
Midwest	26.7
South	28.0
West	18.7
Overall	25.1
*Wave 1, RDD, sample, excludes supplemental sample

Given these observed differences we might expect differences in the match rate by PSU. Indeed, the match rates range from less than 10% to almost 50% in some PSUs. Table D1 in Appendix D, lists all match rates by quarter, PSU, and region. Given this variability, we should be cautious to examine the possible bias that may be introduced by the introduction of the letter experiment. If the letter significantly introduces the response rate, it does so only for Consumer Units (CUs) that are matched to an address.

Advance Letter: Remembering Receipt of Letter

Certainly we would not expect that all CUs that were mailed a letter received the letter and that all CUs receiving the letter have a recollection of that letter. Returned letters were not tracked, but a question in the instrument asked CUs if they remembered receiving the advance letter. Table 6 presents the frequency distribution of those responses. Note that there is a large amount of missing data. This appears to be an error in data collection and although none of the “valid missing” are completed interviews, 232 of the “invalid missing” are completed interviews. Refusals are well represented in both of these missing categories, with 462 “valid missing” and 2,471 “invalid missing.” A large proportion of both types of missing are ineligible sample phone numbers (businesses or out of geographic area). The remaining missing values are distributed between noncontacts and unknown households. While this is cause for concern, and can certainly bias the estimate of response rates, there was no apparent fix to correct this problem. In addition, it is quite probable that a large amount of both the “valid” missing and the “invalid” missing a survey break-offs.

Table 6: Percent of Respondents Remembering Receiving Letter and Missing*
	Percent	Count
Remember Letter	33.4	7,997
Do Not Remember Letter	22.4	5,369
Don’t Know	1.2	296
Refused	0.1	22
“Valid Missing” †	7.1	1,692
“Invalid Missing”‡	35.7	8,547
*Respondents who were sent a letter and on path to receive question. All quarters combined † There should be no “Valid Missing” ‡ Probable break-offs before question

Table 7 reports the percentage of CU’s that are sent a letter that remember receiving the letter – eliminating the refusals, invalid, and valid missing and incorporating the “don’t know” responses into the “no” category – we estimate that approximately 60% of CU’s remember receiving the letter. This decreased somewhat over the study period from 62% to 58%.

Table 7: Percent Recalling Receipt of Letter by Quarter*
Quarter	Percent
Q042	61.8
Q043	60.2
Q044	60.3
Q051	60.8
Q052	59.5
Q053	57.6
Overall	59.8
*Wave 1, RDD, sample that were sent a letter, excludes supplemental sample

Advance Letter: Response Rates

Overall, the percentage of sample records that end in a completed interview is significantly higher for those who are sent a letter. In addition, the percentage of sample records terminating in a refusal is significantly lower. The patter is consistent across all quarters (although the difference in completed interviews is only statistically significant from Q044 to Q053). Table 4 shows the final disposition of sample records by quarter and over all quarters.

Table 4: Percent Final Call Attempt Disposition by Advance Letter Treatment and Quarter†
Disposition*	Letter /Control	Quarter						Overall‡
		Q042	Q043	Q044	Q051	Q052	Q053
Completed Interview	L	42.1	44.6	44.9	47.2	49.9	46.7	46.8
	C	37.3	40.0	40.1	40.8	42.3	40.1	40.4
Refusal	L	18.8	19.0	20.4	20.1	20.8	20.7	20.3
	C	25.8	25.6	26.2	27.9	28.4	28.5	27.4
Ineligible	L	17.9	19.1	22.3	19.8	16.3	18.8	19.3
	C	15.2	18.7	20.2	19.7	15.7	17.4	18.3
Eligible Noncontact	L	14.1	10.4	8.8	8.9	9.3	10.4	9.5
	C	14.9	11.0	9.1	8.0	10.0	10.0	9.5
Unknown Eligible	L	7.1	7.0	3.7	4.0	3.8	3.5	4.2
	C	6.9	6.7	4.5	3.6	3.6	4.0	4.3
† Wave 1 Sample, only those with matched addresses * AAPOR compliant disposition where possible ‡Excluding Q042 Boldface with underline indicates statistically significant difference between letter and control at p=0.05

Table 5 shows response rates, refusal rates, and cooperation rates by quarter and over all quarters for both the letter and control groups. All differences between these two groups are statistically significant at 95% confidence. The calculation of response rates is clarified in Appendix C. The four different response rates consistently show that those receiving the advance letter have a higher response (58% overall) rate than those who did not receive a letter (49.5% overall). In addition, the refusal rate is lower for the letter group and the cooperation rate, which does not take into account noncontacts and unknown eligible CUs, shows a large difference of approximately 10 percentage points over all quarters combined. This is consistent with the results of Table 4, showing that the impact of the advance letter is primarily felt in cooperation, not in contact. While the response rates vary by region, the difference in the response rates between the letter and control remain statistically significant except in the North, where a 9% increase in the response rate (RR2) was not statistically significant due to the relatively large standard errors in that region. The South experienced the smallest difference (6%) while the Midwest received the largest increase (12%) The use of the advance letter increased response and cooperation in all regions. Table E1 in Appendix E reports all key rates by region.

Table 5: Response, Refusal, and Cooperation Rates by Advance Letter Treatment and Quarter†
Rate	Letter /Control	Quarter						Overall*
		Q042	Q043	Q044	Q051	Q052	Q053
RR2‡	L	51.2	55.1	57.8	58.8	59.6	57.6	58.0
	C	43.9	46.7	50.2	50.8	50.2	48.6	49.5
SMDRR2	L	51.3	54.9	57.6	58.4	59.3	57.2	57.7
	C	43.8	46.4	50.0	50.4	50.1	48.3	49.2
RR4	L	54.7	58.8	59.8	61.1	61.6	59.4	59.6
	C	46.7	49.7	52.3	52.5	51.8	51.8	50.9
SMDRR4	L	54.4	61.2	59.4	60.4	61.1	58.9	59.4
	C	46.4	51.5	51.8	51.9	51.5	49.8	50.7
REF1	L	22.9	23.5	26.2	25.1	24.8	25.4	25.1
	C	30.4	31.6	32.8	34.7	33.7	34.5	33.6
SMDREF1	L	22.5	23.4	26.1	24.9	24.7	25.3	25.0
	C	30.1	31.4	32.7	34.5	33.7	34.3	33.4
COOP4	L	69.1	70.1	68.8	70.1	70.6	69.4	69.8
	C	59.1	59.7	60.4	59.4	59.8	58.5	59.5
† Wave 1 Sample, only those with matched addresses *Excluding Q042 ‡ See Appendix B for calculation of response rates All differences between letter and control groups are statistically significant at p =0.05

Examining these same rates by whether a CU remembers receiving the letter produced even more drastic results. As Table 8 shows, both of these groups have relatively high response rates, but this is primarily due to the fact that a large proportion of refusals refuse earlier in the interview, prior to the question on remembering the letter. Those that remember the letter are much more likely to complete the interview than those who do not remember. The response rate for CUs that remember the letter is almost 90%, while the cooperation rate slightly exceeds the 90% mark. There are at least four possible explanations for this difference. Firstly, the receipt of the letter may be selecting respondents that are more likely to respond to an interview. For example, home owners rather than renters may be more likely to actually receive the advance letter that was sent to their dwelling and home ownership is positively related to survey response. Secondly, the tendency to open and read letters (and therefore remember them) of the nature of the TPOPS advance letter may be correlated with a willingness to participate in surveys. Thirdly, simply answering this question positively may indicate a level of complicity on the part of the CU. Reluctant CUs may be more likely to provide a negative response a means to indicate their reluctance or terminate the interview. Finally, the letter could have a positive effect on compliance, confirming the efficacy of advance notification. It is difficult to assess the viability of these competing explanations. Later in this section the effect of remembering the letter on data quality will be examined and control variables introduced. Control variables unfortunately, are not available for nonrespondents, making it impossible to assess the spuriousness of the relationship of remembering the letter with survey response.

Table 8: Response, Refusal, and Cooperation Rates by Respondent Remembering Advance Letter†
	Remembers Letter	Does not Remember
RR2	89.3	73.3
SMDRR2	89.3	73.2
RR4	89.5	73.5
SMDRR4	89.4	73.3
REF1	9.5	22.2
SMDREF1	9.5	22.2
COOP4	90.4	76.7
† Wave 1 Sample, only those with matched addresses who were sent a letter and received question on recollection of letter. Both of these groups received the Full OMB introduction. All quarters combined excluding Q042. All differences between letter and control groups are statistically significant at p =0.05

Advance Letter: Effort

In an attempt to estimate the amount of effort expended per case, estimates of the number of call attempts and contacts were produced by advance letter status. Table 6 shows the mean number of call attempts and contacts by whether the CU was sent an advance letter. While the letter group shows a general pattern of requiring fewer call attempts and fewer contacts, the only statistically significant results are for contacts for all quarters of the study period combined and for Q044.

Table 9: Mean Number Of Call Attempts by Advance Letter Treatment and Quarter
	Mean Call Attempts		Mean Contacts
Quarter	Letter	Control	Letter	Control
Q042	5.66	5.88	2.60	2.77
Q043	5.89	6.46	2.43	2.70
Q044	5.49	5.85	2.34	2.63
Q051	5.47	5.73	2.29	2.53
Q052	6.30	6.76	2.38	2.57
Q053	6.44	6.85	2.40	2.53
Overall*	5.91	6.32	2.36	2.58
*Excluding Q042 Boldface with underline indicates statistically significant difference between letter and control at p=0.05

Examining only completed interviews, we see that the pattern of fewer required call attempts and contacts persists, with statistically significant differences for both when combining all quarters. In some individual quarters (Q051 and Q052) a statistically significant difference between the letter and control groups is observed for both measures, while in Q044, only a significant difference in contacts are observed. Although there is a clear pattern for the advance letter group requiring fewer attempts and contacts, the differences are quite small. On average the letter group completions require .5 fewer attempts than the control and about .25 fewer contacts.

Table 10: Mean Number Of Call Attempts by Advance Letter Treatment and Quarter for Complete Interviews Only
	Mean Call Attempts		Mean Contacts
Quarter	Letter	Control	Letter	Control
Q042	4.16	4.92	2.18	2.57
Q043	4.20	4.73	2.04	2.28
Q044	3.79	4.32	1.93	2.22
Q051	3.72	4.38	1.90	2.22
Q052	4.12	4.78	1.93	2.19
Q053	4.45	4.66	2.00	2.14
Overall*	4.04	4.56	1.95	2.21
*Excluding Q042

Advance Letter: Data Quality

Data quality is assessed in two areas of the survey: the amount and quality of outlet (store) information and the amount of missing demographic information. Table 11 shows the mean number of outlets and the mean number of unique outlets reported by the CU.⁵ Although the general pattern is favorable towards the advance letter, there are no significant differences in the number of outlets reported. In addition, the percentage of outlets rejected due to inadequate address information is also not statistically significant.

Table 11: Mean Number of Outlets and Proportion Rejected by Treatment and Quarter­­­
Quarter	Mean Number of Outlets				Mean Percent Rejected
	Gross		Unique
	Letter	Control	Letter	Control	Letter	Control
Q042	5.19	5.19	4.23	4.18	8.73	11.06
Q043	5.11	5.13	4.20	4.28	8.52	7.47
Q044	5.36	5.13	4.36	4.19	8.74	10.33
Q051	5.40	5.24	4.40	4.25	8.42	8.80
Q052	5.39	5.02	4.39	4.04	8.52	8.25
Q053	5.33	5.02	4.37	4.07	8.65	8.49
Overall*	5.34	5.11	4.36	4.16	8.57	8.77
*Excluding Q042 No differences between letter and control are statistically significant at p=0.05 in this table

Examining these numbers by whether the CU remembers receiving the letter provides somewhat more pronounced results. As Table 12 indicates, the differences between those that do not remember the letter and those that do remember the letter are statistically significant for gross outlets, unique outlets, and the percent of outlets rejected. Although the difference is smaller, those that are not sent a letter report significantly fewer total and unique outlets than those that remember the letter. There are no significant differences between those that don’t remember the letter and CUs that were not sent a letter.

Table 12: Mean Number of Gross and Unique Outlets and Mean Percent Rejected by Respondent’s Recollection of Advance Letter†*
Mean	Remembers Letter	Does not Remember	Not Sent Letter
Gross Outlets	5.67	4.88	5.11
Unique Outlets	4.60	4.02	4.16
Percent Rejected	7.32	9.88	8.77
† Wave 1 Completions, only those with matched addresses who were sent a letter received question on recollection of letter. Both of these groups received the Full OMB introduction. *Excluding Q042 Boldface with underline indicates statistically significant difference between the estimate and the estimate for the “Remembers Letter” group at p=0.05. There are no statistically significant differences between the “Does not Remember” and “Not Sent Letter” groups.

There are no significant differences between the control and letter group in the amount of missing demographic information (Table 13). However, when comparing those that remember the letter to those that do not remember the letter we see that CUs remembering the letter are more likely to give their demographic information, including questions about race and Hispanic origin.

Table 13: Percentage of Completed Interviews with Missing Demographics by Advance Letter Treatment†
	Letter	Control
Missing on race or Hispanic origin	2.91	3.37
Missing on any other demographic	4.07	4.63
† Wave 1 Completions with matched addresses. All quarters combined excluding Q042. No differences between letter and control are statistically significant at p=0.05 in this table

Table 14: Percentage of Completed Interviews with Missing Demographics*
	Remembers Letter	Does not Remember	Not Sent Letter
Missing on race or Hispanic origin	3.02	5.71	4.63
Missing on any other demographic	2.21	4.07	3.37
* Wave 1 Completions with matched addresses. All quarters combined excluding Q042. Boldface with underline indicates statistically significant difference between the estimate and the estimate for the “Remembers Letter” group at p=0.05. There are no statistically significant differences between the “Does not Remember” and “Not Sent Letter” groups.

Comparing those that remember the letter with the group that was not sent a letter, we see that those that were not sent a letter are significantly less likely to provide the interviewer with their race or Hispanic origin. As with the number of outlets, there are no significant differences between those that do not remember the letter and those that were not sent a letter.

To address the issue of spuriousness - that is the idea that the effect of remembering the letter on data quality is actually due an underlying relationship between characteristics that both make a person likely to respond to the survey and more likely to receive, open, and read mail of this type - we control for a number of demographic variables (such as age and owner/renter status) and re-examine the relationship between remembering the letter and data quality. For outlet information as well as demographic information (excluding the control variable), the relationship between these measures and remembering the letter is attenuated by the introduction of age and ownership status. In the case of missing race data, the relationship is greatly attenuated by home ownership status, where renters who are not sent the letter are not significantly different than those remembering the letter.

Advance Letter: Possible Biases

Biases may be introduced by the use of an advance letter. Table E1 shows the percentage of the letter and control groups that are in a given demographic category. Significant departures in the demographic profile of the two groups may indicate possible bias. However, there are no significant differences in the percentage CUs in any given demographic category by advance letter status.

When examining the demographic profile of those who remember the letter compared to those who did not and those that were never sent a letter, some differences are in demographic categories are apparent. Those that do not remember the letter are more likely to be renters, younger, Hispanic, and non-White, and are less likely to have someone in their CU over the age of 62. Those that were not sent a letter are also more likely to be non-White and are less likely to have someone in their CU over the age of 62 than those that remember the letter. Once again there are not significant differences between CUs that do not remember the letter and CUs that were not sent a letter. These results are shown in Table E2.

It is unlikely that the advance letter experiment introduces any significant bias (by raising the response rate of a select group). There are no differences observed by advance letter status, and the differences that occur by recollection of the letter are quite small.

Advance Letter: Effects on Panel

Respondents in the first wave of the TPOPS are inducted into a panel where an interview is attempted in three subsequent quarters of the survey. Table 16, shows the key rates by wave of interview and attrition (% of wave 1 interviews completed in that wave). Attrition is not true attrition, in that wave 2 and 3 nonrespondents are solicited for an interview in subsequent quarters, so that nonrespondents can re-enter the panel. As shown in Table 16, there appear to be lasting positive effects of the advance letter on the response rate and cooperation rate. There is some evidence that the letter decreases attrition in the panel, as well.

Table 16: Response Rate, Refusal Rate, Cooperation Rate And Attrition by Wave of Interview*
	Letter /Control	Wave 1	Wave 2	Wave3	Wave4
RR2†	L	56.7	69.4	65.4	68.3
	C	48.8	65.5	59.4	61.1
REF1	L	25.1	12.8	15.4	12.4
	C	32.3	15.2	18.4	16.2
COOP4	L	69.3	84.4	81.0	84.6
	C	60.1	81.2	76.3	79.0
Attrition‡	L	100.0	77.9	71.1	67.8
	C	100.0	74.4	66.8	61.0
*Cohorts with quarters Q043 to Q052 and Q044 to Q053 combined. †AAPOR RR2 (non-SMD) ‡Defined as the percentage of Wave 1 completed interviews that are completed in subsequent waves. Boldface with underline indicates statistically significant difference between letter and control at p=0.05

There are small differences between the letter and control groups for the number of outlets and unique outlets volunteered by respondents or in the proportion of rejected outlets in Waves 2 through 4. Over all waves, the mean number of outlets for the advance letter treatment group is 20.2 compared to 19.6 for the control group. The mean number of unique outlets is also slightly higher for the treatment group (16.5 compared to 16.2), while the proportion of outlets that are rejected is slightly smaller (7.5% compared to 8.4%). These differences, while consistent with previous results are not statistically significant.

Interestingly, later panel waves tend to have higher missing rates on demographic variables. At each wave the control group has a larger percentage of missing data than the letter group but none of these differences are statistically significant. Table E4 in Appendix E summarizes these results.

Examining those who remember the letter compared to those that did not, we see that over all waves combined, that those that do not remember receiving a letter (but who were sent one) are similar to those who were never sent a letter in terms of the number of outlets, unique outlets, and the proportion of rejected outlets. Overall the mean number of outlets reported by those that remember receiving the letter is 21.2, compared to 18.7 for those that did not remember the letter and 19.6 for those not sent a letter. Similarly, the number of unique outlets is higher for CUs remembering the letter (17.4) compared to those not remembering (15.3) and those never sent a letter (16.2). The proportion of outlets rejected is 7.1% for those that remember the letter while the the proportion of outlets rejected is 8.7% and 8.4% for those that did not remember the letter and those that were not sent a letter, respectively.

Although the letter treatment group consistently requires fewer contacts and attempts on all waves, the differences in the mean are very small. The same is true for contacts and attempts for completed interviews in waves 2 through 4.

Confidentiality Statement

Only 18% of all refusals received the confidentiality statement in either form, most refusals occur prior to the confidentiality statement. Of those receiving the letter, the control groups can be divided by whether they include those that do not remember the letter, or not. Only those remembering the letter are eligible to receive the short statement. As shown in Table 19 the results of the confidentiality statement experiment are counter-intuitive. While there is no significant difference between those who were read the short confidentiality statement and remember the letter and those that are read the longer statement and may or may not remember the letter, there is a significant difference between those read the short statement and those read the long, when examining only those who remember the letter. The results, however, are in the opposite direction as expected with the response rate higher for those in the control group. The cooperation rate is also significantly higher, while the refusal rate is significantly lower for the control group.

Table 19: Key Rates by Confidentiality Statement Treatment Group†‡
Rate	Short Statement*	OMB Statement
		Control 1**	Control 2*
RR2	92.3	90.6	96.9
RR4	92.5	90.9	96.8
REF1	6.3	6.1	2.3
COOP4	93.6	93.7	97.7
* This control group includes only households that remember receiving the advance letter. ** This control group includes both households that do remember and do not remember receiving the advance letter. † SMD rates are identical to corresponding corrected rates. ‡ Wave 1 sample with matched address that received confidentiality statement combined across quarters, excluding Q042. Boldface with underline indicates statistically significant difference between the estimate and the estimate for the “Short Statement” group at p=0.05

Although significant differences are only found in Q052 and Q053, Table 20 shows that this phenomenon is consistent across all quarters of the study.

Table 20: Response, Refusal, and Cooperation Rates by Confidentiality Treatment and Quarter†
Rate	Short /OMB	Quarter						Overall*
		Q042	Q043	Q044	Q051	Q052	Q053
RR2	Short	92.1	94.5	96.0	97.8	87.7	87.2	92.3
	OMB	95.5	97.1	98.8	99.3	95.6	93.7	96.9
RR4	Short	92.5	94.8	96.0	97.8	88.1	87.3	92.5
	OMB	95.5	97.1	98.9	99.3	95.8	94.0	96.8
REF1	Short	5.3	3.8	2.8	1.5	10.1	11.5	6.3
	OMB	3.3	1.9	0.5	0.4	3.6	4.6	2.3
COOP4	Short	94.5	96.1	97.1	98.5	89.7	88.4	93.6
	OMB	96.6	98.1	99.5	99.6	96.4	95.3	97.7
† Wave 1 sample with matched address that received confidentiality statement and remembered advance letter. *Excluding Q042 Boldface with underline indicates statistically significant difference between letter and control at p=0.05

There are no differences in the number of outlets and unique outlets reported or the percentage of outlets rejected by type of confidentiality statement. In addition there is little difference in attrition or subsequent wave response rates. Of course, this is expected given that the confidentiality statement is only given to CUs in the first wave of interviewing. Subsequent waves receive the OMB statement in all cases.

Table 21: Total Interview Time and Interview Segment Time by Confidentiality Treatment and Quarter†
Segment	Short /OMB	Quarter						Overall*
		Q042	Q043	Q044	Q051	Q052	Q053
Front of Instrument	Short	3.8	3.5	3.0	3.1	3.2	3.1	3.2
	OMB	4.6	4.0	3.5	3.5	3.7	3.6	3.6
Outlet Questions	Short	10.9	8.8	8.4	8.2	8.4	8.2	8.4
	OMB	11.2	8.8	8.9	8.6	8.5	8.3	8.6
Total Time	Short	17.6	14.6	13.8	13.6	13.9	13.7	13.9
	OMB	18.7	15.3	14.7	14.4	14.4	14.1	14.5
† Wave 1 completed interviews with matched address that received confidentiality statement and remembered advance letter. Times in minutes. *Excluding Q042 Boldface with underline indicates statistically significant difference between letter and control at p=0.05

In conversation with a supervisor at one of the Census call centers that conducts surveys for the TPOPS, it was suggested that the counter intuitive finding that the short statement had a lower response rate may be partially explained by question flow and wording of the short statement. Immediately preceding the confidentiality statement is the question about remembering the advance letter. Immediately following the confidentiality is notification that the supervisor may be listening in. According to the supervisor interviewers having a great deal of experience with the OMB statement are able to use its length as opportunity to emphasize certain persuading arguments within that long statement, adding caveats, and other persuasive language. Being unfamiliar with the new confidentiality, their ability to improvise was hampered and the flow and persuasiveness suffered. However, if this was the case, we would expect that later administrations of the experiment would have smaller differences in response rates. Instead, the opposite is true. It was also mentioned that the short introduction emphasized the wrong aspects of the survey, mentioning, in this order: the quarterly nature of the survey (repeated interviews), the purpose (very briefly), the length of the interview, it being voluntary, and confidential.

IV. Conclusions and Recommendations

The advance letter has proven to be effective in soliciting response and encouraging cooperation from the entry wave RDD sample. The 7-8% improvement in response among this sample is encouraging. However, the overall effect of the advance letter due to a low match rate is quite small. If all matched sample were to be administered the advance letter of the period of the experiment RR2 would be 43.4% for the Wave 1 RDD sample, if no matched sample were administered the advance letter RR2 would be 39.7% for the Wave 1 RDD sample. Similarly, the overall response rate for the entire quarter, based on the results from each wave of the cohort, would be 50.4% if all matched sample were administered the letter compared to 47.8% in the advance letter were not administered at all.

Given the relatively small increase in the response rate we might question whether using the advance letter is worth the cost. Although cost figures are difficult to ascertain, the reason for the diminished effect of the advance letter, the unmatched sample, does not contribute to the cost. No letters are sent to unmatched sample units. Therefore, it is the recommendation of this author that the advance letter be administered to all units in the entry wave RDD sample that are able to be matched to an address.

The confidentiality statement experiment produced some counter intuitive findings. While using the short confidentiality statement saved interview time and decreased respondent burden, the response rate was lower for this group, compared to those that received the standard OMB confidentiality statement. It was suggested from a interview supervisor that the statement may be emphasizing points that are not very persuasive and by so short in length as to allow little non-scripted persuasive arguments by the interviewer. It is the author’s recommendation that the wording of the abbreviated confidentiality statement be revised, and perhaps lengthened to allow for more salient arguments.

Appendix A: Advance Letter

{DOL/BLS letterhead}

Dear Smith Household,

Sometime in the next week or so someone will be calling from the US Census Bureau. The reason for the call is that your household has been selected to participate in a brief survey that helps estimate the cost of living in the United States. Since this is a study about where people shop, we felt it was important to assure you that the call is for government research.

Although participation is voluntary, we would really appreciate your help. The information you provide will be kept confidential. Additional information about the survey is on the back of this letter. Thank you in advance for your participation.

FREQUENTLY ASKED QUESTIONS

WHAT IS THE PURPOSE OF THE POINT OF PURCHASE SURVEY?

The purpose of the Point of Purchase Survey is to provide information on the stores and businesses where people shop and the products they buy. This information is used to update the Consumer Price Index or the “cost of living index.” This index directly affects almost everyone at one time or another.

The President, the Congress, and the Federal Reserve use trends in the cost of living to aid in formulating fiscal and monetary policies. Adjustments to wages, Social Security Payments, retirement benefits and other compensation such as child support and payments received by millions of Americans are based on changes in the Consumer Price Index. In addition, businesses, labor organizations and private citizens use the index as a guide in making economic decisions.

HOW WILL THE INFORMATION BE USED?

In order for the Bureau of Labor Statistics to update the Consumer Price Index, it is necessary to maintain an up-to-date list of the stores and businesses where people shop. The Census Bureau conducts the Point of Purchase Survey to provide statistical data needed to update this list.

WHY WAS I SELECTED?

Participation is voluntary and there are no penalties for not answering questions, but to ensure the validity and accuracy of the information it is very important that we have your cooperation in the survey. Your household was selected randomly from a list of phone numbers. The information that you provide will be used only to identify the stores and businesses where Americans shop and the products they buy.

WHAT CONFIDENTIALITY PROTECTION DO I HAVE?

This survey is authorized by Office of Management and Budget under project 1220-0044. The Bureau of Labor Statistics and the Census Bureau will use the information you provide for statistical purposes only. The confidentiality of the information you provide is assured to the full extent permitted by law in accordance with the Privacy Act of 1974 and the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347), Your responses will not be disclosed in identifiable form without your informed consent.

WHY IS CENSUS RATHER THAN BLS CONDUCTING THIS SURVEY?

Title 29, Section 2 and Title 15 of the United States Code authorizes the Bureau of the Census to conduct household studies for other Federal Agencies..

HOW LONG DOES THIS INTERVIEW TAKE?

The typical interview lasts between 7 and 15 minutes.

Appendix B: Interview Script Modifications

Note: Modifications listed below do not include all changes in branching.

---------------------------------------------------------------------------------------------------------

>HELLO_RS<

I’d like to speak with a resident who is 18 or older, who normally uses this phone, and knows about household expenses.

IF APPROPRIATE: Would that be you?

(1) Respondent is resident 18+ - IF QUARTER = 1 AND TREATMENT GROUP = 3 OR 4 GO TO >LETTER<, ELSE GOTO >INTRO_1ST<

(2) Resident 18+ called to phone – IF QUARTER = 1 AND TREATMENT GROUP = 3 OR 4 GO TO >LETTER<, ELSE GO TO >INTRO_1ST<

(3) Eligible person not home now or not available now – GO TO >HELLOWHO<

(4) No one in household is 18+ - GO TO >INT_OTH2<

(5) Other outcome/problem interviewing respondent including teenage phone line. – GO TO >HELLO_2<

---------------------------------------------------------------------------------------------------------

IF QUARTER = 1 AND TREATMENT GROUP=3 OR 4,

>LETTER< * New question

Not long ago, we mailed a letter to your household telling you a little bit about this survey.

Do you remember receiving this letter?

(1) YES – IF TREATMENT GROUP = 3 GOTO >INTRO_1ST<

IF TREATMENT GROUP = 4 GOTO >INTRO_LET<

(2) NO – GO TO >INTRO_1ST<

---------------------------------------------------------------------------------------------------------

>INTRO_1ST<

IF NECESSARY: Hello, this is iname from the U.S. Census Bureau. We are conducting a survey for the Bureau of Labor Statistics.

This survey is conducted quarterly and is used to update the Consumer

Price Index by identifying where consumers currently shop. The survey will take about 11 minutes and is voluntary. Without Office of Management and Budget approval under project 1220-0044, we could not conduct this survey. The BLS and the Census Bureau will use the information you provide for statistical purposes only and will protect the confidentiality of the information in accordance with the Privacy Act of 1974.

Title 29, Section 2 and Title 15 of the United States Code authorizes us to conduct this survey.

ENTER (P) TO PROCEED – GOTO >INTRO_B<

<R>REFUSED – GOTO >HELCOM<

---------------------------------------------------------------------------------------------------------

>INTRO_LET< * New question

This study is conducted quarterly and is used to update the Consumer Price Index. On average it takes about 11 minutes. Participation is voluntary and the information you provide will be kept confidential.

ENTER (P) TO PROCEED – GOTO >INTRO_B<

<R>REFUSED – GOTO >HELCOM<

---------------------------------------------------------------------------------------------------------

>INTRO_B<

READ: My supervisor is working with me today and may listen in to evaluate my performance.

PERSUADE RESPONDENT TO COMPLETE INTERVEIEW NOW IF POSSIBLE.

(1) Inconvenient time, callback needed – GOTO >INTRO_B1<

(2) Refused to participate – GO TO >HELCOM<

(3) Language problem OR Refer to Supervisor – GOTO > INTRO_B2<

PRESS (P) TO PROCEED – GO TO >INTRO_C<

Appendix C: Calculation of Response Rates

General Form of Key Rates

Response rates are AAPOR compliant (AAPOR 2006) rates where possible. The difference in the SMD rates and the Corrected rates are that the corrected rates define the final call dispositions differently than SMD. The calculation of the rates is the same.

Where I = Interviews

R = Refusal

NC = Noncontacts

UK = Unknown eligibility

Where e, is the estimate of the proportion of the unknown eligible numbers that are actually eligible. For TPOPS e is estimated to be 0.27.

Estimated Response Rate for Entire New RDD Sample

Given that

and

where,

I=Completions

R=Refusal

NC = Noncontact

UK = Unknown eligibility

According to AAPOR

Samptype = [Letter (L), Control (C), Unmatched (UM)]

TS is the total sample

The overall response rate is calculated by,

If we set the response rate of the control group to be equal to that of the letter group,

then it follows that,

so that RR₂ now can be written as:

In a similar way, the refusal rate is defined as

and the cooperation rate is:

where TCS = Total Contacted sample given by

Adjusted Response Rate for Entire Sample:

The adjusted response rate for the entire sample is estimated by the weighted average of RR2’, where each wave in sample is estimated by the corresponding wave of the experiment cohort.

Where RR’_2Q is the adjusted response rate for the entire quarterly sample, TES_Q’ is the estimated eligible sample size for a given survey implementation (Estimated by total sample size in experimental cohort Q044-Q053), and TES_x is the estimated eligible sample size for wave x in given quarter (Estimated by sample size of wave x in experimental cohort).

Appendix D: Address Match Rate by Region and Primary Sampling Unit

Table D1: Address Match Rate by Region and PSU
Region	PSU	Q042	Q043	Q044	Q051	Q052	Q053	All Quarters
1	All PSU	19.84	19.14	29.55	28.96	27.10	26.79	25.49
1	1102	22.69	19.48	25.21	28.46	22.34	21.51	23.39
1	1103	15.54	17.71	34.35	34.38	35.03	32.12	28.64
1	1104	13.16	9.39	26.75	30.24	23.85	24.62	22.17
1	1109	21.93	19.16	25.49	24.69	24.76	21.59	23.01
1	1110	16.37	15.25	29.62	30.93	27.64	28.04	24.92
1	1111	26.34	25.71	27.66	24.87	25.47	24.65	25.77
1	2102	13.16	13.39	29.05	26.09	22.45	27.59	22.47
1	2104	14.57	26.06	33.33	30.36	24.50	26.58	26.10
1	2106	16.07	20.77	34.48	27.27	32.65	28.76	27.23
1	2108	19.39	22.17	34.78	31.34	31.40	32.72	28.98
1	2110	28.51	29.49	35.07	31.10	23.64	35.68	30.52
1	2112	12.28	12.14	32.94	25.70	30.86	29.27	25.13
1	2114	28.18	26.79	33.65	34.48	36.07	42.02	33.67
1	2116	15.44	13.33	33.91	36.78	31.22	32.02	27.60
2	All PSU	21.56	22.84	29.20	29.92	27.64	27.29	26.67
2	1207	18.02	19.85	24.04	23.63	22.21	22.38	21.84
2	1208	22.01	24.38	26.65	27.31	21.54	27.77	25.01
2	1209	18.02	15.99	33.33	34.11	24.76	23.71	25.49
2	1210	24.54	26.49	23.40	28.05	25.96	28.04	26.11
2	1211	11.16	14.63	29.48	31.07	29.83	28.83	24.83
2	1212	27.40	26.01	32.79	32.45	27.15	28.57	29.13
2	1213	19.66	22.65	31.37	35.07	30.77	29.22	28.52
2	1214	12.18	19.21	30.48	33.96	29.09	31.14	26.90
2	2218	29.63	37.40	38.78	38.52	38.36	27.70	35.19
2	2220	33.05	26.40	31.88	36.96	36.43	31.97	32.88
2	2222	11.61	6.03	30.56	27.70	28.28	27.92	23.08
2	2224	25.23	30.53	30.14	28.17	31.17	23.93	28.23
2	2226	33.05	28.29	30.92	36.05	29.41	26.86	30.53
2	2228	13.64	13.79	34.38	29.79	36.17	25.50	25.80
2	2230	26.24	23.75	29.27	28.92	28.86	26.84	27.52
2	2232	26.32	22.39	25.85	29.41	26.90	25.15	26.07
2	2234	28.24	30.07	33.73	27.50	21.12	27.21	27.97
2	2236	20.18	23.87	40.10	32.77	34.25	33.33	31.74
2	3212	9.45	16.77	29.94	28.74	32.48	36.92	26.74
2	3216	45.54	39.29	36.70	39.62	36.36	37.00	39.07
2	3218	15.13	26.50	22.43	21.74	29.03	19.85	22.90
2	3222	33.07	26.35	26.95	38.04	27.89	30.88	30.43
3	All PSU	22.89	22.72	31.91	31.18	28.87	28.78	28.04
3	1312	27.64	29.39	26.30	28.57	24.85	22.99	26.56
3	1313	23.18	33.05	29.89	31.23	25.42	28.15	28.52
3	1316	16.01	13.40	29.66	27.92	25.34	26.36	23.48
3	1318	16.61	16.31	28.07	26.59	27.48	24.94	23.74
3	1319	12.84	11.34	24.69	24.50	23.27	23.87	20.91
3	1320	17.93	18.70	29.03	27.95	26.31	22.91	24.00
3	1321	18.70	17.67	30.50	31.50	30.31	24.32	26.07
3	2338	18.90	14.45	25.85	33.94	29.44	30.19	26.43
3	2340	25.38	29.70	36.81	31.36	29.05	31.69	30.84
3	2342	17.53	14.06	21.34	25.43	27.75	28.71	23.22
3	2344	31.85	19.44	27.52	28.67	26.97	22.84	26.10
3	2346	25.20	33.59	35.86	34.04	27.66	33.33	31.72
3	2348	34.85	29.05	33.33	25.37	30.77	38.28	31.84
3	2350	32.80	24.14	34.21	26.42	27.95	28.93	28.97
3	2352	22.30	25.58	37.10	34.42	32.03	33.53	30.87
3	2354	32.52	29.45	35.10	39.33	36.02	45.16	36.32
3	2356	51.55	41.96	43.22	44.62	47.93	51.88	47.07
3	2358	32.26	32.91	39.62	30.77	37.93	26.67	33.33
3	2360	21.13	23.86	44.84	35.78	33.99	38.04	33.76
3	2362	30.20	27.71	32.78	31.15	30.93	29.02	30.33
3	2364	24.49	22.05	48.20	41.71	30.67	30.77	33.85
3	2366	29.84	28.99	32.41	31.69	31.14	28.77	30.47
3	2368	23.42	26.96	37.60	45.86	31.78	29.77	32.93
3	2370	22.50	23.57	36.00	29.59	32.32	22.92	28.31
3	2372	13.16	14.17	30.28	30.53	24.31	26.09	23.57
3	2374	18.18	18.01	35.00	27.01	32.08	33.12	27.41
3	2376	29.55	25.32	36.07	35.26	28.33	28.49	30.71
3	2378	26.80	30.20	32.34	29.38	30.89	28.98	29.89
3	2380	15.22	15.25	22.80	28.64	23.21	26.51	22.52
3	3328	23.21	23.35	31.89	35.53	27.98	33.33	29.94
3	3332	12.10	15.53	36.24	38.62	33.11	35.66	28.75
3	3334	39.81	43.86	46.67	46.58	42.11	49.14	44.88
3	3344	22.70	0.00*	23.49	24.66	20.72	23.20	22.88
4	All PSU	17.05	17.10	20.74	19.97	18.27	18.64	18.67
4	1419	15.34	17.20	15.26	17.48	14.16	15.92	15.86
4	1420	16.67	15.80	16.96	20.65	13.94	18.32	17.06
4	1422	17.56	14.92	18.67	16.26	16.52	14.73	16.42
4	1423	15.52	21.80	22.49	20.54	25.00	22.64	21.42
4	1424	23.08	20.00	17.84	17.02	15.61	19.02	18.67
4	1425	14.42	13.64	21.79	21.85	18.86	17.69	18.31
4	1426	15.99	17.59	12.88	17.04	13.13	13.62	14.97
4	1427	9.49	8.97	11.74	8.87	7.39	6.94	8.89
4	1429	20.76	20.76	24.20	19.87	26.64	25.00	22.83
4	1433	6.82	5.43	26.44	23.36	18.55	22.43	17.68
4	2482	17.78	27.78	28.44	30.51	26.96	24.53	26.32
4	2484	27.55	26.37	33.66	28.57	29.81	27.68	28.97
4	2486	19.01	21.88	23.64	21.95	20.59	11.68	19.50
4	2488	20.00	14.94	37.34	32.43	28.06	36.29	28.28
4	2490	21.26	18.38	19.89	22.22	18.44	12.18	18.78
4	2492	35.29	32.73	43.08	37.19	36.36	32.14	36.33
4	3450	18.64	25.00	32.08	28.00	30.67	29.05	27.71
4	3456	16.78	15.86	13.29	13.21	13.14	18.64	15.27
Overall		20.49	20.63	28.25	27.92	25.89	25.78	25.05
*Not mailed or matched to address due to Hurricane Katrina

Appendix E: Supplemental Tables

Table E1: Response, Refusal, and Cooperation Rates by Advance Letter Treatment and Region†
Rate	Letter /Control	Region				Overall
		Northeast	Midwest	South	West
RR2‡	L	53.3	64.4	55.2	60.4	58.0
	C	44.3	52.8	49.1	51.9	49.5
SMDRR2	L	52.9	64.2	54.9	60.0	57.7
	C	44.1	52.5	48.9	51.7	49.2
RR4	L	55.9	65.9	51.4	53.7	59.6
	C	46.6	54.1	57.7	62.7	50.9
SMDRR4	L	55.6	65.9	57.7	62.3	59.4
	C	46.3	54.0	51.4	53.3	50.7
REF1	L	27.8	23.6	25.5	22.7	25.1
	C	36.8	33.9	31.8	32.7	33.6
SMDREF1	L	27.6	23.5	25.4	22.6	25.0
	C	36.6	33.7	31.7	32.6	33.4
COOP4	L	65.7	73.1	68.4	72.6	69.8
	C	54.6	60.9	60.7	61.4	59.5
† Wave 1 Sample, only those with matched addresses. Excluding Q042 ‡ See Appendix B for calculation of response rates

Table E2: Demographics (Percentage*) of Completed Interviews by Treatment†
	Valid and Missing		Valid Only
	Letter	Control	Letter	Control
Owner	79.9	78.4	81.6	80.8
Renter	18.0	18.7	18.4	19.2
Missing owner/renter	2.1	2.9
Age <= 25	3.4	3.7	3.5	3.8
26 <= Age <= 35	12.4	13.4	12.9	14.0
36 <= Age <= 50	29.7	31.5	30.8	32.8
51 <= Age <= 65	28.1	26.9	29.2	28.0
Age >= 66	22.7	20.5	23.6	21.4
Missing age	3.8	4.0
Married	60.0	58.6	61.5	60.5
Not married	37.6	38.3	38.5	39.6
Missing marital status	2.4	3.1
Male	34.8	36.3	35.5	37.2
Female	63.3	61.2	64.5	62.8
Missing sex	1.9	2.6
Hispanic	7.4	8.1	7.6	8.4
Not Hispanic	90.3	89.0	92.4	91.6
Missing Hispanic Origin	2.3	2.9
Non-White	15.1	16.3	15.5	16.7
White	82.2	80.5	84.5	83.1
Missing race	2.7	3.2
HH size: One	25.1	24.6	25.2	24.8
Two	37.6	36.3	37.8	36.6
Three	14.3	15.5	14.4	15.7
Four or more	22.5	22.7	22.6	22.9
Missing HH size	0.5	0.9
Presence of over 62	35.2	31.8	36.8	33.5
None over 62	60.4	63.1	63.2	66.5
Missing presence of over 62	4.3	5.1
More than one telephone line	12.6	11.1	12.9	11.4
One telephone line	85.4	86.3	87.1	88.6
Missing number of tel. lines	2.0	2.7
*Valid percent unless percent missing †Wave 1 completions, all quarters excluding Q042 No differences between letter and control are statistically significant at p=0.05 in this table

Table E3: Demographics (Percentage) of Completed Interviews by Treatment†
	Remembers Letter	Does not Remember	Not Sent Letter
Owner	83.6	76.4	80.8
Renter	16.4	23.6	19.2
Age <= 25	2.6	5.2	3.8
26 <= Age <= 35	12.6	14.3	14.0
36 <= Age <= 50	30.5	31.4	32.8
51 <= Age <= 65	30.0	27.4	28.0
Age >= 66	24.3	21.7	21.4
Married	62.8	59.7	60.5
Not married	37.2	40.3	39.6
Male	35.2	35.8	37.2
Female	64.8	64.2	62.8
Hispanic	6.0	11.0	8.4
Not Hispanic	94.0	89.0	91.6
Non-White	12.2	20.2	16.9
White	87.8	79.8	83.1
HH size: One	26.0	24.4	24.8
Two	38.2	36.4	36.6
Three	14.0	14.9	15.7
Four or more	21.8	24.2	22.9
Presence of over 62	38.7	34.6	33.5
None over 62	61.3	65.5	66.5
More than one telephone line	86.8	87.5	88.6
One telephone line	13.2	12.6	13.2
†Wave 1 completions, all quarters excluding Q042 Boldface with underline indicates statistically significant difference between the estimate and the estimate for the “Remembers Letter” group at p=0.05. There are no statistically significant differences between the “Does not Remember” and “Not Sent Letter” groups.

Table E4: Percentage of Completed Interviews with Missing Demographics by Advance Letter Treatment by Wave*
	Letter /Control	Wave 1	Wave 2	Wave 3	Wave 4
Missing on race or Hispanic origin	L	2.9	6.8	12.3	16.1
	C	3.6	8.9	14.5	19.4
Missing on any other demographic	L	4.0	7.6	12.9	16.7
	C	4.8	9.5	15.5	20.0
*Cohorts with quarters Q043 to Q052 and Q044 to Q053 combined. No differences between letter and control are statistically significant at p=0.05 in this table

1 A Consumer Unit is defined as a person living alone or a group of two or more persons that are related or, if unrelated, share major living expenses. Persons are considered to be sharing major living expenses if they share any two of housing expenses, food expenses, other expenses such as transportation, clothing, medical, or educational expenses. A Consumer Unit is not necessarily an entire household. In some cases more than one Consumer Units reside in a household sharing a single phone line.

2 The respondent is required to see all of the information in the OMB statement prior to conducting an interview. Because of this only respondents remembering receiving the letter, which duplicated all of the information in the OMB statement, were able to be given the short statement.

3 Due to some confusion on behalf of those implementing the survey, the allocation of the matched sample in Q042 is allocated differently among the three groups (30,30,40).

4 SAS Procedures, SURVEYREG and SURVEYMEANS were used to assess statistical significance.

5 A respondent may purchase a variety of goods at the same store. Therefore, in the course of an interview they may mention that outlet more than once. Any time an outlet is mentioned, irregardless it being mentioned previously in the same interview, will count towards the gross outlet total, while only outlets being volunteered for the first time count towards the unique outlet total.