ACS Methods Panel 2020 SMMT Study Plan

American Community Survey Research and Evaluation Program

DATE

ACS Research & Evaluation Analysis Plan (REAP)

2020 Specialized Mail Materials Test

TABLE OF CONTENTS

1. Introduction 1

2. Background 1

2.1 2020 ACS Data Collection Strategy 1

2.2 ACS Baseline Mail Materials in 2020 3

2.3 ACS Specialized Mail Materials in 2020 4

2.3.1 Cognitive Testing 4

2.3.2 Final Design of 2020 Specialized Mail Materials 5

2.4 2020 Census Contact Strategy 8

2.4.1 Mailout Strategy 8

2.4.2 Additional 2020 Census Contacts 9

3. Literature Review 11

3.1 2010 ACS Response Rates and Check-in Rates 11

3.2 2010 Evaluation of ACS Revised Mail Materials 13

3.3 ACS Messaging Research by Reingold 14

4. Research Questions and Methodology 15

4.1 Experimental Design 15

4.2 Research Questions 15

4.2.1 Question 1 – Self-response Return Rates 16

4.2.2 Question 2 – Final Response Rates 17

4.2.3 Question 3 – Blank and Insufficient Forms 18

4.2.4 Question 4 – Form Completeness 18

4.3 Sample Design and Weighting 19

5. Assumptions and Limitations 20

5.1 Assumptions 20

5.2 Limitations 20

6. Sample Table Shells 21

7. Potential Changes to ACS 25

8. References 25

Appendix A. Images of the 2020 ACS Baseline Mail Materials 27

A.1 Mailing 1: Initial Mail Package 27

A.2 Mailing 2: Initial Reminder 32

A.3 Mailing 3: Paper Questionnaire Package 34

A.4 Mailing 4: Reminder Postcard 40

A.5 Mailing 5: Final Reminder Letter 41

Appendix B. Images of the 2020 ACS Specialized Mail Materials 43

B.1 Mailing 1: Initial Mail Package 43

B.2 Mailing 2: Initial Reminder 47

B.3 Mailing 3: Paper Questionnaire Package 49

B.4 Mailing 4: Reminder Postcard 55

B.5 Mailing 5: Final Reminder Letter 56

Appendix C. Form Numbers on 2020 ACS Mail Materials 58

Appendix D. Mail Delivery Dates for ACS and Decennial, January-July 2020 59

LIST OF TABLES

Table 1: Differences between Baseline and Specialized Mail Materials 7

Table 2: 2020 Census Mailout Strategy and In-Home Delivery Dates 9

Table 3. Self-Response Return Rates Before the Third Mailing 21

Table 4. Self-Response Return Rates Before the Fifth Mailing 21

Table 5. Self-Response Return Rates Before CAPI 22

Table 6. Final Overall Response Rates and Distribution by Mode 23

Table 7. Overall Rate of Blank and Insufficient Forms 24

Table 8. Overall Form Completion 24

Table 9. Basic Person Demographics Section Form Completion 24

Table 10. Housing Section Form Completion 25

Table 11. Detailed Person Section Form Completion 25

Table 12: Form Numbers for Baseline and Specialized Mail Materials 58

Table 13. Target In-Home Delivery Dates for ACS Mailings and Decennial Self-Response Mailings 60

TABLE OF FIGURES

Figure 1. Overview of the 2020 ACS Self-Response Mail Contact Strategy and Mailing Universes 2

Figure 2. FAQs Added to ACS Specialized Letters in 2020 8

Figure 3. Differences in the National Mail Check-in Rates – December 2009 through July 2010 Panels Compared to Previous Year’s Panel 12

Figure 4. Differences in Mail Form Completeness – December 2009 through July 2010 Panels Compared to Previous Year’s Panel 13

Figure 5. Front of Envelope ACS-46IM(2020)(02-07-2019) 27

Figure 6. Back of Envelope ACS-46IM(2020)(02-07-2019) 27

Figure 7. Front of Letter ACS-13(L)(2020)(08-02-2019) 28

Figure 8. Back of Letter ACS-13(L)(2020)(08-02-2019) 29

Figure 9. Front of Multilingual Brochure ACS-9(2020)(05-31-2019) 30

Figure 10. Back of Multilingual Brochure ACS-9(2020)(05-31-2019) 30

Figure 11. Front of Instruction Card ACS-34IM(2020)(02-13-2019) 31

Figure 12. Back of Instruction Card ACS-34IM(2020)(02-13-2019) 31

Figure 13. Inside of Letter ACS-20(L)(2020)(08-02-2019) 32

Figure 14. Outside of Letter ACS-20(L)(2020)(08-02-2019) 33

Figure 15. Front of Envelope ACS-46(2020)(02-07-2019) 34

Figure 16. Back of Envelope ACS-46(2020)(02-07-2019) 35

Figure 17. Front of Letter ACS-14(L)(2020)(08-02-2019) 36

Figure 18. Back of Letter ACS-14(L)(2020)(08-02-2019) 37

Figure 19. Front Cover of ACS Questionnaire ACS-1(2020)(07-13-2019) 38

Figure 20. Front of Business Reply Envelope 6385-47(2020)(12-12-2018) 39

Figure 21. Front of Postcard ACS-29(2020)(05-21-2019) 40

Figure 22. Back of Postcard ACS-29(2020)(05-21-2019) 40

Figure 23. Inside of Letter ACS-23(L)(2020)(08-05-2019) 41

Figure 24. Outside of Letter ACS-23(L)(2020)(08-05-2019) 42

Figure 25. Front of Envelope ACS-46IMZ(2020)(08-09-2019) 43

Figure 26. Back of Envelope ACS-46IMZ(2020)(08-09-2019) 43

Figure 27. Front of Letter ACS-13(LZ)(2020)(08-27-2019) 44

Figure 28. Back of Letter ACS-13(LZ)(2020)(08-27-2019) 45

Figure 29. Front of Multilingual Brochure ACS-9(2020)(05-31-2019) 46

Figure 30. Back of Multilingual Brochure ACS-9(2020)(05-31-2019) 46

Figure 31. Inside of Letter ACS-20(LZ)(2020)(08-23-2019) 47

Figure 32. Outside of Letter ACS-20(LZ)(2020)(08-23-2019) 48

Figure 33. Front of Envelope ACS-46(Z)(2020)(08-09-2019) 49

Figure 34. Back of Envelope ACS-46(Z)(2020)(08-09-2019) 50

Figure 35. Front of Letter ACS-14(LZ)(2020)(08-27-2019) 51

Figure 36. Back of Letter ACS-14(LZ)(2020)(08-27-2019) 52

Figure 37. Front Cover of ACS Questionnaire ACS-1(2020)(07-13-2019) 53

Figure 38. Front of Business Reply Envelope 6385-47(2020)(12-12-2018) 54

Figure 39. Front of Postcard ACS-29(2020)(05-21-2019) 55

Figure 40. Back of Postcard ACS-29(2020)(05-21-2019) 55

Figure 41. Inside of Letter ACS-23(L)(2020)(08-05-2019) 56

Figure 42. Outside of Letter ACS-23(L)(2020)(08-05-2019) 57

1. Introduction

The U.S. Census Bureau conducts the American Community Survey (ACS) continuously, including during decennial census years. All addresses in sample for the ACS in 2020 will be asked to complete both the ACS and the census, which can be confusing and burdensome to respondents. ACS response rates were impacted during both the 2010 Census and the 2000 Census; ACS response rates were higher than usual in the first few months of the year (January, February and March) but were lower than usual in the spring and summer months (April, May and June) (Chesnut and Davis 2011, Baumgardner 2013).

In preparation for fielding the ACS during the 2020 Census, the Census Bureau considered ways to minimize the anticipated negative impact on ACS response rates in 2020. Since the majority of the households sampled for the ACS will receive an initial request to respond in the mail, the mail materials were revised in order to help legitimize the ACS and distinguish it from the 2020 Census. The ACS mail materials that include 2020-specific features are called the 2020 specialized mail materials. The specialized mail materials will be sent to the majority of the ACS housing unit sample from March through September of 2020. A small portion of ACS addresses will receive ACS mailings that do not explicitly delineate it from the decennial census; these will be called the baseline mail materials. The objective of this test is to compare response rates and related metrics for these two treatments for each monthly sample from March through September of 2020.

2. Background

This section presents information on:

• the 2020 ACS data collection strategy in Section 2.1,

• the two sets of 2020 ACS mail materials in Section 2.2 and Section 2.3, and

• the decennial census data collection strategy in 2020 in Section 2.4.

1. 2020 ACS Data Collection Strategy

Every year, the Census Bureau contacts over 3.5 million households across the country to participate in the ACS. The yearly sample for the ACS is distributed evenly across the calendar year, with approximately 290,000 new households being contacted at the beginning of each month. The sampled households are first contacted by mail, when possible, and could receive five mailings within two months if a survey response is not received. Each month’s sample is referred to as a panel. The ACS mailing strategy for 2020 will be the same for all 2020 panels; no changes in strategy will be made to months that overlap with the decennial census data collection. Figure 1 shows the mail contact strategy for 2020 ACS.

Figure 1. Overview of the 2020 ACS Self-Response Mail Contact Strategy and Mailing Universes

All mailable addresses that are sampled for the ACS in 2020 will be sent a letter in the mail, inviting residents of the address to participate in the ACS online. Along with the letter, the first mailing will also contain a multilingual brochure and an instruction card that facilitates online response.¹ About seven days later, a pressure seal mailer will be sent to the same addresses. The letter inside the pressure seal mailer tells the household to either respond online, wait for a paper questionnaire, or call with questions. Recipients who call can have their questions answered and can also complete the survey with an interviewer in the Telephone Questionnaire Assistance (TQA) call center.

A reduced universe of only addresses that have not yet responded to the ACS is identified eleven days after the second mailing; the paper questionnaire package is then sent to those addresses a few days later. The addresses in this new universe will receive both a third and fourth mailing. The third mailing will contain a paper questionnaire, another letter, and a return envelope. About four days after the questionnaire package is mailed, these same addresses will be mailed a reminder postcard.

Roughly two weeks after the fourth mailing is sent, a third universe of the remaining nonrespondents is identified; a few days later, these addresses are sent a fifth mailing (another pressure seal mailer). Once the fifth mailing has been sent, the recipients have approximately two more weeks to respond before the processing begins for the personal visit data collection operation.

Approximately two months after the first mailing, addresses that have not responded are eligible for the Computer-Assisted Personal Interview (CAPI) nonresponse follow-up operation.² Nonmailable addresses that were not sent the ACS mailings are also eligible for the CAPI operation; these are addresses that are either P.O. boxes, a non-residential ZIP code, or otherwise determined to not be mailable. From these two universes (nonresponders and nonmailable addresses), a subsample is selected for the CAPI operation. Field representatives (FRs) will visit the addresses sampled for CAPI to conduct in-person interviews (or encourage self-response). The CAPI operation lasts approximately four weeks and completes the data collection effort for that monthly panel.³

This multi-mode contact strategy lasts a maximum of three months for each monthly panel. Self-responses are accepted throughout the entire three-month period.

2. ACS Baseline Mail Materials in 2020

The content of the 2020 mailings has changed substantially from those used in 2019 as part of a continual effort to improve the ACS respondent experience and self-response rates. These improvements, to be fielded for the first time in the January 2020 ACS panel, include:

• A change to the size of the paper questionnaire and the size of the envelopes,

• A “new look and feel” that was a byproduct of the 2020 specialized materials development and built on designs that were tested in the 2018 Mail Materials Test (MMT), and

• Additional changes that were tested and found successful in the 2018 MMT.

The implementation of these changes created the baseline 2020 ACS materials and are described more in this section.

The Census Bureau decided in the summer of 2019 to move to a standard questionnaire size. The 2019 ACS questionnaire was 10-1/4” x 10-1/2" and the 2020 ACS questionnaire size will be 8-1/2” x 11". The new questionnaire size is an easier size to print but increases the number of pages in the questionnaire from 28 to 48 pages. The increased size of the questionnaire means that it cannot be folded to be placed in an envelope but instead needs to be mailed flat. This change in questionnaire size requires a change to the questionnaire envelope as well. The outgoing questionnaire envelope will change from 11.5” x 6-1/16” to 9” x 11-5/8”. The business reply envelope will change from 10-5/8" x 5-3/4" to 9” x 11.5”. The new questionnaire envelopes will have a side flap instead of a top flap.

Mail material changes were also made to reflect a “new look and feel”. The new look and feel was designed to enhance the connection between the ACS and the U.S. Census Bureau and to declutter the mail pieces. These changes include:

• moving the logos on letters to the top left corner instead of the top right corner,

• dropping references to the Department of Commerce and Economics and Statistics Administration from the letterhead,

• moving the form number from the top left corner to the bottom left corner of each letter,

• removing “An Equal Opportunity Employer (EOE)” from the envelopes, and

• adding “National Processing Center” as part of the return address.

Changes were also made to the mail materials as a result of the 2018 MMT, which tested changes to mandatory messaging in the mail materials along with improvements to other aspects of the mail materials (Risley et al, forthcoming). Based on results from this test, the 2020 ACS mail materials will differ from 2019 materials in the following notable ways:

• Updated line spacing and bullet points as well as font type and style for ease of readability,

• Emphasized the mandatory nature of the ACS in various ways, including by the placement of the mandatory language text in the letter, use of bold font, or use of enlarged bold font,

• Added “Open Immediately” to the exterior of the initial mailing envelope and the paper questionnaire package envelope,

• Redesigned the questionnaire cover page

  • Included the Census Bureau logo in the top left of the header,

  • Added icons to illustrate the possible modes of self-response, and

  • Added a bold-face sentence, “Your response is required by law.”

• Removed the FAQ brochure from the first and third mailing and removed the instruction card from the third mailing, and

• Added “Final Notice Respond Now” to the exterior of the final pressure seal mailer.

See Appendix A for images of all baseline 2020 ACS mail materials.

3. ACS Specialized Mail Materials in 2020

The baseline 2020 ACS mail materials contain a number of permanent improvements to the overall ACS mail materials. However, additional temporary changes to the 2020 materials were desired in order to specifically address the overlap with the decennial census. The 2020 ACS mail materials that include 2020-specific features, in addition to the baseline improvements, are called the specialized materials.

An initial draft of the specialized mail materials was cognitively tested in the fall of 2018. Final decisions about the design of the specialized ACS mail materials were made based on the results of cognitive testing. In this section, we discuss the results of the cognitive testing and the final design decisions.

1. Cognitive Testing

The first draft of the specialized mail materials that were cognitively tested included the following elements.

• Revising the phrase “U.S. Census Bureau - The American Community Survey” on the outside of the envelopes to be in a more prominent position and font.

• Addition of variations on the sentences, “The American Community Survey is a separate part of the 2020 Census program. Some households, including yours, will receive both the American Community Survey and the 2020 Census this year. Your response to both is required by U.S. law.”

• Addition of FAQs to the back of the letters sent in both the initial mailing and the paper questionnaire package to explain the differences between the ACS and the 2020 Census.

• Addition of text to the back of the paper questionnaire envelope printed on the envelope flap. Two options were tested:

  • “The American Community Survey is an official survey conducted by the U.S. Census Bureau. You are required to respond to both the American Community Survey and the 2020 Census. Thank you for your participation.”

  • “2020 is a census year. Some addresses, including yours, will receive both the 2020 Census and the American Community Survey this year. Please respond to both. Thank you for your participation.”

Twenty cognitive interviews were conducted in October of 2018. Participants were shown all ACS and all 2020 Census mail materials in the order that they would receive them if they were part of the April 2020 ACS panel and 2020 Census Internet Push Cohort 3 (see Section 2.5 for a description of the 2020 Census mail contact strategies).⁴ A draft flyer for use in the 2020 CAPI operation was also tested during the cognitive interviews, but was shown to respondents after they reacted to all mail materials.

Recommendations after cognitive testing were to (Holzberg et al, 2019):

• Consider opportunities to further emphasize the ACS in the letters (e.g., bolding “the American Community Survey” in letter text).

• Drop or de-emphasize the word “both” from the letters, such as in the tested language, “Some households, including yours, will receive both the American Community Survey and the 2020 Census this year.” This phrasing led some respondents to believe that some of the ACS mailings were sent on behalf of both the ACS and the census.

• Consider removing the text from the envelope flap since it was not noticed by respondents.

2. Final Design of 2020 Specialized Mail Materials

The final content of the ACS mail materials in 2020 was determined based primarily on two factors: (1) the results of cognitive testing and (2) the timing of management decisions to implement new mail material design features into these materials. All three recommendations from cognitive testing were implemented in the final materials. However one notable feature that tested well, having “American Community Survey” in a more prominent place on the first and third mailing envelopes, was not implemented in the final envelopes. This decision was the result of the change to the envelope size, which was made late in the printing schedule. The new, larger envelopes had to go to print immediately upon a decision being made, before a specialized envelope with the more prominent placement of the words ‘American Community Survey’ could be finalized in the new size. As a result, the envelopes used in the first and third mailings will look the same in both the baseline materials and the specialized materials. See Appendix B for images of all specialized 2020 ACS mail materials

Table 1 shows the contents of each mailing and a brief description of the changes made that are intended to help distinguish ACS from the 2020 Census. The form numbers for each mailing can be found in Appendix C.

Table 1: Differences between Baseline and Specialized Mail Materials

Mailing	Mail Material	Differences between Specialized Materials and Baseline Materials	Specific Wording Differences
Initial Mailing	Envelope	No difference
	Letter	2020 Census text and FAQs included on back of letter	Specialized letter has added text: “The American Community Survey is not the 2020 Census. This survey asks questions about topics not on the 2020 Census, such as veteran status, transportation, and internet access.”
	Instruction Card	No difference
	Multilingual Brochure	No difference
Pressure Seal Mailer #1	Exterior	Mentions the ACS on the exterior	Specialized version has the added text: U.S. Census Bureau American Community Survey
	Interior	2020 Census text included	Specialized version has the added text: “Some households, including yours, will receive both the American Community Survey and the 2020 Census this year.”
Paper Questionnaire Package	Envelope	No difference
	Letter	2020 Census text and FAQs included on back of letter	Specialized version has the added text: “This year, the Census Bureau is also conducting the 2020 Census. The American Community Survey is different from the 2020 Census.” The letter also changed “this survey” to “the American Community Survey”.
	Questionnaire	No difference
	Return Envelope	No difference
Postcard	Postcard	No difference
Pressure Seal Mailer #2	Exterior	No difference
	Interior	No difference

The first and third mailing letters will contain FAQs on the back of the letter. The baseline materials will have FAQs that discuss confidentiality, privacy, and the mandatory nature of the survey. The specialized materials will have those FAQs plus two additional FAQs, shown below.

Figure 2. FAQs Added to ACS Specialized Letters in 2020

4. 2020 Census Contact Strategy

The Census Bureau estimates contacting over 150 million housing units in the 2020 Census. While the vast majority of those housing units will be contacted by mail in 2020 and asked to self-respond to the census, others will only be contacted in person. In this section, we briefly discuss the mail contact strategy for the 2020 Census, as well as other ways that a housing unit may be contacted as part of the 2020 Census.

1. Mailout Strategy

The Census Bureau estimates that over 140 million housing units will receive mail invitations to respond to the 2020 Census. This census will be the first time that the delivery of decennial census mailings is staggered across the country, instead of being delivered on (or close to) the same day everywhere. This staggered mail contact strategy reduces the burden on the United States Postal Service (USPS), call centers, and the internet application by distributing the workload. This approach also seeks to maximize the response rate.

In addition to the staggered timing of mailings, there are also two different mail contact strategies being employed in the 2020 Census: Internet First and Internet Choice. In the Internet First contact strategy, the content of the first two mailings encourages online response; a paper questionnaire is only sent in the fourth mailing to those who have not yet responded. In the Internet Choice contact strategy, the very first mailing contains a paper questionnaire while also encouraging online response; a paper questionnaire is sent again in the fourth mailing to those who have not yet responded. Census tracts will receive the Internet Choice contact strategy if the tract meets certain criteria that indicate low internet connectivity or with characteristics that make it less likely the recipients will complete a census questionnaire online.⁵

The Internet First contact strategy will be used for the majority of the country. Housing units targeted for the Internet First contact strategy have been distributed into four cohorts, which have different target in-home delivery dates. Table 2 shows the target in-home delivery date of each mailing by mail contact strategy (Internet First or Internet Choice), cohort (if Internet First), and the format of each mailing, for the housing units in the mailout universe. The 2020 Census contact strategy (including time between mailings and the type of mailing) shown in Table 2 differs from the ACS contact strategy shown in Figure 1.

Table 2: 2020 Census Mailout Strategy and In-Home Delivery Dates

Mailing Strategy	Cohort	Allocation of Mailout Universe	Mailing 1 Letter (Internet First) or Letter + Questionnaire (Internet Choice)	Mailing 2 Letter	Mailing 3* Postcard	Mailing 4* Letter + Questionnaire	Mailing 5* “It’s not too late” Postcard
Internet Choice	NA	22%	March 13	March 17	March 27	April 9	April 20
Internet First	1	23%	March 12	March 16	March 26	April 8	April 20
Internet First	2	18%	March 13	March 17	March 27	April 9	April 20
Internet First	3	20%	March 19	March 23	April 2	April 15	April 27
Internet First	4	17%	March 20	March 24	April 3	April 16	April 27

* Sent only to nonresponding housing units

Appendix D shows the target dates for when ACS mailings and 2020 Census mailings will arrive in homes, based on which month an ACS housing unit is sampled for and to which 2020 Census mail contact strategy the housing unit is allocated.

2. Additional 2020 Census Contacts

The majority of housing units across the country will be classified into one of the mail contact strategies above. If no self-response is received in a timely fashion from these housing units, they are eligible for a personal visit as part of the Nonresponse Followup (NRFU) operation. A small universe of addresses could be visited in person as early as April 9;⁶ most nonresponding addresses, however, will receive a first visit in mid-to-late May with subsequent visits possible until the end of July. After completing a NRFU interview, an address could be visited again by a quality control worker between April and July.

Addresses in the NRFU universe could also receive another postcard in the mail (not shown in Table 2), telling the address that this is their last chance to self-respond. Housing units that are suspected (based on administrative records) to be vacant or not a viable housing unit will receive a postcard around May 18, before an enumerator visits. Housing units that are suspected to be occupied (based on administrative records) will be visited once by an enumerator; if no contact is made, then a postcard will arrive one to two weeks after the first contact attempt (sometime between May 18 and July 27, though ideally all first contacts will be made by late June).

There are some housing units in the country that will not be contacted for the 2020 Census through one of the mail contact strategies described in Section 2.4.1. For these housing units (in either the Update Leave (UL)⁷ or Update Enumerate (UE)⁸ operations)⁹, their first contact by the Census Bureau will be a personal visit from an enumerator. The UL and UE 2020 Census operations will have enumerators in the field from mid-March to mid/late April. The UL addresses will get a personal visit in March or April, in which a census questionnaire is hand-delivered, and then two reminder mailings in April if no census response was received. The UE addresses receive personal visits until a successful enumeration in person has been obtained. Addresses in UL and UE areas are often nonmailable and so do not receive ACS mailings, if in the ACS sample. They would be contacted in person during the ACS CAPI phase, so ACS sampled housing units in these areas will have an atypical experience of contacts for the decennial census and ACS.

Housing units might also be contacted by the Census Bureau later in the spring or summer of 2020 by phone or in person in order to clarify information on a census response or as part of the 2020 Census coverage measurement program.¹⁰

3. Literature Review

This section presents a summary of relevant literature that informs this research, including:

• Response rates and check-in rates from the ACS in 2010,

• The experiment conducted in 2010 on distinguishing ACS from the 2010 Census, and

• An ACS messaging survey conducted by Reingold, Inc. in 2014.

1. 2010 ACS Response Rates and Check-in Rates¹¹

Data are available on two ACS self-response metrics from 2010 (when self-response options were only by mail or TQA): self-response rates and self-response check-in rates. Check-in rates track when ACS responses are received by the Census Bureau (without regard for content), whereas response rates omit any responses from the numerator that, upon data capture, are shown to be blank questionnaires. Check-in rates also look at the universe of all mailable addresses while response rates look at mailable addresses that were not identified as a business, UAA, or vacant.

Self-response rates are available for every ACS monthly panel since the ACS was fully implemented in 2005. From January 2005 to December 2017, the month with the highest self-response rate was March 2010, which was at the height of 2010 Census advertising and when most people received their census form in the mail (Baumgardner, 2018). However, May 2010 had one of the lowest self-response rates of any month in that thirteen-year timespan.

Self-response check-in rates show the same pattern as self-response rates. Figure 3 shows the self-response check-in rate for the ACS December 2009 to July 2010 monthly panels compared to the equivalent monthly panels from the previous year. Figure 3 was originally published in the report Tracking American Community Survey Mail Response During the 2010 Census (Baumgardner, 2013).

The increase in self-response early in the year was attributed to the 2010 Census communications campaign while the decrease later in the year was attributed to respondent confusion or respondent burden, as respondents had already filled out their census form.

Figure 3. Differences in the National Mail Check-in Rates – December 2009 through July 2010 Panels Compared to Previous Year’s Panel

Source: Tracking American Community Survey Mail Response During the 2010 Census (Baumgardner, 2013)

In addition to tracking the 2010 ACS mail response, Baumgardner (2013) also studied the ACS mail form completeness rates as a measure of respondent cooperation. Form completeness of the ACS mail forms for the December 2009 through July 2010 panels was compared to the ACS mail forms for the equivalent panel from the previous year, at the national level. For all monthly panels, the December 2009 and 2010 completeness rate was significantly lower than the corresponding month in the previous year. The April 2010 panel saw the greatest negative effect on mail form completeness of any 2010 monthly panel (Baumgardner, 2013).

Figure 4. Differences in Mail Form Completeness – December 2009 through July 2010 Panels Compared to Previous Year’s Panel

Source: Tracking American Community Survey Mail Response During the 2010 Census (Baumgardner, 2013).

2. 2010 Evaluation of ACS Revised Mail Materials

An evaluation of ACS mailing materials and mailing strategy during the 2010 Census tested the use of language, color, and branding to create a distinct identity for the ACS during the census year and attempted to minimize the negative impact of 2010 Census activities on ACS response. The test had three experimental treatments, which employed two new envelopes and one new letter as part of the experimental mail materials. The 2010 ACS sample was delineated into three time periods based on the timing of the first mailing: pre-census (January and February), census (March through May), and post-census (June through November).

The experimental materials consisted of two different envelope treatments and one set of new letters. Images of the materials can be found in the Evaluation of the ACS Mail Materials and Mailing Strategy during the 2010 Census (Chesnut and Davis, 2011):

• Green Envelopes – These envelopes were used for the pre-notice letter, initial questionnaire mailing, and replacement questionnaire mailing¹². They contained a new box with a green background with “U.S. Census Bureau The American Community Survey” above the address window. There was an existing box on the questionnaire package that said, “Your Response is Required by Law,” which was also filled with a green background. The font size was also increased. Green was selected because it matched the primary color of the ACS questionnaire.

• White Envelopes – These envelopes had the same boxes and text as the “Green Envelopes”, but without the green background.

• New Set of Letters – The new set of letters included a revised pre-notice letter as well as revised letters in both the initial and replacement questionnaire packages. The new letters contained information informing the housing unit about the requirement to respond to both the ACS and the 2010 Census.

The results of the test found that:

• All of the test treatment letter and envelope combinations improved participation compared to the current ACS letters and envelopes across the three census time periods.

• Of the three combinations, the new letters paired with the new white envelopes resulted in the greatest increase in respondent participation across all three time periods.

• The new white envelopes with messaging linking the ACS to the Census Bureau improved respondent participation compared to that of the current envelopes prior to and during the census and maintained the same level of participation post-census.

• Adding green color to the new envelopes reduced participation across the census time periods compared to that of the new white envelopes.

• The new letter with text addressing possible confusion between the 2010 Census and ACS improved participation compared to the current letter across the census time periods, when they were sent with the new white envelopes.

These results and materials were utilized when planning for 2020. For instance, since adding color to the envelopes did not lead to increased response rates, that change was not considered for use in 2020 ACS materials.

3. ACS Messaging Research by Reingold

In early 2014, a quantitative study called the Benchmark Messaging Survey was conducted by Reingold, Inc. to gather attitudinal data about the ACS and identify the best message themes surrounding ACS participation (e.g., civic duty, importance for governance, community benefit). Over 1,000 telephone interviews were conducted with a representative sample of the U.S. population. The Benchmark Messaging Survey found that only 11 percent of respondents had previously heard of the ACS (Hagedorn, Green, and Rosenblatt, 2014). Conversely, they found in the same study that 90 percent of respondents had heard of the census of the United States (the decennial census). This research confirms the need to clarify for ACS respondents that it is distinct from the decennial census.

4. Research Questions and Methodology

This section discusses the experimental design of the 2020 Specialized Mail Materials Test (SMMT), the research questions and metrics, and the sample design. The goal of this test is to assess what, if any, impact the specialized content has on response metrics in 2020. The 2020 specialized ACS mail materials are now very different from the 2019 ACS mail materials, due to the numerous changes implemented in the baseline ACS mail materials (described in Section 2.2). In order to isolate the effect of the specialized content on 2020 response, an experimental treatment is needed in 2020.

1. Experimental Design

In the 2020 SMMT, the Census Bureau will compare two treatments using a randomized controlled experiment. The test will be administered in seven ACS monthly panels, from March to September. The monthly ACS production sample consists of approximately 290,000 housing unit addresses and is divided into 24 nationally representative groups (referred to as methods panel groups) of approximately 12,000 addresses each. From March to September, approximately 2,020,000 housing unit addresses will be sent a request to respond to the ACS.

Each month for this test, two methods panel groups (approximately 24,000 addresses) will receive the specialized mail materials and another two methods panel groups will receive the baseline mail materials. The remaining twenty methods panel groups in each of the March to September panels will receive the specialized mail materials but will not be included in analysis for this test.¹³ The 2020 ACS baseline materials are planned for use in ACS mailings the remainder of the year (January, February, October, November and December 2020 panels).

2. Research Questions

The objective of this research is to learn more about the impact of the decennial census on response to the ACS. The research questions for this test are:

1. What is the impact on self-response return rates of including specialized 2020 language in the ACS mail materials?

2. What is the impact on final response rates of including specialized 2020 language in the ACS mail materials?

3. What is the impact on the rate of blank or insufficient forms of including specialized 2020 language in the ACS mail materials?

4. What is the impact on form completeness of including specialized 2020 language in the ACS mail materials?

Prior to answering these research questions, we will investigate the underlying data to ensure there are no differences between treatments in metrics that could impact the research question results. One comparison will examine the rate that addresses are flagged by the USPS as being Undeliverable as Addressed (UAA), as return rates and response rates can be influenced by UAA rates. A second comparison will check that the distribution of ACS cases by 2020 Census mail contact strategy (Internet First or Internet Choice, plus cohort number) is similar between treatments. A difference in the distribution of the 2020 Census mail contact strategy could contribute to a difference seen in the ACS response rates.

Additional analysis will look at major demographic distributions, as sample size allows, of Person 1 (who is typically the respondent) from sufficiently complete responses. We assume that respondents in both treatments will have similar demographic characteristics so similar demographic distributions are expected. Differences in the demographic distributions could be attributed to the experimental difference in the treatments. Only significant findings from these analyses will be reported.

1. Question 1 – Self-response Return Rates

What is the impact on self-response return rates of including specialized 2020 language in the mail materials to distinguish the ACS from the 2020 Census?

To assess the impact on self-response of adding the specialized 2020 language to the mail materials, we will calculate and compare self-response return rates of the Specialized and Baseline treatments. We will compare self-response return rates just before the Paper Questionnaire Package mailing, before the fifth mailing, and just before the start of CAPI. A change in self-response return rates at these points in time has workload implications for subsequent mailings and for the CAPI operation. All results will be reported by ACS panel month.

We will make comparisons using a two-tailed t-test. The null hypothesis will be H₀: Baseline = Specialized and the alternative hypothesis H_A: Baseline ≠ Specialized.

The rates will be calculated for total self-response and separately by mode. The self-response return rates will be calculated using the following formula:

Self-Response Return Rate	=	Number of mailable and deliverable sample addresses that either provided a non-blank14 return by mail or TQA, or a complete or sufficient partial15 response by internet	* 100
		Total number of mailable and deliverable sample addresses16

Given the possibility of higher than usual respondent confusion, there could be an increase in the number of calls to the call center and an associated increase in the number of TQA interviews. We will examine how many TQA cases there are each month; if there is a significant difference in TQA cases between treatments, they will be reported separately from other modes. If not, then they will be combined with mail responses due to the typically low volume of TQA cases.

Results will also be graphed over time.

2. Question 2 – Final Response Rates

What is the impact on final response rates of including specialized 2020 language in the ACS mail materials?

We will compare the final response rates for each ACS panel month in the test, and the distribution of response by mode. We will make comparisons using a two-tailed t-test. The null hypothesis will be H₀: Baseline = Specialized and the alternative hypothesis will be H_A: Baseline ≠ Specialized.

The final response rates will be calculated using the following formula:

Final Response Rate	=	Number of eligible sample addresses that either provided a non-blank return by mail, internet or TQA, or a complete CAPI interview	* 100
		Total number of sample addresses eligible to reply to the survey and not sampled out of CAPI

The denominator does not include UAAs (unless the address did respond or is in the CAPI sample) and does not include addresses that are found to be a business, demolished, under construction, etc.

If more than one response is received from an address, then the response received first will be considered the primary response. In the rare case that two responses are received on the same day, then the primary response will be chosen based on mode in the following order: (1) mail, (2) TQA, (3) internet, and (4) CAPI.

3. Question 3 – Blank and Insufficient Forms

What is the impact on the rate of blank or insufficient forms of including specialized 2020 language in the ACS mail materials?

2. Among addresses that were sent the mailings, we could receive either (1) no response at all, (2) a blank or almost blank form, or (3) a response that is sufficiently complete if not entirely complete. If we receive a complete or sufficient return, it demonstrates that the respondent opened the ACS mailings and was motivated to respond at least sufficiently for data collection purposes. A blank or insufficient return shows some engagement with the ACS; the respondent opened the mailings even if they did not provide a sufficient survey response. Insufficient returns will be contacted again if still within the contact window.

3. Insufficient returns provide a way to measure how successful the ACS mailings were at accomplishing a critical goal of mail contacts: being opened and reviewed by the respondent.

4. A blank form is a mail questionnaire or internet response that has minimal information on it; it might not be completely blank. A form is considered blank when it has no persons with sufficient response data and there is no telephone number. Internet returns are classified as insufficient if the respondent either (1) has not reached the Pick Next Person screen for a household with two or more individuals on the roster, or (2) has not gone through the place of birth question for a one-person household. Also included in this numerator are responses that indicate the sampled address represents a business.

Analysis will be done by mode and overall for each ACS monthly panel in the test. We will make comparisons using a two-tailed t-test. The null hypothesis will be H₀: Baseline = Specialized and the alternative hypothesis will be H_A: Baseline ≠ Specialized.

Rate of Blank and Insufficient Forms	=	Number of addresses that either provided a blank return, an insufficient partial response, or were a business	* 100
		Total number of mailable and deliverable sample addresses

4. Question 4 – Form Completeness

What is the impact on form completeness of including specialized 2020 language in the ACS mail materials?

Form completeness provides a way to assess how much of the survey a respondent completed, by quantifying the answers provided across a housing unit’s entire response. Form completion measures the number of questions on the form that were answered among those that should have been answered.¹⁷ Calculations are made using the following formula:

Overall Form Completion Rate	=	Number of questions answered	*100
		Number of questions that should have been answered

The calculations will only be made with complete or sufficient self-responses, as interviewer-assisted responses (CAPI and TQA) are influenced by the interviewer as well as the respondent. If more than one response is received from an address, the form with the most data provided is chosen for analysis.

The number of questions that should have been answered is influenced by a number of factors, most notably by the number of people in a household. We will confirm that the distribution of household size is comparable between treatments in each monthly panel and will report household size if significantly different.

The ACS has three question sections: a basic person-level demographic section, a housing section, and then detailed person-level questions. We will calculate completeness rates for each section of the questionnaire. The calculation for each section completeness rate is below.

Analysis will be done by mode and overall for each ACS monthly panel in the test. We will make comparisons using a two-tailed t-test. The null hypothesis will be H₀: Baseline = Specialized and the alternative hypothesis will be H_A: Baseline ≠ Specialized.

3. Sample Design and Weighting

The sample size will be able to detect differences of approximately 1.25 percentage points between the self-response return rates of the treatments (with 80 percent power and α=0.1). We will use a significance level of α=0.1 when determining significant differences between treatments. For analysis that involves multiple comparisons, we will adjust for the Type I familywise error rate using the Hochberg method (Hochberg, 1988).

All self-response analyses will be weighted using the ACS base sampling weight (the inverse of the probability of selection). The CAPI response analysis will include a CAPI subsampling factor that will be multiplied by the base weight.

We will estimate the variances of the point estimates and differences using the Successive Differences Replication (SDR) method with replicate weights—the standard method used in the ACS (see U.S. Census Bureau, 2014, Chapter 12). In calculating the estimates, we will use replicate base weights, which only account for sampling probabilities. We will calculate the variance for each estimate and for the difference between estimates using the formula below:

Where:

the estimate calculated using the r^th replicate

the estimate calculated using the full sample

The standard error of the estimate (X₀) is the square root of the variance.

5. Assumptions and Limitations

   1. Assumptions

1. A single ACS monthly sample is representative of an entire year (twelve panels) and the entire frame sample, with respect to both response rates and cost, as designed.

2. A single methods panel group (1/24 of the full monthly sample) is representative of the full monthly sample, as designed.

3. We assume that there is no difference in mail delivery timing or subsequent response time across samples of similar size using the same postal sort and mailout procedures.

1. Limitations

1. Group quarters are not included in the sample for the test.

2. Housing unit addresses from remote Alaska and from Puerto Rico are not included in the sample for the test.

3. Only English-language mail materials are included in this test, which limits the conclusions that can be drawn.

4. We will not assess how or when a response to the 2020 Census is received from an ACS sampled address.

5. We will not know exactly when each mail piece is delivered to an address or when a recipient notices it, opens it, or processes it.

6. Addresses and individuals across the country will receive a varied amount of 2020 Census advertising and communications, based on where they live, what news they consume, and how they engage with their community. We cannot control for the type and quantity of news that an individual consumes about either the 2020 Census, the ACS, or the Census Bureau. We assume that randomization in the sample selection will control for this but we will be unable to confirm this assumption.

6. Sample Table Shells

Below are samples of tables that will be used in the final report to show results from this test.

Table 3. Self-Response Return Rates Before the Third Mailing

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 4. Self-Response Return Rates Before the Fifth Mailing

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 5. Self-Response Return Rates Before CAPI

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 6. Final Overall Response Rates and Distribution by Mode

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
Mail
Internet
CAPI
April Panel	####	####	####	####
Mail
Internet
CAPI
May Panel	####	####	####	####
Mail
Internet
CAPI
June Panel	####	####	####	####
Mail
Internet
CAPI
July Panel	####	####	####	####
Mail
Internet
CAPI
August Panel	####	####	####	####
Mail
Internet
CAPI
September Panel	####	####	####	####
Mail
Internet
CAPI

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 7. Overall Rate of Blank and Insufficient Forms

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 8. Overall Form Completion

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 9. Basic Person Demographics Section Form Completion

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 10. Housing Section Form Completion

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

Table 11. Detailed Person Section Form Completion

ACS Monthly Panel	Specialized	Baseline	Difference	P-Value
March Panel	####	####	####	####
April Panel	####	####	####	####
May Panel	####	####	####	####
June Panel	####	####	####	####
July Panel	####	####	####	####
August Panel	####	####	####	####
September Panel	####	####	####	####

Source: U.S. Census Bureau, American Community Survey, 2020 Specialized Mail Materials Test. DRB Approval Number: ####.

Note: Minor additive discrepancies are due to rounding. Standard errors are in parentheses. An asterisk (*) indicates a statistically significant result. Significance was tested based on a two-tailed t-test at the α=0.1 level.

7. Potential Changes to ACS

Based on the results of this research, the Census Bureau may consider revising the strategy of modifying ACS production mail materials during the 2030 Census.

8. References

Baumgardner, S. (2013). “Tracking American Community Survey Mail Response During the 2010 Census,” 2013 American Community Survey Research and Evaluation Report Memorandum Series #ACS13-RER-16. Retrieved on September 6, 2019 from https://www.census.gov/library/working-papers/2013/acs/2013_Baumgardner_01.html

Baumgardner, S. (2018). “When is the Best Time to Field Your Survey? Trends in American Community Survey Response Rates,” AAPOR 2018 Conference Poster.

Chesnut, J., Davis, M. (2011). “Evaluation of the ACS Mail Materials and Mailing Strategy during the 2010 Census,” 2011 American Community Survey Research and Evaluation Report Memorandum Series #ACS11-RER-09. Retrieved on September 6, 2019 from https://www.census.gov/library/working-papers/2011/acs/2011_Chesnut_01.html

Hagedorn, S., Green R. and Rosenblatt, A. (2014). “ACS Messaging Research: Benchmark Survey. Washington, D.C.: U.S. Census Bureau.” Retrieved on September 6, 2019 from https://www.census.gov/library/working-papers/2014/acs/2014_Hagedorn_01.html

Heimel, S. (2016). “Postal Tracking Research on the May 2015 ACS Panel.” 2016 American Community Survey Research and Evaluation Report Memorandum Series #ACS16-RER-01, April 1, 2016.

Hochberg, Y. (1988). “A Sharper Bonferroni Procedure for Multiple Tests of Significance,” Biometrika, 75 (4), 800-802. Retrieved on September 6, 2019 from http://www.jstor.org/stable/2336325?seq=1#page_scan_tab_contents

Holzberg, J., Luck, J., Katz, J., Martinez, M., Davis, M. (2019). “Cognitive Testing of the 2020 American Community Survey (ACS) Mailing Materials,” FORTHCOMING.

Marasteanu, I. (2019). The Research and Methodology on Staggering the 2020 Census Mailings. In JSM Proceedings, Statistical Computing Section. Alexandria, VA: American Statistical Association. FORTHCOMING.

Risley, M., Berkley, J. (2020). “2018 ACS Mail Materials Test Report” FORTHCOMING.

U.S. Census Bureau (2014). “American Community Survey Design and Methodology, Chapter 12: Variance Estimation.” Retrieved on September 6, 2019 from https://www.census.gov/programs-surveys/acs/methodology/design-and-methodology.html

 

1. Images of the 2020 ACS Baseline Mail Materials

1. Mailing 1: Initial Mail Package

Figure 5. Front of Envelope ACS-46IM(2020)(02-07-2019)

Figure 6. Back of Envelope ACS-46IM(2020)(02-07-2019)

Figure 7. Front of Letter ACS-13(L)(2020)(08-02-2019)

Figure 8. Back of Letter ACS-13(L)(2020)(08-02-2019)

Figure 9. Front of Multilingual Brochure ACS-9(2020)(05-31-2019)

Figure 10. Back of Multilingual Brochure ACS-9(2020)(05-31-2019)

Figure 11. Front of Instruction Card ACS-34IM(2020)(02-13-2019)

Figure 12. Back of Instruction Card ACS-34IM(2020)(02-13-2019)

2. Mailing 2: Initial Reminder

Figure 13. Inside of Letter ACS-20(L)(2020)(08-02-2019)

Figure 14. Outside of Letter ACS-20(L)(2020)(08-02-2019)

3. Mailing 3: Paper Questionnaire Package

Figure 15. Front of Envelope ACS-46(2020)(02-07-2019)

Figure 16. Back of Envelope ACS-46(2020)(02-07-2019)

Figure 17. Front of Letter ACS-14(L)(2020)(08-02-2019)

Figure 18. Back of Letter ACS-14(L)(2020)(08-02-2019)

Figure 19. Front Cover of ACS Questionnaire ACS-1(2020)(07-13-2019)

Figure 20. Front of Business Reply Envelope 6385-47(2020)(12-12-2018)

4. Mailing 4: Reminder Postcard

Figure 21. Front of Postcard ACS-29(2020)(05-21-2019)

Figure 22. Back of Postcard ACS-29(2020)(05-21-2019)

5. Mailing 5: Final Reminder Letter

Figure 23. Inside of Letter ACS-23(L)(2020)(08-05-2019)

Figure 24. Outside of Letter ACS-23(L)(2020)(08-05-2019)

2. Images of the 2020 ACS Specialized Mail Materials

1. Mailing 1: Initial Mail Package

Figure 25. Front of Envelope ACS-46IMZ(2020)(08-09-2019)

Figure 26. Back of Envelope ACS-46IMZ(2020)(08-09-2019)

Figure 27. Front of Letter ACS-13(LZ)(2020)(08-27-2019)

Figure 28. Back of Letter ACS-13(LZ)(2020)(08-27-2019)

Figure 29. Front of Multilingual Brochure ACS-9(2020)(05-31-2019)

Figure 30. Back of Multilingual Brochure ACS-9(2020)(05-31-2019)

2. Mailing 2: Initial Reminder

Figure 31. Inside of Letter ACS-20(LZ)(2020)(08-23-2019)

Figure 32. Outside of Letter ACS-20(LZ)(2020)(08-23-2019)

3. Mailing 3: Paper Questionnaire Package

Figure 33. Front of Envelope ACS-46(Z)(2020)(08-09-2019)

Figure 34. Back of Envelope ACS-46(Z)(2020)(08-09-2019)

Figure 35. Front of Letter ACS-14(LZ)(2020)(08-27-2019)

Figure 36. Back of Letter ACS-14(LZ)(2020)(08-27-2019)

Figure 37. Front Cover of ACS Questionnaire ACS-1(2020)(07-13-2019)

Figure 38. Front of Business Reply Envelope 6385-47(2020)(12-12-2018)

4. Mailing 4: Reminder Postcard

This is the exact same postcard as will be used in the baseline materials.

Figure 39. Front of Postcard ACS-29(2020)(05-21-2019)

Figure 40. Back of Postcard ACS-29(2020)(05-21-2019)

5. Mailing 5: Final Reminder Letter

Figure 41. Inside of Letter ACS-23(L)(2020)(08-05-2019)

Figure 42. Outside of Letter ACS-23(L)(2020)(08-05-2019)

3. Form Numbers on 2020 ACS Mail Materials

Table 12 documents the form numbers for all mail materials that will be used in the 2020 SMMT.

Table 12: Form Numbers for Baseline and Specialized Mail Materials

Mailing	Mail Material	Baseline Form Number	Specialized Form Number
Initial Mailing	Envelope	ACS-46IMZ (2020)	ACS-46IM (2020)
	Letter	ACS-13(LZ) (2020)	ACS-13(L) (2020)
	Instruction Card	ACS-34 IM	ACS-34 IM
	Multilingual Brochure	ACS-9 (2020)	ACS-9 (2020)
Second Mailing	Pressure Seal Mailer	ACS-20(LZ) (2020)	ACS-20(L) (2020)
Third Mailing	Envelope	ACS-46(Z) (2020)	ACS-46 (2020)
	Letter	ACS-14(LZ) (2020)	ACS-14(L) (2020)
	Questionnaire	ACS-1 (2020)	ACS-1 (2020)
	Return Envelope	6385-47 (2020)	6385-47 (2020)
Fourth Mailing	Reminder Postcard	ACS-29 (2020)	ACS-29 (2020)
Fifth Mailing	Pressure Seal Mailer	ACS-23(LZ) (2020)	ACS-23(L) (2020)

4. Mail Delivery Dates for ACS and Decennial, January-July 2020

Table 13 shows the target dates when ACS mailings and decennial mailings will arrive in homes, based on which month an ACS address is sampled for and which decennial mail contact strategy the address is allocated to. An address will only receive all ten contacts if they do not respond to either the census or the ACS during the mailout period.

In order to reduce the size of the table, decennial Internet Push cohorts 1 and 2 are grouped together, and cohorts 3 and 4 are grouped together. However, Cohort 2 mailings will arrive 1 day after those of Cohort 1. Cohort 4 mailings will arrive one day after those of Cohort 3.

ACS mailings are planned for a given mailout date, not a target delivery date; delivery dates were approximated by adding three mail delivery days after the scheduled mailout date.

The table does not reflect all possible contacts to an address from the Census Bureau in 2020, such as the possible receipt of a NRFU postcard mailing, possible NRFU enumerator or ACS Field Representative personal contacts, or other Census Bureau contacts. With these limitations in mind, the table does provide an overview of the intensity of mail contacts that sampled ACS addresses could experience.

Table 13. Target In-Home Delivery Dates for ACS Mailings and Decennial Self-Response Mailings

ACS Panel	Census Cohort	Timespan	Contact 1	Contact 2	Contact 3	Contact 4	Contact 5	Contact 6	Contact 7	Contact 8	Contact 9	Contact 10
January	Internet Push Cohort 1&2	112 days	December 30, 2019 ACS Initial Mailing	January 6 ACS Pressure Seal Reminder	January 21 ACS Qnaire package	January 24 ACS postcard	February 14 ACS Final Pressure Seal	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard
January	Internet Push Cohort 3&4	119 days	December 30, 2019 ACS Initial Mailing	January 6 ACS Pressure Seal Reminder	January 21 ACS Qnaire package	January 24 ACS postcard	February 14 ACS Final Pressure Seal	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard
January	Internet Choice	112 days	December 30, 2019 ACS Initial Mailing	January 6 ACS Pressure Seal Reminder	January 21 ACS Qnaire package	January 24 ACS postcard	February 14 ACS Final Pressure Seal	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard
February	Internet Push Cohort 1&2	77 days	February 3 ACS Initial Mailing	February 10 ACS Pressure Seal Reminder	February 25 ACS Qnaire package	February 28 ACS postcard	March 12 Decennial letter	March 16 Decennial letter	March 20 ACS Final Pressure Seal	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard
February	Internet Push Cohort 3&4	84 days	February 3 ACS Initial Mailing	February 10 ACS Pressure Seal Reminder	February 25 ACS Qnaire package	February 28 ACS postcard	March 19 Decennial letter	March 20 ACS Final Pressure Seal	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard
February	Internet Choice	77 days	February 3 ACS Initial Mailing	February 10 ACS Pressure Seal Reminder	February 25 ACS Qnaire package	February 28 ACS postcard	March 12 Decennial qnaire	March 16 Decennial letter	March 20 ACS Final Pressure Seal	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard
March	Internet Push Cohort 1&2	49 days	March 2 ACS Initial Mailing	March 9 ACS Pressure Seal Reminder	March 12 Decennial letter	March 16 Decennial letter	March 23 ACS Qnaire package	March 26 ACS postcard	March 26 Decennial postcard	April 8 Decennial qnaire	April 17 ACS Final Pressure Seal	April 20 Decennial postcard
March	Internet Push Cohort 3&4	56 days	March 2 ACS Initial Mailing	March 9 ACS Pressure Seal Reminder	March 19 Decennial letter	March 23 ACS Qnaire package	March 23 Decennial letter	March 26 ACS postcard	April 2 Decennial postcard	April 15 Decennial qnaire	April 17 ACS Final Pressure Seal	April 27 Decennial postcard
March	Internet Choice	49 days	March 2 ACS Initial Mailing	March 9 ACS Pressure Seal Reminder	March 12 Decennial qnaire	March 16 Decennial letter	March 23 ACS Qnaire package	March 26 ACS postcard	March 26 Decennial postcard	April 8 Decennial qnaire	April 17 ACS Final Pressure Seal	April 20 Decennial postcard

ACS Panel	Census Cohort	Timespan	Contact 1	Contact 2	Contact 3	Contact 4	Contact 5	Contact 6	Contact 7	Contact 8	Contact 9	Contact 10
April	Internet Push Cohort 1&2	64 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	March 30 ACS Initial Mailing	April 6 ACS Pressure Seal Reminder	April 8 Decennial qnaire	April 20 ACS Qnaire package	April 20 Decennial postcard	April 23 ACS postcard	May 15 ACS Final Pressure Seal
April	Internet Push Cohort 3&4	57 days	March 19 Decennial letter	March 23 Decennial letter	March 30 ACS Initial Mailing	April 2 Decennial postcard	April 6 ACS Pressure Seal Reminder	April 15 Decennial qnaire	April 20 ACS Qnaire package	April 23 ACS postcard	April 27 Decennial postcard	May 15 ACS Final Pressure Seal
April	Internet Choice	64 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	March 30 ACS Initial Mailing	April 6 ACS Pressure Seal Reminder	April 8 Decennial qnaire	April 20 ACS Qnaire package	April 20 Decennial postcard	April 23 ACS postcard	May 15 ACS Final Pressure Seal
May	Internet Push Cohort 1&2	99 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	May 4 ACS Initial Mailing	May 11 ACS Pressure Seal Reminder	May 26 ACS Qnaire package	May 29 ACS postcard	June 19 ACS Final Pressure Seal
May	Internet Push Cohort 3&4	92 days	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard	May 4 ACS Initial Mailing	May 11 ACS Pressure Seal Reminder	May 26 ACS Qnaire package	May 29 ACS postcard	June 19 ACS Final Pressure Seal
May	Internet Choice	99 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	May 4 ACS Initial Mailing	May 11 ACS Pressure Seal Reminder	May 26 ACS Qnaire package	May 29 ACS postcard	June 19 ACS Final Pressure Seal
June	Internet Push Cohort 1&2	127 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	June 1 ACS Initial Mailing	June 8 ACS Pressure Seal Reminder	June 22 ACS Qnaire package	June 25 ACS postcard	July 17 ACS Final Pressure Seal
June	Internet Push Cohort 3&4	120 days	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard	June 1 ACS Initial Mailing	June 8 ACS Pressure Seal Reminder	June 22 ACS Qnaire package	June 25 ACS postcard	July 17 ACS Final Pressure Seal
June	Internet Choice	127 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	June 1 ACS Initial Mailing	June 8 ACS Pressure Seal Reminder	June 22 ACS Qnaire package	June 25 ACS postcard	July 17 ACS Final Pressure Seal
July	Internet Push Cohort 1&2	155 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	June 29 ACS Initial Mailing	July 7 ACS Pressure Seal Reminder	July 20 ACS Qnaire package	July 23 ACS postcard	August 14 ACS Final Pressure Seal
July	Internet Push Cohort 3&4	148 days	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard	June 29 ACS Initial Mailing	July 7 ACS Pressure Seal Reminder	July 20 ACS Qnaire package	July 23 ACS postcard	August 14 ACS Final Pressure Seal
July	Internet Choice	158 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	June 29 ACS Initial Mailing	July 7 ACS Pressure Seal Reminder	July 20 ACS Qnaire package	July 23 ACS postcard	August 14 ACS Final Pressure Seal
August	Internet Push Cohort 1&2	191 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	August 3 ACS Initial Mailing	August 10 ACS Pressure Seal Reminder	August 24 ACS Qnaire package	August 27 ACS postcard	Sept 19 ACS Final Pressure Seal
August	Internet Push Cohort 3&4	184 days	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard	August 3 ACS Initial Mailing	August 10 ACS Pressure Seal Reminder	August 24 ACS Qnaire package	August 27 ACS postcard	Sept 19 ACS Final Pressure Seal
August	Internet Choice	191 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	August 3 ACS Initial Mailing	August 10 ACS Pressure Seal Reminder	August 24 ACS Qnaire package	August 27 ACS postcard	Sept 19 ACS Final Pressure Seal
September	Internet Push Cohort 1&2	219 days	March 12 Decennial letter	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	August 31 ACS Initial Mailing	September 8 ACS Pressure Seal Reminder	September 21 ACS Qnaire package	September 24 ACS postcard	October 17 ACS Final Pressure Seal
September	Internet Push Cohort 3&4	212 days	March 19 Decennial letter	March 23 Decennial letter	April 2 Decennial postcard	April 15 Decennial qnaire	April 27 Decennial postcard	August 31 ACS Initial Mailing	September 8 ACS Pressure Seal Reminder	September 21 ACS Qnaire package	September 24 ACS postcard	October 17 ACS Final Pressure Seal
September	Internet Choice	219 days	March 12 Decennial qnaire	March 16 Decennial letter	March 26 Decennial postcard	April 8 Decennial qnaire	April 20 Decennial postcard	August 31 ACS Initial Mailing	September 8 ACS Pressure Seal Reminder	September 21 ACS Qnaire package	September 24 ACS postcard	October 17 ACS Final Pressure Seal

1 The body of the letter also lets respondents know that a paper questionnaire will be sent in a few weeks to address the concerns of those who are unable to (or prefer not to) respond online.

2 CAPI interviews start at the beginning of the month following the fifth mailing.

3 CAPI interviewers also attempt to conduct interviews by phone when possible.

4 Respondents were shown all ten mail pieces within a one-hour session. If a housing unit in the April 2020 ACS sample does not respond to either the ACS or the census, they will receive ten mailings from the U.S. Census Bureau over two months (mid-March to mid-May), potentially even receiving an ACS mailing and decennial mailing on the same day.

5 Historical ACS data was used to identify tracts where mail response rates exceeded Internet response, where there was a high proportion of people in older age groups, and where there was low internet access.

6 These are addresses for which the NRFU operations starts earlier than usual in order to enumerate college students or other populations that might not be living at a given address in May or June but lived at the address on Census Day.

7 UL occurs in areas where the majority of housing units do not have a city-style address to receive mail.

8 UE enumerates at housing units in areas where the initial visit requires enumerating while updating the address frame (primarily remote geographic areas that have unique challenges associated with accessibility).

9 The Census Bureau estimates 6.75 million housing units will be contacted in the UL operation and 8,000 will be contacted in UE (excluding Remote Alaska).

10 The coverage measurement program conducts a separate enumeration of housing units and people to check the quality of the census. A sample of housing units across the country will be included in this research and will receive additional contacts from a Census Bureau worker in 2020. There is the possibility of a housing unit being contacted by decennial, ACS, and coverage measurement, but the likelihood is small.

11 Chesnut and Davis allude to results from 2000 but we could not find formal documentation of ACS/decennial interaction from 2000. From Chesnut and Davis: “During Census 2000, the American Community Survey (ACS) experienced an increase in mail response in the months prior to Census Day (April 1, 2000) and a decline in response in the months following.”

12 The ACS mail contact strategy in 2010 was different than it is now.

13 Previous research indicates that in ACS experiments postal procedures alone could cause a difference in response rates at a given point in time between smaller experimental treatments and larger control treatments, with response for the small treatments having a negative bias (Heimel, 2016). Thus, the treatments are structured to be of similar size. The specialized treatment will be sorted and mailed separately from the rest of the production cases so that the specialized and baseline treatments will have similar mail delivery timing.

14 A blank form is a form in which there are no persons with sufficient response data and there is no telephone number listed on the form.

15 A sufficient partial internet response is one in which the respondent reached the Pick Next Person screen for a household with two or more individuals on the roster or has gone through the place of birth question for a

1-person household.

16 We will remove addresses deemed to be Undeliverable as Addressed by the Postal Service if no response is received.

17 The number of questions that should have been answered is determined based on questionnaire skip patterns and respondent answers. If it is not clear if a question should have been answered (because a prior question was left blank), it is excluded from the calculation.