Contact Strategies Testing

Attachment K

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2022 Economic Census

OMB Control Number 0607-0998

Contact Strategies Testing

Component of the Collection Strategy	Survey Collection and Treatment(s) Tested		Summary of Results		Implementation Decision
Advance notice1	2012 Economic Census: Advance letter tested among SUs in industries with historically low response rates. Advance request to verify contact information tested among MUs with fewer than 1,000 employees.		Differences in check-in rates were not statistically significant for either experiment.		No. However, Account Managers will contact selected MUs in advance of mailout. (See Section 3 subsection on “Outreach”)
Full-scale pilot of later mailout and due date	2015 ASM: Initial mailout date moved to Late January of year following reference year rather than Late December. Selected large MUs were offered a later due date (in May 2016) than SUs and smaller MUs, whose due date was mid-March 2016.		May 2016 due date for large MUs resulted in later 2015 ASM responses compared to 2014 ASM reporting.		Partial implementation, consisting of late January mailout and mid-March due date for all units.
Rolling extension dates	2020 BERD, COS/ASM: Full-scale pilot allowing respondents to choose extension date within upcoming 2 weeks throughout extension window to avoid clustering of extensions around fixed max extension dates.		Smoothed out incoming call spikes and data processing of completed responses throughout data collection.		Yes. Rolling time extensions will be used to improve flow of incoming calls and completed cases for data processing.
Messaging in Mailings	2014 COS/ASM: Emphasis on electronic reporting options versus standard messaging in letters Emphasis on electronic reporting options explained in letters versus placed in flyers.		General improvement in uptake of electronic mode and decrease in requests for paper forms; statistical significance varies depending on whether cases are in COS only, in both COS and ASM, as well as employment size.		Yes. Electronic reporting will be emphasized in letters, along with mandatory requirement, confidentiality pledges, purpose & uses of data collected, per OMB requirements.
Automated Messaging using Respondent Portal	2020 ACES, BERD: Customized content on the Respondent Dashboard Email customized Welcome Packet after respondent first accesses survey Thankyou email after respondent completes survey 2021 ARTS: After TFU contact, email to follow-up on the call		TBD		TBD, pending results of 2020 ACES, BERD and 2021 ARTS research
Flyers2		2015 SAS: Three treatments consisted of different flyers, each with a different type of motivational message, enclosed with initial and follow-up mailings. 2021 ACES: For initial mailing, three treatments to new sample cases with 1/3 sheet insert containing graphics and information about respondent portal, uses of data, or both	2015 SAS: No statistically significant effect of flyers on check-in rates amongst prior NR or respondents. Some inconsistent, small statistically significant improvements in check-in rates or response times for different flyers amongst different subgroups.	TBD, pending results of 2021 ACES research. Including flyers in all 2015 SAS mail contacts did not improve overall response. Mixed results of different flyers with different industry subgroups are inefficient and not cost effective to implement in production.
Due date reminder 2	SQ-CLASS (2014 2nd qtr): Reminder letter mailed 3 weeks prior to due date 2014 ARTS: Reminder letter mailed 2 weeks prior to due date		Improved timeliness and statistically significant increase in response maintained through to the end of the collection period.	Yes. Improved check-in rate and increased timeliness of response provides cost savings, reducing cases requiring more expensive follow-up (e.g., certified mail and telephone follow-up).
Accelerated follow-up with and without due date reminder 2		2014 ARTS: 1st nonresponse post-due-date follow-up reminder letter mailed 2 weeks earlier than traditional mail follow-up	Improved timeliness and statistically significant increase in response maintained through to the end of the collection period.	Yes. Improved check-in rate and increased timeliness of response provides cost savings, reducing cases requiring more expensive follow-up (e.g., certified mail and telephone follow-up).
Red ink on envelopes 2	2014 AWTS: Using red ink versus standard black ink for imprinted due date / past due notice on the envelope. Applied in initial mail and all follow-up reminders.		Overall difference in check-in rates not statistically significant. However, statistically significant interaction effects of red ink treatment with selected subgroups.		Yes. Using red ink for imprinted due date / past due notices on envelopes appears to improve response rates among certain subgroups, particularly prior NRs, without reducing response from other subgroups, and it is cost neutral.
Half-page envelope size2	2015 ARTS: Findings from focus groups with past Economic Census respondents suggested that a larger envelope may get respondents’ attention more effectively. Half-page-sized envelopes compared with standard letter-sized envelopes used in all mail contacts.		Some statistically significant results, but of no practical significance: Statistical significance in check-in rates only at due date, but not at close-out. Statistically significant difference of ½ day between mail-out and receipt.		No. Differences, if any, in overall or subgroup response of no practical significance.
Pressure-sealed envelopes	2016 SQ-Class, Refile, ASM: Proposed use for due-date reminders and follow-up mailings. They will not be used for initial mailout. Using pressure-sealed will reduce time lag between producing mailing lists and mailout, improving ability to remove responding cases prior to mailout. Concern that pressure-sealed envelopes may be perceived as “junk” mail and discarded or ignored, reducing effectiveness.		Using pressure-sealed envelopes gained processing improvements with minimal effect on check-in rates		Yes. Pressure-sealed mailings will be used for the Due Date Reminder and standard mail follow-ups to improve processing of the large workloads.
Certified mail for targeted subsample of SU nonrespondents3	2012 Economic Census: Compare non-certified to certified mail follow-up for 2nd or 3rd Follow-up to improve response by communicating legitimacy of official survey request 2015 ASM: Compare use of certified mail follow-up amongst a targeted subsample of SUs with non-targeted SUs getting non-certified follow-up. Compare approach that combines the two approaches (targeted certified plus non-certified for the remainder) with full non-certified follow-up for all nonresponding SUs.		2012 Economic Census: Certified follow-up increased check-in rate more than non-certified More cost effective to use certified in later follow-up 2015 ASM: The combined approach improved data quality.	Yes. Targeted certified mail follow-up will be implemented amongst nonresponding SUs for the 3rd Follow-up.  The nonresponding SUs not selected to receive a certified follow-up will be sent non-certified follow-up letters.
Priority Class Follow-up	2020 ABS, 2022 ABS: Compare Priority Class to Certified Class to reduce Undeliverable As Addressed (UAA) rate while still gaining response by communicating legitimacy of official survey request		2020 ABS: Certified had higher check-in rate but higher UAA rate Certified completed sooner after mailing Given the UAA rates, have concerns about certified, especially during COVID		TBD, pending results of 2022 ABS research
Office of General Counsel (OGC) Letter	2020 COS/ASM: 2x2 experimental design Compare follow-up letter from OGC to letter that emphasizes legal reporting requirements without referring to OGC Compare 1st Class to Priority Class		TBD		TBD, pending results of 2020 COS/ASM research
Email follow-up sequence	2020 ARTS: 3 panels to explore impact of conducting Mail and then Email Follow-up, Email and then Mail Follow-up, or Mail and Email Follow-up concurrently		Email then mail panel had earlier bumps in response, requiring fewer mail packages Mail & email together panel showed larger bumps than panel with mail then email Response for 3 panels converged and ended about the same		Use email then mail follow-up sequence. Different email follow-up sequences ended with about the same response but email than mail had earlier bump in response, reducing mail follow-up and costs.
Opting into email preference	2020 ACES,2021 COS/ASM: Full-scale pilot to allow respondents to choose email as preferred contact method (reverting to mail after 2nd follow-up if don’t respond) 2021 ACES: Respondents that have previously opted into email preference will get initial email instead of letter (reverting to mail if don’t respond by due date) Continue to allow respondents to opt into email only follow-up		TBD		TBD, pending results of 2020 and 2021 ACES, 2021 COS/ASM research
Follow-up Robocalls	2021 COS/ASM: Full-scale pilot pairs follow-up robocalls with email follow-up prior to TFU 2021 SAS: Cases not selected for TFU assigned to 2 panels: 1st pairs Follow-up Robocalls with Email Follow-up and 2nd panel conduct Follow-up Robocalls 2 weeks after Email Follow-up		TBD		TBD, pending results of 2021 COS/ASM and SAS research
Account managers for selected medium MUs	2021 COS/ASM: Expanding Account Manager Program to include selected Medium MUs (in addition to selected Large MUs) Single point of contact to provide outstanding customer service and create customized respondent reporting arrangements		TBD		TBD, pending results of 2021 COS/ASM research
CEO letter for selected large & medium MUs	2021 COS/ASM: For selected MUs, 4th Follow-up will be letter to CEO instead of OGC letter (some may later get an OGC letter, if needed)		TBD		TBD, pending results of 2021 COS/ASM research
Overlap of large collection activities for Economic Census & ABS	2022 ABS collection activities overlap with 2021 COS/ASM 4th Mail Follow-up, Targeted Telephone Follow-up, and Final Email Follow-up (similar to ABS/ Economic Census overlap): For cases in both surveys (with same contact information), ABS Initial Mail, Initial Email, 1st Follow-up Email, and Respondent Portal and COS/ASM TFU will address separate survey requests so respondents know both need to be completed ABS and COS/ASM response priorities will be assessed to determine which overlap cases to exclude from COS/ASM 4th Mail Follow-up and Final Email Follow-up		TBD		TBD, pending results of 2021 COS/ASM and 2022 ABS overlap research

1 Marquette, Erica, Michael E. Kornbau, and Junilsa Toribio. 2015. Testing Contact Strategies to Improve Response in the 2012 Economic Census. In JSM Proceedings, Government Statistics Section. Alexandria, VA: American Statistical Association. 2212-2225.

2 Tuttle, Alfred D. 2016. Experimenting with Contact Strategies to Aid Adaptive Design in Business Surveys. In JSM Proceedings, forthcoming. Alexandria, VA: American Statistical Association.

3 Kaputa, et al., 2016 ICES-V