Coordinated Collection - High Level Plan

ODYSSEY

Coordinated Collection High-Level Plan

Version 1.0

October 18, 2019

Table of Contents

Introduction…………………………………………………………………………………………………………..…..3

Coordinated Collection Pilot -Focus.………………………….……………………………………………….3

Coordinated Collection Pilot – Goals………………………………………………………………………....4

Case Overlap……………….………………………………….………………………………………………………...4

Proposed Method of Collection………………….……………………………………………………………..4

Collection Strategy Proposal..………………………………………………………..……………………..…..5

Coordinated Collection Timeline…………....…………………………………………………………………6

Approvals…………………………………………………………………………………………………………………..7

Introduction

The Coordinated Collection Team was formed to define what it means to coordinate communication with respondents across programs, specifically the annual business programs. This team will work on implementation strategies to meet Recommendation 4.7 of the study to re-engineer the annual economic surveys, conducted by the National Academies of Sciences, Engineering, and Medicine (NAS). The recommendation states: Census bureau should work to harmonize its communications and fieldwork strategies across its annual economic surveys, leading to a strategy for an Annual Business Survey System (ABSS) aimed at efficiently achieving accurate, complete, and timely responses from sampled businesses. This strategy would be informed by available information from pretest, post-field evaluations, experiments to test alternative approaches, and active monitoring of responses through paradata. A future goal might be to develop a paradata-based monitoring system (or process control framework) that supports a responsive (or adaptive) design for fieldwork for an ABSS.

This particular document outlines a high-level plan aimed at planning a pilot to better understand the processing, setup requirements, and data collection operations necessary to implement coordinated collection. This pilot will focus on coordinated collection implementation for three annual surveys with a future goal of expansion to the remaining appropriated annual current surveys and potentially other surveys (indicators, reimbursables) down the road.

Coordinated Collection Pilot - Focus

For the pilot, the team will focus on implementing the coordinated collection process for overlapping cases, based on the matching criteria specification, for the Annual Retail Trade Survey (ARTS), Annual Wholesale Trade Survey (AWTS), and/or the Service Annual Survey (SAS). These three surveys were chosen for the following reasons:

1. Start small - Starting with three surveys will allow us to test the new coordinated collection processes and programs to ensure success and efficiencies as well as make necessary modifications before expanding to more surveys.

2. These 3 programs are BSR surveys with IDs that are easily matched

3. ARTS, AWTS, and SAS utilize the same NPC support – this will lead to focused training for particular groups of NPC staff with the ability to test and improve processes prior to expanding.

Future phases will expand to cover the remaining in-scope annual surveys for the project: Annual Capital Expenditures Survey (ACES), Annual Survey of Manufacturers (ASM), and the Manufacturers, Shipments, Inventories, and Orders Unfilled Orders Survey (M3UFO).

This pilot will include the addition of an advanced contact to notify respondents of the changes as well as debriefings afterwards to help us analyze the impacts, benefits, and issues of implementing a coordinated collection effort.

Coordinated Collection Pilot - Goals

• Coordinate respondent communication across programs. Communication with respondents in multiple programs is combined, to collectively address initial survey requests and follow-up reminders.

• Eliminate duplication of mail packages, email messages, and telephone calls for overlapping surveys to save money and reduce respondent burden and confusion.

• Develop a framework system for consolidated collection with the flexibility to expand to additional annual surveys in the future.

• Implement a new consolidated strategy with various scenarios to investigate different alternatives to consolidating collection and measure the success of each in order to inform decisions for future years.

Case Overlap

Internal programs will perform a match across ARTS, AWTS, and SAS to get a count of the overlapping cases and better understand what to expect for consolidation options. The counts of matched cases within these surveys are below:

Surveys	Unique Companies	IDs (Forms)	Street Match	Contact Match
ARTS/AWTS/SAS	74	501	33 (197)	7 (42)
ARTS/AWTS	160	401	115 (268)	78 (179)
ARTS/SAS	166	902	120 (414)	70 (190)
AWTS/SAS	337	1,244	208 (643)	100 (304)
TOTAL	737	3,048	476 (1,522) 65%	255 (715) 35%

There will be a hierarchy of the matches to assign the primary company contact, looking at variables such as check-in date for the various surveys and contacts.

Proposed Method of Collection

1. Pre-notification Operations

   • Contact respondents prior to mail-out using a combination of phone and mail.

   • This advanced contact will alert the respondent to changes in the process, including notification of a primary contact responsible for completing all assigned surveys or delegating for completion.

   • This advanced contact will include a letter and/or phone call with the option for the respondent to reach out for additional information or assistance.

   • Phone calls to the companies in all 3 surveys (SAS/ARTS/AWTS), as agreed to by all divisions, will be conducted by full-service account managers.

   • Initial inbound and outbound phone contacts will be able to assist with burning the authentication code for setup of the survey cards and walking respondents through the process of sharing survey access, if desired.

2. Respondents will receive one authentication code for all three surveys

   • Entering the authentication code in the respondent portal will produce up to three survey cards (SAS, ARTS, AWTS), as appropriate, with additional messaging for status, due dates, etc. as well as red text to draw attention to surveys that are past due.

   • Clicking “REPORT NOW” then takes the respondent to the individual instrument, which will not have any consolidated content for this pilot.

   • The respondent can delegate and request time extensions separately, if it meets their reporting needs.

This method gives the user flexibility and with changes to the authorization bank will set us up for a module approach for future collection through the respondent portal. This option assumes that the end state supports handling the consolidation in the respondent portal (not Centurion), meaning that building this out is a step in the right direction.

Collection Strategy Proposal

The proposed strategy for coordinated collection of the overlapping cases between SAS, ARTS, and AWTS includes two groups, combined and staggered, with the following high-level timelines:

• Combined Group (1/2 of the cases)

1. Pre-notification (mid-Dec. 2019 to mid-Jan 2020)

2. Mid-February 2020 Initial Mail (approx. Feb. 18, 2020) when the Centurion instrument for all surveys is available.

3. Coordinated follow-ups (mail, email, and/or phone) that speak to the status of all surveys in which the respondent is assigned.

4. All surveys are due on the same date, approximately March 24, 2020
   
   

• Staggered Group (1/2 of the cases)

1. Pre-notification (mid-Dec. 2019 to mid-Jan 2020)

2. Mid-January 2020 Initial Mail (approx. Jan. 21, 2020)

   1. SAS and AWTS Centurion instrument will be open and available

   2. ARTS Centurion instrument will not yet be available; respondent portal will show when it will be available.

3. Coordinated follow-ups (mail, email, and/or phone) that speak to the status of all surveys in which the respondent is assigned.

4. Staggered due dates:

   

   Follow up each month with a status of the surveys

   1. SAS due 2/25/20

   2. AWTS due 3/24/20

   3. ARTS due 4/28/20

Note: With these three surveys having various due dates between the consolidated and non-consolidated cases, the team made a decision to remove due dates from the Centurion instrument, as there is a possibility of the due date being incorrect. The (correct) due date will be populated on the survey card within the respondent portal, which is how all respondents will access the survey (Centurion instrument), therefore this information is still visible.

Research Questions

What are we hoping to learn from this coordinated collection test?

• Workload measurement: Does consolidating surveys with multiple contacts change the type and amount of work for analysts handling feedback/changes (e.g. inbound calls, customer service assistance requirements, requests for breaking consolidation, etc.)?

• Impact of one contact: Does consolidating surveys with multiple contacts into one contact point affect survey performance indicators and respondent burden?

• Due dates: Do companies that have a single due date respond differently than those who have a staggered due date?

• Data quality: Does combining survey mailings change data quality?

Approvals

Approver	Date Approved
Kimberly Moore, Chief, Economy-wide Statistics Division (EWD)	9/27/19
Lisa Donaldson, Chief, Economic Management Division (EMD)	10/8/19
Carol Caldwell, Chief, Economic Statistical Methods Division (ESMD)	10/16/19
Stephanie Studds, Chief, Economic Indicators Division (EID)	10/21/19
Michelle Karlsson, Assistant Division Chief for Collection (EMD)	10/16/19

8