Contact Strategies Testing

Attachment F 

Department of Commerce 
United States Census Bureau 
OMB Information Collection Request 
2017‐2019 Report of Organization 
OMB Control Number 0607‐0444 

Contact Strategies Testing 

Component of
Survey Collection and
the Collection
Treatment(s) Tested
Strategy
Advance notice 1 2012 Economic Census:
 Advance letter tested among
SUs in industries with
historically low response
rates.
 Advance request to verify
contact information tested
among MUs with fewer than
1,000 employees.
2015 ASM:
Full-scale pilot
of later mailout  Initial mailout date moved
and due date
to Late January of year
following reference year
rather than Late December
of reference year.
 Selected large MUs were
offered a later due date (in
May 2016) than SUs and
smaller MUs, whose due
date was mid-March 2016.
Quarterly Survey of Business
Due date
Professional & Classification
reminder 2
(SQ-CLASS) (reference
period = 2014 2nd qtr):
 Reminder letter mailed 3
weeks prior to survey due
date
2014 ARTS:
 Reminder letter mailed 2
weeks prior to survey due
date
2014 ARTS:
Accelerated
follow-up with
 1st nonresponse post-dueand without due
date follow-up reminder
2
date reminder
letter mailed 2 weeks earlier
than traditional mail followup

Summary of Results

Implementation
Decision

Differences in checkin rates were not
statistically significant
for either experiment.

No. However, Account
Managers will contact
selected large MUs in
advance of mailout.
(See Section 3
subsection on
“Outreach”)

May 2016 due date for
large MUs resulted in
later 2015 ASM
responses compared to
2014 ASM reporting.

Partial implementation,
consisting of late
January mailout and
mid-March due date for
all units.

Improved timeliness
and statistically
significant increase in
response maintained
through to the end of
the collection period.

Yes. Improved check-in
rate and increased
timeliness of response
provides data collection
cost savings since fewer
cases require more
expensive follow-up
techniques (e.g.,
certified mail and
telephone follow-up).

Improved timeliness
and statistically
significant increase in
response maintained
through to the end of
the collection period.

Yes. Improved check-in
rate and increased
timeliness of response
provides data collection
cost savings, reducing
number of cases
requiring more
expensive follow-up
(e.g., certified mail and
telephone).

Component of
the Collection
Strategy
Red ink on
envelopes 2

Half-page
envelope size 2

Survey Collection and
Treatment(s) Tested
2014 AWTS:
 Using red ink versus
standard black ink for
imprinted due date / past
due notice on the envelope.
Applied in initial mail and
all NR follow-up reminders.

2015 ARTS:
 Findings from focus groups
with past EC respondents
suggested that a larger
envelope may get
respondents’ attention more
effectively.
 Half-page-sized envelopes
compared with standard
letter-sized envelopes used
in all mail contacts.

Summary of Results
Overall difference in
check-in rates not
statistically significant.
However, statistically
significant interaction
effects of red ink
treatment with selected
subgroups:
 Increased check-in
rate among prior
nonrespondents
compared to prior
respondents
 Increased check-in
rate among selected
industries
Some statistically
significant results, but
of no practical
significance:
 Statistical
significance in
check-in rates only at
due date, but not at
close-out of data
collection.
 Statistically
significant difference
of ½ day, on
average, between
mail-out and receipt.

Implementation
Decision
Yes. Using red ink for
imprinted due date /
past due notices on
envelopes appears to
improve response rates
among certain
subgroups, particularly
prior NRs, without
reducing response from
other subgroups, and it
is cost neutral.

No. Differences, if any,
in overall or subgroup
response of no practical
significance.

Component of
the Collection
Strategy
Messaging

Survey Collection and
Treatment(s) Tested
2014 Report of
Organization/ASM:
 Emphasis on electronic
reporting options versus
standard messaging in
letters
 Emphasis on electronic.
reporting options explained
in letters versus placed in
flyers.

Summary of Results

General improvement
in uptake of electronic
mode and decrease in
requests for paper
forms; statistical
significance varies
depending on whether
cases are in Report of
Organization only, in
both Report of
Organization and
ASM, as well as
employment size.
2015 SAS:
Flyers 2
 No statistically
 Three treatments consisted
significant effect of
of different flyers, each
flyers on check-in
with a different type of
rates amongst prior
motivational message,
NR or respondents.
enclosed with initial and
 Some small
follow-up mailings.
statistically
significant, but
inconsistent,
improvements in
check-in rates or
response times for
different flyers
amongst different
subgroups.
2015 ASM:
The combined
Certified mail
approach improved
for targeted
 Compare use of certified
subsample of SU
mail follow-up amongst a data quality.
nonrespondents3
targeted subsample of SU
nonrespondents with nontargeted SU NR follow-up
using regular 1st-class mail.
 Compare approach that
combines the two
approaches (targeted
certified plus regular mail
for the remainder) with full
SU NR follow-up using
regular 1st-class mail only.

Implementation
Decision
Yes. Electronic
reporting will be
emphasized in letters,
along with mandatory
requirement,
confidentiality pledges,
purpose & uses of data
collected, per OMB
requirements.

No. Including flyers in
all mail contacts did not
improve overall response.
Mixed results of different
flyers with different
industry subgroups is
inefficient and not cost
effective to implement in
production.

A one-time, targeted
certified mail follow-up
will be implemented
amongst SU NRs for the
2017 Econ Census. The
nonresponding SUs not
selected to receive a
certified follow-up will be
sent follow-up letters
using regular 1st-class
mail.

Component of
the Collection
Strategy
Pressure-sealed
envelopes

1

Survey Collection and
Treatment(s) Tested

Summary of Results

2016 SQ-Class, Refile, and
TBD
ASM:
 Proposed use for due-date
reminders and NR followup mailings. They will not
be used for initial mailout /
contact.
 Using pressure-sealed will
reduce time lag between
producing mailing lists and
mailout, improving ability
to remove responding cases
prior to mailout.
 Concern that pressuresealed envelopes may be
perceived as “junk” mail
and discarded or ignored,
reducing effectiveness of
mail contacts.

Implementation
Decision
TBD, pending results of
SQ-Class, Refile, ASM
research

Marquette, Erica, Michael E. Kornbau, and Junilsa Toribio. 2015. Testing Contact Strategies to
Improve Response in the 2012 Economic Census. In JSM Proceedings, Government Statistics
Section. Alexandria, VA: American Statistical Association. 2212-2225.
2
Tuttle, Alfred D. 2016. Experimenting with Contact Strategies to Aid Adaptive Design in
Business Surveys. In JSM Proceedings, forthcoming. Alexandria, VA: American Statistical
Association.
3
Kaputa, et al., 2016 ICES-V