2020 Census Evaluation: Evaluation of the Group Quarters Advance Contact (GQAC) Operation Study Plan

The memorandum and attached document(s) was prepared for Census Bureau internal use. If
you have any questions regarding the use or dissemination of the information, please contact
the Stakeholder Relations Staff at dcco.stakeholder.relations.staff@census.gov.

2020 CENSUS PROGRAM INTERNAL MEMORANDUM SERIES: <2019.26.i>
Date:

June 11, 2019

MEMORANDUM FOR: The Record
From:

Deborah M. Stempowski (signed June 11, 2019)
Chief, Decennial Census Management Division

Subject:

2020 Census Evaluation: Evaluation of the Group Quarters
Advance Contact (GQAC) Operation Study Plan

Contact:

Jennifer Reichert
Decennial Census Management Division
301-763-4298
jennifer.w.reichert@census.gov

This memorandum releases the final version of the 2020 Census Evaluation: Evaluation of the Group
Quarters Advance Contact (GQAC) Operation Study Plan, which is part of the 2020 Census Program for
Evaluations and Experiments (CPEX). For specific content related questions, you may also contact the
authors:
Jessica Graber
Center for Behavioral Science Methods
301-763-6550
jessica.ellen.graber@census.gov

Anna Sandoval Girón
Center for Behavioral Science Methods
301-763-3575
anna.b.sandoval.giron@census.gov

Dave Tuttle
Center for Behavioral Science Methods
301-763-7809
alfred.d.tuttle@census.gov

census.gov

2020 Census
Evaluation
Evaluation of the Group Quarters
Advance Contact (GQAC) Operation
Study Plan

Jessica Graber
Dave Tuttle
Anna Sandoval Girón
Center for Behavioral Science Methods (CBSM)

June 11, 2019
Version 2.2

Page intentionally left blank.

Evaluation of the GQAC Operation,Version 2.1

Table of Contents
I.

Introduction ......................................................................................................................... 1

II.

Background ......................................................................................................................... 1

III.

Assumptions........................................................................................................................ 3

IV.

Research Questions ............................................................................................................. 4

V.

Methodology ....................................................................................................................... 4

VI.

Data Requirements .............................................................................................................. 7

VII.

Risks.................................................................................................................................... 8

VIII.

Limitations .......................................................................................................................... 8

IX.

Division Responsibilities .................................................................................................... 9

X.

Milestone Schedule ............................................................................................................. 9

XI.

Review/Approval Table .................................................................................................... 10

XII.

Document Revision and Version Control History ............................................................ 10

XIII.

Glossary of Acronyms ...................................................................................................... 11

XIV. References ......................................................................................................................... 11

Evaluation of the GQAC Operation,Version 2.1

Page intentionally left blank.

Evaluation of the GQAC Operation,Version 2.1

I.

Introduction

The Center for Behavioral Science Methods (CBSM) will conduct an evaluation of the 2020
Census Group Quarters Advance Contact (GQAC) operation. The purpose of this research is
twofold: we plan to evaluate the implementation of the “502” group quarters (GQ) type code
during the GQAC operation and understand the magnitude of the mismatch between census
coding of college and university student housing and the true status of an address. Our findings
are intended to inform a program of research that could continue into 2030 Census research and
planning activities and yield significant improvement in both decennial and American
Community Survey (ACS) college or university housing identification procedures.

II.

Background

The Census Bureau defines college and university student housing as a category of “group
quarters” and conducts enumeration activities at those addresses using GQ operational
procedures. This GQ category is inclusive of multiple subtypes of buildings, including
traditional residence halls, fraternity and sorority housing recognized by the college or
university, and apartment-style housing that is designed primarily to house college and university
students. These facilities may be located either on or off campus, and may be
owned/leased/managed either by the college or university or by a private company or agency.
Historically, all college or university student housing have been coded as a single GQ Type 501.
However, the definition and coding employed by the Census Bureau have not kept pace with the
evolution in student housing options, and Census Bureau group quarters research has primarily
focused on college- or university-owned/leased/leased/managed faciliites (Kuwik et al, 2014;
Moore, 2013)
The use of a single category has been shown to be problematic for multiple reasons. First,
stakeholders, through state data centers, have requested additional granularity in GQ data to be
able to distinguish the populations of students residing at the college/university-owned, leased, or
managed vs. privately owned, leased, or managed facilities. Second, the accurate identification
of privately owned, leased, or managed apartments designed for college and university student
residents is often difficult and operational procedures previously used have led to
miscategorization.
For example, Census Bureau listers may incorrectly code a privately owned student housing
facility as a housing unit (HU) instead of a GQ. Or a student residing in a privately owned
facility may report to an American Community Survey (ACS) field representative (FR) that the
unit is an apartment and not a GQ. Further complicating this issue is the coding of these
addresses on the Delivery Sequence File from the United States Postal Service (USPS), where
privately owned/leased/managed student housing facilities are coded a housing units by default.
The prevalence of such errors during census field data collection is expected only to increase.
As student expectations and trends have evolved, the number of private college/university
student housing options have increased, making it more difficult for listers, enumerators and
1

Evaluation of the GQAC Operation,Version 2.1

field representatives to distinguish these student-targeted residences from traditional apartment
buildings. During a decennial census environment, multiple opportunities exist for an address to
be reclassified, including the Address Canvassing operation, GQAC, and GQ enumeration
(GQE).
The issues are further compounded with ACS data collection. Annually, thousands of individual
apartments within private student housing buildings are randomly sampled by the ACS. In past
years, ACS FR, relying on information or categorization from the respondent or building
informant, converted units from GQs to HUs or the reverse. (Please note that while an argument
may be made that such units should not be considered GQs at all and instead be enumerated as
HUs, that is not the matter currently under discussion.)
In light of these issues, the GQ Working Group Definitions Subteam conducted some initial
research to explore the problem, and identify and pretest a potential solution. Specifically, in
2015, the Center for Survey Measurement (CSM) conducted exploratory qualitative research in
three locations with universities (two large public institutions, one private). At each location,
interviews were conducted with the office of student housing at the university and two private
apartment building managers.1 Based on those results, more tailored research was completed the
subsequent year. In 2016, CSM conducted additional targeted interviews with housing
administrators and apartment managers in a different location with multiple colleges and
universities in close proximity to each other. A total of six interviews were conducted, three
with each respondent type.2
Based on the research findings, a proposal was put forth to implement a secondary or
subcategory (GQ Type 502) for college/university student housing that was
owned/managed/leased by a private company or agency. A critical part of the definition was the
description of “by the bed” or “single-liability” leasing, a term that was found to be highly
meaningful during the early research. In short, students are able to sign individual leases for a
single room–or bed–and are therefore not fiscally responsible for their roommates’ portion of the
lease. Students may also opt to rent in an apartment where they do not know their roommates in
advance.
In 2017, the ACS conducted a field test implementing the new and revised definitions of college
and university student housing,3 surveying contacts at 120 privately owned student housing
locations. FRs also completed debriefing questionnaires at the end of the field data collection
period. This test demonstrated that these revised codes could be successfully implemented in the
field, with most FRs reporting no difficulty.
As a result of what has been found to date, the GQAC operation made the decision to implement
the following procedures for the 2020 Census.
Holzberg, Jessica L. “Findings and Recommendations from Group Quarters (GQ) Student Housing Qualitative
Interviews.” Center for Survey Measurement, U.S. Census Bureau, November 30, 2015.
2
Graber, Jessica E. “Findings and Recommendations from Group Quarters Student Housing Research.” Center for
Survey Measurement, U.S. Census Bureau, May 2016.
3
Butler, Nicole ”Field Test on Revised and New Definition of College/University Student Housing.” 2017
American Community Survey Research and Evaluation Report, August 2017.
1

2

Evaluation of the GQAC Operation,Version 2.1

STEP 1:

The GQ Type will be verified (previously assumed to be 501).

STEP 2:

If confirmed as student housing, a follow-up question will be asked of the
college/university respondent:
“Is this student housing owned, leased, or managed by a college, university,
or seminary?
Or, is this student housing owned, leased, or managed by a private company
or agency, which typically offers “by the bed” (or single-liability) leases to
students?
Facilities where respondents answer “yes” to the first question will maintain their
GQ Type 501 code. Private facilities, whose respondents answer “yes” to the
second question, will be updated to reflect the new GQ Type 502.

III.

Assumptions

Below are some assumptions used when developing this study plan.
1. The Living Quarters Type code for any specific address may be updated or revised during a
number of data collection operations–including ACS data collection, 2020 Address
Canvanassing, GQAC (both in-field and in-office), and GQE.
2. We anticipate that GQs identified and enumerated during the 2020 Nonresponse Followup
operation (NRFU), including early NRFU, will be updated in the MAF to reflect a status of
GQ.
3. Addresses identified as GQ Type 502 during the 2020 GQAC operation will be recorded as
such only in the GQAC Production Control System (PCS).
4. The current Master Address File (MAF) does not include a GQ Type of 502. All college or
university student housing, regardless of ownership type is coded as a 501. The MAF will not

3

Evaluation of the GQAC Operation,Version 2.1

be modified to reflect any 502s identified during 2020 GQAC operations. All identified 502s
will remain 501s in the MAF after the 2020 Census.
5. Listers conducting the 2020 Address Canvassing operation will record detailed GQ types,
including both 501 and 502, within the Listing and Mapping Application (LIMA) system.
6. Center for Behavioral Science Methods (CBSM) staff will have access to quantitative data,
as well as interviewer/clerk notes from the GQAC operation.
7. The anticipated dates of the GQAC operations are as follows:
 GQAC In-Office: 2/3/20 – 3/6/20
 GQAC In-Field: 2/20/20 – 3/6/20
 GQ Enumeration: 4/2/20 – 6/5/20

IV.

Research Questions

The GQAC team is seeking to understand the factors that lead to the misclassification of
college/university student housing, and to understand the magnitude of the mismatch between
census coding of college and university student housing and the true status of an address. Our
specific research questions include:
1. Are the 2020 Census GQAC college or university student housing questions and materials
effective in soliciting accurate information from student housing providers?
2. Are there reporting differences found during the GQAC operation by method of contact (infield vs. in-office)–in respondent cooperation, accuracy of data collected, effectiveness of
staff ?
3. What is the prevalence of shifting between codes, that is addresses shifting between
classification as a HU to a GU, or the reverse, from a 501 to a 502 or the reverse, and through
which census operations.
4. How do 2020 Census final GQ Type codes compare to the alternative frame that will be
developed as part of this proposed research?
5. Are there differences between college/university student housing and private
college/university student housing organizations with regards to cooperation and accuracy in
reporting?

V.

Methodology

Analyses will make use of a combination of qualitative and quantitative data analysis using a
variety of sources that include: ethnographic observation of the data collection operations,
reinterviews of GQ housing informants, analysis of 2020 Census data collected prior to and
4

Evaluation of the GQAC Operation,Version 2.1

during 2020 operations, and comparisons of census-collected data to those from alternative
sources. This effort is described as five unique, yet interrelated tasks. Each is described below.
A. Design
Task 1. Expert Review of Group Quarters Advance Contact Materials
The expert review of the GQAC materials will serve as the foundation for the subsequent proposed
work. While this review will not be completed in time to impact 2020 Census operations, this
review is critical to provide context to the evaluation. We will document all suggestions and
integrate these recommendations into our final evaluation report. CBSM will evaluate the
following materials:
 Address Canvassing training materials.
 GQAC in-office screening questions.
 GQAC in-office clerk training materials, including calling scripts, FAQs, letters, other jobaids.
 GQAC in-field screening questions.
 GQAC in-field screening interviewer training materials, including scripts, FAQs, letters,
other job-aids.
Data collected during Task 1 will be important when answering research question #1.
Task 2. Observation of GQAC Staff During 2020 Census Operations
CBSM proposes visiting approximately three-to-six sites around the country and observing clerks
and field staff conducting GQAC calls and visits. The purpose of this is to observe first-hand the
functionality of the instrument and evaluate the level of staff training (e.g., the degree to which
clerks and census field supervisors [CFS] need to respond to questions, provide clarification or
guidance, probe, and perhaps assist respondents in coming to a final response).
CSBM will work closely with the GQ team and Field Division to select locations with multiple
colleges/universities and high numbers of student housing. During GQAC interactions with
respondents, CBSM staff will only observe and not speak or participate in any way. During these
visits, CBSM will:
 Listen in on GQAC in-office screening calls.
 Observe GQAC in-field staff conducting visits to student housing locations that are
unresolved from in-office operations.
 Conduct debriefing interviews (either one-on-one or as a group) with clerks and field staff
to solicit their feedback on the instrument, probe their knowledge about classifying student
housing, frequent areas of difficulty encountered by GQ respondents, and gain insight into
GQAC field staff strategies for helping respondents understand the classification task, and
identify any other issues or common outcomes from the GQAC operation.
Data collected during Task 2 will be important when answering research questions #s 1, 2, and 5.
Task 3. Development of an Alternative Frame from External Sources
CBSM proposes developing a separate frame of privately owned college/university student
housing to compare against and evaluate 2020 Census results. Given available resources, we
5

Evaluation of the GQAC Operation,Version 2.1

consider this frame development to be a prototype and will limit it in scope to (1) major national
college/university student housing providers, and (2) geographies where we conducted qualitative
research to capture locally owned entities. The frame will be focused on college or university
student housing that can be positively confirmed as privately owned, mangaged, or leased and
meet the Census Bureau’s definition of a GQ Type 502. We recognize that not all addresses–such
as private homes that rent rooms to students–will be represented in our frame. Further, as the
frame is not intended to be a national representation of all privately owned student housing
providers, we will not produce prevalence rates of these entities.
The frame will be generated and/or validated using webscraping tools, known lists of college or
university student housing, third party vendors, and other methods to be explored. We will reach
out to subject matter experts in the Geography Division for any input or collaboration, as
appropriate.
Data collected during Task 3 will be important when answering research question #4.
Task 4. Analysis of 2020 Census Data
CBSM will analyze quantitative data collected during GQAC to identify addresses that changed
codes at any point between the 2010 and 2020 censuses, including any updates made during
American Community Survey (ACS) operations. As possible, we will also identify post-2020
Census MAF updates resulting from NRFU and early NRFU operations where GQs were identified
and enumerated. We will review patterns in status changes by contextual factors, including those
describing the associated university and regional or local census offices. We will also review case
notes for select cases.
We will identify cases where the housing type code has changed since the 2010 Census, either
during ACS operations or the 2020 Census. We will identify addresses from the MAF universe,
as well as any added cases identified during GQ and NRFU and early NRFU operations. If part
of the sampled geography, the case will also be compared to the alternative frame. For each case,
the change in coding will be identified, focusing on four specific living quarter types:
 Housing Unit (HU)
 GQ -501
 GQ – 502
 Unknown – 999
Additional analyses include identification and examination of the following:
 Alternative Frame addresses determined to be 502s that were miscoded in 2020 Census
operations.
 ACS-sampled addresses where individual units were coded as GQs, but not the entire
building.
 Confirmation of Basic Street Address and geocoding of all 502 additions during the 2020
GQAC operation.
Data collected during Task 4 will be important when answering research questions #s 2, 3, 4, and
5.
6

Evaluation of the GQAC Operation,Version 2.1

Task 5. 2020 Census GQAC Student Housing Unit Mismatch Debriefings
Based on the results of the Task 4 data analyses, CBSM proposes conducting follow-up interviews
with GQAC respondents who incorrectly coded their student housing type. These interviews will
be conducted over the telephone. The purpose of these interviews will be to debrief the respondent
on their recollection of the GQAC interaction, probe on their understanding of the questions and
terminology, and identify possible strategies to facilitate accurate classification. We will use
contact information collected or updated during 2020 operations (GQAC or other) to re-contact
these respondents.
Data collected during Task 5 will be important when answering research questions #s 1, 2, and 5.
Data collection from the public will require clearance from the Office of Information and
Regulatory Affairs (OIRA) at the Office of Management and Budget (OMB). CBSM has a generic
clearance to conduct such research and we can seek approval under this mechanism. Alternatively,
CBSM can work with staff in the Decennial Directorate staff to submit a request under other
available clearance channels.
B. Interventions with the 2020 Census
This research involves direct observation of 2020 GQAC clerks for the in-office operation and
the census field supervisors for the in-field and in-office operations. We will also request to
conduct staff debriefings. CBSM research staff will not interfere or participate in
interviewer/clerk-respondent interactions.

C. Implications for 2030 Census Design Decisions and Future Research and Testing
The findings from this multitiered evaluation are intended to inform a program of research that
could continue into 2030 Census research and planning activities and yield significant
improvement in both decennial and ACS student housing identification procedures.

VI.

Data Requirements

Data File/Report

Source

Purpose

Expected
Delivery Date

2010 GQ Student Housing Data
Mid-decade updates to GQ Type

2010 Census
ACS

mm/dd/yyyy
mm/dd/yyyy

2020 GQAC Quantitative data

2020 Census

2020 GQAC Call notes

2020 Census

Post-2020 MAF data

2020 Census

Baseline GQ Type code
Identify any ACS updates
post-2010
To compare changes in
classisfication over time and
identify any additions or
deletions to GQ lists.
Provides context to
quantitative data review
To compare changes in
classisfication over time and
7

mm/dd/yyyy

mm/dd/yyyy
mm/dd/yyyy

Evaluation of the GQAC Operation,Version 2.1

Alternative Frame of Privately
Owned/Managed College or
University Student Housing
Providers

identify any additions or
deletions to GQ lists from
Early NRFU or NRFU
To compare classifications
of addresses from census
data sources

CBSM

mm/dd/yyyy

VII. Risks
1. IF we are unable to secure sufficient funding THEN we will be unable to complete this
research.
2. IF CBSM is unable to gain access to the data necessary to conduct our analyses THEN we
will be unable to conduct our stated research plans.
3. IF Census Bureau regional offices do not make GQAC clerks and field staff available for
observation THEN we will be unable to conduct our stated research plans.

VIII. Limitations
1. This research is intended to evaluate the 2020 GQAC operation and not produce a solution
to a specific problem. It is also designed to be exploratory and inform an ongoing research
agenda in support of ACS operations and planning for the 2030 Census. Therefore the
outcomes will likely be modest, but useful for long-term planning and will provide a
perspective that has been missed to date.
2. Task 2 requires the selection of three to six sites for observation. The selection criteria will
be based on the presence of colleges and universities to increase the yield of addresses
coded as student housing. We recognize that this method of selection is not statistically
representative and may not represent all college and university student housing.

8

Evaluation of the GQAC Operation,Version 2.1

IX.

Division Responsibilities

Division or Office
CBSM

Responsibilities
 Project management
 Protocol development
 OMB clearance for Task 5
 Incentive request for Task 5
 Data collection
 Data analyses
 Report preparation

DCMD/Special
Enumerations




Review of and comment on materials
Providing CBSM access to data

FLD/Oversight Branch
FLD/Group Quarters Branch



Assist in gaining access to Area Census Office staff for
observations

X.

Milestone Schedule

Evaluation of the GQAC Student Housing Operation Milestone

Date

Complete Task 1 – Expert Review of GQAC Materials

09/30/2019

Complete Task 2 –Observation of GQAC Staff during 2020 Census Operations

03/6/2020

Receive, Verify, and Validate Data For Evaluation of the GQAC Operation for
College or University Student Housing

mm/dd/yyyy

Complete Task 3 – Development of an Alternative Frame from External Sources

09/30/2020

Complete Task 4 – Analysis of 2020 Census Data

09/30/2021

Complete Task 5 – 2020 GQAC Student Housing Unit Mismatch Debriefings

09/30/2021

Distribute Initial Draft Evaluation of the GQAC Operation for College or University
Student Housing Report to the Decennial Research Objectives and Methods (DROM)
Working Group for Pre-Briefing Review

09/01/2022

Decennial Census Communications Office (DCCO) Staff Formally Release the
FINAL Evaluation of the GQAC Operation for College or University Student Housing
Report in the 2020 Memorandum Series

09/15/2022

9

Evaluation of the GQAC Operation,Version 2.1

XI.

Review/Approval Table

Role

Approval Date

Paul Beatty, Division Chief, CBSM

mm/dd/yyyy

Decennial Census Management Division (DCMD) ADC for Nonresponse,
Evaluations, and Experiments

mm/dd/yyyy

Decennial Research Objectives and Methods (DROM) Working Group

mm/dd/yyyy

Decennial Census Communications Office (DCCO)

mm/dd/yyyy

XII. Document Revision and Version Control History
Version/Editor
1.0/Graber
1.0/Graber
2.0/Graber
2.1/Graber
2.2/Graber

Date
5-3-2019
5-8-2019
5-30-2019
5-30-2019
6-11-2019

Revision Description
Initial version
Integrated comments from GQAC team
Integrated comments from DROM
Integrated comments from DROM
Integrated edits from Stakeholder relations staff

10

Evaluation of the GQAC Operation,Version 2.1

XIII. Glossary of Acronyms
Acronym
ACO
ACS
ADC
CBSM
CFS
CSM
DCCO
DROM
DSF
DSSD
EXC
FR
GQ
GQAC
GQE
HU
IPT
MAF
NRFU
PCS
R&M
RCC
USPS

Definition
Area Census Office
American Community Survey
Assistant Division Chief
Center for Behavioral Science Methods
Census Field Supervisor
Center for Survey Methods
Decennial Census Communications Office
Decennial Research Objectives and Methods Working Group
Delivery Sequence File
Decennial Statistical Studies Division
Evaluations & Experiments Coordination Branch
Field Representative
Group Quarters
Group Quarters Advance Contact
Group Quarters Enumeration
Housing Unit
Integrated Project Team
Master Address File
Nonresponse Follow Up
Production Control System
Research & Methodology Directorate
Regional Census Center
United States Post Office

XIV. References
Butler, Nicole “Field Test on Revised and New Definition of College/University Student
Housing.” 2017 American Community Survey Research and Evaluation Report, August 2017.
Graber, Jessica E. “Findings and Recommendations from Group Quarters Student Housing
Research.” Center for Survey Measurement, U.S. Census Bureau, May 2016.
Holzberg, Jessica L. “Findings and Recommendations from Group Quarters (GQ) Student
Housing Qualitative Interviews.” Center for Survey Measurement, U.S. Census Bureau,
November 30, 2015.
Kuwik, Chris, Bonnie Moore, and Kim Jonas. “Recommendations for the NHIS College Housing
Fame, Sample and Operations.” Demographic Statistical Methods Division, U.S. Census
Bureau, April 22, 2014.
11

Evaluation of the GQAC Operation,Version 2.1

Moore, B., 2013. Improving Coverage of New College Housing in the Group Quarters Frame for the
Household Surveys. In JSM Proceedings, Government Statistics Section. Alexandria, VA: American
Statistical Association.

12