2016 Census Test OMB Supporting Statement Part B_final

Department of Commerce

United States Census Bureau

OMB Information Collection Request

2016 Census Test

OMB Control Number 0607-XXXX

Part B – Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The Census Bureau will conduct a mandatory survey in 2016 to test data collection methods for the 2020 Census. To conduct the 2016 Census Test, the Census Bureau will contact up to 250,000 housing addresses in each of two selected areas; one within parts of Los Angeles County, California, and one within parts of Harris County, Texas. The 2016 Census Test includes a Nonresponse Followup (NRFU) operation that will include a subset of approximately 60,000 housing units in each area.

All sampled addresses within the areas in the two sites will be included in the initial sample mailouts. To reduce burden on the public, any housing units previously selected for the 2016 American Community Survey (ACS) or the 2015 National Content Test will be excluded from sample selection. No known group quarters (GQ) will be selected for the sample.

The Census Bureau estimates an overall self response rate of 55 to 60 percent in the Los Angeles County site and 45 to 50 percent in the Harris County site. These estimates are based on analyses from prior testing, including the 2012 National Census Test, the 2014 and 2015 Census Tests, the 2010 Census, and literature reviews of other studies of self response methods.

2. Procedures for Collecting Information

Census Day for the 2016 Census Test will be April 1, 2016. The test will employ four response modes: Internet self response on a secure web site, self response using a paper questionnaire, Computer Assisted Telephone Interviewing (CATI) via Telephone Questionnaire Assistance, and Computer Assisted Personal Interviewing (CAPI) in NRFU.

As in previous censuses and tests, the Census Bureau will accept responses from respondents who do not have a unique Census ID for their living quarters. These responses, designated as Non-ID responses, will be checked against the Census Bureau’s address frame. In addition, Census staff will review each address not matched during automated processing as part of a manual Non-ID processing operation. This may require contacting respondents in some instances to obtain further address information to help obtain a match to the frame, or to request assistance to accurately locate their living quarters in the absence of a match. Finally, an estimated 5,000 of the Non-ID responses may be selected for a follow-up interview to validate the response data. This operation will be conducted concurrently with NRFU.

Self Response Operation for 2016 Census Test

All addresses in each of the selected portions of Los Angeles County, CA and Harris County, TX will be contacted to participate in the test, except those addresses excluded from the sample because of their inclusion in previous tests or surveys.

On or about March 23, 2016 (mailout date of March 21, 2016), five experimental panels of addresses will receive direct invitations to respond to the Census test, either via the Internet or by calling the Telephone Questionnaire Assistance number to provide their responses via a telephone interview. The mailout sample size will be equal for each of the five panels. One panel will receive a bilingual “frequently asked questions” sheet as an insert or complement to the letter. The bilingual design is three-fold: English/Spanish with sentences in Chinese and Korean on where to get more information in those languages, English/Chinese with sentences in Spanish and Korean on where to get more information in those languages, and English/Korean with sentences in Spanish and Chinese on where to get more information in those languages. Another panel will receive a multi-lingual brochure with information on the outer envelope designed to assist respondents who speak languages other than English. Three days after the initial contact notification, the Census Bureau will send a reminder message (by mail as a letter or postcard) to all sampled addresses.

A second reminder (mail postcard) will be sent to non-respondents the following week. Approximately three weeks after the initial notification, the Census Bureau will send a final reminder with a paper questionnaire (one of three bilingual questionnaire versions) to non-respondents, asking them to complete the census questionnaire and return it in the business reply envelope provided. They will also be informed they can still choose to respond online or by telephone.

One panel will receive a questionnaire and the cover letter in the first contact with reminders following. A second questionnaire will be sent to housing units who do not respond by paper, telephone, or through the Internet in the fourth contact.

Nonresponse Followup

Between May 2 and June 21, 2016, Census enumerators will contact the non-responding housing units to conduct the NRFU operation. The enumerators will determine the Census Day occupancy status of the housing units, and will enumerate the members of housing units determined to be occupied. Data will be collected using a smartphone application.

Enumerators will be trained on how to convince reluctant respondents to participate, such as reassuring them of their data privacy, and explaining the goals of the test and its importance in developing a cost effective 2020 Census. Enumerators will also be trained on how to verify whether an address is vacant, determine if an address has changed, or whether the address refers to something that no longer meets the Census Bureau’s definition of a housing unit.

The data collection device will contain instruments translated into languages other than English. Cases where the enumerator and respondent do not speak the same language will be reassigned to bilingual enumerators with the required language skills. Efforts will be made to recruit and assign bilingual field staff for areas known to be bilingual. The specific languages that have a high prevalence in these areas will be determined using American Community Survey (ACS) results and translations will be completed in advance of data collection. “Notice of Visit” forms left by enumerators will also be translated into additional non-English languages and will include Telephone Questionnaire Assistance numbers and the web address for the Internet collection site. Additional information about language materials is provided in supplemental materials titled, “Language Materials Provided for 2016 Census Test.”

Under specific circumstances, data may be collected from proxy respondents, such as a hard refusal by a household member. Sampled addresses believed to be occupied become proxy eligible after a variable number of unsuccessful contact attempts. Sampled addresses believed to be vacant or not a housing unit will become proxy eligible for a period of time after each unsuccessful contact attempt.

As described in OMB Supporting Statement Part A, we will use administrative records to reduce the workload for the Nonresponse Followup operation. The 2016 Census Test will use a similar predictive identification approaches to indentify vacant and occupied units as were used in the 2015 Census Test. The 2015 Census Test used predictive models with optimization techniques to identify vacant and occupied units. This was a change from the approach used in the 2014 Census Test where rule-based approaches were used. The Census Bureau will continue to research the predictive identification approaches in the 2016 Census Test. For vacant and occupied cases removed from the 2016 Census Test workload as a result of our administrative records modeling, we will perform a supplemental letter mailout, giving the household a final chance to respond before we resort to enumeration using administrative records.

A sample of completed NRFU cases will be selected for a quality control reinterview, where a different enumerator will attempt to make contact with and collect data from the original respondent, that will later be compared to the originally received data. This comparison process will help to determine potential falsification by NRFU enumerators, and ensure quality in the NRFU operation.

Coverage Reinterview

Decisions associated with how we collect housing unit population and the approaches that the Census Bureau takes to creating an accurate roster of persons at a housing unit on census day are integral to an accurate census count. In the 2016 Census Test, the Census Bureau is exploring various approaches to the presentation of our residence rule and the utility in asking undercount and overcount questions to ensure an accurate enumeration. The quality of the final household roster created from the panels with experimentally applied questions will be evaluated by a coverage reinterview conducted by telephone to the sample of 24,500 housing units. Specifically, the reinterview will re-contact 20,000 Internet responders to determine if any people may have been left off the roster or erroneously included on the roster during the initial response in order to determine the quality of the household roster that results from each experimental version of the questionnaire. The reinterview will also re-contact 3,000 NRFU responders and 1,500 paper responders using the same reinterview questionnaire so that the Census Bureau can estimate the quality of household rosters collected in these other modes. The reinterview will be conducted via a Computer Assisted Telephone Interviewing (CATI) operation in the Census Bureau's call centers. Telephone interviewers will be specially trained to ask probing questions and elicit responses from respondents that will help to determine the status of all members of the housing unit. The different treatments to collecting the initial roster will be assessed using the reinterview data. The training will be a mix of online and in-person and will focus on role-play, common situations and overcoming respondent reluctance. Bilingual telephone operators will be available to complete interviews in multiple languages.

Focus Groups

Participant recruitment for focus groups will be conducted by Census Telephone Center staff using existing phone number information we have for a sample of addresses in the 2016 Census Test. Staff will use screener questions to ensure that focus group participants: 1) personally responded to the 2016 Census Test, and 2) are not employed by the Federal Government.

The focus groups will be conducted in person, and all interviews will be tape-recorded to facilitate later analyses of the results. Participants will be asked to sign consent forms and give permission to be recorded. All participants will be informed that their response is voluntary and that the information they provide is confidential. Respondents will receive a $75 stipend after the group concludes.

3. Methods to Maximize Response

Self Response

One of the primary objectives during the 2016 Census Test is to experiment with different methods of maximizing self response. Each strategy is designed to increase the number of households that respond online and to gain cooperation of non-English speaking respondents.

Four of the five panels incorporate the “Internet Push” approach, in which sample housing units will not initially receive a paper questionnaire. Sampled households receive some combination of (1) a letter that describes the survey and includes instructions about how to respond to the survey online (including a non-English insert to complement the instructions); and (2) a multi-lingual brochure or insert and/or envelope with (bi)multi-lingual banner. The full contact strategy panel design is summarized in Table 1.

The fifth panel represents the Internet Choice option for respondents who receive in the first contact both the paper questionnaire and the explanation of the choice to respond on the Internet.

Paper questionnaires will be sent to non-respondents in each of the panels about three weeks after the initial contact notification. The paper questionnaire is an opportunity for those without Internet access or a telephone, or with reservations about providing their data online or by phone, to respond. However, households that receive the paper questionnaire will still be given another opportunity to respond online or on the phone.

The full contact strategy panel design is summarized in Table 1.

Table 1. Self Response Contact Strategy Panel Design

	Panel	Contact #1 Mailout March 21	Contact #2 (3 days later) March 24	Contact #3* April 4	Contact #4* April 11
1	Internet Push (with alternate wording for benefits to community)	Letter	Postcard	Postcard	Mail Q’nnaire +Letter
2	Internet Push with reminder letter	Letter	Letter	Postcard	Mail Q’nnaire +Letter
3	Internet Push with language brochure (URL on envelope)	Brochure +URL	Postcard	Postcard	Mail Q’nnaire +Brochure /URL
4	Internet Push with language insert	Letter +Insert	Postcard	Postcard	Mail Q’nnaire +Letter / Insert
5	Internet Choice	Mail Q’nnaire +Letter	Postcard	Postcard	Mail Q’nnaire +Letter

* Targeted only to nonrespondents.

Note: materials will be bilingual (English/Spanish, or English/Korean or English/Chinese, based on ACS estimates for each census tract and the estimates for data on “Ability to Speak English”)

The Internet data collection instrument will also include various ways to collect and confirm the number of persons residing at an address.  Respondents will see one of three screens about the existence of people on the roster: one that displays the residence rule and asks for the number of people in the household, one that asks for the number of people who live in the household but puts the residence rule in the help text, and one that asks if any other people live at the household with the residence rule in the help text.  After the names of the roster members are collected, the respondent will then see one of three series of undercount detection questions: one series asks about additional people on two separate screens, another series asks about additional people on only one screen, and a third asks no undercount questions.  After the demographic items are collected, the respondent will then see either one person-based overcount question that asks if a specific person stayed in a number of types of places or no overcount question at all.  Respondents who receive no undercount question will always receive no overcount question, and respondents who receive no overcount question will always receive no undercount question. The quality of the final household roster created from these experimentally applied questions will be evaluated by a coverage reinterview conducted by telephone that will contain extensive probes about missed roster members or other places that people sometimes stay. Table 2 illustrates the paths or panel design within the Internet instrument.

Table 2. Coverage Content Testing (Internet only)

Initial Roster Creation	Undercount Probes	Overcount Probes
P1 - Rules and population count question	U1 - 2015 design of two separate questions	V1 – Person-based question
P1 - Rules and population count question	U2 – One question	V1 – Person-based question
P1 - Rules and population count question	U3 – No questions	V2 – No questions
P2 - Just the population count question with rules in the help text	U1 - 2015 design of two separate questions	V1 – Person-based question
P2 - Just the population count question with rules in the help text	U2 – One question	V1 – Person-based question
P2 - Just the population count question with rules in the help text	U3 – No questions	V2 – No questions
P3 - The “others” question with rules in the help text	U1 - 2015 design of two separate questions	V1 – Person-based question
P3 - The “others” question with rules in the help text	U2 – One question	V1 – Person-based question
P3 - The “others” question with rules in the help text	U3 – No questions	V2 – No questions

On the Internet instrument only, we will include testing on wording and terminology for response categories in the race/ethnicity and relationship questions. The change to the race and ethnicity question is in the “Black and African American” category, where we will compare the use of “American” to the abbreviated “Am.” This addresses a problem posed by this abbreviation found in software that makes the web accessible for persons with visual disabilities and supports Section 508¹ compliance for the Census Internet data collection instrument.

For the relationship question, the 2016 Census Test will include testing new response categories for opposite sex and same sex husband/wife/spouse and unmarried partner. In addition, the Internet data collection instrument will also provide two versions of the relationship question, with one version eliminating the response categories associated with unrelated household members (“roomer or boarder” and “housemate or roommate”). Tables 3 and 4 illustrate these paths or the panel design within the Internet data collection instrument.

Table 3. Race and Ethnicity Content Testing (Internet only)

A1	Race: Black or African Am. (control)
A2	Race: Black or African American

Table 4. Relationship Content Testing (Internet only)

R1	Relationship: Control
R2	Relationship: Remove “roomer or boarder”
R3	Relationship: Remove “roomer or boarder” and “housemate or roommate”

4. Tests of Procedures or Methods

In developing these tests, the Census Bureau consulted with a variety of stakeholders, including, but not limited to, academics, national researchers, community and organizational leaders, and the Census Bureau’s Advisory Committees. In addition, external consultants from the National Academy of Sciences shared information about other relevant studies and provided quarterly feedback about the Census Bureau’s research plans and objectives for the 2016 Census Test. The objectives for the 2016 Census Test are largely building on findings and lessons learned from the 2015 Census Tests. The results from these tests will be shared widely with decennial census stakeholders.

To help evaluate and assess the results of these tests, the Census Bureau will conduct debriefings with enumerators and others who work on the tests, and will conduct focus groups, as described earlier in this supporting statement. Cognitive testing participants will be recruited from outside the Census Bureau to provide their views on the wording of questionnaire items, the use of text messages, and the navigation of the Internet application. The cognitive testing participants will be asked to provide their reactions to the content of the messages and the mode of delivery in order to test for comprehension, saliency, and sensitivity of the messages.

5. Contacts for Statistical Aspects and Data Collection

For more information, contact Robin A. Pennington (301) 763-8132, robin.a.pennington@census.gov.

1 Section 508, an amendment to the U.S. Workforce Rehabilitation Act of 1973, is a federal law mandating that all electronic and information technology developed, procured, maintained, or used by the federal government be accessible to people with disabilities

5