05__2014_Census_Test_OMB_Supporting_Statement_B_(05-09-2014) Final

SUPPORTING STATEMENT

U.S. DEPARTMENT OF COMMERCE

U.S. CENSUS BUREAU

2014 Census Test

OMB CONTROL NUMBER 0607-XXXX

Part B – Collections of Information Employing Statistical Methods

1. Universe and Respondent Selection

The Census Bureau will conduct a mandatory survey in 2014 to test data collection methods for the 2020 Census. To conduct the 2014 Census Test, the Census Bureau will select a sample of up to 190,500 housing addresses. The test will be limited to housing units in mailout/mailback areas in a defined geographic area. The area selected will contain two strata defined by response propensities. Group quarters and units in other types of enumeration areas are not eligible. Address sampling will be a two-part design: select the areas containing the two strata and then sample the addresses within those areas. Further, to reduce burden on respondents, any housing units selected for the 2014 American Community Survey (ACS) will be excluded from sample selection. No quality control operations will be tested during this census test. There will be a Nonresponse Followup (NRFU) portion of the 2014 Census Test that will include no more than 50,000 housing units. If response rates are not as high as we expect resulting in a larger NRFU workload, we will subsample so as not to exceed 50,000 housing units NRFU.

The Census Bureau will attempt to select different block groups that are based on a diversity of socio-economic populations. The sample will require use of a supplemental address frame, which includes addresses from the Census Bureau’s living quarters inventory that have been linked to an email address or phone number via commercial vendor sources.

The Census Bureau estimates an overall self-response rate of 55 percent in the portion of the site designed to compare NRFU strategies, and self- response rates of 35 percent to 55 percent across the eight panels designed to compare different self-response strategies.  These estimates are based on analyses from prior testing, including the 2012 National Census Test, and literature review from other studies of self-response methods. 

The sample size for the panels designed to compare different self-response strategies in the 2014 Census Test is based on a power analysis of expected measurable differences on key measures for response rates and analysis of responses by demographic group. With an alpha level of 0.10 and power of 0.80, we expect to be able to detect statistically significant differences between two contact strategy panels of 1.2 percentage points. The self-response panels will be stratified so as to take into account the supplemental contact frame information (presence of email address and telephone number). One half of each contact strategy panel will be randomly selected from those housing units with at least one landline phone number and at least one email address. Doing so will ensure comparability between panels for statistical analysis by using the same sampling universe. The remaining half of each panel will be randomly selected based on the needs of each individual panel (for example, the remaining half of the cold contact email invitation panel (E1) will be selected from housing units with at least one email address regardless of whether or not they have a landline phone number).

The sample size for the area of the test designed to compare nonresponse followup strategies in a specific geographic area is driven by the need to ensure a sufficient workload for NRFU for comparing key metrics of efficiency and quality, such as miles per case, productivity rates (measured in cases completed per hour) and completion rates.

2. Procedures for Collecting Information

The 2014 Census Test Census Day will be July 1, 2014. The test will employ four response modes; Internet self-response on a secure web site, self-response on a paper questionnaire, Computer Assisted Telephone Interviewing (CATI) for both self-response and NRFU, and Computer Assisted Personal Interviewing (CAPI) in NRFU.

The Census Bureau will accept responses from respondents who do not have their unique Census ID at the time of the response. By accepting responses without a unique ID, the Census Bureau is testing its ability to increase self-response rates and reduce respondent burden. Non-ID cases need to be validated against the Census Bureau’s address frame. The key survey measure for this data collection is the number of Non-ID cases successfully resolved (i.e., matched to an address on the frame) through automated matching and geo-coding batch processes. Additionally, the test provides an early opportunity to test address enhancements to the automated Non-ID processing conducted by the Census Bureau.

All addresses in the 2014 Census Test will be sent a notification asking them to respond to the census questionnaire on a secure Internet site. The notification in three panels will contain a Census ID that is unique to the address in sample. The initial notification in five panels will include the link for Internet data collection, but these respondents will not use an ID and are considered part of the Non-ID workload. One week after the initial contact notification, the Census Bureau will send a reminder message (by mail postcard, email, or text message). A second reminder will be sent (by mail postcard, email message, or text message) the following week. Approximately three weeks after the initial notification, the Census Bureau will send a final reminder along with a paper questionnaire to non-respondents. Respondents will be instructed to complete the census questionnaire and return it in a business reply envelope provided, or they can still choose to respond online or by telephone. The respondents in the sample/panel for pre-registration are invited to provide their prefered contact information (email or cellphone/text) beginning on June 5 with the Census Bureau sending the initial contact (and followup reminders) according to their preferred contact method. Refer to Section 3 (Methods to Maximize Response), below, for more details on the self-response options/panels.

The NRFU period falls between August 14 and September 25, starting approximately 6 weeks after the initial mailing date of July 1. During NRFU, census enumerators will contact non-responding housing units to conduct the NRFU operation. The enumerators will determine the Census Day status of the housing units, and collect the test questionnaire data for occupied housing units. A difference from procedures in the 2010 Census is that the enumerators will use an automated handheld device for the data collection. Another difference is that telephone numbers will be provided to enumerators (when possible) with their case assignment. These numbers will come from a supplemental contact frame developed from commercial data sources obtained and compiled by the Census Bureau.

During this nonresponse followup field operation, newly recruited and hired enumerators will contact non-responding households in person or by telephone. However, even though the enumerators for this test will be newly recruited and hired, they might come with previous decennial or survey interviewing experience. For this test, The Census Bureau will be providing traditional paper-based verbatim in-classroom training as was done for the 2010 Census, but we are looking to conduct more automated trainee-administered training for the 2020 Census. The Census Bureau is unsure what type of workforce to expect in 2020 – as much depends on the economy – but changes to the work (automated instruments, etc.) may attract different applicants than have previously been seen in the census.

Some enumerators will be able to use the telephone numbers provided to them with assignment to try to collect the data by phone using the CAPI instrument. However, enumerators are encouraged to make a personal visit interview their main contact mode unless instructed to do something different. Enumerators will be trained on how to convince reluctant respondents to participate, such as reassuring them of the privacy of their data and explaining the benefits of this work. The automated instrument will not contain any translations into Spanish or any other languages. Cases with language barriers can be reassigned to be completed by bilingual enumerators with those language skills and these enumerators will translate the questionnaire at the time of the interview. Efforts will be made to recruit and assign bilingual field staff for areas known to be bilingual. If a household member for a sample address is still not willing to provide information, enumerators can utilize proxy respondents (e.g., nearest neighbors or others who might know about the housing unit such as building managers of apartment buildings, etc.) to gather the necessary information. Enumerators will also be trained on how to verify whether an address is vacant, or determine if an address has changed, or whether the address or housing unit no longer exists.

3. Methods to Maximize Response

Self-Response

Experimenting with different methods of maximizing self-response to the decennial census is one of the primary objectives of the 2014 Census Test. The unifying theme underlying each of the strategies is the goal of pushing households to respond online. Eight contact strategy panels will be implemented utilizing various strategies. The eight panels will be randomly assigned to each housing unit, after taking into account the availability of email address and phone number on the supplemental contact frame. Half of the panels will receive up to four contacts, while the other half will receive up to five contacts.

All panels incorporate the “Internet push” approach, in which sample housing units will not initially receive a paper questionnaire. Instead, sample households either receive (a) a letter that describes the survey and includes instructions about how to respond to the survey online; or (b) an email with a description of the survey and a prompt to respond to the survey online with the URL for Internet collection. Households in three of the eight panels will also receive a pre-notice, notifying them of the forthcoming invitation to participate in the 2014 Census Test

Traditional paper questionnaires will be sent to nonrespondents in the eight panels about three weeks after the initial contact notification. The paper questionnaire is an opportunity for those without Internet access or a telephone, or with reservations about providing their data online, to respond to the data collection. Households that receive the paper questionnaire will still have the option of responding to the survey online or through the telephone questionnaire assistance line. The eight self-response contact strategy panels are:

Pre-registration: Sample households in this panel will receive a pre-notice postcard inviting them to visit the 2014 Census Test “Notify Me” website to sign up to receive a notification when it is time to respond to the 2014 Census Test. Pre-registrants can opt to be notified by text or email. Notifications will be sent up to three times in the chosen format. If there is no response after the third notification/reminder, a paper questionnaire will be sent. Sample households in the pre-registration panel who do not pre-register will be sent the standard Internet Push mail materials (described below).

U-Count: This panel is designed to test the Census Bureau’s automated matching and geo-coding processes for cases without a pre-assigned, unique Census ID. Sample households in this panel will receive standard Internet Push mail materials, however they will not be provided with a Census ID. Respondents enter the website using their email address and provide their mailing address as part of the response process. After the initial contact letter, sample households receive up to two additional reminder postcards by mail. If there is no response, a paper questionnaire will be sent. Magnitude of respondent engagement in Non-ID response will also be studied.

Internet Push: For this panel, the first contact is a letter along with an instruction card on how to respond online using the household’s Census ID. After the initial contact letter, sample households receive up to two additional reminder postcards by mail. If there is no response, a paper questionnaire will be sent.

Internet Push with Email as 1^st Reminder: Sample households in this panel receive the standard first contact letter along with an instruction card on how to respond online using the household’s Census ID. Email addresses are obtained from the supplemental contact frame, developed from commercial data sources. After the initial contact letter, sample households will receive a reminder by email. If no response is received after the 1^st reminder email, a 2^nd reminder will be sent by mail (postcard). If there is no response, a paper questionnaire will be sent.

Internet Push with AVI (Automated Voice Invitation): Sample households in this panel receive standard Internet Push materials. The first contact is a letter along with an instruction card on how to respond online using the household’s Census ID. After the initial contact letter, sample households receive up to two additional reminder postcards by mail. If there is no response, a paper questionnaire will be sent. If there is still no response, an AVI phone call will be made to the household.

Cold Contact Email Invitation and 1^st Reminder: Sample households in this panel receive their first contact by email. Email addresses are obtained from the supplemental contact frame, developed from commercial data sources. The email contains instructions and a link for responding online. A second email is sent about one week later. If no response is received, the household is sent a postcard reminder. Finally, if there is still no response, a paper questionnaire will be sent.

Letter pre-notice, Email Invitation and 1^st Reminder: Sample households in this panel receive a pre-notice letter in the mail letting them know about the upcoming data collection. These households are then sent the initial survey information (description and website link) by email. Email addresses are obtained from the supplemental contact frame, developed from commercial data sources. A second email is sent about one week later. If no response is received, the household is sent a postcard reminder. Finally, if there is still no response, a paper questionnaire will be sent.

AVI Pre-notice and Email Invitation: Sample households in this panel receive a pre-notice phone call by AVI. These households are then sent the initial survey information (description and website link) by email. Email addresses are obtained from the supplemental contact frame, developed from commercial data sources. A second email is sent about one week later. If no response is received, the household is sent a postcard reminder. Finally, if there is still no response, a paper questionnaire will be sent.

The full contact strategy panel design is summarized in the following table:

Contact Strategy Panel Design

Panel	Sample Size	Pre-Notice	Contact 1 (June 23)	Contact 2 (July 1)	Contact 3* (July 8)	Contact 4* (July 15)	Contact 5* (July 22)
1) Pre-registration (P1)	10,000	Postcard invite to register (June 5)	Choice of email or text	Choice of email or text	Choice of email or text	Mail questionnaire
2) U Count (U1)	10,000		Letter (no ID)	Postcard (no ID)	Postcard (no ID)	Mail questionnaire
3) Internet Push (M1)	110,250 (NRFU portion) 10,000 (Optimized Self-Response contact strategy panel)		Letter	Postcard	Postcard	Mail questionnaire
4) Internet Push with Email as 1st Reminder (M4)	10,000		Letter	Email	Postcard	Mail questionnaire
5) Internet Push with AVI as 3rd Reminder (M5)	10,000		Letter	Postcard	Postcard	Mail questionnaire	AVI
6) Cold Contact Email Invite and 1st Reminder (E1)	10,000		Email	Email	Postcard	Mail questionnaire
7) Letter Prenotice, Email Invite and 1st Reminder (E2)	10,000	Letter (June 17)	Email	Email	Postcard	Mail questionnaire
8) AVI Prenotice, Email Invite and 1st Reminder (E3)	10,000	AVI (June 17)	Email	Email	Postcard	Mail questionnaire

* Targeted only to nonrespondents.

AVI = automated voice invitations

Non-response Followup (NRFU)

Another goal of the 2014 Census Test is to evaluate the difference in mileage and productivity rates (measured in cases completed per hour) between the control panel and alternative NRFU treatments across the other three panels. The NRFU panels and associated mailout sample size, as well as the NRFU field workload, are shown in the table below. Throughout this section, when describing the use of administrative records, it should be noted that administrative records will be supported with commercially-available data when possible, but that we will continue to refer to these data as only “administrative records”.

2014 Census Test Nonresponse Followup Panels

		Staff Training		Workload Stratum 1 District of Columbia	Workload Stratum 2 Montgomery County	Total
Treatment 1 (NR1)	Control – similar to 2010 NRFU operations	Control	NRFU Mailout Size NRFU Size NRFU Fieldwork	14,000 7,000 7,000	17,500 7,000 7,000	31,500 14,000 14,000
Treatment 2a (NR2)	Reduced Contact Strategy – 2 PV contacts (PV-T-PV/Proxy)	Fixed	NRFU Mailout Size NRFU Size NRFU Fieldwork	14,000 7,000 7,000	17,500 7,000 7,000	31,500 14,000 14,000
Treatment 2b (NR4)	Reduced Contact Strategy/ AdminRec2 Interaction (remove occupied and unoccupied cases)	Fixed	NRFU Mailout Size NRFU Size NRFU Fieldwork	8,400 4,200 3,150	10,500 4,200 3,360	18,900 8,400 6,510
Treatment 3 (NR7)	Adaptive Design/AdminRec2 Interaction (remove unoccupied cases, conduct one personal visit, then remove occupied cases)	Adaptive	NRFU Mailout Size NRFU Size NRFU Fieldwork	12,600 6,300 5,387	15,750 6,300 5,686	28,350 12,600 11,073
		Total	NRFU Mailout Size NRFU Size NRFU Fieldwork	49,000 24,500 22,537	61,250 24,500 23,046	110,250 49,000 45,583

NOTES:

All NRFU cases will result from nonresponse to the Internet Push Panel (M1) from the self-response portion of the test.

The original design of this test included only a control (Treatment 1) and a fixed contact strategy (Treatment 2a). Over the course of months, we modified the design of the test to incorporate findings from the 2020 Census Research and Testing teams. First, we added an adaptive contact strategy (as employed in Treatment 3). Then, we split the fixed and adaptive contact strategies by differing uses of administrative records – resulting in 7 panels. We then designed the panel sizes to fit our testing needs for those seven panels. Based on additional findings from the 2020 Census Research and Testing teams, we increased the power of certain panels by collapsing the seven panels into the four panels shown in the table above. Even though we were able to collapse panels, we could not move sample between panels as we had already begun delineating field activities and recruiting for positions based on the seven panel design thereby resulting in different panel sizes. Still, we are able to meet our testing needs with the current panels and panel sizes.

Treatment 1 (NR1) is the control panel. It will employ similar NRFU procedures to those used in the 2010 Census. The instructions to enumerators will be to make no more than six contact attempts with the first being in-person and no more than two additional attempts made in person. There are two major differences from the 2010 Census – enumerators will be using an automated instrument instead of paper and they may be provided telephone numbers (if available from the supplemental contact frame) for their cases.

Treatment 2a (NR2) will employ a strategy to reduce the number of contact attempts in a fixed manner. The instructions to enumerators will be to make no more than two in-person contact attempts, with the first contact being a personal visit, the next being a telephone contact if possible and then a final personal visit. Enumerators will be using an automated instrument that will remove cases from their workload after the second in-person contact attempt. Telephone numbers (if available from the supplemental contact frame) will be provided to enumerators.

Treatment 2b (NR4) will employ the same fixed strategy from Treatment 2a (NR2) to reduce the number of contacts for each case. In addition, the initial workload for NRFU will be reduced to exclude any housing units that were identified as vacant using administrative records AND any occupied housing units that could be reasonably enumerated using administrative records. The method to identify a vacant housing unit is a lack of administrative records data coupled with the mailing for that case being returned as undeliverable as addressed (UAA) because of vacancy, and the method to identify housing units that can be enumerated by administrative records is that multiple administrative record sources have the same number of household members counted at the housing unit. In 2014, there will not be a field followup operation to validate the results of this procedure, but we will analyze information from other panels where housing units that can be enumerated with administrative record are not removed from the workload. With this information, we can compare the results we would have predicted for those housing units to the information collected in the field.

Treatment 3 (NR7) will employ a strategy to reduce the number of contacts in an adaptive manner for each case (referred to as adaptive design). The instructions to enumerators will be to attempt to contact the “priority” cases identified for that day on their case list – either by telephone, if applicable, or by personal visit. These priorities will be determined using information from administrative records and the contact history information for each individual case. This strategy will be similar to the strategy employed in the 2013 Census Test. The goal of using an adaptive design strategy is to tailor contact strategies to individual housing units. The initial workload for NRFU in this treatment will be reduced to exclude any housing units that were identified as vacant using administrative records. For all housing units that have not been removed due to vacancy, enumerators will make one personal visit. After that initial personal visit, the remaining NRFU workload will be further reduced to remove any housing units that could be reasonably enumerated using administrative records. The methods to reduce the NRFU workloads using administrative records are the same as those employed in Treatment 2b (NR4). For remaining NRFU cases at that point, we will employ the adaptive design contact strategy described above.

For all NRFU panels mentioned above, if the enumerator cannot reach someone during an in-person contact attempt, the enumerator will leave a Notice of Visit with information for the respondent to either respond via Internet or Telephone Questionnaire Assistance (TQA) for each unresolved case. If the NRFU respondent does respond via the Internet or TQA after receiving the Notice of Visit, we will remove those cases from the NRFU workload on a daily basis so that enumerators will no longer see that case as part of their assignment.

4. Tests of Procedures or Methods

The Census Bureau consulted with a variety of data users including, but not limited to, academicians, national researchers, and the Census Bureau’s Advisory Committees.

Cognitive testing participants will be recruited from outside the Census Bureau to provide their views on the wording of the email and text messages, but will be covered under the Center of Survey Measurement’s Generic Clearance for Pretesting.  The respondents will be asked to provide their reactions to the content of the messages and the mode of delivery, in order to test for comprehension, saliency, and sensitivity of the messages. Usability testing participants will be recruited from outside the Census Bureau to provide feedback on the Internet data collection instrument. In addition, external consultants from the National Academy of Sciences shared information about other relevant studies and provided quarterly feedback about the Census Bureau's research plans and objectives for the 2014 Census Test.

The paradata collection screens used in this NRFU operation are not public-facing. They are adapted from the Contact History Instrument employed by current Census Bureau Field Representatives for the American Community Survey and other current surveys conducted by Census. Additional questions are adapted from the CATI instrument used by census interviewers for telephone contacts from call centers. This adaptation of questions will undergo usability testing before being released for the operation.

Analysis of Content Testing

The 2014 Census Test will contain some content testing within the Internet data collection instrument including two versions of a combined race and Hispanic origin question and separate race and Hispanic origin questions, and new response categories for the relationship question (for opposite sex and same sex husband/wife/spouse and unmarried partner) compared to the traditional version.

Because the 2014 Census Test is a site test that is not nationally-representative, the analysis of the new terminology and response categories is a secondary goal of the test and is not specifically intended to draw inferences regarding differences between question versions. In particular, since the sample was not designed to reach certain small demographic groups, such as same sex relationships, we do not expect to detect any meaningful differences for the relationship questions. For the race and Hispanic origin questions, depending on the category of interest (e.g., percent Hispanic or Latino, percent White, etc.) with an alpha level of 0.10 and power of 0.80, we will be able to detect statistically significant differences in the response distribution between two question versions of 0.7 to 1.3 percentage points. The aim, however, is to deploy the new versions for the first time in a field test to refine for further research and development and not to draw definitive conclusions at this time.

5. Contacts for Statistical Aspects and Data Collection

For more information, contact Erin Love (301-763-2034; erin.s.love@census.gov), Magdalena Ramos (301-763-4295; magdalena.ramos@census.gov) for the NRFU aspects of the test, Jane Ingold (301-763-4646; jane.h.ingold@census.gov) for the Self-Response aspects of the test, or Frank McPhillips (301-763-8781; francis.c.mcphillips@census.gov) for the Non-ID aspects of the test.

Page 7 of 7