OMB Memo

Generic Information Collection Request

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct the fourth round of testing of the Small-Scale Response Testing (SmaRT) Program. We are seeking approval for cognitive and usability testing conducted by Census Bureau staff.

Purpose: This project is modeled after the small-scale mailout testing program the Decennial directorate established in 2015 to prepare for the 2020 Census. The plan for the 2030 Census is to initiate and maintain an ongoing and iterative small-scale response testing (SmaRT) program by using a small, thoughtfully selected sample throughout the decade. This is the fourth round in the 2024 SmaRT test. Objectives regarding cognitive testing of mailing materials and usability testing of the Decennial online instrument (2024 National Census Survey) are outlined below:

Mailing Materials

1. Identify major comprehension issues and review spontaneous feedback of the mailing materials focusing on:

   1. Envelope message about the opportunity to record a message to future generations

   2. “Respond by” dates

   3. Census ID format

   4. Census ID location

2. Suggest revisions addressing comprehension issues

Decennial online instrument (2024 National Census Survey)

1. Test wording of a question that allows respondents to enter a message to future generations (sometimes referred to as the Time Capsule question)

2. Test a method of asking for household member contact information to forward the survey to those household members who were not originally listed on the roster

3. Find any usability problems which interfere with a respondent’s ability to answer the 2024 National Census Survey questions accurately and with satisfaction

Decennial combined race question on paper (2024 National Census Survey)

1. Test “Enter, for example” instead of “Print, for example” for the write-in instruction for the combined race/ethnicity question administered on the paper questionnaire. Participants will answer the combined race question for themselves on a piece of paper with the instruction of “Enter, for example” at the end of the session. 

Population of Interest: The planned cognitive and usability evaluation will assess and improve the user experience for the general population, and the following household (HH) composition characteristics:

1. Unrelated HH composition

2. Related HH composition

3. Single person HH

4. Two person HH

5. Apartment dwellers

We will have both English speakers and monolingual Spanish speakers participate in testing. We are interested in recruiting a diverse sample of participants. We are also interested in recruiting participants with a diversity of education levels.

Sample: We plan on testing with approximately 20 people total: ten English and ten Spanish speaking participants. We will aim to recruit participants that have not participated in previous testing for the decennial census; however, if we are struggling with meeting our recruitment goals, particularly for Spanish speaking participants, we may recruit participants who have participated in previous studies. We will recruit participants who have at least one year’s experience using the internet for tasks other than reading or writing email. Respondents that would be likely to complete the decennial census online will likely be familiar with the internet, so for this recruitment, familiarity with the internet is necessary.

We plan to do half of our testing in-person and half of our testing virtually for both languages. For the in-person sessions, participants will reside either in the metropolitan DC area or in the Houston and Brownsville areas of Texas. The remote session participants can live anywhere in the 50 U.S. states and Washington DC.

Language: Testing will be conducted in English and Spanish.

Timeline: Testing will be conducted in June and July 2024.

Questions: Most of the questions to be tested are the same as those included in the last round of SmaRT pretesting. There are two new questions. The screen that asks the respondent to enter contact information so the Census Bureau can forward the survey to individuals not included on the roster is found in the spec (Enclosure 2) as FORWARD_SURVEY (page 47). The time capsule question is found in the spec as TIME_CAPSULE (page 52). The entire questionnaire is programmed in Qualtrics. The Enter for example instruction on the combined race/ethnicity question will be on a piece of paper.

Protocol: The SmaRT testing of the mailing materials and the online instrument will employ a think-aloud protocol as well as retrospective probing. Participants will be asked for their consent to participate in the study and given the privacy notice (Enclosure 1). In-person participants will be given the mail materials and remote participants will have the mail materials mailed to their homes (Enclosures 8A, 8B, 8C, 8D, 8E, 8F, 8G, 8H). All participants will be asked to read through them, one by one, while thinking aloud; then they will be asked a series of debriefing questions (Mail Materials Protocol, Enclosures 3). They will then be asked to complete the 2024 National Census Survey (online questionnaire) while thinking aloud (Instrument Protocol, Enclosure 4). Then they will be asked satisfaction questions about their experience (Enclosure 5). They will answer debriefing questions (Enclosure 6). They will answer questions to assess their understanding of the census (Enclosure 7). And finally, they will answer the combined race/ethnicity question on piece of paper while thinking aloud and answer a concurrent probe on their understanding of what “Enter, for example” means to them (Enclosure 9).

Below is a list of materials to be used in the study.

1. Informed Consent and Privacy Notice (Enclosure 1)

2. Specifications of the updated decennial census (Enclosure 2)

3. Protocol for cognitive testing of mailing materials including tasks & debriefing probes (Enclosure 3)

4. Protocol for instrument testing (Enclosure 4)

5. Satisfaction questions (Enclosure 5)

6. Debriefing screens (Enclosure 6)

7. Knowledge Check (Enclosure 7)

8. Mailing materials (Enclosures 8A-8N)

9. Combined race/ethnicity question and debriefing probe (Enclosure 9)_

Length of interview: The session should take 90 minutes for each of the 20 participants. We anticipate 5 minutes per participant for the technology check for remote participants.

Respondents will be screened using the Census Bureau’s generic screener. We estimate that we will screen 3 people for each successful recruit, resulting in 60 participants screened. The total estimated respondent burden for this request is 26 hours. See Table 1.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	60	5 minutes	5 hours
Tech check	10	5 minutes	1 hour
Usability Sessions	20	90 minutes	30 hours
Totals			36 hours

Use of Incentive: The total incentive for this project is $60 per participant. For virtual interviews, the test administrator will send the incentive money to the participant’s address using approved USPS priority mail. If the session is conducted in person, the test administrator will give the participant cash.

The contact person(s) for questions regarding data collection and statistical aspects of the design of this research is listed below:

Marcus Berger

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-8335

marcus.p.berger@census.gov