OMB Letter

Generic Information Collection Request

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct the fifth round of testing of the Small-Scale Response Testing (SmaRT) Program. We are seeking approval for cognitive and usability testing conducted by Census Bureau staff.

Purpose: This project is modeled after the small-scale mailout testing program the Decennial directorate established in 2015 to prepare for the 2020 Census. The plan for the 2030 Census is to initiate and maintain an ongoing and iterative small-scale response testing (SmaRT) program by using a small, thoughtfully selected sample throughout the decade. This is the fifth round of SmaRT testing. Objectives regarding cognitive testing of mailing materials and usability testing of the Decennial online instrument (2025 Census Survey) are outlined below:

Mailing Materials

1. Identify major comprehension issues and review spontaneous feedback of the mailing materials focusing on:

   1. An instruction on the paper questionnaire pushing large households to complete the census online instead of by paper.

   2. An alternative instruction for entering the age of infants less than a year old on the paper questionnaire.

   3. The answers to the frequently asked questions about how the census is used to direct federal funding and where to find more information about data privacy.

   4. Instructions for where to find the Census ID on the letter included with the paper questionnaire (the ID is not on the letter, but under the barcode on the questionnaire).

   5. A message about how the census can be used for ancestry research on the reminder letter.

   6. A new every-day direct mailer (EDDM postcard) as the second mailing.

   7. The placement of the instruction to turn the mailing over for Spanish content on the letters.

2. Suggest revisions addressing comprehension issues

Decennial online instrument (2025 Census Survey)

1. Test different versions of a satisfaction question at the end of the online census.

2. Test a close-ended follow-up question (OTHER_VACANCY) that would gather more information about different types of vacant addresses.

3. Test a FAQ search tool on the 2025 Census Survey census.gov landing page.

4. Find any usability problems which interfere with a respondent’s ability to answer the 2025 Census Survey questions accurately and with satisfaction.

Population of Interest: The planned cognitive and usability evaluation will assess and improve the user experience for the general population, and the following household composition characteristics:

1. Households with 6 or more members

2. Spanish speakers

3. Households with babies under a year old

4. Complex households

We will have both English speakers and monolingual Spanish speakers participate in testing. We are interested in recruiting a diverse sample of participants. We are also interested in recruiting participants with a diversity of education levels.

Sample: We plan on testing with approximately 25 people total: ten English and fifteen Spanish speaking participants. We will aim to recruit participants that have not participated in previous testing for the decennial census; however, if we are struggling with meeting our recruitment goals, particularly for Spanish speaking participants, we may recruit participants who have participated in previous studies. We will recruit participants who have at least one year’s experience using the internet for tasks other than reading or writing email. Participants who would be likely to complete the decennial census online will likely be familiar with the internet, so for this recruitment, familiarity with the internet is necessary.

For both languages, we plan to conduct most of our testing in-person, with some testing virtually if necessary. For the in-person sessions, participants will reside either in the metropolitan DC area or in the Houston and Brownsville areas of Texas. The remote session participants can live anywhere in the 50 U.S. states and Washington DC.

Language: Testing will be conducted in English and Spanish.

Timeline: Testing will be conducted in February and March 2025.

New Questions: Most of the questions to be tested are the same as those included in the last round of SmaRT pretesting. There are two new questions in the online questionnaire. The OTHER_VACANCY question seeks to gather additional information on vacant addresses for respondents who selected “Other” on the VACANCY question. The satisfaction question at the end of the online questionnaire to gauge respondents’ satisfaction with the survey. The entire questionnaire is programmed in Qualtrics.

The paper questionnaire included in the mailing materials will be modified to fit the study’s definition of a large household. Person 6 and above will be removed from the paper questionnaire so that any households greater than five are pushed to web. The paper questionnaire will also have a revised age question so that there is an option for respondents to report the age of babies under 1 year old in months.

The census.gov landing page is including a search for frequently asked questions (see Enclosure 9 for a mockup of the landing page).

Protocol: The SmaRT testing of the mailing materials and the online instrument will employ a think-aloud protocol as well as retrospective probing. Participants will be asked for their consent to participate in the study and given the privacy notice (Enclosure 1). In-person participants will be given the mail materials and remote participants will have the mail materials mailed to their homes (Enclosures 8A-8G). All participants will be asked to read through them, one by one, while thinking aloud; then they will be asked a series of debriefing questions (Mail Materials Protocol, Enclosure 3). They will then be asked to complete the 2025 Census Survey either online or using the paper form while thinking aloud (Survey Protocol, Enclosure 4). Then they will be asked satisfaction questions about their experience (Enclosure 5). They will answer debriefing questions (Enclosure 6). Finally, they will answer questions to assess their understanding of the census (Enclosure 7).

Below is a list of materials to be used in the study.

1. Informed Consent and Privacy Notice (Enclosure 1)

2. Specifications for the updated decennial census online questionnaire (Enclosure 2)

3. Protocol for cognitive testing of mailing materials including tasks & debriefing probes (Enclosure 3)

4. Protocol for survey testing (Enclosure 4)

5. Satisfaction questions (Enclosure 5)

6. Debriefing screens (Enclosure 6)

7. Knowledge Check (Enclosure 7)

8. Mailing materials (Enclosures 8A-8G)

Length of interview: The session should take 90 minutes for each of the 25 participants. We anticipate 5 minutes per participant for the technology check for remote participants.

Respondents will be screened using the Census Bureau’s generic screener. We estimate that we will screen 3 people for each successful recruit, resulting in 75 participants screened. The total estimated respondent burden for this request is 45.83 hours. See Table 1.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	75	5 minutes	6.25 hours
Tech check	25	5 minutes	2.08 hours
Pretesting Sessions	25	90 minutes	37.5 hours
Totals			45.83 hours

Use of Incentive: The total incentive for this project is $60 per participant. For virtual interviews, the test administrator will send the incentive money to the participant’s address using approved USPS priority mail. If the session is conducted in person, the test administrator will give the participant cash.

The contact person(s) for questions regarding data collection and statistical aspects of the design of this research is listed below:

Marcus Berger

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-8335

marcus.p.berger@census.gov

4