OMB Memo

National Telecommunications and Information Administration (NTIA) Internet Use Survey Cognitive Interviews

Request:

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct an iterative cognitive evaluation of the National Telecommunications and Information Administration (NTIA) Internet Use Survey, which is part of the Current Population Survey (CPS), and is sponsored by NTIA in the Department of Commerce. As part of this current submission, we are seeking approval for cognitive interviews conducted by Census Bureau staff from the Center for Behavioral Science Methods (CBSM).

Purpose:

Most of the current NTIA Internet Use Survey items have already been cognitively evaluated because the most recent iterations of the survey underwent cognitive testing before being implemented. For this iteration, NTIA is interested in shortening the questionnaire, which will next be fielded in November 2025. The purpose of this cognitive testing research is to test a shortened questionnaire where select items have been removed and existing items have been revised to decrease the amount of time to administer the NTIA Internet Use Survey. In addition, select items have been revised for comprehension.

In data collection, the survey is currently administered via computer assisted personal interviewing (CAPI) and computer assisted telephone interviewing (CATI).

The full supplemental questionnaire is attached and includes core questions from the CPS used to roster the household members. Rostering household members is done to determine household member eligibility for the NTIA Internet Use Survey (see Enclosure 1: CPS Roster; Enclosure 2: NTIA Internet Use Survey).

We plan to cognitively test all items in the NTIA Internet Use Survey using specific probes designed as part of the cognitive test (see Enclosure 3: NTIA Internet Use Survey Protocol). We will also evaluate how easy it is for participants to answer questions on behalf of other people in their household.

The results of the cognitive test will inform the final revised NTIA Internet Use Survey CAPI/CATI instrument content that will be used to collect data in November 2025.

Populations of Interest:

The planned cognitive pretesting evaluation will focus on assessing and improving the questionnaire content for the general population.

Timeline:

Testing will be interviewer administered beginning approximately March of 2025 and ending in May of 2025. Recruitment will begin in early March of 2025.

Language:

Testing will be conducted in English only.

Method:

We will conduct the interviews with a maximum of 20 participants across in a single round with some changes after a subset of interviews. Testing will be conducted by an interviewer who will read aloud the questions to mimic CAPI; we will use a Qualtrics instrument to administer the questionnaire.

The interviews will be conducted remotely via Microsoft Teams. We will allow interviews to be conducted via telephone for participants who are not able to use Microsoft Teams. Each interview will be 60 minutes long. Before the start of the interview, we will conduct a short “tech check” for those participants who will be using Microsoft Teams.

During the interview, the interviewer may share their screen with participants to gather feedback on some of the question wording during retrospective probing. There will be no screen sharing when the questionnaire is being administered.

Sample:

Our recruiting efforts will target participants with the following characteristics:

We will target participants who are either aged 65 or older, as we are interested in how participants aged 65 or above comprehend the questions, especially those where terminology has been modified to reflect the current technology landscape changes. We are also interested in households that include another member that is 65 or older, households with extended family (e.g., grandparent/grandchild, cousins) and households with unrelated members. These interviews will focus on comprehension as well as the feasibility of participants answering the questions for themselves and other household members. See Table 1 below for the recruitment goals of each interview:

Table 1: Number of interviews by participant type (n = 20).

65+ years old	Lives in household with another 65+ year old	Lives in a household with other distantly related people	Lives in a household with unrelated people
8	4	4	4

Recruitment:

Participants will be recruited using several methods: CBSM’s existing database of participants, word-of-mouth, advertisements on different web sites (e.g., social media, Craigslist.com, NextDoor), and broadcast messages (see Enclosure 4: Communication Materials) distributed through the Census Bureau. All interviews will be conducted via Microsoft Teams. We will allow interviews to be conducted via telephone for participants who are not able to use Microsoft Teams. The advertisements on web sites and broadcast messages will have a national outreach, and as a result, we will recruit participants from the entire U.S.

People who respond to the advertisements will be screened with the universal screener and other questions attached (see Enclosure 4). They will be selected based on meeting the age and household composition characteristics described above as well as other characteristics associated with different patterns of technology use.

Participants will be screened either by telephone or will complete a self-administered web form. They will receive a reminder about the interview appointment after they agree to participate.

Protocol:

We will inform participants that their response is voluntary, confidential, and will be accessed only by employees involved in the research project. The consent form (see Enclosure 4) will also indicate that the participant agrees that the interview can be video and audio-recorded to facilitate analysis of the results. Interviewers will ask participants to give consent orally to be recorded. Participants who do not consent to be audio-recorded will still be allowed to participate.

Participants will first be asked a list of roster questions to obtain information about their demographics and household size (enclosure 1). They will then be asked the complete list of questions from the NTIA Internet Use Survey. We will conduct the cognitive interviews using a protocol (enclosure 3) with retrospective methods which allow the participant to complete the questionnaire in its entirety and respond for all eligible household members before being asked probes about particular questions of interest. During probing, participants will be asked about the feasibility of answering the questions for both themselves and on behalf of other household members and their comprehension of specific terminology in the questionnaire.

The interview protocol includes probes regarding whether: (a) the wording of the revised questions allows participants to understand the meaning and purpose of the questions, (b) participants can answer the revised questions in the NTIA Internet Use Survey accurately and with little to no difficulty (c) participants can answer the revised questions on behalf of other household members.

* While a convenience sample of this size is not intended to be representative of a random sample of the U.S. population, we intend to recruit participants with some demographic heterogeneity. As such, we will be collecting race and ethnicity in the roster part of the testing. Since we are not creating estimates from the race and ethnicity question, a less detailed version is sufficient for the purposes of this study (the collection of demographic information for potential cognitive interview participants, from a convenience sample), while limiting respondent burden. Therefore, we will use the question wording and response options for the race and ethnicity question from Figure 3, rather than the more detailed version in Figure 1.

Incentive:

We are requesting a $40 incentive for each remote participant. Payment will be mailed in cash to an address the participant provides on the payment voucher after the interview is complete.

Length of Data Collections:

We estimate that each interview will take approximately 60 minutes. The screening questionnaire will take approximately 15 minutes per person.

We expect to screen five people for each successful recruit. In total, we will screen about 200 people to arrive at 20 successful recruits. Thus, we estimate the burden for this research is 70 hours: 20 hours for interviewing (1 hour X 20 interview participants), and 50 hours (0.25 hours X 200 people) for screening.

Table 2: Estimated Burden

Research Activity	Number of Participants	Participation Time (minutes)	Burden Time (hours)
Screening for Interviews	200	15	50 hours
Interviews	20	60	20 hours
Total			70 hours

The materials to be used in this study are enclosed with this letter and include:

• CPS Roster (Enclosure 1)

• NTIA Internet Use Survey (Enclosure 2)

• Cognitive interview protocol (Enclosure 3)

• Communication Materials (Enclosure 4)

The contact person for questions regarding data collection and technical aspects of the design of this research is listed below:

Kathleen Kephart

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

301-763-8891

kathleen.m.kephart@census.gov

3