OMB Letter

Generic Information Collection Request

Cognitive Testing of Select Items from the SIPP Redesign Questionnaire

Request: The U.S. Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We propose to conduct cognitive interviews to pretest proposed revised questions for the Survey of Income Program Participation (SIPP). We are seeking approval for this project.

Background: The Survey of Income Program Participation (SIPP), is conducted by the Census Bureau. The SIPP is interviewer-administered through Computer Assisted Personal Interviewing (CAPI) and through telephone interviews. The survey is undergoing a redesign to accommodate self-administration using a Web survey. The Census Bureau’s Demographic Statistical Methods Division (DSMD) Survey Methodology (SM) team was tasked with adapting existing content for self-administration. The SM team made recommendations for proposed new items and will prepare a Qualtrics questionnaire to test sets of questions within four sections of the SIPP instrument; those on 1) where people live, 2) employment and income, 3) receipt of Food Stamps or SNAP benefits, and 4) health insurance. The next step is to conduct cognitive interviews to evaluate how well the new content performs and to identify improvements.

Purpose: The purpose of this research is to interview adults from the general U.S. population to assess their understanding of the proposed questions and response options. In the cognitive interviews, we will assess the clarity and potential effectiveness of the questions and response option structure and identify any potential respondent issues with comprehension or interpretation, with the goal of producing valid survey items that minimize the potential for measurement error attributable to the questions themselves. The results of the cognitive interviews will influence decisions about final content and wording of questionnaire items for the redesigned SIPP instrument.

Population of Interest: Participants will be screened during the study recruitment effort to ensure they have characteristics relevant to the study. These include (by topic):

Section A - Residency 

• People who (1) have, and (2) have not moved within the reference period getting, aiming for similar proportions of movers and non-movers among the participants.

Section B - Employment 

• People who work multiple jobs and/or “gig” jobs 

• People who are employed now – some with a job change within the past 6 months or currently unemployed but had a job within the past 6 months 

• People with occupations that are paid in ways other than an annual salary (for example, Teachers, realtors, wait staff, etc.)

 

Section C - SNAP 

• People who received Food Stamps or SNAP benefits within the past 6 months 

• People who live with others who received Food Stamps or SNAP benefits within the past 6 months

 

Section D – Health Insurance 

• People who (1) did not have insurance or (2) had a lapse in coverage within the past 6 months 

• People who changed insurance type/coverage within the past 6 months 

• People who live with at least one other person in their household

Language: The interviews will be conducted in English.

Timeline: Cognitive interviews are planned for February through May 2024.

Project scope: The cognitive testing of the proposed revisions to the NSCH topical questionnaires will be carried out according to the schedule described in Table 1.

Table 1. Estimated Project Schedule

Activity	Dates
Recruiting (Ongoing)	2/26/2024
Conduct Cognitive Interviews	2/28/2024 – 5/23/2024
Data Analysis – Preliminary Results	6/7/2024
Develop Final Content Recommendations	6/21/2024
Deliver Report	7/8/2024

General Protocol: DSMD will conduct this research remotely, via Microsoft Teams. Teams is a Census Bureau approved platform and allows for secure video conferencing capabilities and screen-sharing to maximize both the respondent and interviewer experience. The interviews will be conducted in accordance with guidelines and policies outlined in the internal Census Bureau policy memo Use of Microsoft Teams for Conducting Title 13 Qualitative Research Remotely.

At the start of the interview, the participant will electronically sign an online consent form via Qualtrics that references the OMB approval for the study, the confidentiality of the session, the voluntary nature of the study, and acknowledges that the session will be recorded.

Once the consent form has been signed, any participant questions will be answered, and audio recording will begin. The participant will complete the relevant survey questions in Qualtrics. The researcher will walk the participant through the questions using a think aloud technique, verbally expressing their thoughts to allow researchers to gain insight into how they interpret and respond to questions. The cognitive interviewer will administer both concurrent and retrospective probes to gain a full understanding of respondent perceptions of specific elements of the survey.

Sample: Up to 30 participants will be interviewed over two rounds (15 per round) throughout the U.S. Participants will be screened during the study recruitment to ensure they meet criteria of relevance to the cognitive testing goals, described under the Population of Interest section of this document.

Recruitment: We will use a multi-pronged strategy to recruit participants, such as posting ads on internet and social media platforms (e.g., Facebook), utilizing broadcast messages to Census Bureau employees, and using personal networks. While not intended to be nationally representative, an attempt will be made to recruit participants with varying demographic characteristics (e.g., sex, education, age, race and ethnicity).

Use of Incentive: Due to the length and complexity of the task, we will offer an incentive of $50 per participant. The incentive value is intended to compensate participants sufficiently to motivate them to participate and pay for any potential childcare arrangements needed to do so.

Length of interview: Based on past experience with similar pretesting efforts, it will require approximately 3 minutes for potential participants to complete the online cognitive interviewing screener questionnaire. We estimate up to 120 screener completions to result in 30 participants. Therefore, total recruitment burden is 6 hours. We estimate it will take 1 hour per participant to complete the cognitive interview. For 30 participants, the estimated burden for the interviews is therefore 30 hours, bringing the total burden to no more than 36 hours with recruitment.

The materials to be used in this research, listed below, are included as attachments:

1. 2024 SIPP Redesign Cognitive Interviewing Protocol_OMB

2. 2024 SIPP Redesign Cognitive Interviewing Participant Screener_OMB

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

M. Christopher Stringer

Demographic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

m.christopher.stringer@census.gov

3