2023 Race and Ethnicity Testing for ABS OMB Request Letter

Generic Information Collection Request:
Cognitive testing for the 2023 Race and Ethnicity Questions on the Annual Business Survey

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). As part of the Testing Team that is under the U.S. Office of Management and Budget Interagency Technical Working Group on Race and Ethnicity Standards, the U.S. Census Bureau plans to conduct presting activities on the newly revised race and ethnicity questions released in the Federal Register (88 FR 5375). These pretesting activities will take place with business respondents for the Annual Business Survey (ABS).

The ABS is conducted annually as a supplement to the economic census collected by the Census Bureau and sponsored by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF).

The ABS provides a detailed, regularly collected source of data on the status, nature, and scope of women-, minority-, and veteran-owned businesses. The ABS provides estimates for the number of employer firms, sales and receipts, annual payroll, and employment by owners’ gender, ethnicity, race, and veteran status. The ABS also provides information on research and development activity and cost for businesses with 1-9 employees and includes survey questions to measure business innovation in firms of all sizes. For the 2023 ABS, estimates will be available for the U.S., states, and select metropolitan statistical areas (MSA). Among other uses, the Small Business Administration (SBA) and the Minority Business Development Agency (MBDA) use results from the ABS to assess business assistance needs and allocate available program resources. Data collected on research and development and innovation may be used to compare R&D costs across industries, determine where R&D activity is conducted geographically, and identify the types of businesses with R&D, to contribute to the Bureau of Economic Analysis (BEA) system of national accounts; to increase investments in research and development, strengthen education, and encourage entrepreneurship; and to compare US business innovation with innovation in other countries, including those in the European Union.

The ABS is primarily collected via a self-administered questionnaire using the Census Bureau’s online survey reporting system. Respondents are mailed a letter informing them of the requirement to complete the survey and providing them with access information. A preview of the questionnaire is available for respondents to download to review the survey questions and instructions. For more information regarding ABS: https://www.census.gov/programs-surveys/abs.html

Purpose: The United States Office of Management and Budget (OMB) has proposed changes to the way information on race and ethnicity is collected by the Federal government. The testing of the race and ethnicity questions on the ABS will help us better understand business establishments’ ability to provide such data if the changes are implemented. During these pretesting activities, respondents will only see a select number of questions from the owner characteristics section during the interview. Respondents who participate will not need to view or discuss a majority of the questions that are typically asked on the ABS. Testing will include evaluations of both the revised short and long race/ethnicity questions.

The results from the cognitive testing will be recorded and a report produced that outlines the findings of the pretest and recommendations for improvement to questions. This report will include:

• Documenting how respondents comprehend the proposed long and short version of the revised race/ethnicity question

• Identifying respondents’ use of records or other response strategies for answering the long and short versions of the revised race/ethnicity question

• Assessing respondents’ ability to answer the long and short versions of the revised race/ethnicity question

• Identifying difficulties in completing the long and short versions of the revised race/ethnicity question

• The prevalence of proxy reporting within the business establishment and it’s affect on responding to the revised race/ethnicity question.

Population of Interest: Sample members from the 2019 and 2020 ABS. Businesses should be representative of all business sizes and cover a variety of industries.

Timeline: Testing will be conducted from March through May 2023.

Language: Testing will be conducted in English only.

Method: We will conduct cognitive interviews with up to 40 respondents. These interviews will be conducted via Microsoft Teams. During the interviews, respondents will view both versions of the draft questions (see Attachment C). Interviews will be conducted by researchers from the Data Collection Methodology and Research Branch. Special Sworn Status staff from the NCSES and/or Economic Reimbursable Division may observe if available. We will use probes (see Attachment A) to assess respondents’ understanding of the questions and the flow of the questionnaire.

We will also conduct online testing via the Qualtrics platform with up to 150 respondents. These respondents will be split into 2 panels with one seeing the long version of the revised race/ethnicity question first and one seeing the short version first of the question first. They will then be asked to answer a limited number of probes from the cognitive interviewing protocol (see Attachment B).

All participants will be informed that their response is voluntary and that the information they provide is confidential and will be seen only by Census Bureau employees and those with special sworn status (see Attachment E).

Sample: We plan to conduct a maximum of 190 interviews – cognitive interviews combined with online testing via Qualtrics. We plan to conduct interviews with a variety of sizes and types (i.e., industries) of businesses. This number of interviews was selected because it is a manageable number of interviews for the time period allotted, it should adequately cover target companies, and should be large enough to provide reactions to the questions in order to identify meaningful findings.

Recruitment: We intend to target up to half of respondents who may answer as proxies for the owners of the business. Additionally, using data from our frame assignment process that identifies country of birth, we would like to target 1/4 to 1/3 of respondents to test for the Middle Eastern and North African (MENA) category. Participants will be recruited via email. Once interviews are scheduled, researchers will send participants a confirmation email. Before beginning the interviews, we will inform participants that their response is voluntary and that the information they provide is confidential under Title 13. Researchers will confirm that the respondents have read, understand, and agree to the contents of the consent form (see Attachment E).

Protocol: A copy of a draft interview protocol and a draft questionnaire for testing purposes are enclosed (see Attachment A).

Cognitive interviews will be conducted via Microsoft Teams. Participants will be asked to read the draft ABS questions. Probes will be asked concurrently. We anticipate these sessions will take 30 minutes to complete, and only one participant will be interviewed at a time.

Online testing will be conducted via Qualtrics. Participants will be asked to read the draft ABS questions and to answer a few probes from the protocol. We anticipate these sessions will take 15 minutes to complete.

Use of incentive: Monetary incentives for participation will not be offered.

Enclosures: Below is a list of materials to be used in the current study:

Attachment A: Draft protocol for moderated testing

Attachment B: Draft protocol for unmoderated testing

Attachment C: Questions for testing

Attachment D: 2022 ABS worksheet

Attachment E: Consent form

Length of interview: We expect that each cognitive interview will last no more than 30 minutes (40 cases x 30 minutes per case = 20 hours). Additionally, to recruit respondents we expect to reach out via email and to make up to 3 phone contacts per completed case. The recruiting calls are expected to last on average 3 minutes per call (3 attempts per phone call per completed case x 40 cases x 3 minutes per case = 2 hours). Thus, the estimated burden for the cognitive interview portion of this project is 22 hours (20 hours for interviews + 2 hours for recruiting). The online testing instrument will be administered to 150 respondents and will take approximately 15 minutes to complete. This results in a total burden of 37.5 hours for online testing. The total burden for all testing is 59.5 hours.

Contact: The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Krysten Mesner

Data Collection Methodology and Research Branch

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-9852

Krysten.Mesner@census.gov

cc.:

Nick Orsini (ADEP) with enclosures

Stephanie Studds (ADEP) with enclosures

Kevin Deardorf (ERD) with enclosures

Aneta Erdie (ERD) with enclosures

Patrice Hall (ERD) with enclosures

Kimberly Moore (ESMD) with enclosures

James Hunt (ESMD) with enclosures

Amy Anderson Riemer (ESMD) with enclosures

Jasmine Luck (ADRM) with enclosures

Danielle Norman (PCO) with enclosures

Mary Lenaiyasa (PCO) with enclosures