OMB Memo

Generic Information Collection Request:
Cognitive Interviewing for Phase 2 of the Small Business Pulse Survey (SBPS)

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting cognitive interviews by phone lasting no longer than 45 minutes to test newly developed questions about the effect of changing business conditions due to the Coronavirus pandemic and other social changes on small businesses in the United States.

On March 13, 2020, the President of the United States declared a national state of emergency in response to the Coronavirus pandemic. Additionally, individual states began declaring state-wide emergencies as early as February 29, 2020. One of the steps state governments have taken in response to the public health emergency is to close the physical locations of all businesses deemed ‘non-essential,’ and to advise or mandate the population of the state to shelter in place, remain at home, or engage in other restrictions of movement in order to social distance.

Early analysis suggests that these closings have had a disproportionate effect on small businesses in the United States. Even with the passage of the Coronaviruse Aid, Relief, and Economic Security (CARES) Act, including the Paycheck Protection Program (PPP) – designed to bolster small business payrolls, and cover rent, mortgage interest, and utilities – initial rounds of data collection indicate that about half of small businesses nationally (51.2 percent) reported a decrease in the total number of hours worked by paid employees for the week of April 24.¹

In response to the rapid changes in the economic landscape, the U.S. Census Bureau has conducted weekly surveys to measure the effect of changing business conditions due to the Coronavirus pandemic on small businesses. These weekly surveys – called the Small Business Pulse Surveys (SBPS) – focused on the changes in small businesses’ incoming revenue and outgoing expenses, staffing, and business models, including adapting to curb-side pickup or delivery of goods and shifting production in response to the virus.

However, conditions for small businesses continue to unfold. In consultation with other stakeholders, the Census Bureau has updated the Phase 1 questions to incorporate new content, but also to retain core concepts. Researchers from the Census Bureau’s Economic Statistcal Methods Division (ESMD) will conduct a series of 45 minute phone-based cognitive interviews with small businesses.

Purpose: The purpose of cognitively testing newly proposed questions for the SBPS is to ensure that the questions are measuring the underlying constructs of interest and to better understand the accessibility of the requested data and the burden of compiling responses to the questions. The feedback from these interviews will be used to refine question wording and decide whether (or not) to include these new questions.

Population of Interest: Representatives of small businesses across all sectors. These businesses will meet the following criteria (as determined from their responses to the 2017 Economic Census), in accordance with eligibility parameters for the PPP²:

• Any small business concern that meets SBA’s size standards (either the industry based sized standard or the alternative size standard);

• Any business, 501(c)(3) non-profit organization, 501(c)(19) veterans organization, or Tribal business concern (sec. 31(b)(2)(C) of the Small Business Act) with the greater of:

  • 500 employees, or

  • That meets the SBA industry size standard if more than 500

• Any business with a NAICS Code that begins with 72 (Accommodations and Food Services) that has more than one physical location and employs less than 500 per location; and/or

• Sole proprietors, independent contractors, and self-employed persons

Timeline: Testing will be conducted during June and July, 2020.

Language: Testing will be conducted in English only.

Method: The method of research will be cognitive interviewing. The purpose of cognitively testing the second round of SBPS questions is to minimize measurement error and maximize the validity of these questions by ensuring that the questions accurately measure the underlying construct of interest. Cognitive interviewing is a method of pre-testing instruments that involves in-depth interviewing, paying particular attention to the mental processes respondents use to respond to questions³. Cognitive interviewing uses a framework dependent on evaluating questions against their outcome objectives, including accurately eliciting the underlying construct of interest, and to what level of accuracy respondents can provide data in response

All interviews will be conducted over the telephone. The interviewers will follow a semi-structured interview protocol (Attachment A). Interviewers will send a link to the survey online to respondents prior to calling so that respondents can work through the survey on the phone.

Sample: We plan to conduct a maximum of 40 interviews total over two rounds of data collection. In the first round, we will target 20 to 25 respondents across all industries. The research staff will then review the resultant data and make adjustments to the interviewing protocol and to the questions if necessary. We will then engage a second round of interviewing targeting 10 to 15 respondents, again across all industries. This number of interviews is targeted because it is a manageable number of interviews for the time period allotted, it should adequately cover targeted small businesses, and it should be large enough to provide reactions to the questions in order to identify meaningful findings.

For this research, we will target sample from the Phase 1 SBPS from weeks 5, 6, and 7. We have specifically chosen these respondents because their last contact with the Census Bureau was long enough ago to minimize satisficing due to familiarity with the questions, but recent enough that the research is still salient. We will include both respondents to the first round of the SBPS, and non-respondents, if they agree to participate.

Recruitment: Participants will be recruited using a sample file from the Phase 1 of SBPS. First, we will send an email to the contact information listed in the 2017 Economic Census data. This email will include instructions for respondents to schedule an interview time and date with a Census Bureau researcher. We will verify the appointment time, and respond by email to the respondent with confirmation of scheduling; in that email, we will also verify the best number to reach the respondent. About 30 minutes before their appointment time, we will email the respondent again reminding of the upcoming appointment and including a link to the survey. The first screen of the online survey will be a Paperwork Reduction Action (PRA) and Privacy Act (PA) statement, informing participants that their response is voluntary and that the information they provide is confidential under Title 13 and asking for consent to be interviewed. Respondents will need to click a checkbox indicating that they understand these rights and agree to be interviewed.

Protocol: A copy of a draft interview protocol is enclosed.

Use of incentive: Monetary incentives for participation will not be offered.

Below is a list of materials to be used in the current study:

Attachment A: Draft protocol outlining intended questions to guide the interviews

Attachment B: Draft instrument, including PRA/PA statements

Length of interview: For cognitive interviews, we expect that each interview will last no more than 45 minutes (40 cases x 45 minutes per case = 30 hours). Additionally, to recruit respondents we expect to reach out via email and to make up to 3 phone contacts per completed case. The recruiting emails and calls are expected to last on average 3 minutes per call (3 attempts per phone call per completed case x 40 cases x 3 minutes per case = 6 hours). Thus, the estimated burden is 36 hours (30 hours for interviews + 6 hours for recruiting).

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Melissa A. Cidade, Ph.D.

Data Collection Methodology and Research Branch

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-8325

Melissa.cidade@census.gov

cc:
Nick Orsini (ADEP) with enclosures

Cathy Buffington (ADEP)     “          ”

Lucia Foster (CES)     “          ”

Carol Caldwell             (ESMD)     “          ”
Amy Anderson Riemer  (ESMD)     “          ”

Diane Willimack (ESMD)     “          ”

Jennifer Hunter Childs (ADRM)     “          ”

Jasmine Luck (ADRM)     “          ”

1 United States Department of Commerce, Bureau of the Census. “Small Business Pulse Survey Results: Week 1.” Retreived from: https://portal.census.gov/pulse/data/#data, June 16, 2020.

2 See United States Small Business Administration. “Paycheck Protection Program.” Retrieved from: https://www.sba.gov/funding-programs/loans/coronavirus-relief-options/paycheck-protection-program, June 16, 2020.

3 Campanelli, P. 2007. “Methods for Testing Survey Instruments.” Short Course, Joint Program in Survey Methodology (JPSM). Arlington, VA.