OMB Memo

Memorandum
Date:

April 10, 2018

To:

Margo Schwab, Desk Officer
Office of Management and Budget

From:

Emilda B. Rivers, Acting Division Director
National Center for Science and Engineering Statistics
National Science Foundation

Via:

Suzanne Plimpton, Reports Clearance Officer
National Science Foundation

Subject:

Request for Approval for Cognitive Testing of Innovation Activities as Input to Annual
Business Survey

This memorandum is to request approval for the National Science Foundation’s (NSF) National Center
for Science and Engineering Statistics’ (NCSES) plans to conduct cognitive interviews under the
generic clearance for survey improvement projects (OMB number 3145-0174). This research will
focus on the ability of small and medium sized companies (between five and 500 employees) to
answer questions on innovation and innovation activities. This activity is associated with the Annual
Business Survey to be conducted in partnership with NCSES and the U.S. Census Bureau (Census).
Background
NCSES is broadly tasked with measuring the role of science and technology (S&T) in the United States’
economy and abroad. A major component of this activity is its sponsorship of the Business Research
and Development (R&D) and Innovation Survey (BRDIS), which collects information annually on R&D
and innovation by industrial firms within the United States. BRDIS consists of multiple questionnaires,
the most prominent being the BRDI-1 questionnaire for companies with five or more employees. Last
year another questionnaire was added, the Business Research and Development and Innovation
Survey for Microbusinesses (BRDI-M), for companies with fewer than ten employees.
The BRDI-1 and BRD1-M surveys are conducted through a partnership NCSES has with Census. NCSES
is continuing this partnership through a new survey called the Annual Business Survey (ABS). The ABS
is designed to combine multiple existing firm-level collections to reduce respondent burden, increase
data quality, reduce operational costs, and operate more efficiently. The ABS replaces the Survey of
Business Owners (SBO), the Annual Survey of Entrepreneurs (ASE), and the Business R&D and
Innovation for Microbusinesses (BRDI–M) surveys.
The ABS questionnaire contains sections or modules on research and development, financing,
company and owner characteristics, technology and intellectual property, and innovation. Testing has
been conducted on the innovation questions. This request is to focus on the innovation activities, the
1

ability of respondents to provide cost estimates, and the reasons why some businesses do not
innovate.
Proposed Methodology
NCSES is working with cognitive researchers from Westat to conduct no more than 40 interviews with
respondents from a variety of businesses with five to 500 employees. Interviews will be conducted
either in person or utilizing WebEx conferencing, and will assess the ability of respondents to provide
data. All interviews will be based on the attached protocols (Attachments 2 and 3). It is possible that
the protocol will be modified in minor ways as the interviews progress; e.g., if an item is not being
understood properly, then alternative wordings may be offered to see if they are more effective.
These interviews will allow NCSES to better understand the question-response process, examining
whether participants understand the questions, whether the questions are interpreted consistently
across respondents, whether data are available for answering the questions, what strategies
participants use to develop answers, and whether the participants are willing and able to respond
accurately. In most cases, an NCSES subject matter expert will accompany the researchers to observe
the interviews.
The in-person and WebEx interviews will be conducted in the same manner and based on the same
protocol, except that not all people will be in the same room. In-person interviews are an ideal
format because it is possible to view a participant’s facial expressions, and it is easier to establish a
personal relationship with the participant. However, WebEx interviews are still effective, they allow
greater geographic disparity among the participants while controlling costs, and they are easy to
schedule and less invasive for the participants. It is expected that in-person interviews will be
conducted at the participants’ offices.
Businesses will be selected to ensure representation across a variety of characteristics such as
geographic region, number of employees, and industry. It is anticipated that either the Small Business
Administration (SBA) database (http://dsbs.sba.gov/dsbs/search/dsp_dsbs.cfm) or the Dun &
Bradstreet (D&B) database will be used to select businesses to recruit. Preliminary review of each
database suggests that while D&B’s database is more comprehensive, the SBA database may be
updated more frequently. Thus, both will be used.
To recruit respondents, businesses will be contacted by phone and e-mail utilizing the Recruiting
Script in Attachment 1. The best respondent for the interview will be “the person in your company
who is most knowledgeable about innovations your company has made or is working on”, which may
or may not be the business owner. After respondents agree to participate, an e-mail will be
forwarded thanking them for their willingness to participate and confirming the appointment.
Interviews will be recorded, but only after written (Informed Consent Form - Attachment 4) or verbal
consent from the respondent is obtained. The protocol used during the interview will depend on how
the respondent answers the questions in the recruitment script.
Incentive Payments
NCSES proposes offering respondents an incentive of $40 per interview, an amount that was offered
in earlier cognitive interviews with the same population. This is a standard amount for cognitive
interviews, which are more intensive than ordinary surveys and which require people to schedule a
2

significant amount of time to be away from their normal activities (in contrast, surveys can be
completed in small portions at a time at a person’s convenience).
The incentive payments will be made after the interview is completed. For in-person interviews, the
participants will receive $40 in cash immediately at the end of the interview. For WebEx interviews, a
check will be mailed after the interview is completed.
Response Burden
Less than 74 burden hours are estimated for this project. The estimated time for completion of the
interview is about 60 minutes each for a total of 40 hours (40 interviews x 60 minutes = 40 hours). It
is anticipated that 400 companies will be contacted for recruiting purposes. The recruiting process
should take on average five minutes per company resulting in 33.3 hours (400 company contacts x 5
minutes = 33.3 hours).
Contact Person
Audrey Kindlon
Project Officer
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
akindlon@nsf.gov
703-292-2332
Attachment 1: Recruiting Script
Attachment 2: Innovation Activities Cognitive Testing Interview Protocol
Attachment 3: Non-Innovators Cognitive Testing Interview Protocol
Attachment 4: NSF/NCSES Consent Form

3