Request for Approval for Cognitive Testing of Innovation Activities as Input to Annual Business Survey

Attachment 3 - Non-innovators Cognitive Testing Interview Protocol
Introduction
[To be given by NSF representative if present, or Westat interviewer if NSF is not available]
Hello, we are [introduce attendees from NSF, Westat].
The National Science Foundation (NSF) is in the process of developing a survey to understand what
kind of external and internal factors can affect the ability of small and medium sized businesses in the
U.S. to innovate.
The survey is currently in the design phase. Today, we would like to ask questions about your
company and get your opinions about some questions -- what you think the questions are asking for,
whether these questions are relevant for your company, and how easy or difficult it is to answer
them and why. There are no right or wrong answers.
We would like to record this interview so that we have all your comments. Would that be okay?
[Have respondents read and sign consent forms.]
Background
Please tell me, in 1-2 sentences, what your company does. [If respondent does not mention, ask
about the company’s goods and services.]
Tell me what innovation means to you.
Some companies consider themselves innovative while others say it not a priority and companies
have a variety of reasons for both. Do you think of your company as an innovative company or not as
an innovative company?
Has your company considered making any major changes to the goods or services your company
offers?
[IF YES]What has your company considered?
What, if anything, do you expect to come out of that?
[IF NO] What has prevented your company from doing so?
Has your company considered making any major changes in its processes, such as marketing or
manufacturing?
[IF YES]What has your company considered?
What, if anything, do you expect to come out of that?

13

[IF NO] What has prevented your company from doing so?
What do you see as the biggest external factors that affect your company?
What do you see as the biggest internal factors that affect your company?
What do you consider to be the biggest challenges your company faces in the current economy?
What major changes, if any, have your competitors made that your company has faced?
[IF APPLICABLE]: What impact have changes from competitors had on your company?
What actions, if any, has your company taken to respond to those changes?
What major changes, if any, have there been in what your customers expect?
[IF APPLICABLE]: What impact have changes in customer expectations had on your company?
What actions, if any, has your company taken to respond to those changes?
Innovation and factors affecting it
Now I would like to discuss innovation and factors that can impact innovation. More specifically, I
would like to ask you some questions about how different factors can or cannot affect your
company’s ability to innovate.
Please answer the following question [USE ATTACHMENT 3A].
During the three years 2015 to 2017, how important were the following factors in preventing this
business from innovating?
A.
B.
C.
D.
E.
F.
G.
H.
I.
J.
K.

Lack of internal finance
Lack of credit or private equity
Innovation costs too high
Lack of skilled employees within the business
Lack of collaboration partners
Difficulties in obtaining government grants or subsidies
Uncertain market demand for your ideas
Too much competition in your market
Legislation/regulation that generated excessive burden
Legislation/regulation that created uncertainty
Legislation/regulation that lacked consistency across the United States

•

In your own words, what do you think this question is asking you?
14

•

What does innovating mean to you in the context of this question?

•

To you, what does “preventing this business from innovating” mean?

•

What time period were you thinking of when you answered this question? How easy or
difficult was it to think of factors in this time period only?

•

How did you decide which boxes to check?

•

I noticed you noticed you spent a lot of time thinking before you answered this question/line,
can you tell me about that?

•

Which parts of this question were the most difficult to answer? Why?

•

Which parts of this question were easiest to answer? Why?

•

Which parts of this question, if any, were not relevant for your company? Why?

•

Do these items get at the real reasons why companies do not innovate? What other reasons
should be added to the list?

•

[If there are external or internal factors that the respondent mentioned previously please
bring them up here] Previously you mentioned [note respondent’s previous response]. Do any
of the options listed address this factor? If so, which one? If not, do you think it should be
included?

Reasons for not innovating
Again, I would like to ask your thoughts on another question.
[USE ATTACHMENT 3B]
How important were the following reasons for this business not to conduct innovation activities
during the three years 2015 to 2017?
A.
B.
C.
D.

Low demand for innovations in your market
No need to innovate due to previous innovation
No need to innovate due to very little competition in the business’s market
Lack of good ideas for innovations

•

In your own words, what do you think this question is asking you?

•

What does innovation activities mean to you in the context of this question?

15

•

What time period were you thinking of when you answered this question? How easy or
difficult was it to think of factors in this time period only?

•

How did you decide which boxes to check?

•

I noticed you noticed you spent a lot of time thinking before you answered this question/line,
can you tell me about that?

•

Do you think this question is asking something different from the previous question or are
they the same? Why?

•

Which parts of this question were the most difficult to answer? Why?

•

Which parts of this question were easiest to answer? Why?

•

Which parts of this question, if any, were not relevant for your company? Why?

•

Are there other reasons, that are not included, that affected your company’s ability to
innovate?

•

Do these items get at the real reasons why companies do not innovate?

Thank you

16

ATTACHMENT 3A
During the three years 2015 to 2017, how important were the following factors in preventing this
business from innovating?
Very
Somewhat
Important Important

Not at all
Important

A. Lack of internal finance
B. Lack of credit or private equity
C. Innovation costs too high
D. Lack of skilled employees within the business
E. Lack of collaboration partners
F. Difficulties in obtaining government grants or subsidies
G. Uncertain market demand for your ideas
H. Too much competition in your market
I. Legislation/regulation that generated excessive burden
J. Legislation/regulation that created uncertainty
K. Legislation/regulation that lacked consistency across the
United States

17

ATTACHMENT 3B
How important were the following reasons for this business not to conduct innovation activities
during the three years 2015 to 2017?
Very
Somewhat
Important Important

Not at all
Important

A. Low demand for innovations in your market
B. No need to innovate due to previous innovation
C. No need to innovate due to very little competition in the
business’s market
D. Lack of good ideas for innovations

18