Protocol

Services Annual Survey (SAS) E-Commerce Debriefing Interview

DRAFT PROTOCOL

September 18, 2018

Aryn Hernandez/Kristin Stettler

(NOTE: This protocol is a guide – the questions presented here will not necessarily be asked exactly as worded in the protocol or in this order. Not all questions will be asked in every interview. As much information about interview participants and enterprise/establishments should be obtained prior to the interview as possible. Conduct background research on each enterprise/establishment prior to each interview. In some cases, probing may need to be adjusted based on the background research that was conducted.)

Expected Length of Interview: 30 minutes

General probes that may be used throughout the interview:

• Reflect back on respondent’s answer: “you said…”

• In your own words, what is this question asking?

• How did you arrive at this number/answer this question?

• Was this data easy to access?

• What records (if any) did you look at? What line or lines were of interest?

• Specifically, what did you include in this number? What did you exclude?

• Would you consult other people to obtain this answer?

Introduction:

• Explain purpose of meeting: Thank you very much for your time. Let me start by telling you a little about what we will be doing today. We are getting feedback on revisions to the Services Annual Survey. We are in the process of redesigning portions of this form in order to better meet our data users’ needs. We would like to get your input on the revised questions, in order to make them as clear and easy to complete as possible.

• This is not a test of you! We want to understand how you are interpreting this form to make sure we are doing our jobs right and asking questions that can easily be answered.

• As we go through the interview, we will ask both general and specific questions. If at any time a question seems odd to you, please let us know. We encourage all feedback.

Respondent Background

1. Before we start with more specific questions, I would like to ask you a few questions about this company and your role in it.

1. (For each person on the call): What is your formal job title? And, could you tell me a little about your role and responsibilities?

2. Have you completed the Services Annual Survey in the past?

3. How about other government surveys?

Company Background

1. What would you consider to be the main operating activity of this company?

2017 E-Commerce Question A

Now, I wanted to review a few questions from the Services Annual Survey. Please, take a look at the document with the question text I emailed to you earlier. Let’s start with 2017, question A under ‘Revenues from Electronic Sources.’

A. Did this firm have any revenues from customers entering orders directly on the firm’s websites or mobile applications in 2017? Yes / No

1. Could you tell me what this question was asking for in your own words?

2. You answered ______________, how did you arrive at this answer?

   1. What sort of orders were you thinking off when you answered this question? Can you give me any examples?

   2. Can you give me some examples of the websites or mobile applications you were thinking of?

3. Did you consult any records or anyone else in your organization in order to answer this question?

   1. Does this wording match how it is recorded in your records?

4. In your own words, what does the term “orders” mean to you? Do you use that term in your business, or is there another term you use?

5. How long would you estimate it took to answer this question?

6. Do you have any suggestions to improve this question? Would any instructions or clarifications be helpful?

2017 E-Commerce Question B

Let’s move to question B.

B.  Did this firm have any revenues from customers entering orders directly on third-party websites or mobile applications in 2017? Yes / No

1. Could you tell me what this question was asking for in your own words?

2. You answered ______________, how did you arrive at this answer?

   1. What sort of orders were you thinking off when you answered this question? Can you give me any examples?

   2. Can you give me some examples of the third-party websites or mobile applications you were thinking of?

3. Did you consult any records or anyone else in your organization in order to answer this question?

   1. Does this wording match how it is recorded in your records?

4. How long would you estimate it took to answer this question?

5. Do you have any suggestions to improve this question? Would any instructions or clarifications be helpful?

2017 E-Commerce Question C

Now, let’s look at question C.

C.  Did this firm have any revenues from customers entering orders via any other electronic systems (such as private networks, dedicated lines, kiosks, etc.) in 2017? Yes/ No

1. Could you tell me what this question was asking for in your own words?

2. You answered ______________, how did you arrive at this answer?

   1. What sort of orders were you thinking off when you answered this question? Can you give me any examples?

   2. Can you give me some examples of the other electronic systems you were thinking of?

3. Did you consult any records or anyone else in your organization in order to answer this question?

   1. Does this wording match how it is recorded in your records?

4. How long would you estimate it took to answer this question?

5. Do you have any suggestions to improve this question? Would any instructions or clarifications be helpful?

2017 E-Commerce Question D

Let’s move to question D.

D.  Of the total 2017 revenues reported in the SALES, RECEIPTS, AND REVENUES item, ($xx,xxx,xxx), what was the dollar amount (or percentage) that was from the revenues identified in A-C above? Please provide an estimate if exact figures are not available. $____ OR _____ %

1. Could you tell me what this question was asking for in your own words?

2. You answered $_____________, how did you arrive at this answer?

   1. In 2016, the SAS e-commerce question was different from this year’s questions. In 2016, you answered $__________. What would you say was the main reason your answer [increased/decreased]?

      1. If needed: Was this a difference in the actual amount of sales in total? Are you including [more/less] sources of revenue based on questions A-C as compared to last year’s question (provided in email)?

      2. For NAICS 481111: Are you including bookings from 3^rd party websites and your own website?

      3. Do you have any comments on the changes to the e-commerce questions from 2016 to 2017?

   2. You chose to answer the question [in dollars/as a percentage], why did you chose that answer format?

3. Did you consult any records or anyone else in your organization in order to answer this question?

   1. Does this wording match how it is recorded in your records?

4. How long would you estimate it took to answer this question?

5. Do you have any suggestions to improve this question? Would any instructions or clarifications be helpful?

End/Wrap up

1. Overall, what do you think of the questions we discussed today?

2. What sorts of records does your business keep related to electronic sales? Does your business track electronic sales separately from all other sales? If so, are these records broken down by product?

3. Are there other words or terms you use to describe this concept?

4. Would you say that someone in your role was the appropriate person to answer questions like these? If not, who would be the best person to answer questions like these?

5. Do you have any other comments, questions or suggestions for us?

Thank you very much for your time today!