Protocol

2017 Commodity Flow Survey

Cognitive Testing Protocol

(This protocol is a guide – the questions presented here won’t necessarily be asked exactly as worded in the protocol or in this order. It also is important to note that not all questions will be asked in every interview.)

Introduction

Introduce observers and their background

Purpose of visit

Let me start by telling you a little about what we will be doing today. The Commodity Flow Survey collects data on the movement of goods within the United States. It will be mailed out quarterly in 2017. We are in the process of updating the questions on this survey, and we need some feedback on the changes we would like to make. Our goals is to fins out how well the changes work for people, and you are going to help us with that process.

Permission to audio tape discussion? Have R sign consent form

Before we get started I’d like to audio tape this interview so I don’t have to rely on my memory later. This session is confidential. Only persons connected with this project will have access to your tape. If that is all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Background Information

Respondent Background

• What is your role here at the company?

• How long have you worked in your current position?

• Have you completed other government surveys?

  • If so, which ones?

  • Have you ever filled out any of these government surveys online?

  • (If yes) How did you find the process? Easy? Difficult?

General Probes

• In your own words, what was that question asking?

• What are you thinking about?

• Can you tell me more about that?

• You answered, “…” because…?

• I want to make sure I understand, can you explain that again?

• Can you tell me how you arrived at that answer? What did you include? What did you exclude? What records did you use?

• Reflect back on R’s answer and ask R to correct (“I want to make sure I have it right. I think you said, “…?”)

Shipping records and other information

Before we start talking about the questionnaire, I would like to find out a little bit about your company keeps records of your shipments.

• What do they look like?

• What information is contained within them?

• How your shipping records are kept? (Are they paper-based, spreadsheet-based, etc? If paper based, do they get lost or misplaced and how are they stored? What system do you use for creating spreadsheets? Are records in more than one format?)

• How are they organized? By carrier? By mode of transportation? By customer, etc.?

• Are they easily accessible or is there a process you need to go through in order to access them? Who has access to these records?

• How long do you keep the records on hand?

Filling out the Form

Now I’d like to show you the questionnaire. I would like you to go through it and complete it as you normally would. This is not a test, so just proceed in whatever way makes the most sense to you. If you need to get information from records or others within the company in order to answer a question, just let me know what you would need to do. If at any point something isn’t clear or you have questions about something please ask and we’ll try our best to answer them.

I’ll periodically stop you and ask you questions and get your feedback on some of the questions..

(Note: We will employ both concurrent and retrospective probing techniques during the interviews. We will ask respondents about each question and/or group of questions after s/he has answered. We will also follow up on any difficulties that respondents had with the overall process of filling out the questionnaire once they have completed it)

Item A – Verification of Shipping Address

Observe R to see if they respond, and if responding “No” has to make corrections. Probe as necessary.

Item B – Verification of Mailing Address

Observe R to see if they have to enter a different mailing address. Probe as necessary.

Item C – Operating Status

• How did you arrive at your answer to this question?

• How easy or difficult was it to come up with your response to this question?

Item D – Total Number of Outbound Shipments

• How did you arrive at your answers to this question?

• What records, if any, do you consult in order to come up with an answer?

• How easy or difficult was it for you to answer this question?

• Did you happen to notice the definition of an outbound shipment?

• How well or how poorly does this definition match your own definition of “outbound shipment”?

• (If R didn’t read the definition before providing an answer) Does this definition change your answer? In what ways? Does it change the process that you would use to come up with an answer?

• Did you happen to notice the bullet instructions after the definition of outbound shipments?

• Were there any problems incorporating those instructions to generate your response?

• Did you happen to notice the instructions after the response options in the second question?

Item E – Sampling Instructions

Observe how R works through Item E. Note if they have difficulty working through the item.

• What number did you come up with? What made you choose that number?

• In your own words what is this Item asking you to do?

• Example: Did you notice the example? Did you read any of it? (If yes: Did the example help to clarify the instruction or make them more difficult?

• Is there a better way to help you understand the instruction in Item E?

Item F – Shipment Characteristics

Ask R to complete 2-3 lines of shipment data on Item F, using his/her own data. Observe R’s navigation and completion of pages 4-5. Is R following the arrows across the page or is he/she completing page 4 before starting page 5?

• Column D: If R says “don’t know value,” ask about range/approximation.

• Column F: How helpful is the SCTG code manual? How easy/difficult was it for you to find the codes you were looking for? What do you think of the way the codes are organized? Do you have any suggestions for improvement?

• Column H: (See if R reads the * at the bottom of the page). How did you arrive at your answer? Is this information available within your shipping records? Was the statement at the bottom of the page helpful?

• Column J: How did you arrive at your answer? Is this information available within your shipping records or not?

• Columns K and N: How do you track the modes? Do you know if the modes vary within shipments? What does your answer represent (just when it left the facility vs. all modes until it reaches recipient?) What do each of the transport codes mean to you?

Item G – Verification of Industry Classification

• In your own words, what do you think this item is asking?

• How did you arrive at your answer?

• How easy or difficult was it to understand the description of you business? (If difficult, find out what made it difficult)

• If respondent answers “no”: you indicated that this was not the industry in which you do business. Can you tell me about more about that?

• (If not obvious) How accurate or inaccurate was the description? (If inaccurate, find out what made it inaccurate)

• Would you change anything about this description? If so, what would you change?

Item H – Monthly Value of Outbound Shipments

• In your own words, what did you think this question was asking?

• How did you come up with this answer? (Records? Estimate? “Usual” month?)

• What period of time does this amount refer to?

Item I – Time to Complete

• How did you come with your answer to this question?

• What kinds of activities were you including in your estimate? (Find out if R is including time it takes to gather the data.)

Other shipment information

We are interested in finding out about some other information you might be able to provide about your shipment.

Customers

• What kind of information, if any, do you have about the customers that receive your shipments? (IF not obvious, probe: Does that information contains names? Do you have information on whether is it an individual or another company?)

• Where is that information kept in your records?

• If information is available: Would you be willing to provide that information if we asked for it?

Country of origin (to be asked only of certain NAICS)

• Do you keep any records on shipments that are shipped form other facilities?

• In what format are those records? What kind of information do you have about them? (If not obvious, probe: Do your records contain any information of the country from which your shipments originate?)

• If information is available: Would you be willing to provide that information if we asked for it?

Large shipments

Because we are only asking for a sample of the shipments that you have during the week the questionnaire asks about, we may miss out on some shipments that we would like to know about.

• Do you ever have any large shipments that might not occur that frequently, or may fall outside of the week on which you are reporting (high weight)?

• If yes: how often do those types of shipments occur?

• How do you keep track of those shipments? Would you be able to provide similar information about those shipments as you do for smaller shipments?

Web Reporting Option

• If R responded on paper: It looks like you responded on paper for at least some of the quarters in 2012. Can you tell me about your decision to respond on paper? How can we encourage companies like yours to respond on the Web?

• What are the benefits, if any, for completing the questionnaire online?

• What is the downside, if any, to completing the questionnaire online?

• If we did not send you a paper questionnaire in the mail, how would that affect your response process?

• What role, if any does the paper questionnaire play in your process for filling this questionnaire out?

• Do you need to keep a copy of your answers? If so, in which format do you prefer: hard copy, pdf, both, or some other format?

Data Transfer Questions

Finally, I want to briefly change topics and discuss with you the idea of transferring files between businesses. Have you heard of the term file transfers before today? (Yes/No) Is this a term that is used within your organization? (Yes/No) If no, what term is used?

1. Are you familiar with the concept of a data transfer or file transfer?

   1. IF YES: Can you briefly tell us about your experiences with file transfers?

      1. Have you ever provided financial or other data to another party outside of your organization in the form of a file transfer [note: need correct term]?

      2. Probe if needed: What type of organization did you transfer the data to?

      3. Probe if needed: What type of data did you send?

   2. IF NO: Do you know of anyone in your company who is involved with file transfers?

2. How are your financial/accounting records normally organized? If you were to transfer a file to us, what type of structure would that file have (e.g., by region, by business unit, by cost center)?

3. Could you provide any geographical information along with that data? For example, country, US state, ZIP code, etc.? If so, what would it be? What is the most granular level of geography in your records?

4. On scale of 1 to 5, with 1 being very unwilling and 5 being very willing, how willing would your company be to provide data to the Census Bureau in this manner, in lieu of completing a survey?

5. What software does your company use for your tracking your shipments?

6. Would you be willing to talk about file transfers with us in more detail at a later time?

Company Name

Contact Name

Address

Phone

Email

City, State ZIP

SURVID

Company Size (L, M, or N):

Survey Name

Wrap-up

• Are you the person who would probably answer these questions, or would someone else answer them?

• Would the information requested in this questionnaire generally be available to you, or would information have to come from someplace else?

• How would you go about getting that information?

• Would you pass the form over to that department, or just get information from them over the phone/e-mail?

• Overall, how easy or difficult was it to complete this questionnaire?

• Any other comments?

Thank you so much for assisting us today.