Protocol

Attachment B BTOS Work from Home Supplement Draft Protocol 5.9.24_OMB Census responses 5.21.24 CLEAN.docx

Generic Clearance for Questionnaire Pretesting Research

Protocol

OMB: 0607-0725

Document [docx]
Download: docx | pdf

Attachment B

Cognitive Interviewing Protocol

Business Trends and Outlook Survey – Work from Home Supplement

5.9.24 draft (revised 5.21.24)



Interviewer instructions are in red.



Research Questions to Address:

  • Are respondents able to answer the questions as intended?

  • Are there any items that cause respondents problems?

    • What can be done to fix those problems?

  • Estimate of average time to complete?



Introduction:

  • Hello. Thank you for your time today. We are looking to obtain feedback on a survey that the Census Bureau has been conducting to provide insight into the state of the economy by providing timely data for key economic measures and business expectations about future conditions.

  • We plan to use your feedback to improve the survey questions and make sure they make sense to respondents like you. We are not testing you– we only want to evaluate the questionnaire.

  • Today, we will have you complete the short questionnaire, then you will be asked some questions to get your feedback.

  • Thank you for signing the consent form. As noted, this study is being conducted under the authority of Title 13 USC. (If they did not yet sign: I just sent you an email. Can you please click on the link to the Qualtrics survey and read over the first page? This study is being conducted under the authority of Title 13 USC. Do you agree to participate? Thank you. Please click on the checkbox at the bottom of the screen, but do not advance the screen just yet.)



Before the Questionnaire:

  • Before we begin, could you tell me a little bit about your business -- what types of goods or services does this business provide? How is the business organized?

  • Thank you. And what is your role in the business? (What kind of responsibilities do you have?)





Have Respondent Complete the Questionnaire:


Now I’d like for you to go through the survey question by question. Please tell me your answer to each question as you go along. After you are done, we will go back and discuss some of the questions in more detail.



Review the Questionnaire:


Thank you for completing the survey. On your screen should be a summary of your responses. You can use this as we talk a little bit more about some of the questions you have just answered. We will be skipping over some questions to focus on some new questions. But first, I have some general questions for you.



General probes that may be used for any question throughout the interview:

  • In your own words, what is this question asking?

  • Reflect back on respondent’s answer: “you said…”

  • How did you arrive at this number/answer this question?

  • What records (if any) would you look at?

  • Specifically, what would you include in this answer? What would you exclude?

  • Would you consult other people to obtain this answer?

  • How easy or difficult is it to answer this question?



First, what is your overall impression of this survey?



How comfortable were you in answering these questions for your company – very comfortable, somewhat comfortable, not very comfortable or not at all comfortable?

  • Very comfortable

  • Somewhat comfortable

  • Not very comfortable (Can you say more about this?)

  • Not at all comfortable (Can you say more about this?)



How confident are you in your answers to these questions – very confident, somewhat confident, not very confident or not at all confident?

  • Very confident

  • Somewhat confident

  • Not very confident (Can you say more about this?)

  • Not at all confident (Can you say more about this?)



If you had been answering these questions without being on the phone with me, would you likely have consulted records or others within your company?

  • Yes, records (What records would you consult?)

  • Yes, others within company (Who would you consult with?)

  • No



About how long do you think it would have taken you to complete this survey (if we hadn’t been on the phone)?



Thinking back through all the questions you answered for your overall company, do you feel you were the best person to answer these questions or would someone else at the company have been a better fit?

  • Yes, I was the best person

  • No, someone else would have been a better fit (What is that person’s job title?)



[For Multi-units only] The questions asked about the business as a whole. How representative do you think your answers were, in terms of the various locations of the business’s establishments – they represented all locations of the business, they represented most locations of the business, they represented some locations of the business or they were not representative of the business?

  • Represented all locations of the business

  • Represented most locations of the business

  • Represented some locations of the business

  • Not representative of the business



[For Multi-units only] How representative do you think your answers were, in terms of the various parts of the business (such as industries or divisions) – they represented all parts of the business, they represented most parts of the business, they represented some parts of the business or they were not representative of the business?

  • Represented all parts of the business

  • Represented most parts of the business

  • Represented some parts of the business

  • Not representative of the business




Overall, how easy or difficult was it to answer this survey – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you say more about what was difficult?)

    • Very difficult (Can you say more about what was difficult?)




Item 1


Now let’s go back and look at Item 1.


Overall, how would you describe this business’s current performance? Please include all domestic operations of this consolidated business, including all locations, subsidiaries, and divisions.

  • Excellent

  • Above average

  • Average

  • Below average

  • Poor



  • How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • How did you determine which parts of your business to answer for?


  • Do you find the instruction after the question helpful or not helpful? Do you have any suggestions for improving this instruction?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Items 3-9

The following questions are core content but are included in this testing to give the respondent context for the new/supplemental content.


Next let’s look at Item 3.


Between MMM DD – MMM DD, how did this business’s number of paid employees change?

  • Increased

  • Decreased

  • No change


  • How did you come up with your answer to this question?

  • What does “paid employees” mean to you? Who are you including?


Question 4


Between MMM DD – MMM DD, did this business use Artificial Intelligence (AI)1 in producing goods or services? (Examples of AI: machine learning, natural language processing, virtual agents, voice recognition, etc.)

  • Yes

  • No

  • Do not know



  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “Artificial Intelligence” mean to you?


  • What does “in producing goods or services” mean to you?


  • Would you likely click on the hyperlink for “Artificial Intelligence”?



  • Do you find the examples after the question helpful or not helpful? Do you have any suggestions for improving these examples?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Now let’s look at question 5.

In the last six months, did this business experience any monetary losses due to an extreme weather event (for example, hurricane, flood, drought, heat wave, etc.)?

Include incurred costs and losses of sales or assets.

  • Yes

  • No (skip to 7)





  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “experience any monetary losses” mean to you?


  • What does “an extreme weather event” mean to you?


  • Do you find the examples helpful or not helpful? Do you have any suggestions for improving these examples?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Question 6.

What type of extreme weather event caused the monetary losses? Select all that apply.

  • Wildfire

  • Earthquake

  • Hurricane

  • Tornado

  • Flood

  • Drought

  • High winds

  • Severe storm/hail

  • Snow/blizzard

  • Ice

  • Extreme temperature (heat or cold)

  • Other (please describe ___________)


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • Are there any answer choices potentially missing from this list?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Next let’s look at Item 7.


During the next six months, do you think this business will be using Artificial Intelligence (AI)2 in producing goods or services? (Examples of AI: machine learning, natural language processing, virtual agents, voice recognition, etc.)

  • Yes

  • No

  • Do not know


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Item 10 (Work from Home Supplement begins)


Next let’s look at Item 10. This question begins the Work from Home supplement. What do you think of the introductory text above Item 10? Do you find it helpful or not helpful?



Does this business currently have any full-time employees who work from home for at least 6 hours in a day?

  • Yes

  • No (skip to 12)



  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “work from home” mean to you?


  • Are you able to easily separate out “full-time employees”?


  • What does “for at least 6 hours in a day” mean to you?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Item 11


Approximately what percentage of this business’s full-time employees currently work from home for at least6 hours in a day)?


____% do so rarely

____% do so ~1 day per week

____% do so ~2 days per week

____% do so ~3 days per week

____% do so ~4 days per week

____% do so ~5 or more days per week

____

100%


  • Do not know/Not applicable


  • How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • How easy or difficult was it to report percentages for each of the categories?


  • Were there any categories that would be difficult for you to report? Tell me more.


  • Is it clear that the percentages should add to 100%?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Alt-11a


I would like you to review an alternative version of this question:


Does this business currently have full-time employees who work from home for at least 6 hours in a day for any of the following options? Select all that apply.


  • Rarely

  • ~1 day per week

  • ~2 days per week

  • ~3 days per week

  • ~4 days per week

  • ~5 or more days per week

  • Do not know/Not applicable



  • Compared to the previous question, do you find this version easier or more difficult to answer? Do you have a preference?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 12


Five years ago (in 2019), approximately what percentage of this business’s full-time employees worked from home for at least 6 hours in a day?


____% did so never

____% did so rarely

____% did so ~1 day per week

____% did so ~2 days per week

____% did so ~3 days per week

____% did so ~4 days per week

____% did so ~5 or more days per week

____

100%


  • Do not know/Not applicable


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What time period were you thinking about?


  • Is there another time period that would make more sense? What about 1 year ago? 3 years ago?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 13


Looking forward to five years from now (in 2029), approximately what percentage of this business’s full-time employees do you think will work from home for at least 6 hours in a day?

____% will do so never

____% will do so rarely

____% will do so ~1 day per week

____% will do so ~2 days per week

____% will do so ~3 days per week

____% will do so ~4 days per week

____% will do so ~5 or more days per week

____

100%


  • Do not know/Not applicable


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What time period were you thinking about?


  • Would another time period make more sense? What about 1 year from now? 3 years from now?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 14


Does this business currently have any full-time employees who work all their paid hours from home (fully remote)?


    • Yes

    • No (skip to 16)


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “full-time employees who work all their paid hours from home” mean to you? Is there another term you use for this concept?


  • Do you find the clarification (fully remote) in parentheses helpful or not helpful?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 15 and 15a

Half of respondents will receive the original version of the question, and half will receive version 31a, please note which version the respondent views.


Does this business pay its fully remote full-time employees a local rate that varies by city, state, or region?

    • Yes

    • No



Alt15a


Does this business pay its fully remote full-time employees based partly on where they live?

    • Yes

    • No


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • For 15: What does “a local rate that varies by city, state or region” mean to you? Or for 15a: What does “based partly on where they live” mean to you?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 16


What factors limit the ability of this business’s full-time employees to work from home? Select all that apply.


    • Parts or all of the job cannot be performed from home

    • Management of employees working from home too costly/complicated

    • Efficiency/productivity concerns

    • Challenges around mentoring/learning

    • Legal/regulatory/tax reasons

    • Security concerns (IT or other)

    • Other (please describe ___________)

    • No factors limit work from home


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • Are there any answer choices potentially missing from this list?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 17

Does this business have an official work from home policy that includes minimum onsite requirements?



    • Yes

    • No (skip to 19)


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “official work from home policy” mean to you?


  • What does “minimum onsite requirements” mean to you?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 18

How does this business ensure employees meet the minimum onsite requirements? Select all that apply.



  • Badge swipe or sign-in sheet

  • Attendance at in-person meetings

  • Manager checks

  • Other (please describe: ___________________ )

  • This business does not track in-office presence


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • Are there any answer choices potentially missing from this list?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 19


Does this business monitor the productivity of full-time employees?


    • Yes

    • No (skip to 23)


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What does “monitor the productivity of full-time employees” mean to you?


  • If Yes, What types of monitoring are you thinking about?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 20

How does this business monitor full-time employees working from home? Select all that apply.

  • Login/logoff times

  • Computer activity

  • Attendance/participation in online meetings

  • Specific measures of output (e.g., number of customers served or calls answered, sales, units produced, etc.)

  • Other (please describe ___________)

  • This business does not monitor employees working from home.


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • Are there any answer choices potentially missing from this list?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 21

How does this business monitor full-time employees working onsite? Select all that apply.



  • Arrival/departure times

  • Login/logoff times

  • Computer activity

  • Attendance/participation in online meetings

  • Attendance/participation at in person meetings

  • Specific measures of output (e.g., number of customers served or calls answered, sales, units produced, etc.)

  • Other (please describe ___________)

  • This business does not monitor employees working onsite.


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • Are the response options clear or unclear?


  • Are there any answer choices potentially missing from this list?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Item 22 and 22a

Half of respondents will receive the original version of the question, and half will receive version 22a, please note which version the respondent views.



Has this business observed differences in productivity of full-time workers based on location?

  • Yes, work from home more productive

  • Yes, onsite more productive

  • No differences in productivity

  • Do not know/Not applicable


Alt22a


Has this business observed differences in productivity for full-time employees based on whether they are working from home or onsite?



  • Yes, work from home more productive

  • Yes, onsite more productive

  • No differences in productivity


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • For 22: What does “differences in productivity based on location” mean to you? Or For 22a: What does “differences in productivity for full-time employees based on whether they are working from home or onsite” mean to you?


  • Are the response options clear or unclear?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)




Item 23


Has the adoption of work from home by other businesses had an impact on this business? For example, this business has experienced lower sales due to a decrease in neighborhood foot traffic, or this business has experienced increased demand for products that facilitate working from home.


  • Yes, positive impact overall

  • Yes, negative impact overall

  • No impact


  • Please tell me in your own words what this question is asking. How did you come up with your answer to this question? Would you need refer to records or reach out to anyone else in your business to answer this question? If yes, Who/what?


  • What types of impacts were you thinking of when answering this question?


  • Do you find the examples after the question helpful or not helpful? Do you have any suggestions for improving these examples?


  • What time period were you thinking about?


  • How easy or difficult was it to answer this question – very easy, somewhat easy, neither easy nor difficult, somewhat difficult or very difficult?

    • Very easy

    • Somewhat easy

    • Neither easy nor difficult

    • Somewhat difficult (Can you tell me more about that?)

    • Very difficult (Can you tell me more about that?)



Wrap Up:


Wrapping up now!


  • Overall, how easy or difficult was it to answer these questions?


  • Do you have any other comments or suggestions about anything we have discussed today?



Thank you very much for your time today. We really appreciate your help.

1 Hyperlink: AI Definition: Computer systems and software that are able to perform tasks normally requiring human intelligence, such as decision-making, visual perception, speech recognition, and language processing.


Types or applications of AI include machine learning, natural language processing, virtual agents, predictive analytics, machine vision, voice recognition, decision making systems, data analytics, image processing, etc.

2 Hyperlink: AI Definition: Computer systems and software that are able to perform tasks normally requiring human intelligence, such as decision-making, visual perception, speech recognition, and language processing.


Types or applications of AI include machine learning, natural language processing, virtual agents, predictive analytics, machine vision, voice recognition, decision making systems, data analytics, image processing, etc.

32


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified0000-00-00
File Created2025-08-12

© 2025 OMB.report | Privacy Policy