Protocol

Differential Probabilities of Omission Across

Individual and Context Factors

Study Protocol

MATERIALS NEEDED FOR INTERVIEW

• Interviewer Protocol Script

• Two Consent Forms (participant may request a copy)

• Demographics questionnaire

• Internet use questionnaire

• Experimental task questionnaire

• Debriefing questions document

• Laptop (with charger)

• Monitor pad (with cable)

• Incentive

• Payment Receipt

• Pens and Paper

Testing Script

Thank you for your time today. My name is XX and I work with the Human Factors and Usability Group at the U.S. Census Bureau. I will be working with you today.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to record your screen and your voice as you complete the task. We do this to get an accurate record of your feedback. Only those of us connected with the project will review the recording and it will be used solely for research purposes. Your name and face will not be associated with the recording or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date. Show participants where the camera is and what part of the screen will be recorded start recording.]

Thank you. I would also like you to answer a few questions about basic demographic information and internet experience.

[If paper task: Hand demo and internet questionnaire]

[If online task: Open demo and internet questionnaire]

Thank you.

Today you will be helping us explore how people in different life situations would respond to the census form. We are interested in what makes people decide who to include or exclude from the census. While you complete the task, I want you to answer exactly as you would answer if you were completing the census form today, but with one major difference: While you complete the task, I want you to think out loud.

Think Aloud: I am interested in your answers to the task, but I am also interested in the process you go through in your mind when you answer the questions. I would like you to tell me everything that you are thinking and feeling as you go answering the questions. This is going to help me know what kind of things come to your mind and what kinds of things you consider when answering the questions. Because we are not used to thinking outloud in front of other people, let’s do an exercise: While you think outloud, please answer the question “How many windows are there in your home?”

[Probe if they fall silent; if they just give a number ask them to do it again but this time so that you can understand how they came up with the number; after they do the think aloud explain why we are asking them to do this, e.g., if the Census was interested in the number of windows, which we are not, but if we were, and someone was looking at their sliding glass door and wondered aloud that they didn’t know if they should include it or not, them saying those words would clue us in that we needed to fix our question, make it clear that sliding glass doors should be included, etc. Probes should be kept to a minimum – trying to not interfere with the timing data, so used sparingly: Keep talking…; Um-hum?; What are you thinking?; Tell me more…

Great that’s what I want you to do throughout our session. I will remind you to think aloud if you get quiet.

At the end of the session I will have some more specific questions for you about how you answered each of the questions in the task.

Experimental task

[If paper task: Write participant’s number on paper questionnaire. All paper questionnaires are printed with random order of items and responses]

[If online task: Open survey on Qualtrics and enter participant number. Online task has been programmed so that the order of items and responses are randomized for all participants]

Here are the questions that I would like to answer. As a reminder, these questions are different types of life situation. While answering each question, I want you to imagine that this is your current life situation. While you think outloud about the things you consider when answering the questions, I will be making notes. If you fall silent, I may nudge you to keep talking by saying something like “what are you thinking?”

Please start by reading the instructions and the first question out loud.

[Administer main experimental task. Monitor responses and make note of vignettes with omission]

[When participant has completed the questionnaire say the following:]

Ok, now we are going to go back to some of the questions so you can tell me a little bit more about what things you considered when deciding who to count or not to count in the census.

[For vignettes where an omission was observed, complete debriefing question]

Debriefing question:

In the situation where [insert here situation in vignette], you answered that you would count [insert here participant’s response].

Can you please tell me a little more about what you took into consideration to determine who to count or not to count in this situation? [take notes of participant’s response]

Finishing up session

Ok, that is it for the task. Do you have any questions for me?

[Answer participants’ questions. Then hand the respondent incentive receipt and envelope with $40 incentive]

Please make sure that there are $40 in the envelope and sign this document stating that you received this compensation.

Thank you very much for your time and for your helpful feedback for this study.