2018 National Survey of Children’s Health
MATERIALS NEEDED FOR INTERVIEW
Interviewer Protocol Script
Consent Form
Mailed Invitation Letter with Login ID
Laptop with the Demographic and Satisfaction Survey
Incentive
Payment Receipt
Pens and Paper
Tobii x260 and mobile stand
Laptop
Guest Wireless Username and Password
Extra Equipment and Charging Cords
INTERVIEWER USABILITY TESTING SCRIPT
Thank you for your time today. My name is XX and I work at the U.S. Census Bureau. I will be working with you today. In this lab, we evaluate how easy or difficult our various web survey products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it before it goes live to a much larger group.
Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to take a video recording of the device as you use it along with the audio from this session to get an accurate record of your feedback. Your name will not be associated with the recording or any of the other data collected during the session.
[Hand NSCH consent form; give time to read and sign; sign own name and date. Show participants where the camera is and what part of the screen will be recorded start recording.]
OK great.
Thank you.
There are several parts to today’s session.
The first task is to complete this questionnaire. It collects background information about you.
[Have participant fill out the usability background materials: demographic questionnaire on the laptop (for English) and on paper for Spanish—simply move away from where they are answering so they don’t feel like we are looking over their shoulder.]
Thank you.
The second part of today’s session is for you to help us evaluate the design of a survey that we are working on for the U.S. Department of Health and Human Services’ (HHS) Health Resources and Services Administration’s Maternal and Child Health Bureau (HRSA MCHB). Today you will be helping us evaluate the design of the 2018 National Survey of Children’s Health by completing it using [your smartphone / this laptop].
If you run into any difficulties as you work on the survey, please don’t blame yourself. Any difficulties are the result of the design of the Web survey, not your skills or abilities. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the survey. I did not create the online survey, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the online survey work well for everyone.
Think Aloud: I want you to respond to the web instrument exactly the way you would if you were at home, but with one major difference. I would like you to think aloud as you work within the instrument. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. I would like you to tell me everything that you are thinking and feeling as you go about answering each question.
Let’s do a practice question before we start: Please think aloud as you answer the question, how many windows are in your home?
[Probe if they fall silent; if they just give a number ask them to do it again but this time so that you can understand how they came up with the number; after they finish the think-aloud exercise, explain why we are asking them to do this, e.g., if the Census was interested in the number of windows, which we are not, but if we were, and someone was looking at their sliding glass door and wondered aloud that they didn’t know if they should include it or not, them saying those words would clue us in that we needed to fix our question, make it clear that sliding glass doors should be included, etc.
Probes that will be used in study
Keep talking
Um-hum?
What are you thinking?
What are you including?]
Great that’s what I want you to do throughout our session. I will remind you to think aloud if you get quiet.
We are also going to record where you look on the screen as we work on our study. Now we will do a simple task that will allow the computer to find your eyes. To calibrate your eyes, please follow the dot with your eyes.
[Calibrate the participants’ eyes and make sure you start on google.com]
[Hand participant the mailing materials (draft invitation letter with their user name/ID info. Make certain to write down names and ages of the children listed and make note of the child to follow up based on the ages given and the assigned panel. If the age range you expected is not listed, you must choose another child and quickly find another letter. ]
This is an example of mailing materials you would receive at your home if you were selected for the 2018 National Survey of Children’s Health. Please take a moment to read it. You can also let me know if anything stands out to you in the letter. Please answer the questions as they apply to you in real life. You may begin.
[Hand participant the correct mailing materials for the panel they are in (invitation letter with their Login information. Make certain to write down the generated PIN or any information that the user may need later in survey]
STOP Respondent on the child review screen (about half way through the survey) and ask them to pretend that they had to leave the instrument to do something else.
Once they are exited, then ask them to resume. You may either text them or type the correct URL for them if they had difficulty in the first round, especially for smartphones.
STOP Respondent on the child review screen (about half way through the survey) and ask them to pretend that they had to leave the instrument to do something else.
Once they are exited, then ask them to resume with a different login ID and PIN/Security Question (depending on what they used). Prime the case beforehand. You may either text them or type the correct URL for them if they had difficulty in the first round, especially for smartphones.
Now pretend that you have this Login ID and PIN/Security code. Within the survey, questions will refer to Jamie Doe as the name of the child. We know that is not your child’s name. We want you to answer this survey for NAME OF CHILD. You will have to pretend that Jamie Doe is that child’s name. That is the only part of the study that is pretend.
After final SUBMIT is clicked, have participant answer satisfaction questions.
The next task is for you to answer a short questionnaire about your experience completing the survey you just finished.
[Open the satisfaction survey on the laptop and have participant answer the questions.]
Debriefing: Ask all the probes using a PowerPoint slide with those screens. We will skip the screens that the participant did not see.
The next task is to go over each of the screens you saw.
Each screen will have the following probe:
(If we observed issues): Please tell me more about your experience as you answered the question on this screen. (If the person forgets, say something like, “I thought I saw you /heard you say… , please tell me more about what you were thinking.”
Is there anything else you’d like to mention that we haven’t talked about?
[Give the respondent incentive envelope [$40], and thank them for their time and effort. Collect materials and leave or escort respondent out of the interviewing room.]
Okay, that’s all that I have today. Many thanks for your work on this project. The last thing I need you to do is sign this form that says I am giving you $40.
NOTES PAGES
P#___
Characteristic of Participant: 0-5 6-11 12-17
TA:______________________
Date: _____________________
Location: Headquarters / Other :________________________
Device: _____________________
WiFi or Dataplan
Browser type used:_____________________________________
Other browsers on device:________________________________
Accessing URL
Used search / typed in URL
No problem/Problems
What happened?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Erica Olmsted Hawala (CENSUS/CSM FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-21 |