IRS Account Usability Test
Test Materials
IRS Account Usability Test Script 2
Moderator Checklist 5
Paperwork Reduction Act Statement 6
Consent & Recording Release Form 7
Pretest Demographic and Background Information Questionnaire 8
Single Ease Question 9
Recruitment Email 10
Confirmation Email 11
Reminder Email 12
Receipt Form: Usability Test Compensation 13
Recruiting Screener 14
User Personas 19
Welcome and Purpose
Thank you so much for coming in today. I want to give you a little information about what you will be looking at and give you time to ask any questions you might have before we get started.
Today we are asking you to evaluate a website and complete a short task. Our goal is to find out how easy it is or how difficult it is for you to use the site
Test Facilitator’s Role
I am here to record your reactions to and your comments about the website. I have a colleague helping me take notes and observe your interactions with the site as well.
During this session, I would like you to think aloud as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.
Demonstrate
how to think out loud.
Test Participant’s Role
Today I am going to ask you to describe what you are seeing on the screen, find some information and then tell me how easy or difficult it was. There are no right or wrong answers. This is an evaluation of the website, not you! If you have any questions, comments or feel confused about anything while you are working, please let me know.
If you ever feel that you lost or that you cannot complete the task, please let me know. I will ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next step in the process.
We have created a simulation of what the website might look like before it is actually built. Therefore, rather than seeing your data, I will provide you with a made up name, address and other credentials to use on the website today. As you work through the tasks, please try and act as if you are at home or work, but use the simulated data in place of your own.
Consent: Request participant read and sign consent/recording release form.
Each session is recorded. We will be recording your face, your voice and what you see on the screen. Your actual name will not be used or reported with data or findings associated with this evaluation.
I’ll ask you questions as we go along and we will have wrap up questions at the end.
Do you have any questions before we begin?
Pre-Task Questions
Ok, let’s start with a few questions.
First, please fill out this questionnaire. Provide pretest demographic and background information questionnaire.
Have you ever visited IRS.gov before? If so, can you recall the purpose of your visit?
Thank you. Let’s get started.
Usability Tasks Types
Generic Task 1 – Comprehension: Describe the purpose and functionality on this page, try to use it without instruction.
Suppose you came to IRS.gov, logged in, and saw this screen.
[Open a web browser to prototype]
Tell me what it looks like you can do here, how it might work, and how it might be useful to you.
Now that you’ve described some of its features, can you please try [describe specific piece of functionality].
Remember, you are pretending to be [USER PERSONA], so don’t expect to see your own personal information, simply pretend you are [USER PERSONA] and make up a response that seems to fit.
Generic Task 2 – Task Completion: Navigation and page functionality
Imagine that you are [USER PERSONA] and you are now on the homepage of your logged-in personalized account area. Suppose you have to __________[e.g., get past tax records as part of applying for a home loan]__________. How would you get to the area of the site that allows you to do that, and then complete the process in order to get what you want?
Single Ease Question: Overall, how difficult or easy was it to complete the sign in process?
[Record navigation path, any difficulties, and outcome]
Post-Task Questions
Thanks. We’ve completed the tasks for our session. I have a few more questions for you.
Ask any follow up questions based on the participants experience with the IRS Account Prototype.
That’s it for our session! Do you have any questions or additional comments?
Post Test Check List:
Incentive
Sign Incentive Receipt
Close and save Morae recording
ID number on notes, consent forms, receipt
Look at and remember person’s name prior to going to meet him/her.
Introduce yourself and other team members who might interact with participant.
Ask if person would like something to drink and/or to use the restroom.
Ask person to read and sign the consent form.
Reference the OMB Control Number and PRA Statement (provide printed copy):
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Review the purpose of the session.
Ask participants if they have any questions.
Ensure that participant completes post-study questionnaire and discuss his/her responses with him/her. Ask if (s)he has any other comments about the site.
Have participant sign receipt form and record honorarium check number on signed form.
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
I agree to participate in the study conducted and recorded by the Internal Revenue Service (IRS). I understand that the study will take approximately 50 minutes.
I understand and consent to the use and release of the audio, video, and/or digital recording by the IRS. I understand that the information and recording is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by IRS without further permission.
I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrator.
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Please sign below to indicate that you have read and you understand the information on this form and that any questions you might have about the session have been answered.
Date |
|
Please print your name |
|
Please sign your name |
|
Thank you! We appreciate your participation.
1. What is the primary language spoken at home? (Please select one)
English Korean
Spanish Russian
Chinese Other (please specify):________
Vietnamese
2. What payment option do you use the MOST to pay your bills?
Mail in payments
In person
Over the phone
Through the Internet via scheduled electronic bill pay
Through the Internet via same day bill pay
3. Do you have a computer at home?
Yes
No
4. Do you own and use a mobile phone?
Yes, regular mobile phone
Yes, Smartphone (mobile phone with applications and internet access)
No
5. Which of the following social media sources do you use? (Check all that apply)
Facebook Tumblr
Twitter Other (specify)______________
YouTube None
Task One
Overall, how difficult or easy was it to complete the login process?
Very Difficult |
|
|
|
|
|
Very Easy |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
|
|
|
|
|
|
|
Task Two
Overall, how difficult or easy was it to complete the login process?
Very Difficult |
|
|
|
|
|
Very Easy |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
|
|
|
|
|
|
|
When possible, we will recruit using listserves or known pools of participants.
SUBJECT LINE: The IRS invites you to participate in a website study and earn $40!
Hello,
My name is [NAME], and I’m helping to run a usability study for the Internal Revenue Service (IRS). In an effort to improve services, we’re looking for people who are interested in exploring new ways to interact with the IRS Online. Afterward we will simply ask for your feedback. If you agree to participate we will pay you $40.
What will you be doing in a usability study?
You will be asked to perform several short tasks using a website. You will also be asked questions about your experience and what you think of the website.
How long is a study session?
50 minutes.
When and where?
The study will be held [DAYS, DATES, LOCATION]. You will need to participate [in person/remotely via screen sharing].
Interested in participating?
Please reply to this email with your contact information or call me at [ADD PHONE]. I will give you a call to ask you some questions so we can decide if you qualify for the study.
If you have any questions, please contact me at [EMAIL].
Thank you for your interest,
[NAME AND TITLE]
NOTE: This will be sent as an email or in an Outlook meeting notice.
SUBJECT LINE: Confirmation: Your participation in our usability study
Dear [PARTICIPANT NAME]:
Thank you for agreeing to participate to test the Internal Revenue Service website. As I mentioned, you will be asked to perform a few tasks on the website and give us thoughts about your experience. There is nothing to prepare before the session.
You are scheduled to participate as follows:
DATE: [DAY, DATE]
TIME: [TIME]
PLACE: [ADDRESS, LINK TO MAP]
A few key reminders:
You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)
During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.
With your permission, the session will be recorded. We will only use the recording in our research to decide how to improve the website. Your name will not be used for any purpose beyond this session.
Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so we can reschedule your session.
Thank you again!
[NAME OF FACILITATOR AND CONTACT INFORMATION]
SUBJECT LINE: Reminder: Website study tomorrow
Dear [PARTICIPANT NAME]:
Thanks again for agreeing to help us out with testing the Internal Revenue Service website. We are looking forward to talking with you.
You are scheduled to participate as follows:
DATE: [DAY, DATE]
TIME: [TIME]
PLACE: [ADDRESS, LINK TO MAP]
A few key reminders:
You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)
During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.
With your permission, the session will be recorded. We will only use the recording to decide how to improve the website. Your name will not be used for any purpose beyond this session.
Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so I can reschedule your session.
Thanks again!
[NAME OF FACILITATOR AND CONTACT INFORMATION]
Please sign below to indicate that you have received the promised compensation for your participation in testing today.
Date |
|
Amount received |
|
Please print your name |
|
Please sign your name |
|
Thank you!
We appreciate your participation.
Hello, I’m [NAME] from [COMPANY] calling on behalf of the Internal Revenue Service. The IRS wants to hear from you about how we are doing. Have you heard of our website IRS DOT GOV? What we would like to do is ask you a few questions to see if you qualify to participate in a 50 minute study we are conducting of our web site. If you do qualify, we’d like to schedule you as a participant during the week of [DATE]. We are very interested in getting taxpayers opinions on the design of our website. These evaluations will be used for research purposes only and any personal information you provide will not be used for anything other than this particular study. We will pay you to come to a testing facility and give us feedback on how we are doing. Are you be willing to answer a few questions to see if you qualify?
IF NO, Can I schedule a time to call you back at a more convenient time? IF NO, terminate. IF YES, schedule a time to call back.
IF YES, CONTINUE.
[Reference the OMB Control Number and PRA Statement as noted below before continuing with the interview]
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to us at:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Because we are seeking a wide variety of people, let me start by asking you some questions about yourself.
Have you ever taken part in a market research interview or usability study, either one on one or in a focus group type setting? IF NO SKIP TO Q4
When was the last time you participated in a market research interview or usability study? TERMINATE IF LESS THAN 6 MONTHS
Do you or any member of your immediate family currently work for or did they ever work for any of the following?
|
Marketing, Market Research Firm |
|
Advertising |
|
Web Site Design Company |
IF YES TO ANY, TERMINATE
|
For a city, state or federal government agency? If so, which one? |
IF IRS, TERMINATE
Do you have access to the internet where
you can use it for your own personal use?
YES
NO
TERMINATE
On a scale of 1 to 5 where 1 means ‘not at all comfortable’ and 5 means ‘very comfortable’, how comfortable are you with… TERMINATE IF 1 OR 2
using a computer mouse |
1 |
2 |
3 |
4 |
5 |
using a computer keyboard |
1 |
2 |
3 |
4 |
5 |
using the Internet |
1 |
2 |
3 |
4 |
5 |
using a scroll bar |
1 |
2 |
3 |
4 |
5 |
|
|
|
|
|
|
Do you use the Internet to do any of the following?
e-commerce (shop online) |
YES |
NO |
Surf the web |
YES |
NO |
pay bills online |
YES |
NO |
work related reasons |
YES |
NO |
research |
YES |
NO |
online banking |
YES |
NO |
pay taxes online/software |
YES |
NO |
TERMINATE IF REPLY NO TO ALL
Are you employed? IF NOT EMPLOYED SKIP TO 8A
If employed:
What is your occupation? |
|
What industry are you in? |
|
GET A GOOD MIX OF DIFFERENT OCCUPATIONS AND INDUSTRIES
8a. If Not Employed:
|
Retired |
|
Student |
|
Homemaker |
|
In between jobs |
What is the highest level of education you have completed?
DO NOT READ ANSWER CHOICES - GET A MIX
|
Some high school or less |
|
High school graduate |
|
Some college, vocational or trade school |
|
College graduate |
|
Graduate school |
Which age group from the following list do you belong?
GET A MIX - NO MORE THAN THREE PARTICIPANTS PER AGE GROUP
Under 18 |
TERMINATE |
18-29 |
|
30-39 |
|
40-49 |
|
50-59 |
|
60-65 |
|
66+ |
|
Have you ever visited the website IRS DOT GOV?
YES
NO SKIP TO Q17
What was the nature of your visit to the website?
|
to find forms |
|
to review tax regulations |
|
to find phone numbers |
|
file downloads |
|
online tax tools |
|
other |
Do you remember if the visit to the site was:
|
less than 15 minutes |
|
15 minutes to 30 minutes |
|
30 minutes to 1 hour |
|
longer than 1 hour |
|
Don’t remember |
Approximately how many times have you visited the website?
What ways have you ever contacted the IRS?
|
visited a local branch |
|
phone call |
|
spoken with a specific representative |
|
other |
Thank you, now, the following questions are for classification purposes only.
What is your current marital status? RECRUIT A MIX
|
Single |
|
Married/Living with partner |
|
Widowed |
|
Divorced/Separated |
RECRUIT TO OBTAIN 30 INDIVIDUALS
RECRUIT A MIX OF MEN AND WOMEN
Thank you for answering my questions. Based on your responses, you appear to qualify for participation in the study. Are you interested in helping by participating in a paid research project?
No POLITELY DISCONTINUE
Yes CONTINUE
The IRS will pay you $40, to test a portion of our web site and share your opinion with us in a 50 minute evaluation.
This is how the process works: As part of our research, we are conducting study sessions with a number of people from your area. If you choose to participate we will ask you to attempt to do specific tasks on a web site and then discuss your experience. We are evaluating our web site so we can better serve taxpayers. We will simply observe your experience and ask your opinions. The IRS will not use any personal information for anything other than this particular study. These evaluation sessions will be held on [DATE, TIME AT LOCATION]. They will last approximately 50 minutes. Would you be available for that particular date and time?
No POLITELY DISCONTINUE
Yes CONTINUE
Do you have any questions up to this point or special needs for coming to our facility?
[Answer inquiries appropriately and thoroughly.]
Again, let me thank you for your decision to participate. This research is very important to us, and your help is highly appreciated.
Now, for the rest of the steps in the process:
After I have gathered your information, we will send you a confirmation email with a map to the testing facility. You will also receive a reminder email prior to the evaluation session. In a moment, I will need your name and email address. I’d also like to get a phone number where we can reach you.
We will send you a reminder email prior to your scheduled appointment on [DATE, TIME]. We schedule one person at a time for these sessions so if you find that you cannot participate on your scheduled day, please give notice to [NAME, EMAIL ADDRESS, PHONE NUMBER] as soon as possible so we can reschedule your session.
Thank you. If you have any questions, please email [NAME] at [EMAIL].
Crystal Glass
YOUR PERSONAL INFORMATION
Name: |
Crystal Glass |
Social Security No. |
211-22-1236 |
Address: |
11568 Golden Ave. Orlando, FL 40706 |
Date of Birth: |
March 12, 1981 |
e-Mail Address: |
|
Phone Number: |
407-558-5524 |
YOUR LEGAL INFORMATION
Tax Return Data: |
Filing Status – Single |
Blue Berry
YOUR PERSONAL INFO
Name: |
Blue Berry |
Social Security No. |
211-22-1237 |
Address: |
55684 Boston Terrace Tampa, FL 44598 |
Date of Birth: |
May 12, 1981 |
e-Mail Address: |
|
Phone Number: |
407-788-5524 |
YOUR LEGAL INFO
Tax Return Data: |
Filing Status – Qualifying Widow |
TEST |
DATE |
LOCATION |
1 |
8/14/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
2 |
8/28/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
3 |
9/11/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
4 |
9/25/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
5 |
10/9/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
6 |
10/23/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
7 |
11/6/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
8 |
11/20/14 |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
9 |
12/4/14 9am-5pm |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
10 |
9am-5pm |
3 Bethesda Metro Center, Suite 700, Bethesda, Maryland, 20814 |
IRS eAuthentication Usability Study | OMB Control Number 1545-2256
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rebecca Scollan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |