eAuthentication Usability Study

Thank you so much for coming in today. I want to give you a little information about what you will be looking at and give you time to ask any questions you might have before we get started.

Today we are asking you to evaluate a website and complete a short task. Our goal is to find out how easy it is or how difficult it is for you to use the site

Test Facilitator’s Role

I am here to record your reactions to and your comments about the website. In a conference room nearby, I have a colleague helping me take notes and observe your interactions with the site as well.

During this session, I would like you to think aloud as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.

Demonstrate how to think out loud.

Test Participant’s Role

Today I am going to ask you to find some information and then tell me how easy or difficult it was. There are no right or wrong answers. This is an evaluation of the website, not you! If you have any questions, comments or feel confused about anything while you are working, please let me know.

If you ever feel that you lost or that you cannot complete the task, please let me know. I will ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next step in the process.

I will provide you with a made up name, address and other credentials to use on the website today. As you work through the tasks, please try and act as if you are at home or work, but use the fake credentials in place of your own.

Consent: Request participant read and sign consent/recording release form.

Each session is recorded. We will be recording your face, your voice and what you see on the screen. Your actual name will not be used or reported with data or findings associated with this evaluation.

I’ll ask you questions as we go along and we will have wrap up questions at the end.

Do you have any questions before we begin?

Pre-Task Questions

Ok, let’s start with a few questions.

First, please fill out this questionnaire. Provide pretest demographic and background information questionnaire.

Have you ever visited IRS.gov before? If so, can you recall the purpose of your visit?

Thank you. Let’s get started.

Usability Tasks

Today you’ll be using the name, address and credentials of [USER PERSONA], a fake persona. Please use these credentials and pretend you are [USER PERSONA] today while you use the website.

Task 1

Let’s say that [USER PERSONA] has a meeting about a loan at the bank later today. You’ve been asked to bring in a copy of your return transcript from 2013.

[Open a web browser to http://www.irs.gov/Individuals/Get-Transcript]

Use this website to get your return transcript from 2013. Remember, you are pretending to be [USER PERSONA], so where needed, use those credentials. For questions where you are not provided information, simply pretend you are [USER PERSONA] and make up a response that seems to fit.

Single Ease Question (on landing on the Get Transcript page, hand the participant SEQ form): Overall, how difficult or easy was it to complete the sign in process?

Ok, thanks. Please complete the task.

Thank you! Go ahead and log out.

Task 2

Now let’s say it is a few days later and the loan officer needs return transcripts from 2012 and 2011. Log in as [USER PERSONA] and get PDFs of the transcripts to email to them.

Single Ease Question (on landing on the Get Transcript page, hand the participant SEQ form): Overall, how difficult or easy was it to complete the sign in process?

Post-Task Questions

Thanks. We’ve completed the tasks for our session. I have a few more questions for you.

Ask any follow up questions based on the participants experience with eAuthentication.

That’s it for our session! Do you have any questions or additional comments?

Post Test Check List:

Incentive
Sign Incentive Receipt
Close and save Morae recording
ID number on notes, consent forms, receipt

Moderator Checklist

Look at and remember person’s name prior to going to meet him/her.
Introduce yourself and other team members who might interact with participant.
Ask if person would like something to drink and/or to use the restroom.
Ask person to read and sign the consent form.
Reference the OMB Control Number and PRA Statement (provide printed copy):

The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:

Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224

Review the purpose of the session.
Ask participants if they have any questions.
Ensure that participant completes post-study questionnaire and discuss his/her responses with him/her. Ask if (s)he has any other comments about the site.
Have participant sign receipt form and record honorarium check number on signed form.

Paperwork Reduction Act Statement

Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224

Consent & Recording Release Form

I agree to participate in the study conducted and recorded by the Internal Revenue Service (IRS). I understand that the study will take approximately 40 minutes.

I understand and consent to the use and release of the audio, video, and/or digital recording by the IRS. I understand that the information and recording is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by IRS without further permission.

I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrator.

The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:

Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224

Please sign below to indicate that you have read and you understand the information on this form and that any questions you might have about the session have been answered.

Date
Please print your name
Please sign your name

Thank you! We appreciate your participation.

Pretest Demographic and Background Information Questionnaire

1. What is the primary language spoken at home? (Please select one)

 English  Korean

 Spanish  Russian

 Chinese  Other (please specify):________

 Vietnamese

2. What payment option do you use the MOST to pay your bills?

 Mail in payments

 In person

 Over the phone

 Through the Internet via scheduled electronic bill pay

 Through the Internet via same day bill pay

3. Do you have a computer at home?

 Yes

 No

4. Do you own and use a mobile phone?

 Yes, regular mobile phone

 Yes, Smartphone (mobile phone with applications and internet access)

 No

5. Which of the following social media sources do you use? (Check all that apply)

 Facebook  Tumblr

 Twitter  Other (specify)______________

 YouTube  None

 LinkedIn

Single Ease Question

Task One

Overall, how difficult or easy was it to complete the login process?

Very Difficult

Very

Easy



Task Two

Overall, how difficult or easy was it to complete the login process?

Very Difficult

Very

Easy



Recruitment Email

When possible, we will recruit using listserves or known pools of participants.

SUBJECT LINE: The IRS invites you to participate in a website study and earn $40!

Hello,

My name is [NAME], and I’m helping to run a usability study for the Internal Revenue Service (IRS). In an effort to improve services, we’re looking for people who are interested in exploring new ways to access their tax records. Afterward we will simply ask for your feedback. If you participate we will pay you $40.

What will you be doing in a usability study?

You will be asked to perform several short tasks using a website. You will also be asked questions about your experience and what you think of the website.

How long is a study session?

40 minutes.

When and where?

The study will be held [DAYS, DATES, LOCATION]. You will need to participate in person.

Interested in participating?

Please reply to this email with your contact information or call me at [ADD PHONE]. I will give you a call to ask you some questions so we can decide if you qualify for the study.

If you have any questions, please contact me at [EMAIL].

Thank you for your interest,

[NAME AND TITLE]

Confirmation Email

NOTE: This will be sent as an email or in an Outlook meeting notice.

SUBJECT LINE: Confirmation: Your participation in our usability study

Dear [PARTICIPANT NAME]:

Thank you for agreeing to participate to test the Internal Revenue Service website. As I mentioned, you will be asked to perform a few tasks on the website and give us thoughts about your experience. There is nothing to prepare before the session.

You are scheduled to participate as follows:

DATE: [DAY, DATE]

TIME: [TIME]

PLACE: [ADDRESS, LINK TO MAP]

A few key reminders:

You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)
During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.
With your permission, the session will be recorded. We will only use the recording in our research to decide how to improve the website. Your name will not be used for any purpose beyond this session.

Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so we can reschedule your session.

Thank you again!

[NAME OF FACILITATOR AND CONTACT INFORMATION]

Reminder Email

SUBJECT LINE: Reminder: Website study tomorrow

Dear [PARTICIPANT NAME]:

Thanks again for agreeing to help us out with testing the Internal Revenue Service website. We are looking forward to talking with you.

You are scheduled to participate as follows:

DATE: [DAY, DATE]

TIME: [TIME]

PLACE: [ADDRESS, LINK TO MAP]

A few key reminders:

You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)
During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.
With your permission, the session will be recorded. We will only use the recording to decide how to improve the website. Your name will not be used for any purpose beyond this session.

Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so I can reschedule your session.

Thanks again!

[NAME OF FACILITATOR AND CONTACT INFORMATION]

Receipt Form: Usability Test Compensation

Please sign below to indicate that you have received the promised compensation for your participation in testing today.

Date
Amount received
Please print your name
Please sign your name

Thank you!

We appreciate your participation.

Recruiting Screener

Hello, I’m [NAME] from [COMPANY] calling on behalf of the Internal Revenue Service. The IRS wants to hear from you about how we are doing. Have you heard of our website IRS DOT GOV? What we would like to do is ask you a few questions to see if you qualify to participate in a 40 minute study we are conducting of our web site. If you do qualify, we’d like to schedule you as a participant during the week of [DATE]. We are very interested in getting taxpayers opinions on the design of our website. These evaluations will be used for research purposes only and any personal information you provide will not be used for anything other than this particular study. We will pay you to come to a testing facility and give us feedback on how we are doing. Are you be willing to answer a few questions to see if you qualify?

IF NO, Can I schedule a time to call you back at a more convenient time? IF NO, terminate. IF YES, schedule a time to call back.

IF YES, CONTINUE.

[Reference the OMB Control Number and PRA Statement as noted below before continuing with the interview]

Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224

Because we are seeking a wide variety of people, let me start by asking you some questions about yourself.

Have you ever taken part in a market research interview or usability study, either one on one or in a focus group type setting? IF NO SKIP TO Q4

When was the last time you participated in a market research interview or usability study? TERMINATE IF LESS THAN 6 MONTHS

Do you or any member of your immediate family currently work for or did they ever work for any of the following?

	Marketing, Market Research Firm
	Advertising
	Web Site Design Company

IF YES TO ANY, TERMINATE

For a city, state or federal government agency? If so, which one?

IF IRS, TERMINATE

Do you have access to the internet where you can use it for your own personal use?
YES
NO TERMINATE

On a scale of 1 to 5 where 1 means ‘not at all comfortable’ and 5 means ‘very comfortable’, how comfortable are you with… TERMINATE IF 1 OR 2

using a computer mouse	1	2	3	4	5
using a computer keyboard	1	2	3	4	5
using the Internet	1	2	3	4	5
using a scroll bar	1	2	3	4	5

Do you use the Internet to do any of the following?

e-commerce (shop online)	YES	NO
Surf the web	YES	NO
pay bills online	YES	NO
work related reasons	YES	NO
research	YES	NO
online banking	YES	NO
pay taxes online/software	YES	NO

TERMINATE IF REPLY NO TO ALL

Are you employed? IF NOT EMPLOYED SKIP TO 8A

If employed:

What is your occupation?
What industry are you in?

GET A GOOD MIX OF DIFFERENT OCCUPATIONS AND INDUSTRIES

8a. If Not Employed:

	Retired
	Student
	Homemaker
	In between jobs

GET AT LEAST 1 NOT EMPLOYED.

What is the highest level of education you have completed?

DO NOT READ ANSWER CHOICES - GET A MIX

	Some high school or less
	High school graduate
	Some college, vocational or trade school
	College graduate
	Graduate school

Which age group from the following list do you belong?

GET A MIX - NO MORE THAN THREE PARTICIPANTS PER AGE GROUP

Under 18	TERMINATE
18-29
30-39
40-49
50-59
60-65
66+

Have you ever visited the website IRS DOT GOV?

YES

NO SKIP TO Q17

What was the nature of your visit to the website?

	to find forms
	to review tax regulations
	to find phone numbers
	file downloads
	online tax tools
	other

Do you remember if the visit to the site was:

	less than 15 minutes
	15 minutes to 30 minutes
	30 minutes to 1 hour
	longer than 1 hour
	Don’t remember

Approximately how many times have you visited the website?

What ways have you ever contacted the IRS?

	visited a local branch
	phone call
	spoken with a specific representative
	other

Thank you, now, the following questions are for classification purposes only.

What is your current marital status? RECRUIT A MIX

	Single
	Married/Living with partner
	Widowed
	Divorced/Separated

RECRUIT TO OBTAIN 30 INDIVIDUALS

RECRUIT A MIX OF MEN AND WOMEN

Thank you for answering my questions. Based on your responses, you appear to qualify for participation in the study. Are you interested in helping by participating in a paid research project?

No  POLITELY DISCONTINUE

Yes  CONTINUE

The IRS will pay you $40, to test a portion of our web site and share your opinion with us in a 40 minute evaluation.

This is how the process works: As part of our research, we are conducting study sessions with a number of people from your area. If you choose to participate we will ask you to attempt to do specific tasks on a web site and then discuss your experience. We are evaluating our web site so we can better serve taxpayers. We will simply observe your experience and ask your opinions. The IRS will not use any personal information for anything other than this particular study. These evaluation sessions will be held on [DATE, TIME AT LOCATION]. They will last approximately 40 minutes. Would you be available for that particular date and time?

No  POLITELY DISCONTINUE

Yes  CONTINUE

Do you have any questions up to this point or special needs for coming to our facility?

[Answer inquiries appropriately and thoroughly.]

Again, let me thank you for your decision to participate. This research is very important to us, and your help is highly appreciated.

Now, for the rest of the steps in the process:

After I have gathered your information, we will send you a confirmation email with a map to the testing facility. You will also receive a reminder email prior to the evaluation session. In a moment, I will need your name and email address. I’d also like to get a phone number where we can reach you.

We will send you a reminder email prior to your scheduled appointment on [DATE, TIME]. We schedule one person at a time for these sessions so if you find that you cannot participate on your scheduled day, please give notice to [NAME, EMAIL ADDRESS, PHONE NUMBER] as soon as possible so we can reschedule your session.

Thank you. If you have any questions, please email [NAME] at [EMAIL].

User Personas

Crystal Glass

YOUR PERSONAL INFORMATION

Name:	Crystal Glass
Social Security No.	211-22-1236
Address:	11568 Golden Ave. Orlando, FL 40706
Date of Birth:	March 12, 1981
e-Mail Address:
Phone Number:	407-558-5524

YOUR LEGAL INFORMATION

Tax Return Data:

Filing Status – Single

Blue Berry

YOUR PERSONAL INFO

Name:	Blue Berry
Social Security No.	211-22-1237
Address:	55684 Boston Terrace Tampa, FL 44598
Date of Birth:	May 12, 1981
e-Mail Address:
Phone Number:	407-788-5524

YOUR LEGAL INFO

Tax Return Data:

Filing Status – Qualifying Widow

IRS eAuthentication Usability Study | OMB Control Number 1545-2256

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Rebecca Scollan
File Modified	0000-00-00
File Created	2021-01-28

eAuthentication Usability Study

Collection of Qualitative Feedback on Agency Service Delivery

eAuth Usability Test Script and Materials