Revised Form W4 Usability

Task 7 Taxpayer Usability Testing Protocol

Interview information:

Date:

Time: ________am/pm to _________ am/pm

Interviewer:

Note Taker:

Participant ID:

Condition (circle): S1 S2 C1 C2

INTRODUCTION

Thank you for helping us out today. My name is [NAME] and I work as a researcher with the MITRE Corporation. MITRE is a not-for-profit company that runs federally funded research and development centers, including for the IRS.

I’d like to start out by telling you a little bit about what we’re doing today. As you may already know, the IRS has redesigned the W-4 form for 2020. Anytime the IRS redesigns a form or creates a new form, it is important to test how well the new form works for real taxpayers like you. Today, you’ll be helping us out by using fictitious tax information to fill out a current version of the form and a draft of the new version planned for use beginning in 2020. We will observe you while you complete the forms, and after you are done, I’ll ask a few questions.

I want to stress that my purpose here today is not to collect your personal information, but rather to understand if there are any problems or difficulties you encounter while filling out the new form, so we can make recommendations to the IRS on how to improve the form or better support taxpayers as they fill out the new form. As such, there are no right or wrong answers. Our team did not design the new form, so please don’t feel like you need to hold back or worry that we will take your criticisms personally. Also, MITRE has no commercial interest in this form.

Before we get started, I’d like you to read and sign this consent form. This consent form explains your rights as a participant in this research study. It explains that your information will be kept confidential, and that your name will not be associated with anything you tell me when I prepare my report for the IRS. It also explains that you have the right to end the interview at any time.

[GIVE CONSENT FORM AND HAVE PARTICIPANT SIGN]

Thank you. Again, I want to remind you that your participation in this interview is completely voluntary. We will be here for about 60 to 90 minutes. At the completion of the interview, I will give you a prepaid VISA card for $100.

Do you have any questions before we start?

USABILITY SECTION

Form – Trial 1

Our first activity is for you to fill out a W-4 form. Again, we are not looking for information about you personally, but rather, your experience as you fill out the form – what you are thinking and how you decided on your answers. To protect your confidentiality, I am going to give you a fictitious test scenario. The test scenario might not reflect your real-life situation.

[GIVE TEST SCENARIO DOCUMENTS]

Here you will find a write-up about the scenario and some fictitious documents like the ones you might receive in real life from an employer or other sources. I’d like you to pretend that you are filling out the W-4 at home as the person in the scenario. Please fill out the form as though we were not here. I understand that sometimes you might want to ask me a question about how to fill out the form, but again, my goal is to understand how you would fill out the form if I weren’t here. If you have questions for me, we can discuss those when the interview is over.

[GIVE TRIAL 1 W-4 FORM]

Here is the first W-4 form. Please go ahead and fill it out using these documents. I will observe you and take some notes, and then I will have some questions about your experience when you finish the form.

[IF PARTICIPANT ASKS IF FORM IS NEW OR OLD]: “I cannot tell you at the moment whether this is the old or the new form. Towards the end of our session I can answer that question.”

[OBSERVE PARTICIPANT FILL OUT FORM.

TAKE NOTE OF THE FOLLOWING:

• HOW OFTEN PARTICIPANT LOOKED AT INSTRUCTIONS

• FOR WHAT ITEMS PARTICIPANT LOOKED AT INSTRUCTIONS

• ITEMS WHERE PARTICIPANT HESITATED OR EXPRESSED CONFUSION

• ITEMS WHERE PARTICIPANT MADE ERRORS]

Trial 1 Debriefing

RECORD ANSWERS TO QUESTIONS BUT DO NOT ALLOW PARTICIPANT TO CHANGE THEIR ANSWERS

Walking through line by line

1. FOR EACH LINE:

   1. Can you walk me through how you came up with this number?

   2. How confident are you about this number?

      1. If participant responds with 5 or below, ask them: What was the greatest source of uncertainty?

2. (If participant entered an amount for Additional Amount Withheld From Each Paycheck) Can you walk me through your decision to enter an amount here and how you decided on this amount?

Overall experience of filling out form

3. On a scale of 1-10, with 1 being extremely easy and 10 being extremely difficult, how difficult would you say filling out the W-4 form was? Why do you say that?

4. If you were confused at any point, what aspects of the form or instructions provided you the greatest confusion? What was confusing about it?

5. If you found it challenging to compute any numbers at any point, what portion was the most challenging? What was challenging about it?

[TAKE ALL TRIAL 1 MATERIALS FROM PARTICIPANT]

Form – Trial 2

Now, I will ask you to complete a different version of the W-4 form. This will be another fictitious test scenario. Once again, please pretend that you are filling out the W-4 at home as this person in the scenario. Please fill out the form as though we were not here. If you have questions for me, we can discuss those when the interview is over.

[GIVE TRIAL 2 W-4 FORM]

Here is the second W-4 form. Please go ahead and fill it out using these documents. I will observe you and take some notes, and then I will have some questions about your experience when you finish the form.

[OBSERVE PARTICIPANT FILL OUT FORM.

TAKE NOTE OF THE FOLLOWING:

• HOW OFTEN PARTICIPANT LOOKED AT INSTRUCTIONS

• FOR WHAT ITEMS PARTICIPANT LOOKED AT INSTRUCTIONS

• ITEMS WHERE PARTICIPANT HESITATED OR EXPRESSED CONFUSION

• ITEMS WHERE PARTICIPANT MADE ERRORS]

Trial 2 Debriefing

Walking through line by line

1. FOR EACH LINE:

   1. Can you walk me through how you came up with this number?

   2. How confident are you about this number?

      1. If participant responds with 5 or below, ask them: What was the greatest source of uncertainty?

2. (If participant entered an amount for Additional Amount Withheld From Each Paycheck) Can you walk me through your decision to enter an amount here and how you decided on this amount?

Overall experience of filling out form

3. On a scale of 1-10, with 1 being extremely easy and 10 being extremely difficult, how difficult would you say filling out the W-4 form was? Why do you say that?

4. If you were confused at any point, what aspects of the form or instructions provided you the greatest confusion? What was confusing about it?

5. If you found it challenging to compute any numbers at any point, what portion was the most challenging? What was challenging about it?

Overall Debriefing

Contrasting the Two Forms

[GIVE PARTICIPANT BOTH W-4 FORMS TO LOOK AT]

Here are the two W-4 Forms you worked on earlier. Note that the forms are labeled “A” and “B” [POINT OUT LETTERS TO PARTICIPANTS]. Please take a minute to look them over to re-familiarize yourself with the two versions. When you are ready, I have a couple of questions about the forms.

1. Overall, which of the two W-4 versions (Version A or Version B) do you prefer? Why?

2. Overall, which of the two instructions versions (Version A or Version B) do you prefer? Why?

[GIVE PARTICIPANT THE CONTRAST FORM]

Here is a sheet of questions I’d like you answer regarding the forms. Please proceed and answer the items. If at any point you have any questions for me, I will be happy to answer them.

Overall Reactions

[GIVE PARTICIPANTS THE NEW W-4 FORM; REMOVE THE OLD W-4 FORM]

This is the new version of the W-4 form. I have a few questions about it.

1. What, if anything, did you know about the new W-4 prior to today’s session?

2. If you were filling out the W-4 by yourself and couldn’t figure something out, what would you normally do?

3. Are there any specific wording changes or other changes you recommend to the form?

4. Are there any changes to the instructions that you would recommend to make the instructions easier to understand?

5. Is there anything that the IRS could do to make the new version of the W-4 easier?

6. Does the proposed new form change your confidence that the proper amount of income will be withheld from your paychecks? More confident, the same amount of confidence, or less confident? If more confident or less confident: Why do you say that?

7. Do you have any privacy concerns regarding this new form? If yes: Why do you say that?