SOI-329 - Supporting Statement

Earned Income Tax Credit (EITC) Research Studies:
Website Usability Evaluation of www.irs.gov/eitc
TIRNO-05-Z00003, Task Order 11
OMB Summary Package
I. Introduction
Background/overview. The Internal Revenue Service (IRS) estimates that 20-25% of taxpayers

who are eligible to claim the Earned Income Tax Credit (EITC) don’t file for it on their tax
returns. It is suspected that some taxpayers do not claim EITC due to a lack of knowledge about
the credit and the eligibility criteria. To help taxpayers better understand EITC and determine if
they might be eligible for the credit, the IRS has made available information and a series of
online tools through their website www.irs.gov. The IRS is aware that a high percentage of
taxpayers eligible to receive EITC rely upon third-party tax preparers (professional or volunteer)
to complete their tax returns and schedules. The IRS website offers information and online tools
that can assist tax professionals as they process EITC returns and help them understand their
“due diligence” and other professional compliance responsibilities.
The purpose of Task Order 11 is to plan, conduct, analyze, and report on a website usability
evaluation of the EITC informational webpages posted on www.irs.gov. The project will be
conducted in 3 phases. Phase 1 entails a “heuristic review” of the website during which usability
specialists from Westat’s project staff will match the characteristics of the website (e.g. layout,
formatting, and navigational elements) against professionally recognized website usability
principles (aka, “heuristics”). During Phase 2, “User Testing”, targeted users of the website -- in
this case EITC-eligible taxpayers and professional tax preparers -- will interact with the EITC
pages posted on www.irs.gov in the presence of an interviewer in Westat’s usability lab in
Rockville, Maryland. During Phase 3, lessons learned from Phases 1 & 2 will be applied toward
the development of hardcopy and electronic webpage prototypes of the EITC pages on
www.irs.gov. These prototypes will model for the IRS examples of webpage designs that meet
targeted website users’ needs and expectations, and the design characteristics recommended by
experts in the field of website usability. Enhancements that IRS may make to the EITC
webpages on its website as a result of this evaluation are expected to lead to increased usage of
the website by taxpayers and tax professionals. It is anticipated that increased site usage, in turn,
could motivate more taxpayers to claim the tax credit; and, potentially lead to decreases in the
number of EITC claims filed in error.
Only Phase 2, User Testing, involves human subjects; therefore, the focus of this OMB package
will be on the recruitment and data collection activities planned for that phase. We anticipate
that this project may not require the full OMB review process, since members of two completely
different user groups – EITC-eligible taxpayers and professional tax preparers-- will be
individually tested using separate protocols, and there will be less than 10 participants per user
group. Separate protocols are necessary since taxpayers and professional tax preparers go to
www.irs.gov with different information needs regarding EITC. Details of this plan are discussed
further in Section II: Methodology.
1

TIRNO-05-Z00003, Task Order 11

Objectives of data collection. In Phase 1 of the project, members of Westat’s website heuristic
review team will evaluate the EITC webpages on www.irs.gov to learn how well the site adheres
to the usability standards recommended by experts in the field of website design. The kinds of
information that can be gained by conducting a heuristic review of the EITC webpages include
the identification of elements of the website design (e.g., page layout, menu options) that might
impede users from reaching the information about EITC that they seek on the site. In Phase 2,
User Testing, participants representing EITC-eligible taxpayers and professional tax preparers
will use a PC (personal computer) in Westat’s usability lab in Rockville, MD to search for
information about EITC on the irs.gov website. This kind of “user-centered evaluation” enables
real-time observation of the strategies that participants from the different user groups employ to
carry out different information search tasks, and facilitates identification of portions of the web
site that are associated with participants’ hesitations and confusion. Westat will provide written
recommendations to the IRS about ways the EITC webpages on www.irs.gov could be enhanced
to better meet users’ information needs and expectations considering outcomes from Phases 1
and 2, and will model those recommendations using the webpage prototypes that will be
developed in Phase 3.

II. Methodology
Sample design. Only Phase 2 of the project, User Testing, involves human subjects. Names of
potential participants for the taxpayer and professional tax preparer groups will be drawn from
databases provided by the IRS. Participants from both user groups will represent Englishspeaking persons living in the Baltimore/Washington DC metro areas. Nine participants will be
selected for each user group. The nine taxpayers who participate in the user testing will represent
persons who have been identified as eligible to file for EITC per IRS records; however, these
persons may or may not have ever actually filed for EITC. The nine tax preparers who
participate in Phase 2 will represent tax professionals who the IRS has on record as having
claimed EITC on client tax returns.
In recruiting participants, we will strive to include individuals who represent:


EITC-eligible taxpayers who have filed for the tax credit;



EITC-eligible taxpayers who have not filed for the tax credit;



Taxpayers residing in areas served by IRS community partners, such as organizations that
support the IRS Volunteer Income Tax Assistance (VITA) Program;



Tax preparers who represent large tax preparation firms (e.g. H&R Block, Jackson
Hewitt);



Tax preparers who represent small tax preparation businesses (e.g., self-owned “Mom &
Pop” businesses); and,



Taxpayers and tax professionals who have used a computer and the Internet before.

2

TIRNO-05-Z00003, Task Order 11

These two user groups have different reasons for searching for information about EITC on the
IRS website; therefore, a specialized protocol has been created for each group.
Data to be collected. During the user testing in Phase 2, taxpayers and professional tax
preparers will search for information about EITC on www.irs.gov using a PC in Westat’s
usability lab in Rockville, MD. Each participant will work individually (one participant per
session) with the website with a researcher present in the testing room. Through various research
efforts conducted with taxpayers and tax preparers over the past few years, the IRS has learned
that taxpayers and professional tax preparers go to www.irs.gov with different information needs
about EITC. Taxpayers are largely expected to use the website to search for information that tells
them what EITC is and how to determine whether they might be eligible to claim it on their tax
return. Professional tax preparers are expected to use www.irs.gov to search for EITC
regulations and policies, professional filing requirements, and detailed information about
determining eligibility for their clients in special circumstances (e.g., for a taxpayer who receives
disability benefits.)
Because members from the two website user groups approach their search for information about
EITC differently, we have developed a separate testing protocol for each group. The
organization of the interview is similar for each protocol; however there are differences in the
questions that are asked of each user group. Attachment A contains the protocol to be used with
taxpayers, while the Tax Preparer Protocol is included as Attachment B. The structure for the
interview protocols is broken down across the following 7 components, with documents tailored
for the particular user group:
1.

An Interview Script that directs the different steps of the user testing: Introduction
and Consent, Website Testing, Post-test and Questions from Observers, and
Debriefing and Wrap-up;

2.

Consent for Participation and Session Recording;

3.

A Pre-test Questionnaire to collect demographic data and information about
experience with Internet searches about tax information;

4.

Task Scenarios designed for each user group that participants will follow to
search for information about EITC on the irs.gov website;

5.

A brief “Post-test Questionnaire” to collect quantitative data on the participant’s
experiences using the website;

6.

Presentation of any questions from observers (IRS or Westat project staff); and,

7.

A Debriefing Protocol to collect qualitative feedback about the participant’s
experiences using the website.

During the website testing, the researcher will prompt the participants with questions that explore
their thinking, expectations and strategies as they each carry out the Task Scenarios. Participants
will be asked to describe their reasons for clicking a link, conducting a search, or taking some
other action. They will also be encouraged to express whether the behavior of the web site
matched their expectations. We will observe the strategies that participants follow to carry out
3

TIRNO-05-Z00003, Task Order 11

the tasks, and the specific actions and portions of the web site that are associated with the
participants’ hesitations and confusion. We will be alert to the “friction points” on the web site,
where users do not understand how to proceed. With participants’ signed consent, we will video
record their on-screen movements to identify the specific strategies they try when they
successfully and unsuccessfully try to predict where information about EITC can be found. The
video recordings will contain audio of the participant’s interactions with the interviewer, but will
not contain shots of their faces.
How data will be used. Results from Phases 1 & 2 of the evaluation are expected to provide the
IRS with information about how well the agency’s website communicates information about
EITC to taxpayers and tax preparers, and what the experience is like for those users when they
search for information about EITC on the website. Westat will provide written recommendations
to the IRS about ways the site could be enhanced to meet users’ information needs and
expectations considering outcomes from Phases 1 and 2, and will model those recommendations
with the webpage prototypes that will be developed in Phase 3. Enhancements that IRS may
make to the website as a result of this evaluation are intended to lead to increased usage of the
website by the targeted groups. It is anticipated that increased site usage, in turn, could motivate
more taxpayers to claim the tax credit; and, lead to decreases in the number of EITC claims filed
in error.
How data will be analyzed. Each of the Westat usability specialists conducting the heuristic
review of the website during Phase 1 will inspect the website independently using an evaluation
protocol approved by the IRS project team to identify design features that could potentially
interfere with users’ abilities to reach information about EITC that they seek on the site.
Analyses conducted during this phase will include assessment of elements of the website’s
navigation and information architecture - such as page flow, labeling, page layout and design –
to learn how these elements may affect users’ interactions with the website. Once independent
evaluations are complete, the evaluators will develop a Phase 1 summary report that aggregates
their findings.
The following activities will be conducted when analyzing data collected during Phase 2, User
Testing:


Quantitative analyses of Pre-test and Post-test Questionnaire data will be conducted
to identify frequencies and trends among these respondent data;



Qualitative analyses will be conducted of interviewer and notetaker session notes
taken during the website testing to identify problems that participants in the two user
groups encountered while completing their Task Scenarios, and the strategies they
employed to find information about EITC on the website;



Qualitative analyses will be conducted of interview and notetaker session notes taken
during administration of the Debriefing Protocol to evaluate participant feedback
about the ease of use and design features of the EITC webpages.

Westat project staff will develop a Phase 2 summary report that describes outcomes from the
user testing sessions as they relate to the information search experiences of each user group, and
recommendations for revising the EITC webpages considering participants’ feedback. Outcomes
from Phases 1 and 2 will direct the development of webpage prototypes in Phase 3 that model
4

TIRNO-05-Z00003, Task Order 11

ways that information about EITC could be displayed online so that the pages better meet
website users’ needs and expectations.
Dates of data collection. All project activities will take place at Westat’s Rockville, Maryland
campus. Phase 1 activities are expected to be completed in January 2009. Data collection for
Phase 2, User Testing, is expected to be complete by early February 2009. The design of
webpage prototypes, Phase 3, is expected to conclude in early March 2009.
Who is conducting the research. The research is being conducted by Westat. Westat will not
be using any subcontractors. Westat is located at 1650 Research Blvd., Rockville, MD 20850.
Cost of study. The total estimated cost of this research effort is $138,751, which will be broken
down across 5 tasks.
Stipend. Westat will provide a cash honorarium of $150 to taxpayers and $200 to professional
tax preparers at the end of each user testing session. An honorarium of $150 is expected to help
cover travel and child care expenses that taxpayers may encounter in order to attend the testing
session. The honorarium offered to tax preparers is slightly higher to account for their
professional status and for time they may need to take from their business to attend the usability
session.
Recruitment efforts. Human subjects will only be required for Phase 2 of the project (User
Testing.) Westat will recruit 9 taxpayers and 9 tax preparers for the user testing by telephone to
solicit their participation. Potential participants will be identified using files provided by the IRS
of EITC-eligible taxpayers and professional tax preparers who have processed EITC claims. We
will ask potential participants from each user group a series of screening questions which will
help identify whether they are eligible to participate.
Screening criteria for taxpayers include:


Identification of the person in the household who is responsible for preparing
taxes;



Identification of persons who have and have not filed a tax return in the past
year so as to interview taxpayers with varying filing experiences;



Identification of taxpayers with experience using personal computers; and,



Identification of taxpayers with experience using the Internet.

Screening criteria for professional tax preparers include:


Learning the number of preparers in the person’s office/firm so as to
interview those from a range of firm sizes;



Identification of tax professionals who prepare returns as part of a volunteer
effort;

5

TIRNO-05-Z00003, Task Order 11



Identification of preparers who have processed EITC claims in the past year;
and,



Identification of tax preparers with experience using personal computers and
the Internet.

The screening instrument for the taxpayer group is included in Attachment A. The screening
instrument for the tax preparer group is included in Attachment B.
Following agreement to participate, recruited persons will be sent a hardcopy letter confirming
the information discussed over the telephone. The confirmation letter will include an agenda,
schedule, and location of the testing session. It will also include directions to the testing site and
a contact telephone number in case of cancellation or questions. We typically make reminder
calls within 24 hours of the scheduled session to boost attendance.
We will track participation rates for each group type and provide reasons that taxpayers and
preparers give for choosing not to participate. We will also track the number of individuals who
were invited to participate from each targeted group and the number who actually ended up
participating.
Location-region/city and facilities. All project activities will take place at Westat’s corporate
campus in Rockville, Maryland.
Methods to maximize response rates. Since the user testing in Phase 2 is a qualitative effort,
the calculation of response rate is not relevant. Only 9 taxpayers and 9 tax preparers will
participate in the testing; therefore, the results from these sessions will not be representative of
the population being studied. Nonetheless, we will still attempt to encourage participation
during our recruiting phone calls. In addition, all participants will: (1) be sent a confirmation
letter reminding them of their interview session date, time and location; (2) receive a reminder
telephone call one business day before their session; and, (3) be provided a stipend when they
show up for the interview session.
Efforts Not to Duplicate Research. This project is part of an ongoing, qualitative data
collection effort that the IRS is conducting with EITC-eligible taxpayers and tax preparers to
learn about their awareness of the tax credit and to gather their feedback about EITC
informational resources.

III. Participants Criteria
Phase 2, User Testing, is the only part of the project that involves human subjects. During this
phase, 9 taxpayers and 9 professional tax preparers will be recruited by telephone to participate
in individual website testing sessions at Westat’s usability testing lab in Rockville, MD. Names
of potential participants will be drawn from databases provided by IRS. Participants from both
user groups will represent persons living in the Baltimore/Washington DC metro areas.
Taxpayers will represent persons who have been identified as eligible to file for EITC per IRS
records; however, these persons may or may not have ever actually filed for EITC. Tax preparers
will represent third-party paid or volunteer professionals who have filed for EITC on behalf of
6

TIRNO-05-Z00003, Task Order 11

their clients. Participants recruited for each group will have affirmed that they have some prior
experience using a compauter and the Internet. All sessions will be conducted in English.

IV. Privacy, Security, Disclosure
Westat personnel are trained in the importance of protecting the privacy of participant data. All
project staff have signed Westat’s “Assurance of Confidentiality Pledge”, a general assurance of
data privacy, and have undergone IRS Security Screening. Data collected about respondents
(e.g., name, address) will be kept in locked cabinets or secured areas when not in use. Once the
recruiting task is completed, all respondent names and addresses contained in the screener will
be shredded.
During the interview sessions, only the first name of participants will be used. In addition, to
protect the privacy of the respondents, no names will appear in any report documentation.
Videotapes of the interview sessions will not include the participants’ faces; only their on screen
movements and session audio. Videotapes will be destroyed at the conclusion of data analysis.

V. Estimated Burden Hours
The estimated time to complete the participant screening is 4 minutes and the estimated time for
each respondent to participate in the testing is 2 ½ hours including travel time (30 minutes each
way). We will assume a 60% percent success rate in soliciting participation.
Screening Burden
Total number of potential participants screened:
Approx. 100 participants screened for taxpayer user group
Approx. 100 people screened for tax preparer user group

200 People

Estimated time to complete screening

4

Minutes

Estimated participant screening burden (200 X 4 minutes = 800 minutes / 60 =
13 hours)

13 Hours

User Testing Participation Burden
Estimated number of participants:
18 participants (9 taxpayers, 9 tax preparers)

18

People

Time to conduct the User Testing session (1.5 hours) plus half hour commuting
time each way (1 hour)

2.5

Hours

Estimated interview participant burden (18 X 2.5 =)

45

Hours

Total burden (screening and interview participation)

58

Hours

7

TIRNO-05-Z00003, Task Order 11

VI. Special Tallies and Other Information
The following information will be provided to the EITC Office at IRS within 60 days after the
close of the Phase 2 data collection operations:
1. SOI Control Number
2. Title of study
3. Purpose of study
4. Findings: a brief summary of significant (important) findings that were evidenced in the
results.
5. Actions taken or lessons learned: a brief summary of actions taken or lessons learned as a
result of the screening and/or interviews.
6. Number of participants screened.
7. Number of participants who were invited to an interview.
8. Number of participants who actually participated in an interview.
9. Date and times of the interviews.
10. Date the data collection ended.
11. Actual burden hours
12. Estimated costs associated with the data collection.

Attachments
Attachment A: Taxpayer Protocol
Attachment B: Tax Preparer Protocol

8