Supporting Statement

© 2008, Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, RoleView, Technographics, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Forrester clients may make one attributed copy or slide of each figure contained herein. Additional reproduction is strictly prohibited. For additional reproduction rights and usage information, go to www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.

I. Introduction

Background

The Joint Operations Center (JOC) supports 25 contact center locations providing enterprise direction and solutions to enhance the delivery of the taxpayer experience in a multi-channel environment. The IRS ITLA/COLDS R2 team is in the process of developing an interactive online tax tool to encourage taxpayers to migrate online to answer their tax law related questions instead of using other more costly channels like call for toll-free telephone assistance or visit IRS walk-in centers.

Accordingly, the IRS is seeking to understand what drives customers to migrate online for self-service, which functionality would enhance the migration, which types of tax law related questions they’d like to answer in the online channel and what barriers to self-service may exist if they have ever navigated the IRS.gov portal.

In order to understand these relationships, the IRS has determined that it must assess customer behavior and preferences to understand the categories best suited to online self-service. The IRS has contracted with Forrester Research (under subcontract to Lockheed Martin, MSD, Inc.) to execute quantitative research with taxpayers who have had a tax law related questions in the past two years or anticipate having a tax law related question in the next year in order to understand which tax law categories should be migrated to an interactive online tax tool.

Objectives for Data Collection

Forrester Research will test the hypotheses developed previously on which tax categories to migrate to online self-service.

The specific objectives of the survey are as follows:

Determine which tax law related questions are most common for taxpayers;
Understand which IRS service channels taxpayers turn to for answers to their tax-law related questions;
Explore attitudes and psychological dispositions that may act as drivers to various channels;
Elicit keywords taxpayers would expect to use to search for an answer in the online channel;
Assess satisfaction with the answers received in response to past questions about tax law by service channel, including IRS.gov;
Evaluate perceived need for confirmation of information via secondary service channels; and
Uncover any barriers to self-service that may exist with regard to the current IRS.gov portal.

II. Methodology

Who Is Conducting Research

The IRS has contracted with Forrester Research under subcontract to Lockheed Martin, MSD Inc. to conduct the research. Forrester Research is partnered with Momentum Market Intelligence to provide design and analysis of custom primary research. Momentum Market Intelligence will subcontract the data collection tasks of the research to an online panel vendor, eRewards, and to a telephone field vendor, Resolution Research.

Sample Design

The primary target audience for this research is taxpayers who have had a tax law related question in the past, or may have one in the future. There are approximately 130 million taxpayers in the United States. From this universe, the study will target (N=) 2000 taxpayers. This sample size is necessary to accurately represent taxpayers who have had or may have a tax question related to the 106 categories of tax law relevant for this research. This sample size will also allow for analysis on a number of dimensions, such as type of service channel, or method of tax return preparation. .

The taxpayer sample (total N=2000) will include representation from two sub-populations:

English speakers (N=1700) who have had a question or issue related to tax law in the past two years, currently, or have reason to think they might have a question or issue related to tax law in the future, and;
Spanish speakers (N=300) who have had a question or issue related to tax law in the past two years, currently, or has reason to think they might have a question or issue related to tax law in the future.

This study will use a combination of an online panel and a random digit dialing (RDD) telephone approach to data collection. An online panel approach will be used as a mode of data collection for the majority of the English-speaking taxpayer sample. A random digit dial (RDD) telephone approach will be used as the mode of data collection for a portion of English-speaking taxpayers, and the entire Spanish-speaking taxpayer sample. Requirements for this sample population are that it must be representative of the US taxpayer population in general.

Resolution Research, Inc. will administer the telephone portion of the study. For the phone portion, there will be no specific sample source. Resolution Research will randomly dial US households and apply screening criteria to identify eligible survey respondents (see survey instrument in Attachment A for specific screening questions).

eRewards will administer the online panel portion of the study. The sample will be drawn from their Consumer panel of approximately 2.5 million participants. The source of this sample is a “closed” or “by-invitation-only” online panel recruitment method, exclusively inviting pre-validated individuals or individuals who share known characteristics to enroll into market research panels. By partnering with a targeted, large, and diverse set of ubiquitous brands, eRewards is able to invite everyday consumers and business decision-makers into its panels. Sampling can be tailored by targeting specific segments, using demographic and behavioral variables. More information about eRewards can be found by contacting them at ask@e-rewards.com or 1-888-203-6245.

Panel members will receive an invitation to participate in the research via email. (See attachment B for example email invitation). They will be asked to answer a series of screening questions to identify eligible survey respondents (see survey instrument in Attachment A for specific screening questions).

Data to Be Collected

Please see the attached questionnaire (see Attachments A) for the details of information to be collected. Data from the surveys will be quantitative in nature. The surveys will consist of a series of closed-ended and open-ended questions designed to meet the objectives stated above and elicit customer preferences for an online self-service tool. The data to be collected is identical for the online and phone surveys.

How Data Will Be Used

The surveys will focus on testing and expanding upon the hypotheses and issues about online customer self service and the Tax Law Categories identified as appropriate for such a tool in previously conducted tax category volume and complexity research. Ultimately, findings from the surveys will be used to inform the IRS about best practices for deploying an online tax tool for use by taxpayers.

How Data Will Be Analyzed

The surveys will assess drivers for usage of an online, self-service tax tool. Expected analytic techniques include frequency distributions, cross-tabulations, mean scores, T-tests, analysis of variance (ANOVA) to determine significant differences between groups, and correlation analysis. Regression, factor analysis and structural equation modeling may also be used for driver analysis,.

Data Collection Date

The surveys will be fielded approximately two-three weeks after OMB approval is received, to enable programming and testing of the surveys prior to launch. Data will be collected over a 30-day period. Ideally, survey fielding will take place between August and October, 2009.

Location

Data will be collected over the phone and online, so there is no specific location for testing.

Cost of Study

The approximate cost of this study is $276,595. This number includes recruitment costs, labor costs, participant stipend, and other direct costs.

Stipend

Each individual participant will receive an incentive as a token of appreciation for their time participating in the voluntary research. Each taxpayer that completes the survey via telephone will receive a cash incentive of $10. Each taxpayer that completes the survey online will receive the typical incentive provided as part of their participation in the eRewards panel (incentive costs are built into overall interview costs; these are not provided separately).

Each eRewards’ panel member earns eRewards currency or points (depending on the panel) for the time he/she spends answering market research surveys. Members can redeem their currency/points for a variety of valuable rewards that are of interest to them. Incentives are based on the value of the sample group (B2B, B2C, or Specialty Group) and estimates of the time and effort required to fully complete a survey.

Recruitment Efforts

This data gathering effort will be carried out telephonically through the use of random digit dialing and over the internet utilizing eRewards’ online panel.

All respondents, whether participating via telephone or online, will be qualified for participation using a short screening section of the overall survey instrument (see Attachment A).

Expected Response Rate

The expected response rate for the phone survey is 25%. This means that 16,000 taxpayers will need to be contacted in order to screen 4000 taxpayers for qualification, which will ultimately result in 800 completed interviews, since we estimate qualifying incidence among those screened to be 20%.

The expected response rate for the online "panel" survey is 20%. This means that 40,000 respondents will be invited to participate, in order to screen 8,000, which will ultimately result in 1200 completed web interviews, since we estimate the qualifying incidence among panel respondents to be approximately 15%.

Justification for Response rate less than 50%

The telephone response rate for this type of project is typically 25% because of the random digit dial methodology being used to reach, get cooperation from, and qualify potential taxpayer respondents. In addition, the majority of respondents in the survey will be drawn from the subset of the taxpayer population that has contacted the IRS within the past two years with a tax-law related question. Since this sub-population represents less than 50% of the general taxpaying population, we anticipate an overall response rate of less than 50% as well. We estimate that the qualifying incidence will be 20%.

The standard response rate for panel sample such as we are using for the online survey is 20%. We are anticipating a somewhat lower qualifying incidence for the online sample because, in addition to the reasons provided for the phone sample, an online population is more likely to have conducted non-IRS online research to get answers to the questions we are asking about, rather than using IRS customer support.

Methods to Maximize Response Rate

Forrester/MMI will strive to maximize response and cooperation rates among those selected in the sample. For the phone method, we will make up to five callbacks (at least one in the evening and one on Saturday), if necessary, to reach a respondent. Once the telephone is answered, we will administer screening questions to identify the person who has contacted the IRS call center or considered doing so. Interviewers will go through study-related training; part of this training will be teaching them refusal avoidance and refusal conversion techniques.

For the online method, potential respondents will receive up to 3 reminder email invitations to participate in the research. Respondents who partially complete surveys will receive extra reminder emails in order to maximize responses.

Efforts to Not Duplicate Research

The goal of the research is to build upon the foundation already laid by previous research efforts. While some similar questions and answer option lists may be used in the current research in order to ensure continuity of data across studies, the research design team has reviewed previous research conducted in the past by the IRS on channel usage and other topics associated with the objectives of this research in order to avoid duplication of the research.

Test Structure/Design

Data collection for taxpayers will occur via a 20 minute phone survey or an online survey of approximately 20 minutes in length.

Sections of the survey instruments include:

Introduction and Screening
Invitation/Termination Scripts
General behavior
Perceptual and behavioral battery on experience with IRS
Relevance of/Behavioral Response to Tax Law Categories
Rating of IRS.gov website
IRS online tax tool name evaluation
Additional demographics

Please see the attached survey instrument for details about the specific questions that will be asked (Attachment A).

III. Participant Criteria

The target population for this test is the universe of people who may call an IRS call center for customer service. Please refer to the attached survey instrument (Appendix A) for more specific qualification criteria. A summary is below:

Taxpayer Qualification Criteria Summary

Must not work in tax preparation, for the IRS, in market research, or in management consulting
Must be between 18 years old and 65 years old
Must have filed (or had a representative file) a tax return in 2009 (for the 2008 year) – either electronically, or via mail
Must have had a question in the past two years, currently, or anticipate having one in the future regarding any of the following tax law issues (WILL SOFT QUOTA FOR MIX OF ISSUES):
- How to file taxes or filing requirements
- Who they can claim on their return
- Money paid, received, or saved for education
- Credits they may be able to take on their return
- What they can take as a deduction or expense
- Claiming various types of income, payments, awards, winnings, etc.
- Work related questions or questions around if they employed others
- Retirement savings including IRAs, pensions & annuities, social security, etc.
- Gains/losses from selling or losing investments or property
- Questions for foreigners living or working in the U.S. or who won property in the US
- Questions for U.S. citizens with foreign income or who paid taxes in another country
- Questions about your responsibility for your spouses or ex-spouses debt

Soft quotas for mix (based on IRS tax filing population data) of:

Age
Income
Method of filing – electronic, mail
Preparation of return – use tax preparer or not

Soft quota of future/anticipated tax law questions only not to exceed 25%

IV. Privacy, Security, Disclosure, Anonymity

We will protect the privacy of participants by not connecting respondent names or other personal identifying information to the data used in analysis and reporting. All respondents will be issued an ID number as the only identifier in the data. We will also control official access to the information and will not allow public access to the information. The questionnaires and any data collected during the survey will be stored in a secure location for three years, and then destroyed.

Forrester Research, Momentum Market Intelligence, and data collection subcontractors adhere to the CASRO (Council of American Survey Research Organizations) standards for privacy and anonymity. These standards can be found in Attachment C.

The IRS will ensure compliance with Taxpayer Bill of Rights II. All participants will be treated fairly and appropriately.

The security of the data used in this project and the privacy of participants will be carefully safeguarded at all times. Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A7B. Physical security measures include a locked, secure office. Data security at the appropriate levels have been accomplished. Systems are password protected, users profiled for authorized use, and individual audit trails generated and reviewed periodically.

The IRS will apply and meet fair information and record-keeping practices to ensure privacy protection of all participants. This includes criterion for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of taxpayer information as well as its release to authorized recipients.

Anonymity will be safeguarded. During the data collection process, participants will not be identified to IRS personnel. In addition, no participant names will be mentioned in the reports or data files. Participants will be advised that comments will be audio taped. Anonymity is assured by virtue of agency policy. The terms of IRS’s contract with the data collection requires that the anonymity of any data be maintained.

V. Burden Hours

The estimated time to complete each survey, from screening to completion, is 20 minutes.

PHONE - We anticipate a 25% response rate. The time to refuse to take the survey is estimated at 2 minutes . We further anticipate that approximately 20% of taxpayers contacted for this study will qualify. The screening portion of the survey is estimated to be 5 minutes. Therefore, we will need to contact 16,000 taxpayers in order to screen 4000 for qualification, resulting in 800 completed phone surveys.

ONLINE- We anticipate a 20% response rate, but we know that not all respondents will open the email invitation. We are assuming that only 50% of those invited will actually spend any time reading the invitation. The time to read and decide to not take part in the the survey is estimated at 1 minute. We further anticipate that approximately 15% of taxpayers screened for this study will qualify. The screening portion of the survey is estimated to be 5 minutes. Therefore, we will need to send out 40,000 email invitations to taxpayers in order to screen 8000 for qualification, resulting in 1200 completed web surveys.

16,000 phone non-participants contacted at 2 minutes each = 32,000 minutes = 534 hours
(40,000 X .50)=20,000 online non-participants contacted at 1 minute each = 20,000 minutes = 333 hours

4000 phone participants screened at 5 minutes each = 20,000 minutes = 333 hours
8000 online participants screened at 5 minutes each = 40,000 minutes = 667 hours

800 phone participants completed at 15 minutes each = 12,000 minutes = 200 hours
1200 online participants completed at 15 minutes each = 18,000 minutes = 300 hours

Total burden hour estimate = 2367 Hours

The OMB number for this study is 1545-1432.
If you have any comments regarding this study, please write to:

Internal Revenue Service, Tax Products Coordinating Committee, SE:W:CAR:MP:T:T:SP, 1111 Constitution Ave. NW, Washington, DC 20224

- 11 -

File Type	application/msword
Author	Kaitlin McDevitt
Last Modified By	qhrfb
File Modified	2009-08-09
File Created	2009-08-05