0990-0321HHSWebsiteUsabilitySurveySupportingStatement[2]

Supporting Statement for
HHS Website Customer Satisfaction Survey

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

Executive Order 12862 (attachment 1) directs Federal agencies that provide significant services directly
to the public to survey customers to determine the kind and quality of services they need and their level
of satisfaction with existing services. The U.S. Department of Health and Human Services (HHS) seeks to
obtain approval to conduct usability surveys on pages of the HHS websites on an ongoing basis.
This collection of information is necessary to enable HHS to respond to The President's Management
Agenda, which calls for government reform that is citizen-centric, results-oriented and market-based and
requires agencies to participate in e-Government initiatives that provide high quality customer service,
provide citizens with readier access to government services, increase access for persons with disabilities
to agency websites and make government more transparent and accountable
(http://www.whitehouse.gov/omb/budget/fy2002/mgmt.pdf, pp. 4 and 25).
Collecting voluntary customer feedback is the least burdensome, most effective way for HHS to
determine whether or not its public websites are useful to and used by its customers. In fact, the
American Customer Satisfaction Index says the collection of such feedback may be more important in
the public sector than in the private sector:
The dissatisfied recipient of Government services can rarely punish a faltering service provider by
taking his or her business elsewhere, or by demanding a lower price. There is no market
feedback (with the possible exception of elections) that forces the [public sector] service provider
to improve, or signals what to improve (ACSI Commentary: Federal Government
Scores, http://www.theacsi.org/government/govt-01c.html).
HHS is requesting a reinstatement with change request in order to continue this mission. Generic
clearance is needed to ensure that HHS can continuously improve its websites though regular surveys
developed from these pre-defined questions.
Surveying HHS websites on a regular, ongoing basis will help ensure that users have an effective,
efficient, and satisfying experience on any of our websites, maximizing the impact of the information
and resulting in optimum benefit for the public. The surveys will ensure that this communication
channel meets customer and partner priorities, builds HHS’s brands, and contributes to HHS health and
human services impact goals. Revisions to the survey have been made to account for changes in online
user behavior since the original request has been made in 2008. In the recent years, online behavior has
become dramatically sophisticated that measuring efficiency and effectiveness of online content
delivery needed to reflect these significant changes.

HHS employees, full-time contractors, or contract vendors will collect the data for these surveys. The
data collected will include background participant information that does not identify individuals, task
identification, and overall satisfaction measures. The data will be collected electronically and stored
digitally on a secure site or computer.
This survey is authorized under the Public Health Service Act (42 USC 241) Section 301. A copy of the
legislation is included (attachment 2).

2. Purpose and Use of the Information Collection
The entire HHS website and other websites highlighting the HHS Secretary’s priority projects contain
over two million pages of information, products, guidelines, and training focused on diseases, health
conditions, public health, and human services. The HHS websites are comprised of multiple major
divisional sites and smaller sites, and the usability survey will focus on evaluating on evaluating the
effectiveness of content delivery on these websites at the site- and page-level. The HHS websites are
accessible to everyone on the World Wide Web and has many different audiences, including public
health and human services professionals, physicians, media, policy makers, and the general public.
By collecting website usability information, HHS will be able to serve and respond to the ever-changing
demands of its users. Additionally, we will be able to determine the best way to present messages on
HHS websites. The HHS sites are one of HHS’s primary channels to raise awareness of important issues
for its target audiences.
These users include individuals (such as patients, the elderly and disabled, educators, students, etc.),
interested communities, partners, healthcare providers, service providers, and businesses.
The purpose of such usability surveys is to judge the Web content and presentation though which HHS
communicates scientific, health and public assistance information to its target audiences to help ensure
that health and human services impacts are maximized through the delivery of a useful, efficient, and
effective website.
Primary objectives are to determine whether the HHS websites:
(1)
(2)
(3)
(4)

Meet the wants, preferences, and needs of its target audiences.
Are effective vehicles for sending messages to target audiences.
Deliver existing services at a satisfying level of quality.
Provide users with the kind and quality of services they need.

Findings have helped and will continue to help HHS to:
(1)
(2)
(3)
(4)

Understand the user community and how to better serve HHS website users.
Identify areas of the website requiring improvement in either content or delivery.
Determine the kind and quality of services our target audiences need.
Determine how to align Web offerings with identified user need(s).

(5) Explore new or refined methods for offering, presenting and delivering information most
effectively, to enable us to present messages as well as serve the needs of people who are
already coming to seek particular information or to learn about a particular topic.
The data collected from this effort will allow us to answer critical usability questions, including:
•
•
•
•
•
•
•
•

•
•
•

What are the Web needs and preferences for our target audiences?
How often and for what purposes (there can be several simultaneously) do our target audiences
typically use the HHS websites?
How satisfied are they with their experience on HHS websites?
What difficulties do they experience when trying to complete typical tasks on HHS websites?
In what ways can we improve their speed and ability to find the information they want, expect
or need on HHS websites?
Were messages on the site presented in such a way that they are noticeable, easy to
understand, easy to remember, and have an impact on the viewer’s behavior plans?
How does their awareness of, knowledge of, and opinions on a health topic change after viewing
HHS websites?
Did they find information/messages about health and human service issues they weren’t initially
looking for when viewing information on the site? Did the message have an effect, e.g. change
their behavior plans?
Are they satisfied with the services offered through HHS websites?
What improvements would the user like to see made to the existing services on HHS websites?
What other services do they need?

The survey will help ensure that HHS websites meet site visitor and agency needs, build HHS’s brands,
and contribute to health and human service impact goals. Feedback from the user base is necessary to
fully judge the performance of HHS’s websites. All data collected through the survey will be used to
determine whether HHS should revise content, labels, structure, or layout of its Web pages. If indicated,
revisions would be intended to increase the success rate of information–seeking website visitors.
3. Use of Improved Information Technology and Burden Reduction
All data will be collected electronically to reduce the burden to the respondent.
For most questions in the survey, the respondent will click on a “radio button” or checkbox that
corresponds to their response. For open-ended questions in usability surveys, the respondent would be
told to enter their answer in the provided text box. We have attempted to keep the format of the survey
simple with short questions and clearly labeled and scaled answer choice-sets.
We will keep the survey short, with no more than 12 questions at any one time, including no more than
two open-ended questions.
The set of survey questions included in this package (attachment 3) were gathered from (1) previous
usability surveys conducted at HHS or (2) recommended usability questions used by other usability
professionals in other organizations. In determining which questions to include in the package, usability

professionals across HHS were consulted and questions that had poor performance in the past or were
not considered best practices were discarded. Because we are requesting a 3-year generic clearance for
a wide variety of possible usability surveys on HHS websites, the list of questions is large enough that
this package can cover all potential survey scenarios needed. However, as stated above, each survey is
limited to a specific number of questions and HHS staff will not incorporate every question (and every
response option) in any one survey.

4. Efforts to Identify Duplication and Use of Similar Information
No similar information exists, although some usability surveys have been conducted on a few, specific
pages of the HHS.gov website. Approval of this package will greatly expand HHS’s ability to perform
usability surveys on the HHS website.

5. Impact on Small Businesses or Other Small Entities
There is no burden on small businesses or small entities.

6. Consequences of Collecting the Information Less Frequently
There are a number of potential negative consequences if these data are not collected. In addition, if the
collection is not conducted frequently, we will not have valuable data needed to routinely revise
messages and reorganize online health information in a way that is most easily understood and accessed
by website visitors. Specifically, without this data there would be:
•

•

•

No performance measures by which to determine effectiveness of the HHS websites as a tool for
our visitors and message channels for the department. This results in lowered user satisfaction,
fewer return visits, and decreased information dissemination.
No user data to include in website design decision-making to ensure that user experience on our
site is efficient, effective, and enjoyable. This results in an unfocused approach to Web design in
which we are unable to determine whether our site is useful or not.
Vital feedback regarding customer and/or partner satisfaction with various aspects of the HHS’s
services will be unavailable.

Usability surveys will only be conducted at intervals considered appropriate to measure the impact of
HHS website changes and to monitor the level of performance. We are only expecting one-time
responses from respondents. Therefore, it is not possible to ask participants to fill out the survey less
frequently. Page-level usability surveys will be available as long as the page is available to the public.
Similarly, we expect one-time responses from respondents.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances with this information collection package. This request fully complies
with the guidelines of 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult
Outside the Agency

The Agency’s 60-day notice appeared in the Federal Register on Friday, January 12, 2011, in Volume 76,
No. 8; page 2121 (attachment 4). No public comments were received in response to the notice.

9. Explanation of Any Payments or Gift to Respondents
There will be no payments or gifts to respondents.

10. Assurance of Confidentiality Provided to Respondents
The HHS Privacy Officer reviewed this submission and determined that the Privacy Act does not apply to
data collections conducted according to procedures described in this application. All questions for the
surveys to be conducted under this OMB approval are included within this Information Collection
Request.
All participants will be informed at the beginning of the survey that their responses will be treated in a
secure manner, that all data will be safeguarded closely, and that no individual identifiers are planned to
be used in survey reports.
All data will be stored in secured electronic files for at least two years and no longer than 10 years.

11. Justification for Sensitive Questions
Questions concerning Race and Ethnicity (language preference) may be considered sensitive by a
portion of respondents. Race and Ethnicity questions are included in the set of Demographic questions
that may be asked of respondents. Where relevant to the evaluation of website usability, Race and
Ethnicity data will be collected consistent with HHS policy and standard OMB classifications.

12. Estimates of Annualized Burden Hours and Costs
The remote survey will last approximately 12 minutes and take place at the participant’s computer.
These estimates were determined through analysis of times from previous usability surveys using similar
questions and a survey of usability professionals to ascertain average times for users to perform tasks.
Estimate of survey respondents was based on an estimate of the ideal number of usability surveys that
HHS would conduct over a 3 year period. Because HHS has not been able to conduct these types of
surveys at the level needed previously, it is anticipated that most of HHS’s websites will require some
sort of usability survey.

Survey Type
Remote Surveys

Number of
Respondents
48,000

Estimates of Annual Burden
Frequency of Response per
Avg. Burden Per
Respondent
Response (hrs.)
One time
12/60

Burden
Hours
9,600

An average hourly salary of approximately $22.57 is assumed for all respondents, including clinicians
and scientific users, based on the Department of Labor (DOL) National Compensation Survey. Because of
the scope of this generic clearance and the variety of the types of participants, the average salary was
utilized rather than attempting to estimate salaries for groups of audiences. With a maximum annual
respondent burden of 9,600 hours, the overall annual cost of respondents’ time for the proposed

interviews is estimated to be a maximum of $216,672. There will be no direct costs to the respondents
other than their time to participate in each survey.
Total Respondent Hours
9600

Hourly pay rate
$22.57

Total Respondent Burden
$216,672.00

13. Estimates of Other Total Annual Cost Burden to Respondents or Record
Keepers
There are no additional costs to the respondents. There is no burden to record keepers.

14. Annualized Cost to the Federal Government
Usability surveys will be prepared by contractors or HHS staff (FTE). An FTE manager will review all
surveys. Usability teams will vary across HHS web teams but typically an FTE and contractor will work
together on survey preparations, coding the surveys electronically, conducting the surveys, and
analyzing of data. Additionally, a senior-level FTE will typically review and approve the activities. The
amount of time staff and contractors spend on surveys will vary depending on the number of
participants for each survey, the number of questions, and the site being surveyed. An average number
of 400 web page survey sites a year were assumed for estimation purposes. Overall time spent by HHS
staff and contractors is lessened as this package provides tasks and questions to be used in the survey;
thus, reducing time staff normally would have spent developing these questions.
Staff or Contractor
Contractor instrument preparation, conduction,
analysis (GS-12/GS-13 equivalent)
FTE survey preparation, conduction, analysis (GS13)
FTE manager survey review (GS-14)
AVERAGE COST PER SURVEY
AVERAGE 1-YEAR COST

Average
Hours per
Study
3

Average
Hourly Rate

Average
Cost

$36.00

$108.00

10

$39.00

$390.00

1

$45.00

$45.00
$543.00
$217,200.00

15. Explanation for Program Changes or Adjustments
Since the initial request for this collection, HHS is revising the survey to reflect changes in online user
behavior. The revision does not change burden hours, which results in a program change.

16. Plans for Tabulation and Publication and Project Time Schedule
Data collected from these surveys may be integrated with other web analytical data to get
comprehensive insights into online user behavior so HHS can improve the efficiency and effectiveness of
its website’s content and delivery. Integration with data from other online tools will be completed
consistent with HHS policy and standard OMB classifications.
ACTIVITY
1. Determine which website will be surveyed.
2. Determine survey questions.
3. Determine target quotas.
4. Completion of surveys.
5. Analysis of surveys.
6. Adjustment of website based on results of the survey.

TIME SCHEDULE
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing
Ongoing

17. Reason(s) Display of OMB Expiration Date is Inappropriate
Exemption is not being sought. The OMB expiration date will be displayed.

18. Exceptions to Certification for Paperwork Reduction Act Submission
There are no exceptions to certification.