Feedback Sessions for FTC Bulk Order Website

Fleishman-Hillard
The Federal Trade Commission (FTC)
Bulk Order - Moderator Guide

Last Updated: 11/20/12

Prepared by:
Walky Goode
Senior User Experience Researcher
SPARK Experience Design
walky@sparkexperience.com
Phone: 202.215.6962

Prepared for:
Lisa Miller
Senior Vice President
Fleishman-Hillard
lisa.miller@fleishman.com
Phone: 202.857.220

Introductions & Background Questions (<5 Minutes)
Why are we here
I’d like to thank you for spending part of your day talking to us. We’ll spend about an hour together. We
work for a company called SPARK Experience Design and we are helping our client to improve their website
in order to make it more helpful and easier to use.

Some ground rules
I want to mention a few ground rules before we get started.
1. Everything you share with us today will be completely anonymous. We promise not to attribute
anything you say or write down with you personally. All information gathered in this process will be
combined with other data to give us a big picture of what everyone had to say.
2. We want you to feel free to speak your mind. This isn’t a test where there is a right or wrong
answer. We are here to listen to what you have to say and talk about your opinions. You can ask
questions at any time.
3. We will be recording the session for research purposes. We can’t possibly remember everything
that you say during the session, so we will be recording today’s session. We promise not to share
the recordings with anyone except those directly involved with this project.
Do you have any questions before we begin?
Great. Let’s begin!

Getting to know you (10 mins)
I would like to find out more about you so that we can have a better understanding of how this website
might help you.
1. Can you briefly describe what you do (i.e. How you assist/help/educate others)? Probe to check area:
(a) Education, (b) Librarian, (c) Local Government, (d) Law Enforcement, (e) Financial Services, (f)
Community Organizers.
2. You mentioned that some of the education materials that you share with others are [______]-related.
(Read from list of materials participant mentioned.) Could you tell me a little more about that? Do you
see a need to have those materials in another language? If so, which?
3. What resources do you primarily use to look up information related to such education materials on the
web?

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

2

Home Page Evaluation (5 minutes)
Activity introduction
Next we are going to have a look at a website together and get your opinions on it.

Homepage
I’m going to pull up the webpage now and I would like you to tell me your first impressions, whatever
comes to mind. You can feel free to scroll up and down on the page, but please refrain from clicking on
anything right now.
Open browser and go to: www.bulkorder.ftc.gov
1. Who is this website designed for?
2. What types of information do you think this site covers?
3. Would you trust this website to offer credible information? Which aspects of this website make it
appear credible? Which aspects make it appear less credible? Who’s responsible for this site?
4. What type of information do you think this site offers?
5. Have you visited this website before?
6. If yes, what tasks did/do you most frequently perform on this website?

Task Performance (30 minutes)
Activity introduction
Now you will have the opportunity to try out the website by performing a few tasks.
Study Flow: Start at homepage. Read task instructions first, have user interact with the web page then ask
post-task questions.
Observe:
1. Where user clicks first and sequence of clicks
2. Which path user follows?
3. Errors
4. Task status:
a) Passed with no difficulty
b) Passed with difficulty

c) Failed

Task 1: Free Exploration
You mentioned earlier that education materials that you share with others are [______]-related. I would
like you to see if the site provides you with information that specifically addresses your needs. Go ahead
and find items that you might need for your organization. Let me know when you’re done exploring.
Stop participant after approximately 5 minutes. Allow participant to add items to a cart, but stop before
checkout.

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

3

Expected Path: User defined.
Post-task Questions:
1. Ask questions about:
Things they clicked on
Comments they made
Things they spent a long time looking at
Things they spent very little time looking at
2. How easy or difficult was it to find this information? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is there a way that they could make it easier to find?
4. Was there any information regarding the cost of ordering items? Probe to see if initial impression is
that items are free.
5. On a scale from 1 to 7—1 being least satisfied, 7 being most satisfied—how satisfied were you with
how well the website addressed your needs?

Task 2: Finding Relevant Item - Part 1: Finding
Find materials on ________________________ and let me know which one(s) interest you more. Select
topic that participant previously mentioned (money, credit, homes, mortgage, jobs, making money, health,
fitness, privacy, identity theft – skip business).
Expected Path: User defined – Path may finalize with Add to Cart.
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change the design to make this easier?

Task 3: Finding Relevant Item - Part 2: Placing an Order.
If previous task was not completed, find a publication on topic and continue…
If previous task was completed successfully, then continue…

Name:
Organization:
Address:
Phone:
email:

Kris Jones
Learn League
1892 Loganberry Ln
Crown Point, IN 46307
(219) 988-6447
kjones@learnleague.org

From the materials you just found, select a publication that interests you most and order around 200
copies for your organization. (Pass fake information to participant) For this task, you can use this fake
customer information.

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

4

Ask participant to enter fake email address.
Expected Path: User defined – Path finalizes with Order Confirmation.
Post-task Questions:
1. How confident are you that you will receive the publication? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Can you track your shipment?
4. Did you agree to receive updates/provide feedback? What type of communication do you expect to
get (if you agree)?
5. Is there anything in particular that you would change in the design to make this easier?

Task 4: Adding Multiple Items for Event
Find materials for ______________________ Applicable mixture of PTA Meeting, Jobs Fair, Consumer
Protection Event. You will be expecting about 100 attendees. Order 1-2 relevant materials but do not
check out yet.
Expected Path: User defined – Path finalizes with Add to Cart.
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Task 5: Adding Multiple Items in Several Languages
You service a community that is composed of people that speak Spanish. Find materials in Spanish that
are relevant for your community. Order about 100 copies of 3-4 relevant publications. Do not check out
yet.
Expected Path: User defined – Path finalizes with Add to Cart.
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Task 6: Finding Specific Item and Ordering Special Quantity
Order 50 copies of “Identity Theft: What To Know, What To Do,” where would you look for this
information?.
Expected Path:
Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

5

A. Homepage >> Identity Theft, Privacy, and Security >> Identity Theft >> Want more copies or fewer
copies? --- Fill out Form
Post-task Questions:
1. What are your thoughts on this task? Is this what you expected?
2. How confident are you that you will receive the publication? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
3. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
4. Is there anything in particular that you would change in the website to make this easier?

Task 7: Checking Out – Part 1
How many items do you have in your order so far?
Expected Path:
A. Homepage
B. Homepage >> View Cart
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Task 8: Checking Out – Part 2
You’ve been told that ________________ Event from Task 4: PTA Meeting, Jobs Fair, Consumer Protection
Event has been rescheduled for the week after next. Find out if you will receive your order in time.
Expected Path:
A. Homepage >> Frequently Asked Questions
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is the information provided useful? Use a scale from 1 to 7, where 1 is not useful at all and 7 is
extremely useful.
3. Is there anything in particular that you would change in the website to make this easier?

Task 9: Checking Out – Part 3
Request a rush delivery so that your order arrives in time.
Expected Path:
A. Top navigation >> Contact Us
Post-task Questions:

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

6

1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Task 10: Post Check Out
You get a confirmation email and notice that you goofed on your address. You need to find a way to get in
touch with the FTC Bulk Order Center to correct your address.
Expected Path:
A. Homepage >> Technical Problems? Contact Us
B. Homepage >> Frequently Asked Questions >> Toll Free Number
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Task 11: Printing an Item
You only want to obtain and print one copy of “A Consumer's Guide to E-Payments”. How would you do
that?
Expected Path:
A. Homepage >> Computers and the Internet >> Online Shopping and E-Payments >> A Consumer's Guide
to E-Payments >> Click for Preview – Path finalizes before actual printing of document
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
2. Is there anything in particular that you would change in the website to make this easier?

Post Experience Evaluation (5 minutes)
Based on your experience with the site, please rate your agreement with the following statements. Circle
the level of agreement that applies (1 means strongly disagree, 4 means neither disagree nor agree, and 7
means strongly agree).
Emotional Evaluation

Strongly
Disagree

I feel confident that I can use the website to find what I am
looking for

1-------2-------3-------4-------5-------6-------7

I would be curious to explore the website further to discover
things I might also be interested in

1-------2-------3-------4-------5-------6-------7

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

Neutral

Strongly
Agree

7

It feels like this website is designed for someone like me

1-------2-------3-------4-------5-------6-------7

Visual Evaluation

Strongly
Disagree

I thought the website had a professional look and feel

1-------2-------3-------4-------5-------6-------7

The site does not look too busy or cluttered

1-------2-------3-------4-------5-------6-------7

The site has a consistent look and feel

1-------2-------3-------4-------5-------6-------7

Functional Evaluation

Strongly
Disagree

The site provides information that is useful to me

1-------2-------3-------4-------5-------6-------7

The site’s navigation was always clear & easy to use

1-------2-------3-------4-------5-------6-------7

The content on the site was easy to skim or read

1-------2-------3-------4-------5-------6-------7

The website presents information that I didn’t realize the
FTC offered

1-------2-------3-------4-------5-------6-------7

Based on my experience today, I would likely use the web
site in the future

1-------2-------3-------4-------5-------6-------7

Neutral

Neutral

Strongly
Agree

Strongly
Agree

Debrief (5 minutes)
1. Now that you have experienced this website, can you summarize for me what this website offers?
2. Do you belief the information that is provided in this website is valuable for your needs? How so?
3. Are there any other types of publications that are relevant to this website and that you might
want?
4. Which task(s) did you find the easiest to perform?
5. Which were some of the more challenging tasks to perform?
6. Let’s talk about the navigation on the website:
a. Did the links seem to be logically organized?
b. Would you organize them differently?
c. Were there any links that you were unsure about?
7. What did you like most about the site?

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

8

8. What did you like least about it?
9. If you could change anything about it, what would you change?
That brings us to the end of the session.
We really appreciate the time you have taken to participate in this study your participation will assist us in
developing a better site.
Thank you once again!

Fleishman-Hillard - FTC: Bulk Order – Moderator Guide

9