FWS.gov Remote Usability Study

FWS.gov Remote Usability Study Plan
June 21, 2019

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

Table of Contents 

​1​ Introduction

3

​2​ Recruitment & Participation

3

​3​ Methods

4

​4​ Results

4

​5​ Testing Tasks

5

​5.1​ General Introduction and preliminary discussion

5

​5.3​ Influencers - Advocate

6

​5.2​ Enthusiasts

5

​5.4​ Influencers - Researcher

7

​5.6​ Managers - Government

9

​5.5​ Influencers - Communicator

8

​5.7​ Managers - Commercial

10

​5.9​ Wildlife I/O

12

​5.8​ Job Seekers

11

​5.10​ 5.9 Urban Youth Coordinators

13

Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 2 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

​1​ Introduction

U.S. Fish and Wildlife (FWS) is currently undergoing a website redesign and is seeking user
feedback in order to improve the usability, findability, and comprehension of information on the
site. The overall goal of acquiring this feedback is to translate users’ actual and potential use of
the site into explicit knowledge and design requirements for the redesign.
In order to obtain this feedback, we first conducted a contextual inquiry with actual users of the
site and produced a report of findings and recommendations. These findings led to a proposed
set of two alternative navigational models for an updated organization of the site. We next
assessed the navigational models using Treejack software. Treejack presented a series of tasks
to participants, and participants each clicked through the two proposed models of the site
organization to show where they would expect to find particular information.
In this round of user research, we will conduct a usability study to measure the effectiveness of
the proposed design and navigational improvements. Our goal is to learn more about the
attitudes and preferences of FWS customers and key stakeholders so that we can improve how
US Fish and Wildlife communicates with and delivery services to the public. The research
focuses on testing a set of key workflows and navigational pathways. The tasks asked of
participants in this round of research are very similar to the tasks represented in the first two
rounds of research. The research plan for this study involves 54 participants that represent each
of the primary audiences identified by FWS: Enthusiasts, Influencers, Managers, Job Seekers,
Wildlife Importers/Exporters, and Youth Program Coordinators.

​2​ Recruitment & Participation

We will be working with UserWorks, Inc., a third party recruiting company based in Silver Spring,
MD. UserWorks performed recruiting for the first two rounds of user research conducted earlier
in this project. They will be identifying and scheduling test participants. To the extent possible
we will be trying to re-recruited the majority of individuals that participated in the previous
rounds of user research. FWS will also provide UserWorks with the names of some candidates
that have consented to the participant and who FWS was able to identify more easily than the
recruiting firm.
We have set the following recruitment goals for this study:
Target Audience Segment/Persona

Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Count

Page 3 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

Enthusiasts - Refuge visitors, hunters, fishers/anglers, birders

8

Influencer - Advocates: Environmentalists, conservationists, FWS partners,
and Friends Of, NGOs

6

Influence - Researchers: Teachers/professors and students 18+

6

Influencer - Communicators: Corporate marketing, journalists, reporters,
bloggers, social media writers, legislative assistants

6

Managers - Government: Federal, state and local government land and
wildlife resource managers

9

Managers - Commercial: Commercial landowners and developers

4

Manager - Individual: Individual landowners and developers

4

Job Seekers: people seeing full-time employment at FWS

4

Wildlife Importers/Exporters: Animal records managers, regulatory affairs,
import/export managers, production assistants

4

Youth Coordinators: Youth programming coordinators

3

​3​ Methods

Participants will be asked to spend one-hour participating in a remote usability test. The effort
involves a two-person usability team, including a test moderator and note taker. The moderator
will guide each participant through a series of questions and the moderator and note taker will
capture written notes on the participant's actions. Each session will be digitally recorded. We will
use GoToMeeting software for screen sharing and recording video of the participant's screen
and an audio recording. We will be asking participants to “think aloud” such that, to whatever
extent possible, they will keep a running monologue of their thoughts and impressions as they
use the prototype.

​4​ Results

The data we collect will include participants comments, success in completing tasks, and notes
on observed performance.
Issues that emerge during the study are categorized as high, medium or low severity as follows:
● High severity issues​ are likely to prevent task completion
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 4 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●
●

Medium severity issues ​do not prevent task completion but are likely to slow
performance or cause frustration
Low severity issues​ do not prevent task completion or slow performance but are likely
to cause momentary confusion.

These results will be presented in a report that documents findings and recommends and will be
used to provide recommendations to improve the site in order to make it more usable. The task
list found in section 5 below represents the tasks that will be requested of participants.​
This test is focused on gathering qualitative information and doesn't not require statistical rigor.
The results of this study will only be shared with internal FWS employees and contractors
working on redesigning FWS.gov

​5​ Testing Tasks
​5.1​ General Introduction and preliminary discussion
●

●
●
●
●
●

●

●

​Thank you for participating in this study. Today, we are going to be trying out a new US
Fish and Wildlife website. The website is a prototype and is not yet fully built out.
Therefore, we may come to points where you can go no further. If we get to those points,
I’ll ask you what you might expect to see if you were able to continue.
Some key things to remember as we go along today:
​Your comments and opinions will only be noted in combination with the feedback that we
get from other people.
​There are no right or wrong answers. I am not testing you or your knowledge. Rather,
you are helping us to see how the site can be improved.
What I am concerned about is things that will help users like you get necessary
information in the near future.
​I am interested in your thoughts and opinions as we proceed. So please “think aloud” as
you go. Tell me your reasoning, your understanding of the information we are dealing
with, and if there is anything that you are not sure of or are confused by.
​As you know, we are making a recording of this session so that we can review the
session later. Your comments and data will be combined with the comments and data
from the other participants to make recommendations to the developers of the site.
I did not create the site, so please do not feel like you have to hold back on your
thoughts to be polite. Please share both your positive and negative reactions to anything
that we look at or discuss.

The questions below are organized by target audience segment and will be used to guide the
testing sessions.
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 5 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

​5.2​ Enthusiasts
This persona group includes Refuge visitor, Hunter, Fisher/Angler or Birder/Bird Watcher
Key tasks and content needs include:
● Refuges research (alerts, conditions, wildlife info, maps)
● Relevant news
● Trip planning
● Learning about plants/animals of interest
● Volunteering on a refuge
TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find information to help you plan a visit to a Refuge
3. Find information on purchasing a Duck Stamp
4. Find information on the types of programs FWS offers for youth
5. Find news stories about FWS volunteers
6. Find information about a plant or animal of interest
7. Research volunteering opportunities
8. Find contact details for a Refuge, Fishery or Hatchery in your area
9. Find information about one of FWS’s national programs such as, Ecological Services,
External Affairs, or Migratory Birds
10. Find images of specific plants or animals of interest
11. Find Federal Register notices
12. Find out about FWS proposed rules and notices that are open for comments

​5.3​ Influencers - Advocate
This persona group includes Advocate, such as Environmentalists, Conservationists, FWS
Partners, NGOs, Non-Profits or “Friends Of”
Key tasks and content needs include:
● Refuge info
● Regulatory info - related to wildlife/plants
● Wildlife data and reports
● Finding specific content types
○ biological opinions
○ habitat conservation plans
○ Peer-reviewed papers
○ Habitat conservation plans
○ species management plans
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 6 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●
●
●
●
●
●
●
●
●
●

Finding images
Stories about species
Permitting decisions/status
Species news
Press releases
FWS projects by species
Candidate/Target species
Partnership organizations
Contact info
Research how “average person” can get involved in conservation (volunteer, attend an
event)

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find information to help you plan a visit to a Refuge
3. Find Federal Register notices
4. Find a type of reference document that is important to you, such as a peer-reviewed
paper, biological opinion or habitat conservation plan
5. Find out about FWS proposed rules and notices that are open for comments
6. Find recent updates on species listing status
7. Find information about a plant or animal of interest
8. Find contact information for a staff person at FWS you are interested in speaking to
9. Find images of a specific species of interest
10. Find press releases
11. Learn about what organizations FWS partners with to help fulfill its mission

​5.4​ Influencers - Researcher
This persona group includes Researcher, such as Teachers/Professors or Students 18
Key tasks and content needs include:
● Conservation project details
● Refuge info - conducting research at a refuge
○ Trail info
○ Events
○ Current conditions
● Regulatory info
○ Federal register documents
○ Decisions made on endangered species
○ Species regs at all stages in the rulemaking process
● Species data sets
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 7 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●
●
●

●

○ Fish population data
SME contact info
Search by Geo
Publications
○ Published/Peer-reviewed info
○ Gray reports
Services
○ Grant info
○ Getting a research permit

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find fish population data sets
3. Find information to help you plan a visit to a refuge
4. Find Federal Register notices
5. Find a type of reference document that is important to you, such as a peer-reviewed
paper, biological opinion or habitat conservation plan
6. Find information on permits
7. Find information about financial assistance/grant options FWS provides
8. Find contact information for a staff person at FWS you are interested in speaking to
9. Find news stories about FWS conservation projects
10. Learn about what organizations FWS partners with to help fulfill its mission
11. Find images of a specific species of interest

​5.5​ Influencers - Communicator
This persona group includes Communicators, such as Corporate Marketing and
Communications Journalists/Reporters, News media, Bloggers, Social media writers/managers
or Legislative Assistants
Key tasks and content needs include:
● Nature photos
● Refuge info
○ Events
○ Wildlife available
○ Trail info
● Regulatory info
○ Proposed, open for comments,
○ Federal register documents
● News stories
○ Nature stories
○ Press releases
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 8 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●
●
●
●
●
●

Species details
Draft legislation and/or policy positions (items open for public comment?)
FWS office Facility Locations
FWS documents
○ Flyway data
Contact info
Duck stamps

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find images of a specific species of interest
3. Find contact information for a staff person at FWS you are interested in speaking to
4. Find information to help you plan a visit to a refuge
5. Find Federal Register notices
6. Find out about FWS proposed rules and notices that are open for comments
7. Find a type of reference document that is important to you, such as flyway data,
biological opinion or habitat conservation plans
8. Find information on purchasing a Duck Stamp
9. Find contact details for a field office in your area
10. Learn about what organizations FWS partners with to help fulfill its mission
11. Find Press Releases about FWS volunteers

​5.6​ Managers - Government
This persona group includes Federal, State, and Local Government Land & Wildlife Resource
Managers
Key tasks and content needs include:
● Regulatory info
● Policy guidelines - Species details and guidance - trying to find tree cutting restrictions
specific to your geography related to species X
● Search by Geo
● SME contact info
● Species info
○ by geo
○ Profiles
○ Recovery plans
○ Survey Data
○ Biological Opinions
○ Listing status
● Services
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 9 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●

●
●

○ Consultation, - Section 7, HCPs, Section 10,
○ Permitting
○ Financial assistance
○ Fish trading
Documents
○ Species distribution maps
○ ECOS document library?
Images of species
Training

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find contact information for a staff person at FWS you are interested in speaking to
3. Find information about a plant or animal of interest
4. Find information about Section 7 Consultation services
5. Find information on permits
6. Find Federal Register notices
7. Find information about Financial assistance options available to support state and local
government project work
8. Find out about FWS proposed rules and notices that are open for comments
9. Find information about learning opportunities/training options provided by FWS
10. Find images of specific plants or animals of interest
11. Find a type of reference document that is important to you, such as a species recovery
plan, biological opinion or habitat conservation plan

​5.7​ Managers - Commercial
This persona group includes Commercial Landowners or Developers
Key tasks and content needs include:
● Species info by Geo
● Services
○ Consultation - Section 10, HCP, CCA or CCAA?
○ Permitting - application and status tracking
■ Take permit form
■ Take permit reporting form
■ Salvage permit
○ Financial assistance
● Regulatory info
○ Read register documents
○ Ruling comments (active and historical?)
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 10 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

●
●
●

SME contact info
Stories - related to consultation, permitting, financial assistance - important to them
Documents
○ Multi-species conservation plans
○ Habitat conservation plans
○ Conservation models
○ Species Occurrence data and distribution maps

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find information about our Conservation Advice and Consultation services
3. Find information on permits
4. Find contact information for a staff person at FWS you are interested in speaking to
5. Find information about a plant or animal of interest
6. Learn about Conservation Consulting services
7. Learn about Financial assistance options available to support your project work
8. Find Federal Register notices
9. Find news stories about successful conservation projects
10. Find images of specific plants or animals of interest
11. Find a type of reference document that is important to you, such as the most recent
Species Status Assessment document or Species Recovery Plan
12. Find out about FWS proposed rules and notices that are open for comments

​5.8​ Job Seekers
This persona group includes people seeking Full-Time employment
Key tasks and content needs include:
● Learn about FWS mission
● National program overviews
● Field office details
● Role descriptions
● Related job responsibilities
● Scientific and technical details related to job responsibilities
● Contact info
● Job listings
● Research future Job responsibilities
● Look for news stories about FWS employees working in the field
● Wildlife photos

Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 11 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Learn out about FWS’s mission and vision
3. Find information about one of FWS’s national programs such as Ecological Services,
External Affairs, or Migratory Birds
4. Find information on a field office where you are interested in working
5. Find a role description for a job you are interested in
6. Find news stories about FWS employees working in the field
7. Search for FWS staff that have the same job title you are applying for
8. Explore topics on Conservation
9. Find rules and regulations about a specific species
10. Find images of a specific species of interest

​5.9​ Wildlife I/O
This persona group includes Animal Records Managers, Regulatory Affairs Managers,
Import/Export Managers or Production Assistants
Key tasks and content needs include:
● Designated port info
● Researching species info
● Understanding rules and regulations
● Species taxonomy/synonyms
● Locating accurate import/export content outside of eDec
● Import/Export related forms
● Permit forms
● FWS news/updates
● Species listing status
● Species recovery info
TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Find a list of designated ports (or a designated port near me)
3. Find common name or scientific name of a species of interest
4. Find information on permits
5. Find rules and regulations about a specific species
6. Find out what species have had recent listing status updates
7. Find import/export information
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 12 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

8. Find information about a plant or animal of interest

​5.10​ Urban Youth Coordinators
Targeting urban youth via organizations, schools and nonprofits that are focused on providing
youth programs targeting this demographic - Inner city after-school programs (schools/after
school program companies - city, region, national) - Boys/Girls Clubs (Jack/Jill)- - Scouting
Organizations - YMCA - Environmental Educational Centers (DC Public/Charter schools) - High
school teacher - City and county rec departments - State FW agencies - National fraternities and
sororities (Latino or African American)
Key tasks and content needs include:
● Educational materials
● Understanding what programs/services FWS provides for youth
● Service/Volunteer opportunities for kids and adults
● Access to knowledgeable staff willing to take time and interact with kids
● Information on how FWS is helping take care of the environment
● Help with species identification (plants and animals)
● Information on Refuges and their programs for kids
● News stories about youth involved in conservation activities
● Events in your area
● Internship opportunities
● Partnership opportunities

TESTING TASKS
1. Give us your initial reactions to the overall site look-and-feel, global navigation and
homepage
2. Learn about the benefits of becoming an FWS partner
3. Find information on the types of programs FWS offers for youth
4. Find news stories about FWS volunteers
5. Find information about a plant or animal of interest
6. Research volunteering opportunities
7. Find contact details for a Refuge, Fishery or Hatchery in your area
8. Find contact information for FWS staff who may be interested in speaking with students
about endangered species and/or conserving the environment
9. Find information on internships with FWS
10. Find educational materials for kids
11. Look for photos of wildlife FWS protects
Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 13 of 14

FWS PMDSI Phase - FWS.gov Navigation Model Testing Plan

12. Learn about FWS’s rulemaking process

Ⓒ U.S. Fish and Wildlife Service - All Rights Reserved​

Page 14 of 14