CCB Memo

Usability Testing of Census Business Builder Online Tool

The Economic Directorate has requested a usability evaluation of the updated release version of the Census Business Builder (CBB: https://cbb.census.gov/sbe/). CBB was developed by Esri, who specialize in geographic information systems (GIS) tools. CBB uses GIS and Census Bureau data to provide key statistics and cartographical (map) information to users for small business development purposes. The intended user base for this tool is small business owners, prospective small business owners, and small business development counselors. CBB allows its users to find key data for different business industries by location and by several filtering options. Results are provided to users in the form of an annotated map and by a “report” which contains relevant statistics and graphical information. Previous usability evaluations of the prototype versions of CBB were conducted by Esri, as well as focus groups focusing on usability of the tool.

In a pilot round of usability testing performed by the Center for Survey Measurement (CSM), we tested standard usability of the tool via task questions and think-aloud.  The participants used the tool to find specified data, to search for a business that was not yet covered (to test if the fall-back link was visible enough), and to search for information that they thought was "relevant".  Usability metrics included effectiveness (task success), efficiency (time to complete tasks) and satisfaction (according to self-report).  Changes that went into the most recent product release (the link above) include the removal of Step 3 from the splash page, the addition of new industries, enlargement of the map-variable overlay upon initiation of the map tool, addition of dividers on the map tool drop-down boxes to make them appear as drop boxes, relocation of the down-arrows in the drop boxes to help users know which box they belong to, addition of radio buttons for individual map variables to make the choices appear selectable, modification of the filter tool to make it a step-by-step process, addition of frequently asked questions and nominal modifications to the "report" feature including color and font changes to some of the text.

Of note, the CBB team has added brief tutorial videos intended to help new users understand the tool. The videos are embedded within the tool and are hosted on YouTube. Although the videos are intended to aid users, it is possible that the end user may not decide to click on the videos. Therefore, the CBB team requests that CSM test the tool with participants both with the assistance of the videos, and without the assistance of the videos. The purpose of this is to test whether the videos increase task completion and user satisfaction, and to test whether interface improvements made since the last usability test will increase task completion and satisfaction as compared to prior results.

Goals for the current testing include testing of tutorial videos that have been made available, including measuring potential increases in effectiveness, efficiency, and satisfaction when users are exposed to the tutorial videos, to test potential increases in the above metrics due specifically to the changes made between rounds (without aid of tutorial videos), and to identify any further usability issues and changes that can be made to further improve the tool.

Study Methodology

A usability test will be conducted on the updated CBB tool. Participants will be recruited via email from small business center contacts, and be randomly assigned to one of the following groups: Video Group, in which participants are instructed to view each embedded video while they are completing the 2 tasks; No-video Group, in which participants will not watch any video while they are completing 2 tasks. The test will consist of a demographic questionnaire, a background questionnaire on computer usage, a think-aloud protocol during which the participants complete two tasks, a satisfaction questionnaire and a debriefing interview.

For comparison to results of prior usability testing, two tasks from the prior study will be included in the protocol.

Respondents

We will test 18 participants external to the Census Bureau who identify as small business owners, prospective small business owners, or small business counselors. The CBB team will assist CSM in identifying qualifying individuals, who will be contacted by small business development center via email and invited to participate in the study. The Census Bureau generic screener will not be used, as participants who are known members of the desired sample and will be invited to participate via email.

Burden

We estimate that the participants will spend no more than one hour taking the study, including the time spent on demographic and satisfaction questionnaires, the two online tasks, and debriefing, for a total of 18 hours.

Incentives

The Census Bureau will offer an incentive to participants- $40 for their participation in the 1-hour interview to offset costs of participation, including parking and child care.

Enclosures

Below is a list of materials to be used in the current study.

1. Consent Form

2. Test Protocol

3. Demographic Questionnaire

4. Background Computer Usage Questionnaire

5. Task Questions

6. Satisfaction Questionnaire

7. Debriefing Questionnaire

8. Participant incentive voucher

Contact

Marylisa Gareau

Center for Survey Measurement

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-1724

Marylisa.gareau@census.gov