FAS Supplier Portal User Testing of the Offer Submission Process -- Req. 36

Background

We want to test the desirability and usability of the redesigned Offer Submission platform (i.e., Supplier Portal) that will replace eOffer.

Goals

• Learn about any desirability or usability issues in the design.

• Gauge whether the new design is effective at allowing users to complete their tasks.

• Confirm that what we're building is intuitive and will lead to a better user experience than exists today.

Method and Desired Participants

Over the course of a few weeks, we would like to interview at least 26 participants from across the following user groups for 60 minute 1:1 moderated virtual interviews. We will focus on testing actual users of the website because they will have real experience of submitting a MAS offer. We would like to get a mix of participants from the following user types:

• New/prospective MAS Offeror (Small, Other Than Small, SEDB)

• Recent MAS Offeror (Small, Other Than Small, SEDB)

• Current MAS contractors (Small, Other Than Small, SEDB)

• Joint Venture Offeror

• MAS/VA Schedule Holders

• APEX Accelerators (formerly Procurement Technical Assistance Center (PTAC) Technicians)

• Third-party Consultants

(Resource: Stakeholder mapping can be useful for determining core, direct, and indirect users impacted.)

Recruitment Strategy

In order to recruit research participants from our target user groups, we will contact people who we either know or suspect have used or would use the website in the future. We will recruit external participants.

• Invitation to Participate Email Template

• Participant Tracker

• MAS Supplier Portal IPT member recommendations

• RFI

Consent to Participate

Hi <participant name>,

Thank you for your interest in our usability testing research. Attached to this email is a participant agreement that describes the kind of information we're looking for while testing the Supplier Portal wireframes. It also outlines what we'll do with the information we collect.

We appreciate your time and want to make sure you have the information you need beforehand. Please read the attached agreement, ask us any questions you might have, and return a signed copy of this agreement before our session.

Thank you,

• Attachment: Participant Consent Form

Test Assets

Slides Wireframes: https://docs.google.com/presentation/d/1F6JhGUN_r_mJ-048tmV4ClwWsmxT-WCtP045UHdlAAs/edit?usp=sharing

PDF Wireframes

https://drive.google.com/drive/folders/1m6rVwPmAll25ch4fDGEeGZsywxZ737eu

Figma wireframes: https://www.figma.com/file/mg3l4zth7rEEM22TYeSMSr/MAS-Portal-Wireframes?type=design&node-id=0-1&t=M1TPPz7y0J6DPQ1s-0

• Group 1 Figma Wireframes Link

Interviewing Guide

Intro

• Thank you for taking the time to participate in our usability test. My name is _______ and I will be conducting today’s interview. My teammate(s), _____, will be listening in to take notes and observe.

• Our goal is to test and understand whether the newly designed Supplier Portal improves usability, and to confirm that what we're building is intuitive and leads to a better experience than exists today.

• We’d like you to walk through a few wireframe screens that we’ve developed to get your candid feedback about the design, usability, and identify any potential issues.

• Please know we’re testing the site design, and not testing you. There are no right or wrong answers.

• We’d ask that you please think aloud (or put your brain on speakerphone) as you navigate through the screens. All feedback is welcome.

• *Recording*: With your permission, we’d like to record this call. The recording will only be used to help us figure out how to improve the site, and it won’t be seen by anyone except those with a need-to-know. Do I have your permission to record this session?

• *Consent*: Also, if you have not done so yet, please digitally sign the participant agreement form. It just states your rights as a participant and that we won't be capturing any personally identifiable information. If you have signed the agreement, please send me a copy. Otherwise, do you agree to the terms of the participant agreement?

• Do you have any questions before we begin?

Background Info (15 min)

• First, tell me a little about yourself. What’s your occupation? What do you do on a daily basis?

• How familiar are you with GSA’s Multiple Award Schedule (MAS) Program?

• Have you participated in the MAS offer submission process before?

• If yes, what was your role in the offer submission process? How was your experience?

Wireframe/Prototype Testing (15 min)

OK, great. We’re done with the questions, and we can start looking at things. The first thing I’d like you to do is share your screen. You can do that by clicking “Present now” at the bottom of our Google Meet video chat. Before you share, make sure to close anything you don’t want recorded.

Next, we’re going to give you access to a few screens from the redesigned Supplier Portal and offer submission process. We’d like you to imagine that you’re logged into the website and are interacting with the pages as if they are live. Not all features will be clickable because it’s a new design, so in this case, we’d like you to think out loud and state what you would do, show us where you would click, and what you would expect to happen if the site was fully functional.

So in this first scenario, we’d like you to imagine that you’re ready to begin submitting your offer and you’ve just logged into the Supplier Portal. Take a moment to review the screen, including the design, layout, and text.

Group 1

Homepage

• Where would you go to access your profile information?

• What are your overall impressions of this page?

• What would you do to start a new offer?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Company/Entity Information

• How would you navigate back to the Supplier Portal dashboard?

• Where would you go to change the SAM information shown?

• How would you add an authorized negotiator? Remove?

• What are your overall impressions of this page?

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

**Proceed to Closeout Questions**

Group 2

SIN Selection - Select SIN & Categories

• Please read the instructions on this page and proceed with SIN selection.

• How would you update a SIN that you’ve entered?

• How would you remove a SIN?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

SIN Selection - NAICS Code Selection

• Please read the instructions on this page and proceed with NAICS Code selection.

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

SIN Selection - TDR Sales and Discounts

• Please read the instructions on this page and proceed with TDR sales & discounts.

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

SIN Selection - Non-TDR Sales and Discounts

• Please read the instructions on this page and proceed with Non-TDR sales & discounts.

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

SIN Selection - EPA Methodology

• Please read the instructions on this page and proceed with EPA Methodology.

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Prior to closeout we would like to ask the following for this Group 2 section:

• When your offer is in review by a Contract Specialist/Contracting Officer, would you prefer requests for clarification occur for each section separately OR after completion of the entire review?

**Proceed to Closeout Questions**

Group 3

CSA & EULA

• Please read the instructions on this page and proceed.

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Financial Responsibility

• Please read the instructions on this page and proceed through each section.

• Where would you upload documents like balance sheets, income statements or financial reports?

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Past Performance

• Please read the instructions on this page and proceed through each section.

• Could you demonstrate how you would add 2 CPARS reports?

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Prior to closeout we would like to ask the following for this Group 3 section:

• When your offer is in review by a Contract Specialist/Contracting Officer, would you prefer requests for clarification occur for each section separately OR after completion of the entire review?

• Is there sufficient guidance information to understand the purpose of this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

**Proceed to Closeout Questions**

Group 4

Subcontracting Plan

• Please read the instructions on this page and proceed through each section of the Subcontracting Plan.

• Is there sufficient guidance information to understand this page?

• Is there anything you’d redesign or change on this page? If so, what would you do?

• Is there anything on the page that you would expect to see that’s not there now?

Prior to closeout we would like to ask the following for this Group 4 section:

• When your offer is in review by a Contract Specialist/Contracting Officer, would you prefer requests for clarification occur after this section OR after completion of the entire review?

**Proceed to Closeout Questions**

Closeout (10 min)

Those are all of the screens that we’d like to cover with you today. We want to thank you for your feedback. It’s very helpful in understanding how we might improve the Supplier Portal and offer process. Before we go:

• Is there any additional feedback that you’d like to share? Possibly about an area we didn’t cover.

• Do you have any other specific areas of the site (or data elements) that you’re interested in providing your feedback on?

• Are you interested in participating in future studies?

• Is there anyone else you recommend we speak with?

Thank you again for speaking with us today. If there’s additional information that comes to mind that you think might be helpful, please feel free to reach out via email.

PRA Statement

Paperwork Reduction Act Statement - This information collection meets the

requirements of 44 U.S.C. § 3507, as amended by section 2 of the Paperwork

Reduction Act of 1995. You do not need to answer these questions unless we display a

valid Office of Management and Budget (OMB) control number. The OMB control

number for this collection is 3090-0297. We estimate that it will be 4 sessions at 60 minutes to read the instructions, gather the facts, and answer the questions. Send only comments

relating to our time estimate, including suggestions for reducing this burden, or any

other aspects of this collection of information to: General Services Administration,

Regulatory Secretariat Division (MVCB), ATTN: Lois Mandell/IC 3090-0297, 1800 F

Street, NW, Washington, DC 20405.

10