Customer Service Collections for DHS - Batch A

FEMA Fact Sheet: Version June 2021 (FEMA Fact Sheet Header – First Page style)

1. FEMA App Beta Testing Process & Questions

   1. What is beta testing?

Beta testing is a form of customer acceptability testing that involves people from outside the organization to test the product and provide feedback. Beta testers install a beta version of the product and use it as a regular user and provide their feedback. The conclusion of beta testing is followed by the final release of the stable product for all users.

Purpose of beta testing:

• Identify and fix corner cases that would be harder to find with manual testing

• Diversify the cross browser and cross device testing to identify bugs related to responsiveness and compatibility

• Increase app quality by gather customer feedback on specific product features

• Help app team prioritize features for future releases

• Refine the app’s marketing strategy

1. Recruiting participants

The goal is to have 100-300 beta testers.

Beta testing requirements and pre-requisites:

• Hardware and software specifications:

• iOS devices: OS 11.0+

• Android: OS 4.4+

• Experience and/or knowledge with beta testing a mobile app or providing specific technical information about bugs or defects

• Indicating Expertise and Areas of Interest

• Interest in testing new or existing accessibility and inclusive design features

• Interest in testing Spanish language content and/or features

• Interest in testing new or existing Tribal Nations content and/or features

• Interest in testing new or existing disaster survivor features

• Interest in testing new or existing Climate Resilience features

• Interest in testing new or existing weather and emergency alerts features

High-level Recruitment Strategy

• Marketing the beta testing program on FEMA’s digital properties

  • FEMA app

  • FEMA.gov homepage and Ready.gov

  • Social Media: Facebook, Twitter, Instagram

• Asking current power users of the app

  • Regular users: People who use the app more frequently than the majority of the user base

  • Vocal users: People who have shared their insights and feedback with us via email or other channels

• Marketing on other social media outlets such as beta testing groups on LinkedIn, Quora, Reddit, etc.

• Partnering with digital counterparts in different federal agencies

• Working with various FEMA programs to reach out to their private and public partners

• Reaching out to “influencers” in the disaster prep space

2. Signing up beta testers

Prospective beta testers will be asked to fill out an application as part of their signup process. The purpose of the sign up is to better understand their relevant expertise and interest, as well as their overall fit for the beta testing process.

The FEMA app beta testing application will live on a webpage off of FEMA.gov. Submitted applications will go directly to the FEMA app shared inbox at fema-app@fema.dhs.gov, which is monitored daily by the FEMA app team.

Beta testing web form fields:

• Name

• Email address

• Device type (Smartphone, Tablet, Other)

• Device model (iOS, Android, Other)

• Device version

• Do you have previous experience beta testing mobile apps?

• Areas of testing interest (Accessibility, Spanish language, Tribal, New Features, Alerts, Climate Resilience, Disaster Survivor)

• Comments

Qualified iOS users will be asked to:

1. Accept two email invitations, one invite from Apple and another invite from the TestFlight app

2. Download the TestFlight app and accept invitations

Qualified Android users will be asked to install the beta app from Google Play via the app’s Play Store URL.

Prospective beta testers who meet the defined requirements and have accepted the invitation will be added to Apple/TestFlight and Google Play as a beta tester.

3. Beta testing process

Once testers have installed the beta app, they’ll automatically be updated to use the latest test version as it becomes ready. If it doesn’t automatically update, testers can manually update by going into TestFlight (iOS) or the app store page in Google Play (Android).

Beta versions of the app will come with a set of testing instructions and/or test scripts. Beta tests for each version will typically run for 7 days, but this is dependent on various factors such as goals for each beta version, tester limitations, etc.

iOS testers will submit their feedback and screenshots through the TestFlight app. Android testers will submit their feedback to fema-app@fema.dhs.gov or the Play Console’s Testing feedback page.

Beta testing journey

4. Questions for beta testers

Testing instructions will vary by build. As new builds are ready for testing, there will be a new set of instructions and/or test scripts related to the updates being made in the app.

The following statement will appear along with all testing instructions and questions:

A Federal agency may not conduct or sponsor an information collection subject to the requirements of the Paperwork Reduction Act unless the information collection has a currently valid OMB Control Number. The approved OMB Control Number for this information collection is 1601-0029 (expires 12/31/2026). Without this approval, we could not conduct this survey. Public reporting for this survey is estimated to be approximately 3-8 minutes per response. All responses to this information collection are voluntary. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden to fema-app@fema.dhs.gov.”

Types of questions we may ask:

• Numerical Answers and Scales. For example, on a scale of 1 to 10 (1 being the least accessible and 10 being very accessible), how would you rate the accessibility of the app home screen?

• Multiple Choice Questions. For example, did you find it easy to delete your main location. A. Very easy; B. Somewhat easy C. Difficult D. Very Difficult

• Specific Details: For example,

Please help us test a new alerts feature, silent notifications.

1. Add “Test Location” as an Other location.

2. Go to app settings (gear icon) and silence notifications for 24 hours

3. You should still receive alerts for Test Location and your other locations today BUT there should not be an associated sound or vibration

• Open-ended catch-all question. For example, please click around the newly redesigned Recover landing page. Let us know if you run into any issues! We look forward to hearing about your results!

Month Year 1