Memorandum
To: Office of Management and Budget
From: Lin Wang, Center for Survey Measurement, U.S. Census Bureau
Subject: Mobile UI Design Framework for Survey Operations – Phase 1, Study 1
Date: April 11, 2016
Background
Mobile devices have become an important tool in survey data collection and other survey operations at the U.S. Census Bureau. Because of the nature of small screen size and touch interface, we are challenged with usability in mobile software, i.e., some mobile software is not as easy to use as one expects. The user interface (UI) design of mobile software is crucial to user/respondent experience, and consequently survey quality, because the UI is the interface through which the user/respondent receives and provides survey information. Poor UI design has resulted in errors in task performance, prolonged time in completing a task, and unsatisfied user experiences. In addition, it is costly to re-design and re-develop user interfaces. To improve user experience in using the Census Bureau’s mobile software and to reduce survey measurement error, the Census Bureau needs a framework of comprehensive mobile UI design that is based on scientific evidence of user research, for developing human-centered mobile software.
Under its Improving Operational Efficiency (IOE) program, the Census Bureau initiated the project of Mobile User Interface Design Framework for Survey Operations (aka, Mobile UI Framework) in 2015. The purpose of the project is to develop (1) standards for basic parameters in mobile software user interface and (2) guidelines for major aspects in mobile UI design. The standards and guidelines to be developed in this project concern three types of mobile software: (1) self-administrated web survey instruments, (2) field operation applications, and (3) data dissemination websites. The project is to be carried out in three phases. In Phase I, standards and guidelines for self-administrated web survey instruments are developed, Phase II for field operation applications, and Phase III for data dissemination websites.
Phase I includes three studies: Study 1 develops standards and generates empirical evidences supporting the standards; Study 2 develops guidelines and collects evidences supporting the guidelines; Study 3 develops a demonstration web survey instrument in two versions, with one version using the current UI design practice (comparison) and the other version using the standards and guidelines (S&G). The present memorandum addresses human performance data collection in Study 1.
Purpose of research
The Census Bureau plans to conduct usability testing (Testing) under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The purpose of the Testing is to provide behavioral evidences that support the mobile UI design standards.
Research design
Standards are to be developed for three categories of basic mobile UI parameters: (1) Touch target size and spacing, (2) Text entry and display, and (3) UI luminance and color. For categories 1 and 2, the following approach is taken to establish the standards: (1) Define a list of basic mobile UI parameters; (2) define user performance metrics for each parameter; (3) design an experiment for each mobile UI parameter or a combination of parameters; (4) collect user performance data through behavioral experiments; (5) establish standards based on the statistical analysis of user performance data collected in (4). For category 3, standards are formed through literature review because considerable research has been done on the subject of luminance and color perception. Usability testing will be conducted only for categories 1 and 2. Table 1 lists the parameters to be investigated.
Table 1. Basic Mobile UI Parameters for Standards
Parameters / Parameter Combinations |
Standard to be established |
Width of square touch target |
Minimum width of a square touch target |
Spacing surrounding a square touch target |
Minimum space surrounding a square touch target with a given width |
Diameter of round touch target |
Minimum diameter of a round touch target |
Spacing surrounding a round touch target |
Minimum space surrounding a round touch target with a given diameter |
Height of a text field |
Minimum height of a text field |
Vertical spacing above and below a text field |
Minimum vertical space above and below a text field |
x height of text display |
Minimum x height of text display |
Five experiments were designed to collect behavioral evidences which the standards will be developed upon. Experiments 1 to 4 concern data entry while Experiment 5 data display. All experiments are within-subject design. Four groups of 30 participants will participate in the study, each group being tested in Experiment 1, 2, 3, and Experiments 4 and 5 combined, respectively. Our past mobile usability studies showed marked individual differences in task performance. We thus determined that a sample size of 30 is needed to improve statistical power. Each experiment session lasts about 90 minutes. Experiments will be conducted in the Usability Lab at the Census Bureau Headquarters.
Refer to Enclosure 1 for detailed experimental design.
Participants
One hundred and twenty (30x4, 60 males and 60 females) participants will be recruited from the public through advertisement or an existing participants pool. Each participant must meet all the inclusion criteria below. The age specification is based on the rationale that older users have more difficulties in operating a mobile device due to age-related mobility. If an UI design is workable for a older user, it should be readily workable for a younger user; but not necessarily the other way around.
Participants’ inclusion criteria:
Age between 60 and 70 years old
Binocular habitual near-vision around 20/20
Normal luminance contrast sensitivity
Education of 8th grade or equivalent
Fluent English speaker
Smartphone user with 12 months experience
Burden
We estimate that each participant will spend 90 minutes to participate in the study, including time for consent, collecting demographic information, practice, and post-test debriefing. Total burden for 120 participants is estimated 180 hours (1.5 hour x 120 participants).
Incentives
The participant will receive $60.00 to offset the costs of participation in the study.
Sponsor
Improving Operational Efficiency (IOE) program, the U.S. Census Bureau.
New items
The standards are new and yet to be tested.
Enclosures
The following materials will be used in the study and are attached with this letter:
Study Plan for Mobile UI Design Framework for Survey Operations
Demographic questionnaire
Mobile experience questionnaire
Post-test debriefing materials
Experimental protocol
Contact
The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:
Lin Wang
Center for Survey Methods Research
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-9069
lin.wang@census.gov
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | Misa |
File Modified | 0000-00-00 |
File Created | 2021-01-28 |