Supporting Statement for Paperwork Reduction Act Submission
Revised Sept. 29, 2010
B. Collections of Information Employing Statistical Methods
The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses or employ statistical methods” is checked, "Yes," the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
Total focus group participants: 88 (44 people in each of 2 markets)
Total usability testing participants: 44 (virtual testing)
Participants will be American citizens chosen from some of the targeted audience segments:
Task-Driven Citizen
Interested Citizen
Informed Taxpayer
Individual Investor
Students and Teachers
Potential Treasury Employees
Academic Economists and Researchers
Financial Institution Analysts and Managers
Business Owners
Advocacy Groups
Press / Media
Government Affairs Professionals
State/Local Government Employees
Compliance Officers
Homeowners
In addition, participants will be chosen that provide a diverse community based on the following criteria:
Registered voter
Government-related activities
Homeownership status
Internet usage
Current work status/occupation
Level of education
Ethnicity
Household income
Disabilities
2. Describe the procedures for the collection of information including:
* Statistical methodology for stratification and sample selection,
* Estimation procedure,
* Degree of accuracy needed for the purpose described in the justification,
* Unusual problems requiring specialized sampling procedures, and
* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Both the focus groups and usability tests are qualitative research techniques conducted on convenience samples that may be selected to be diverse on a number of characteristics, and therefore are NOT statistically representative samples of the population.
Discovery: Focus groups
To “field” test FinancialStability.gov and MakingHomeAffordable.gov usability and design, TMP will conduct qualitative research, specifically focus groups, representative of key user populations; namely homeowners and the general public. Sufficient focus groups will help us engage audiences in a safe (non-threatening) environment, where thorough dialog, group dynamics and observation of tasks, we can understand how these individuals, from specific audience profiles, might use the sites to meet their needs.
The facilitated focus groups will provide probing opportunities so we can truly understand the “why” behind participant behaviors (actions) in the course of seeking information on the sites as well as perceptions of the sites. We believe that this approach can validate the design concepts and content while gathering any constructive feedback to complete implementation in the most timely and efficient manner.
We recommend conducting 8 in-person focus groups (4 each for FinancialStability.gov and MakingHomeAffordable.gov) in 2 geographically regions: East and West/Central in order to target audiences at more than one site.
Validation: Usability Testing
Findings from the focus groups as well as previous research will be culminated into tasks designed to validate users’ experiences interacting with Treasury.gov, FinancialStability.gov and MakingHomeAffordable.gov. Throughout the testing, the moderator will identify how quickly and easily website users navigate key interactive functionality/features and determine key content areas and features of interest.
Areas in need of adjustments including navigation, functionality, content, layout and design will be documented. Each participant will also be asked to provide recommendations to enhance user experience.
Initial testing, as covered in this document, will be used to evaluate the following:
Basic user information location and retrieving
Top-line navigation
Aesthetics
Economy, i.e. activity was worth the effort for reward achieved
Total site architecture
Data visualization components
All usability testing will be conducted remotely in order to gain national representation while minimizing travel costs. Respondents will include persons with perceptual (no vision, low vision, deaf) and motor impairments (speech and spinal).
Paper prototype/usability testing quantities were determined by the following parameters:
1 Target – According to usability expert, Jakob Nielsen, 5 respondents will identify 85% of the usability issues while 15 respondents will address 100% of issues but incur 3 times the cost.
2+ Targets – Recommendations will vary by the number of targets and the degree of similarities/differences amongst user groups and/or geographies. However, industry standards suggest:
3-4 users from each category if testing two different target groups, limited overlap
2-3 users from each category if testing three or more target groups, limited overlap
3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
A participant screener will be developed to ensure focus group participants are within the specified audiences of concern for site success. Participants, except where not allowed for legal or ethical reasons, will receive an industry standard incentive of $100 each, delivered by check upon completion of the focus group. If participating in a usability testing session, the incentive level will be $50.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.
Focus groups will be a two hour guided session that follows a pre-determined discussion guide.
Usability testing will be a one hour virtual session guided by a pre-determined task list. Tasks will include:
Basic user information location and retrieving
Top-line navigation
Aesthetics
Economy, i.e. activity was worth the effort for reward achieved
Total site architecture
Data visualization components
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
OIRA has produced a number of documents that may serve as useful reference material for completing Supporting Statement B. These can be found at:
http://www.whitehouse.gov/omb/inforeg_statpolicy/
Rob Groat
Chief Technology Officer
Office of the CTO
Smartronix, Inc
301.535.7703
Lindsay Wozniak
Senior Vice President
TMP Government
703.269.0095
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | US Department of Treasury |
| File Modified | 0000-00-00 |
| File Created | 2021-02-02 |