ACS Internet Form Pretesting Plan

The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We plan to conduct a usability test of the 2013 American Community Survey (ACS) online instrument to identify issues that are problematic and frustrating to the user. The objective of the usability test is a) to identify if design changes intended to reduce known break-offs work and b) to evaluate how participants interact with using new security questions to reset a forgotten PIN.

Testing will begin in May and go through August of 2013. Staff from the Center for Survey Measurement’s Human Factors and Usability Research Group will conduct a maximum of 15 interviews. There will be two rounds of testing. The first round of testing will have ten participants; after the sponsor has an opportunity to make changes based on the recommended changes from the first round of testing, we plan to conduct a second and final round of testing with five more participants.

The participants we recruit will primarily be general participants of varying ages who could have been selected for the ACS survey. We plan to focus on recruiting respondents living in households with two or more people—aiming for 2/3 of the participants to live in unrelated households (that is households with at least one nonrelative) and 1/3 of the participants will live in related households (that is where people in the household are related by birth, marriage, or adoption). All participants will have a minimum of one year Internet experience and use the Internet at least three times a week on something other than writing emails. Participants will be recruited from the Usability Lab database, which is composed of people from the metropolitan DC area who volunteered to participate after responding to a Craigslist posting or an ad in a local newspaper. Participants may also be recruited by word of mouth, and Census Partner organizations. Participants will come to the Usability Lab at the Census Bureau for the study and will be compensated $40.00 for their participation.

Prior to working on the ACS instrument, participants will be asked to complete a demographic questionnaire, as well as a background questionnaire on their computer use and Internet experience. Then, participants will be asked to think-aloud while they work on completing the survey and will be prompted to think-aloud when they fall silent. After answering all survey questions, participants will answer a questionnaire designed to measure their satisfaction with the survey instrument. Subjective satisfaction ratings will be collected on such design elements as the screen layout, instructions and questions displayed on the screen, and the overall experience of completing the survey. Finally, participants will provide feedback about the instrument during a debriefing. Participants’ eyes will be tracked, when possible, using the eye-tracking monitor and the TOBII software. A copy of the screen shots of the ACS online questionnaire, the demographic questionnaire, the background questionnaire, the final satisfaction questionnaire, the protocol with probes for specific portions of the instrument and the debriefing questions (imbedded in the protocol) are included.

We estimate that users will spend 1.5 hours on average taking the study, including time spent working on the background, demographic and satisfaction questionnaires, the online survey instrument and the debriefing questions. Thus, the total estimated respondent burden for this study is 23 hours.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Erica Olmsted-Hawala

Center for Survey Methods Research

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-4893

Erica.L.Olmsted.Hawala@census.gov