The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting a usability evaluation of the 2017 National Survey of College Graduates (NSCG) online questionnaire.
Usability testing will identify issues that are problematic and frustrating to the user. Testing will evaluate the usability of the screen design for different types of devices including personal computers (PCs), tablets and smartphones. The NSCG will offer a mobile-optimized design in 2017 for smaller devices in addition to the traditional optimized design for PCs. This is the first time these questions will be tested using the mobile-optimized design.
Two rounds of testing will take place with changes to the design occurring between rounds. The first round will occur between April and May 2016 and the second round will take place between July and August 2016. Recruiting will start April 8, 2016. Twenty participants will take part in each round.
If participants are testing on mobile devices, they will use their own smartphones or tablets to complete the survey. If participants are testing on laptop devices, those will be provided by Census Bureau or the contractor. We will test on-site in the usability lab at the Census Bureau and at locations with WiFi access, such as public libraries.
Participants will be from the general public and all participants will hold at least a bachelor’s degree. At least six participants across both rounds will have PhDs; six will have a Master’s degree; and six will have a professional certification. At least half of the participants will be nonwhite and at least a third of the participants will work in a science field. Participants will not have participated in previous testing of the NSCG online instrument or in any recent testing of the decennial census online questionnaires. Participants will have at least one year’s experience with using the Internet for things other than email. For those participants using their own smartphone or tablet, they will know how to use that device to access the Internet. Participants will be recruited using a combination of word-of-mouth, flyers, and online advertisements.
Participants will first be asked to complete a demographics questionnaire. Then participants will be given an NSCG letter with their login credentials for this evaluation. They will then complete the 2017 NSCG online survey using a think-aloud protocol and eye-tracking. After completing the online form, each participant will be asked to complete a satisfaction questionnaire, and asked debriefing questions about different screens.
Due to the length and complexity of the task as well as the necessity to travel to test locations, and use of their data plan for mobile devices, we plan to compensate respondents $60 for 90 minutes of their time.
Below is a list of materials to be used in the current study. Included is a note on whether each attachment is new or has already been approved by OMB.
Protocol used for the study (Enclosure 1)
Additional screening questions needed for this study (Enclosure 2)
Mailing materials (Enclosure 3) (The director’s signature will be added after approved by OMB for the 2017 NSCG.)
PowerPoint slide shows with debriefing probes (Enclosure 4) Placeholder screen shots are concurrently shown from 2015 NSCG.
Demographic questionnaire (Enclosure 5) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)
Satisfaction questionnaire (Enclosure 6) (Previously approved by OMB for usability testing in the spring of 2015 for the 2015 Census Test.)
The estimated burden for the usability sessions is 60 hours. Respondents will be screened using the Census Bureau’s generic screener and the new questions. The additional questions should take three minutes, thus the additional burden for the screening is 2 hours. The total estimated respondent burden for this study is 62 hours.
The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:
Elizabeth Nichols
Center for Survey Measurement
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-1724
Elizabeth.May.Nichols@census.gov
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Erica L Olmsted Hawala |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |