Memorandum
To: Office of Management and Budget
From: Lin Wang, Center for Survey Measurement, U.S. Census Bureau
Subject: Usability evaluation of Administrative Records Messaging
Date: July 12, 2016
Backgroud
The U.S. Census Bureau is developing a message on general administrative data linkage. The message is to be inserted in census.gov. The message shall be findable, understandable, and transparent about the U.S. Census Bureau’s administrative data program. This study is to evaluate the message in terms of findability and comprehension.
Purpose of research
Investigate how easy or difficult it is for respondents to find information about administrative date linkage on census.gov.
Investigate respondents’ comprehension of the information described in 1).
Research design summary
This study includes two tasks. Task 1 investigates how easy or difficult it is for respondents to find information about administrative date linkage on census.gov. Task 2 investigates respondents’ comprehension of the information about administrative date linkage on census.gov. Task 1 will be conducted first, followed by Task 2. The investigation will be conducted on two user populations: Advocates and non-advocates (i.e., general public). The study protocol for the advocates are the same as that for the general public except for a few targeted probes that are highlighted in yellow in the protocol (see Enclosure C).
Testing materials
The portion of census.gov that contains relevant information – see enclosed screenshots (Enclosure A).
A sample invitation letter (Enclosure B)
Test case
The participant will perform a search task to find information about administrative data linkage. The task is described in the protocol (Enclosures C and D).
Data collection
Three types of data will be collected: (1) search navigation path, (2) eye tracking data of participants’ search behavior, and (3) respondents’ think-aloud.
Data analyses
Search navigation path will be analyzed in comparison with the optimal path.
Eye tracking data will be analyzed by examining participants’ gaze pattern during search.
Think-aloud data will be analyzed using qualitative methods to explore participants’ cognitive process during search.
Testing materials
Same as in Task 1.
Test case
The participant will be debriefed by the Test Administrator (TA).
Data collection
Responses to debriefing questions (see Enclosure C and D) will be collected.
Data analyses
Debriefing responses will be analyzed using qualitative methods to summarize participants’ understanding of the information presented in the portion of census.gov.
Participants and sampling
Nine non-advocate participants who meet all of the following inclusion criteria:
Being fluent in English reading.
Having schooling between 8th grade and a two-year college.
Age between 18 and 65, inclusive.
Having never seen the testing materials before.
Non-advocate participants will be recruited from the public through advertisement and personal contact.
Nine advocate participants who meet all of the non-advocates inclusion criteria and the following additional inclusion criteria: Being a Privacy Advocates who have worked with the U.S. Census Bureau in the past few decades on various committees. Advocate participants will be recruited from the U.S. Census Bureau’s advocate pool. The study team will randomly contact this pool of participants through their professional contact information on file and determine interest and availability to participate in this study.
Burden
We estimate that each participant will spend 60 minutes to participate in the study, including time for consent, collecting demorgraphic information, practice, and debriefing. Total burden for 18 participants is estimated 18 hours (1 hour x 18 participants).
Incentives
The participant will receive $40.00 to offset cost of participation in the study.
Sponsor
Policy Coordination Office and Center for Administrative Records Research and Applications of the U.S. Census Bureau sponsor the project.
New items
The messages are new and yet to be tested.
Enclosures
The following materials will be used in the study and are attached with this letter:
Screenshots from census.gov
Sample invitation letter
Test protocol for general public
Test protocol for advocates
Demographic questionnaire
Computer experience questionnaire
Contact
The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:
Lin Wang
Center for Survey Methods Research
U.S. Census Bureau
Washington, D.C. 20233
(301) 763-9069
lin.wang@census.gov
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Misa |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |