Round 2 Protocol Version 1

Attachment Round 2 Protocol

Cognitive Testing of American Community Survey (ACS) Respondent Comment/Feedback

PARTICIPANT ID #: _________________________ DATE: ____ / ____ / _____

INTERVIEWER’S NAME:_________________________________________________

START TIME: ______: ______ AM / PM

Introduction

Hello, my name is ________________, and I work for the Census Bureau. Thank you for agreeing to participate in our study. Let me start by telling you a little about what we will be doing today. The U.S. Census Bureau counts the population in the United States and also conducts various kinds of surveys. Today we will be trying out some new questions for one of these surveys called the American Community Survey, or the ACS.

Before questions are added to surveys, it’s important to try out questions with the help of people such as yourself. It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way.

If you agree to take part in this study, I will ask you to answer the questions as if they were a part of the ACS. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. I will ask you some questions about your answers, or about the questions themselves. There are no right or wrong answers. Our purpose is not to compile information about you. Instead, your interview, along with those of others, will show us how to improve these questions for the survey.

Your participation in this interview is voluntary, and you may decline to answer any question at any time.

[IF OBSERVER PRESENT]: I also want to let you know that this interview is being observed by [DESCRIBE OBSERVER’S ROLE: e.g., someone who works on the American Community Survey]. They are observing the interview because they are also interested in how the questions work for people like you.

Informed Consent

• [HAND RESPONDENT COPY OF THE CONSENT FORM.]

Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant, such as that all information you provide is confidential and we won’t use your name in any report. It also asks for your permission to have this session audio (IF RELEVANT: and video) recorded so that I can concentrate on what you are saying rather than having to take extensive notes. Please ask me any questions you have about this document. Once you have finished reading the document, please sign and date it.

• [PARTICIPANT READS FORM THEN SIGNS AND DATES FORM.]

Thank you for signing the consent form. I will also sign this form to show that I agree to everything in the form.

• [INTERVIEWER SIGNS AND DATES FORM.]

Do you have any questions about the process?

[IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION RECORDED]:

• AUDIO RECORDING ONLY: I will now turn on the recorder. [TURN ON RECORDER]

• VIDEO AND AUDIO RECORDING: I will need to leave the room for a moment to turn on the video recorder. I will be right back. [GO TO OBSERVATION ROOM AND TURN ON, THEN RETURN.] Thank you for your patience. I will now also turn on this audio recorder, so that I have a back-up in case the video recording does not work. [TURN ON RECORDER]

ACS Administration

IF ADMINISTERING PAPER

I am going to give you a paper copy of the American Community Survey. Please go through and answer these questions as you would if this form had been mailed to your home. Please pause when you get to the part of the form that says “Stop here.” I will tell you what to do next. LET RESPONDENT GO THROUGH ROSTER AND HOUSING QUESTIONS.

IF ADMINISTERING INTERNET

I am going to give you a laptop that will have the American Community Survey displayed on the Internet. Please go through and answer these questions as you would if you were completing this questionnaire on the Internet at your home. I’m going to ask you to stop halfway through and give you additional instructions of what to do next. LET RESPONDENT GO THROUGH ROSTER AND HOUSING QUESTIONS.

ALL RESPONDENTS

ONCE RESPONDENT HAS ANSWERED ROSTER AND HOUSING QUESTIONS, STOP AND COUNT HOUSEHOLD MEMBERS (AFTER HOUSING QUESTION 23 ON PAPER QUESTIONNAIRE AND ON PSELECT SCREEN IN INTERNET INSTRUMENT). FOR THE INTERNET INSTRUMENT, IT’S THE SCREEN THAT SAYS “THE NEXT QUESTIONS ARE ABOUT EACH PERSON IN THE HOUSEHOLD...”

IF MORE THAN 3 HOUSEHOLD MEMBERS (INCLUDING RESPONDENT):

• ASSIGN 2 HOUSEHOLD MEMBERS TO ANSWER ABOUT IN ADDITION TO SELF:

  • FIRST PRIORITY: EXTENDED FAMILY MEMBER OR UNRELATED HOUSEHOLD MEMBER

  • SECOND PRIORITY: ONE OF EACH GROUP IN PRIORITY #1 IF THERE ARE MORE THAN ONE

  • THIRD PRIORITY: IMMEDIATE RELATIVE

• IF ADMINISTERING INTERNET: Please tell me how the people listed here are related to you. USE THIS INFORMATION TO ASSIGN 2 HOUSEHOLD MEMBERS.

• IF 3 OR FEWER HOUSEHOLD MEMBERS (INCLUDING RESPONDENT): Thank you. You can continue answering the American Community Survey as you would at your home.

• IF MORE THAN 3 HOUSEHOLD MEMBERS (INCLUDING RESPONDENT): Thank you. If this were the real American Community Survey, you would answer the next questions about everybody that you included. However, in order to make sure we end this interview on time, please answer the remaining questions about [NAMES AS ASSIGNED] only. I will remove [NAMES NOT ASSIGNED] from the roster before you continue.

(ALLOW UP TO 45 MINUTES FOR INTRODUCTION AND ACS ADMINISTRATION. IF RESPONDENT IS TAKING TOO LONG, ENSURE THAT THEY HAVE AT LEAST ANSWERED ALL QUESTIONS FOR PERSON 1 AND 2, THEN SAY: In order to make sure we end this interview on time, we are going to move to the next part of the interview.

INTERVIEWER CHECK: DID THE RESPONDENT LOOK AT THE BACK PAGE? ___________________

AFTER ADMINISTRATION COMPLETE: Thank you. Please answer these questions as well. [HAND RESPONDENT COMPLETION TIME AND BURDEN QUESTIONS]. AFTER RESPONDENT COMPLETES: Great, thank you.

____________________________________________________________

Respondent Comment/Feedback (Question Administration and Probing)

IF FEEDBACK ITEM WAS NOT FILLED IN DURING ADMINISTRATION:

[DIRECT RESPONDENT TO FEEDBACK ITEM AT THE END OF THE QUESTIONNAIRE/INSTRUMENT IF THEY ARE NOT ON THE PAGE. WHEN THEY ARE ON THE PAGE WITH THE FEEDBACK ITEM, CONTINUE BELOW]

• Now I would like you to take a moment and review this page. Imagine you had just finished completing the ACS at home [on paper/on the Internet]. Please tell me what you would do.

  • [IF R INDICATES THEY WOULD ANSWER FEEDBACK ITEM BUT DOESN’T FILL IT OUT] If you would answer this item [POINTING TO FEEDBACK ITEM], please write down what you would say.

• IF PAPER: PROCEED TO VERSION A PROBES BELOW

• IF WEB: PROCEED TO VERSION D PROBES ON PAGE 5

IF FEEDBACK ITEM WAS FILLED IN DURING ADMINISTRATION:

• IF PAPER: PROCEED TO VERSION A PROBES BELOW

• IF WEB: PROCEED TO VERSION D PROBES ON PAGE 5

Version A Probes:

• In your own words, what is this asking you to do?

• IF FEEDBACK ITEM IS FILLED: How did you decide what to write here?

• IF FEEDBACK ITEM IS NOT FILLED: How did you decide to leave this blank? What kind of comments do you think would be appropriate to write here?

Version B Probes:

HAND RESPONDENT VERSION B. I would like you to take a moment and review this page. Please do not fill this out.

• In your own words, what is this asking you to do?

• What kind of comments do you think would be appropriate to write?

Version C Probes:

HAND RESPONDENT VERSION C. I would like you to take a moment and review this page. Please do not fill this out.

• In your own words, what is this asking you to do?

• What kind of comments do you think would be appropriate to write here?

• IF NEEDED: Did you notice the black heading with the word “thank you”? [POINT TO BLACK HEADING ON VERSION C]

• If needed: What do you think of this?

• If needed: Did you notice the “thank you” statement in the other versions?

Comparing the three paper versions:

• Thinking about all three versions of this page you saw today, [POINT TO EACH OF THE THREE VERSIONS], what do you see as the difference, if any, between these pages?

• Do you think these are required items or optional items to fill out?

• IF RESPONDENT THINKS IT’S REQUIRED: Would you write anything if it were optional?

• Which one do you prefer, if any?

• If needed: Tell me more about that

• Is there one that you think is easier to answer than the others?

  • If needed: Tell me more about that

PROCEED TO DEBRIEFING QUESTIONS ON PAGE 6.

Version D Probes:

• In your own words, what is this asking you to do?

• IF FEEDBACK ITEM IS FILLED: How did you decide what to write here?

• IF FEEDBACK ITEM IS NOT FILLED: How did you decide to leave this blank? What kind of comments do you think would be appropriate to write here?

• Would you have filled this item out if this were part of the ACS questionnaire/instrument?

Version E Probes:

HAND RESPONDENT VERSION E. I would like you to take a moment and review this page. Please do not fill this out.

• In your own words, what is this asking you to do?

• What kind of comments do you think would be appropriate to write here?

Comparing the two Web versions:

• Thinking about the two versions of this page you saw today, [POINT TO EACH OF THE TWO VERSIONS], what do you see as the difference, if any, between these pages?

• Do you think these are required items or optional items to fill out?

• IF RESPONDENT THINKS IT’S REQUIRED: Would you write anything if it were optional?

• Which one do you prefer, if any?

• If needed: Tell me more about that

• Is there one that you think is easier to answer than the others?

  • If needed: Tell me more about that

____________________________________________________________

Debriefing Questions

We are almost finished with the interview.

1. Each of the versions you saw today had a space to provide comments. If you wanted to give feedback about the ACS, would you prefer to do it here, or in some other way?

If needed: By sending an email or giving feedback online? By calling someone?

2. IF FEEDBACK ITEM FILLED OUT: Would you expect to receive a response to what you wrote here? IF FEEDBACK ITEM NOT FILLED OUT: Do you think other people who write something here would expect a response?

3. Thinking about the items you looked at today [SHOW THE PAPER FEEDBACK ITEMS OR THE WEB FEEDBACK ITEMS], were there any you think some people might find difficult to answer?

4. What kinds of things do you think other people might write if they were asked to answer any of these statements?

5. Was there anything else that stood out to you about your experience that we didn’t talk about today?

• IF RESPONDENT MENTIONED ANYTHING ELSE STOOD OUT ABOUT THEIR EXPERIENCE: Did you think about adding this in the space provided for comments [SHOW THE PAPER FEEDBACK ITEMS OR THE WEB FEEDBACK ITEMS]?

Those are all of the questions that I had for you today. Thank you very much for your participation.

[TURN OFF THE RECORDER.]

Here is an envelope containing your $40, and a voucher form that verifies I gave you the money. Please complete the information in the highlighted areas and sign and date.

[HAND THE CASH INCENTIVE TO THE PARTICIPANT AND COLLECT VOUCHER FORM.]

7