Various Demographic Pretesting Activities

Draft Protocol for 2008 CPEX Hispanic Origin/Race

Modified HO-R Question (FORM X9a)

Cognitive Interviewing - Probes

April 8, 2010

Participant ID #: |___|___|___|___|___|

Interview Date: |___|___| / |___|___| / |_2_|_0_|_1_|_0_| (mm/dd/yyyy)

Interviewer initials: |___|___|

Location: □ DC □ MD □ VA

Start Time: ____________ AM / PM End Time: ____________ AM / PM

Section I. Cognitive Interviewing

Now we are ready to start the interview. I'll give you the form to fill out. I'd like you to fill it out as you would if you received it in the mail, at home. But with one big difference:

What I would like you to do is to tell me what you are thinking and feeling as you come up with your answers and fill out your Census form. Instead of thinking to yourself, I’d like you to think out loud, and I will remind you to do this from time to time. I will also ask you some questions about your answers and about the questions themselves. I am really interested in how these questions work for you, so there are no right or wrong answers.

Here is the form [HAND CENSUS FORM TO R]. This form asks questions about everyone in your household. Today, for the purposes of this interview, I'd like you to make yourself the first person listed. [INTERVIEWER: POINT OUT THE PERSON 1 SPACE.] I will give you a few minutes for you to fill it out, and remember to tell me what you are thinking as you come up with your answers.

*** START TAPE NOW***

INTERVIEWER: WE ARE PROBING ON QUESTIONS 1-6 TO TRAIN R TO THINK ALOUD AND RESPOND TO PROBES. THESE QUESTIONS ARE NOT A RESEARCH FOCUS. ALL PROBES SHOULD BE ASKED AFTER R PROVIDES A WRITTEN RESPONSE, EXCEPT FOR THINK ALOUD PROBES, WHICH ARE USED ANY TIME THAT R FALLS SILENT OR HAS A VISIBLE REACTION.

DO NOT PROBE OR INTERRUPT ONCE R STARTS TO ANSWER Q8/Q9 ABOUT HISPANIC ORIGIN/RACE. THIS WILL ALLOW US TO OBSERVE DIFFICULTIES IN SEGMENTING THE QUESTIONS AND IN DEALING WITH COMPLEX FORMATS.

TRAINING PROBES

RESIDENCE RULES AND QUESTION 1:

WHILE R IS READING, INTRODUCE THINK ALOUD PROBES

•  (AS NEEDED) What are you thinking?

•  (AFTER R ANSWERS Q. 1) How did you arrive at this answer?

•  (ABOUT THE RULES AND Q.1) Can you tell me what this part is asking you to do?

QUESTION 2:

•  (AS NEEDED) What are you thinking?

AFTER R ANSWERS Q2:

• How did you choose your answer?

• What does “people staying here temporarily” means to you in this question?

QUESTION 3:

• (AS NEEDED) What are you thinking?

• Can you tell me what “occupied without payment of rent” means to you in this question?

QUESTION 4:

• No probes

QUESTION 5:

Remind R to write his/her own name in this space.

• (AS NEEDED) What are you thinking?

• Do you think you would have written your name here if I hadn't asked you to? Why or why not?

QUESTION 6:

• No probes

QUESTION 7:

• (AS NEEDED) What are you thinking?

• What information is this question asking for?

QUESTION 8 AND 9:

• ALL PROBING WILL TAKE PLACE AFTER R HAS COMPLETED BOTH QUESTIONS. (UNLESS R STARTS TO COMPLAIN OR HAVE LOTS OF DIFFICULTY, IN WHICH CASE, ASK ABOUT THE PROBLEM.)

OBSERVATION NOTES: RECORD ANY NOTABLE BEHAVIORS BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY. INCLUDE:

EVIDENCE THAT R READ THE QUESTION PARTIALLY

EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ?

EVIDENCE THAT R SKIPS THE QUESTION, PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES?

DOES R BOTH MARK THE BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?

DOES R FILL BOTH QUESTIONS IN ORDER?

DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?

DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING A RESPONSE?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

INTERVIEWER: ASK R TO STOP AFTER COMPLETING Q8 AND Q9 FOR PERSON 1, SAY OR PARAPHRASE: “I see you have finished answering for yourself. Before you go on, I’d like to ask you some questions. I'm going to ask you about Question 8 first.”

PROBES FOR PERSON 1 (RESPONDENT’S INFORMATION):

Q8. Hispanic origin

INTERVIEWER: If R had a notable problem with Q8, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”

Standard Probes:

• How did you choose your answer for Question 8?

• [AS APPROPRIATE]:

  • Why did you write in _____?[If a person writes in Mestizo, Mixed Race, Multiracial, or Biracial] What does TERM mean to you? Can you tell me more about why you reported TERM instead of reporting multiple races or origins?

  • I notice you didn't write anything in down here. Can you tell me why?

  • What information did you think this question was asking for?

• Did you notice these examples? (If no, direct R to read them now.)

• Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there?

• Do the words “Hispanic, Latino, or of Spanish origin” mean the same thing, or do they mean something different? What do they mean?

Q9. Race

INTERVIEWER: If R had a notable problem with Q9, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”

• How did you choose your answer for Question 9?

• [AS APPROPRIATE]:

• Why did you write in _____? [If a person writes in Mestizo, Mixed Race, Multiracial, or Biracial] What does TERM mean to you? Can you tell me more about why you reported TERM instead of reporting multiple races or origins?

• I notice you didn't write anything in down here. Can you tell me why?

• What information did you think this question was asking for?

• If you can remember, did you read the whole question? Where did you stop? What parts did you see?

• What does “American Indian or Alaska native” mean to you?

• What does “enrolled or principal tribe” mean to you?

• Did you notice these examples? (If no, direct R to read them now. Make sure to point to the Mayan example under the “American Indian” category.)

• Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there?

• Probe: Was there something you were expecting to see in the answer categories that wasn't there?

NOTE TO INTERVIEWER: We are saving probing on instructions for subsequent Persons. If this is a one person household, however, ask about the “mark one or more” instructions and the instruction prior to Question 8 NOW.

PROBES FOR PERSON 2:

INTERVIEWER: ASK R TO COMPLETE THE FORM FOR PERSON 2: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”

INTERVIEWER: DO NOT PROBE ANY OF THE OTHER DEMOGRAPHIC QUESTIONS PRIOR TO RACE AND HISPANIC ORIGIN. NOTE ANY OBSERVED DIFFICULTIES BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY.

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

STOP R AFTER INFORMATION FOR PERSON 2 IS COMPLETE AND PROBE: “I have a few more questions to ask you about Questions 5 and 6, before you go on to the next person.”

Q5. Hispanic origin for Person 2

• Probe any difficulty noted, for example: “I noticed that you spent more time on question 5 this time. Can you tell me what you were thinking?”

• How did you choose your answer(s) for Q5?

• (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)

• Do you recall seeing the part of the instruction about “mark x one or more boxes”? (Point it out if they didn't see it.)

• If yes: what was that instruction telling you about?

• If no: how many boxes did you think you could mark? If you had noticed it, would it have changed your answer?

• Did you see the part about “print one or more origins”? (This is in Yes, another...)

• What was that instruction telling you to do? What does “origin” mean in this instruction?

• FOR PEOPLE WHO WROTE IN ONLY ONE ORIGIN AND DIDN’T SEE THE INSTRUCTION: If you had noticed it, would it have changed your answer?

Q6. Race for Person 2

• Probe any difficulty noted with this question.

• How did you choose your answer(s) for Q6?

• (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)

• Did you read the list of examples given for the answer you chose? What did you think of them? Were these examples helpful or not helpful? NOTE: You will NOT have to ask these for the Asian and Pacific Islander check boxes, but all other categories have examples attached.

• Did you see the part of the instruction about “mark x one or more boxes”?

• If yes: what was that instruction telling you? Who do you think this applies to?

• If no: how many boxes did you think you could mark? If you had noticed it, would it have changed your answer?

• Do you recall seeing the instruction “➜NOTE: Please answer BOTH Question 8 about Hispanic origin and Question 9 about race.”

  0. (GET AN ANSWER; IF NO, POINT TO IT) What was this instruction asking you to do?

• Did you see the instruction “ For this census, Hispanic origins are not races.”

  0. (GET AN ANSWER; IF NO, POINT TO IT) What does that mean to you in this question?

INTERVIEWER: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”

PROBES FOR PERSONS 3 – 6:

NOTE: We are primarily interested in how responses for children are handled in these persons. We are interested in how respondents answer for multiracial children and U.S.-born children of immigrant parents.

Q5. Hispanic origin/Q6. Race for Persons 3 – 6

• AS NECESSARY: Why did you choose that answer (those answers) for Person (fill in)?

• IF APPROPRIATE: I noticed that you answered differently for yourself and for your child. Can you tell me why you decided to do that? Was your child born here or in another country? If another country, where?

Section II. Alternative Forms

NTERVIEWER, SAY/PARAPHRASE: “Thank you, we're done with this part of the interview. Now, I'd like to show you two more forms to compare with the one you just filled out.

SHOW FORM X9

Please take a look at this form. You don't have to fill it out, but please read it over. [Give a few moments for R to read the form].

What are some of the most noticeable differences between this and the form you saw first? Anything else?

[Point to the instructions ➜ NOTE: Please answer BOTH Question 8 about Hispanic origin and Question 9 about race(X9), and ➜NOTE: Please answer Question 8 about Hispanic origin and Question 9 about race (X9a)].

• What do you think of having the word “both” in this form (X9a) and not having the word “both” that form (X9)? Do you prefer one more than the other?

NOW SHOW FORM X4/B2c

And now take a look at this other form [give a few moments for R to read the form].

What are some of the most noticeable differences between this and the form you saw first? Anything else?

What does ‘race or origin’ mean to you in this question?

What do you think of having the write in boxes at the bottom of the form instead of in the middle?

What do you think of having a single list of examples instead of a list for each race/origin?

Is there one of the 3 forms that you like best? Why? Please rank these forms in the order of your preference. Why did you put them in that order?

Section III. Debriefing

Just a few final questions to wrap up –

• Overall, would you say the survey questions were easy or difficult to respond to? Why?

• Do you think there are questions some people would find difficult? Sensitive?

• Were you born here, or in another country? (If another country) Where? Did you report your race and origin differently there than how you report here? (if yes) How?

• What language do you speak at home? What language did you speak as a child?

• Do you have anything else you would like to tell us that you haven’t had a chance to mention, yet?

That’s all the questions that I have. Thank you for your time.

10

X9a