Various Demographic Pretesting Activities

Generic Clearence for Questionnaire Pretesting Research

omb1013AQESpanishraceethnicityenc11

Various Demographic Pretesting Activities

OMB: 0607-0725

Document [doc]
Download: doc | pdf

Protocol for CPEX-OMB Alternative Race Question

(FORM X14/B4a)

April 8, 2010


Participant ID #: |___|___|___|___|___|



Interview Date: |___|___| / |___|___| / |_2_|_0_|_1_|_0_| (mm/dd/yyyy)


Interviewer initials: |___|___|

Location: DC MD VA

Start Time: ____________ AM / PM End Time: ____________ AM / PM



Section I. Cognitive Interviewing


Now we are ready to start the interview. I'll give you the form to fill out. I'd like you to fill it out as you would if you received it in the mail, at home. But with one big difference:


What I would like you to do is to tell me what you are thinking and feeling as you come up with your answers and fill out your Census form. Instead of thinking to yourself, I’d like you to think out loud, and I will remind you to do this from time to time. I will also ask you some questions about your answers and about the questions themselves. I am really interested in how these questions work for you, so there are no right or wrong answers.


Here is the form [HAND CENSUS FORM TO R]. This form asks questions about everyone in your household. Today, for the purposes of this interview, I'd like you to make yourself the first person listed. [INTERVIEWER: POINT OUT THE PERSON 1 SPACE.] I will give you a few minutes for you to fill it out, and remember to tell me what you are thinking as you come up with your answers.


*** START TAPE NOW***


INTERVIEWER: WE ARE PROBING ON QUESTIONS 1-6 TO TRAIN R TO THINK ALOUD AND RESPOND TO PROBES. THESE QUESTIONS ARE NOT A RESEARCH FOCUS. ALL PROBES SHOULD BE ASKED AFTER R PROVIDES A WRITTEN RESPONSE, EXCEPT FOR THINK ALOUD PROBES, WHICH ARE USED ANY TIME THAT R FALLS SILENT OR HAS A VISIBLE REACTION.


DO NOT PROBE OR INTERRUPT ONCE R STARTS TO ANSWER Q8/Q9 ABOUT HISPANIC ORIGIN/RACE. THIS WILL ALLOW US TO OBSERVE DIFFICULTIES IN SEGMENTING THE QUESTIONS AND IN DEALING WITH COMPLEX FORMATS.


TRAINING PROBES


RESIDENCE RULES AND QUESTION 1:


WHILE R IS READING, INTRODUCE THINK ALOUD PROBES


  • (AS NEEDED) What are you thinking? / What are you looking at?

  • (AFTER R ANSWERS Q.1) How did you arrive at this answer?

  • (ABOUT THE RULES AND Q.1) Can you tell me what this part is asking you to do?


QUESTION 2:


  • (AS NEEDED) What are you thinking? / What are you looking at?

AFTER R ANSWERS Q2:

  • How did you choose your answer?

  • What does “people staying here temporarily” means to you in this question?



QUESTION 3:

  • (AS NEEDED) What are you thinking? / What are you looking at?

  • Can you tell me what “occupied without payment of rent” means to you in this question?


QUESTION 4:

  • No probes


QUESTION 5:

Remind R to write his/her own name in this space.

  • (AS NEEDED) What are you thinking? / What are you looking at?

  • Do you think you would have written your name here if I hadn't asked you to? Why or why not?


QUESTION 6:

  • No probes


QUESTION 7:

  • (AS NEEDED) What are you thinking? / What are you looking at?

  • What information is this question asking for?


QUESTION 8 AND 9:

  • ALL PROBING WILL TAKE PLACE AFTER R HAS COMPLETED BOTH QUESTIONS. (UNLESS R STARTS TO COMPLAIN OR HAS NOTICEABLE DIFFICULTY, IN WHICH CASE, ASK ABOUT THE PROBLEM.)


OBSERVATION NOTES: RECORD ANY NOTABLE BEHAVIORS BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY. INCLUDE:

- EVIDENCE THAT R READ Q8 OR Q9 PARTIALLY

- EVIDENCE THAT A QUESTION OR PART OF A QUESTION IS RE-READ?

- EVIDENCE THAT R SKIPS THE QUESTION, PARTS OF A QUESTION OR READS ONLY ANSWER CATEGORIES?

- DOES R BOTH MARK THE BOX AND PROVIDE A WRITE IN, IF APPROPRIATE?

- DOES R FILL BOTH QUESTIONS IN ORDER OR OUT OF ORDER?

- DOES R GO BACK AND CHANGE ANY ANSWER AT ANY POINT?

- DOES R APPEAR TO BE SEARCHING FOR ANYTHING OR HAVE DIFFICULTY IN LOCATING A RESPONSE?

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

INTERVIEWER: ASK R TO STOP AFTER COMPLETING Q8 AND Q9 FOR PERSON 1, SAY OR PARAPHRASE: “I see you have finished answering for yourself. Before you go on, I’d like to ask you some questions. I'm going to ask you about Question 8 first.”


PROBES FOR PERSON 1 (RESPONDENT’S INFORMATION):


Q8. Hispanic origin


INTERVIEWER: If R had a notable problem with Q8, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”


Standard Probes:


  • How did you choose your answer(s) for Question 8?

  • [AS APPROPRIATE]:

    • Why did you write in _____? [If a person writes in Mestizo, Mixed Race, Multiracial, or Biracial] What does TERM mean to you? Can you tell me more about why you reported TERM instead of reporting multiple races or origins?

    • I notice you didn't write anything in down here. Can you tell me why?

  • What information did you think this question was asking for?

  • What does ‘origin’ mean to you in this question?

Did you notice these examples? (If no, direct R to read them now.)

  • Was there something you expected to see, or would have been helpful to you, in the examples that

wasn’t there?

  • Do the words “Hispanic, Latino, or of Spanish origin” mean the same thing, or do they mean

something different? What do they mean?

Q9. Race


INTERVIEWER: If R had a notable problem with Q9, probe that now. For example: “I noticed that you spent a long time looking for an answer (thinking about what to write). I know it might be a little hard to recall, but can you tell me what was going on for you then?”


  • How did you choose your answer(s) for Question 9?

  • [AS APPROPRIATE]:

    • Why did you write in _____? [If a person writes in Mestizo, Mixed Race, Multiracial, or Biracial] What does TERM mean to you? Can you tell me more about why you reported TERM instead of reporting multiple races or origins?

    • I notice you didn't write anything in down here. Can you tell me why?

    • (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)

  • What information did you think this question was asking for?

  • What does ‘race’ mean to you in this question?

    • What does “American Indian or Alaska native” mean to you?

    • What does “enrolled or principal tribe” mean to you?

  • If you can remember, did you read the whole question? Where did you stop? What parts did you see?

  • Did you notice the heading above the Asian and Native Hawaiian and Other Pacific Islander categories? What did you think of that? Are these headings helpful or not helpful?


  • Did you notice these examples after ‘Other Asian’? (If no, direct R to read them now.) Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there? What do you think of the order or the examples?

  • Did you notice these examples after ‘Other Pacific Islander’? (If no, direct R to read them now.) Was there something you expected to see, or would have been helpful to you, in the examples that wasn’t there? What do you think of the order or the examples?

  • How about “Some other race”, who do you think this applies to?



NOTE TO INTERVIEWER: We are saving probing on instructions for subsequent Persons. If this is a one-person household, however, ask about the “mark one or more” instructions and the instruction prior to Question 8 NOW.


PROBES FOR PERSON 2:


INTERVIEWER: ASK R TO COMPLETE THE FORM FOR PERSON 2: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”


INTERVIEWER: DO NOT PROBE ANY OF THE OTHER DEMOGRAPHIC QUESTIONS PRIOR TO HISPANIC ORIGIN AND RACE. NOTE ANY OBSERVED DIFFICULTIES BELOW AND BE SURE TO INCLUDE THEM IN YOUR SUMMARY.

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________

_____________________________________________________________________________


STOP R AFTER INFORMATION FOR PERSON 2 IS COMPLETE AND PROBE: “I have a few more questions to ask you about Questions 5 and 6, before you go on to the next person.”


Q5. Hispanic origin for Person 2


  • Probe any difficulty noted, for example: “I noticed that you spent more time on question 5 this time. Can you tell me what you were thinking?”

  • How did you choose your answer(s) for Q5?


Q6. Race for Person 2


  • Probe any difficulty noted with this question.

  • How did you choose your answer(s) for Q6?

  • [AS APPROPRIATE]:

    • Why did you write in _____?

    • I notice you didn't write anything in down here. Can you tell me why?

    • (IF R MARKED MORE THAN ONE: How did you decide to mark more than one answer?)


  • Did you see the part of the instruction about “mark x one or more boxes”?

    • If yes: what was that instruction telling you? Who do you think this applies to?

    • If no: how many boxes did you think you could mark? If you had noticed it, would it have changed your answer?


  • Do you recall seeing the instruction “NOTE: Please answer Question 8 about Hispanic origin and Question 9 about race.”

    • (GET AN ANSWER; IF NO, POINT TO IT) What was this instruction asking you to do?

  • Did you see the instruction “For this census, Hispanic origins are not races.”

    • (GET AN ANSWER; IF NO, POINT TO IT) What does that mean to you in this question?



INTERVIEWER, SAY/PARAPHRASE: “Thanks for your answers. Please continue filling your form for the next person in your household, and remember to tell me what you are thinking as you come up with your answers.”


PROBES FOR PERSONS 3 – 6:


NOTE: We are primarily interested in how responses for children are handled in these persons. We are interested in how respondents answer for multiracial children and U.S.-born children of immigrant parents.

Q5. Hispanic origin/Q6. Race for Persons 3 – 6

  • AS NECESSARY: Why did you choose that answer (those answers) for Person (fill in)?

  • IF APPROPRIATE: I noticed that you answered differently for yourself and for your child. Can you tell me why you decided to do that? Was your child born here or in another country? If another country, where?


Section II. Alternative Forms


INTERVIEWER, SAY/PARAPHRASE: “Thank you, we're done with this part of the interview. Now, I'd like to show you two more forms to compare with the one you just filled out.


SHOW FORM X16 / B4c: Please take a look at this form. You don't have to fill it out, but please read it over. [Give a few moments for R to read the form].

What are some of the most noticeable differences between this and the form you saw first? Anything else?

How would you have answered this form?

[PROBE AS APPROPRIATE:]

Did you notice that Q9 is different in X16?

What information do you think Q9 in X16 is asking for?

Is it the same or different from X14?

If you had to pick one wording, which do you prefer X14 or X16? Why?


Did you notice that X16 does not have the heading above Asian and NHPI?

Do you prefer the form with the headings above Asian and NHPI or the one without the headings? Why?


Did you notice that the examples for Other Asian are listed differently?

Which do you prefer? Why?


NOW SHOW FORM X17 / B4d

And now take a look at form X17 [give a few moments for R to read the form].

What are some of the most noticeable differences between X17 and X14 (the form you saw first)? Anything else?

How would you have answered this form?


RANKING

Please rank these forms in the order of your preference. Why did you put them in that order?



Section III. Debriefing


Just a few final questions to wrap up –


  • Overall, would you say the survey questions were easy or difficult to respond to? Why?


  • Do you think there are questions some people would find difficult? Sensitive?


  • Were you born here, or in another country? (If another country) Where? Did you report your race and origin differently there than how you report here? (if yes) How?


  • What language do you speak at home? What language did you speak as a child?


  • Do you have anything else you would like to tell us that you haven’t had a chance to mention, yet?


That’s all the questions that I have. Thank you for your time.

7


CPEX-OMB FORM X14/B4a


File Typeapplication/msword
File TitleProtocol for 2008 CPEX Hispanic Origin/Race FORM B1b+B1c+B1d
AuthorBureau Of The Census
Last Modified ByRodney Terry
File Modified2010-04-22
File Created2010-04-12

© 2024 OMB.report | Privacy Policy