Cognitive Testing of Translated ACS Language Assistance Guides
Protocol Guide - English
Cognitive Testing of Translated ACS Language Assistance Guides
Protocol Guide – English
“B”
Phase 1 - Round 1
PARTICIPANT ID #: |_P_|___|_R_|___|___|___|___|___|___| DATE: ____ / ____ / 2010
LANGUAGE (CHECK ONE): 1 CHINESE Specify dialect: ____________________
2 KOREAN
INTERVIEWER: 1 Ginny 2 Hyunjoo 3 Jiyoung 4 Suzie 5 Younhee
6 Gretchen 7 Guanya 8 Mandy 9 Michelle 10 Lu 11 Grace
LAG CONTENT: Start Here box, Persons 1 and 2, and Housing Questions 12 – 21
START TIME: ____________ AM / PM
SECTION I: INFORMED CONSENT
|
PLACE THE CONSENT FORM IN FRONT OF PARTICIPANT |
A. Before we start, I would like you to read over the document in front of you. This document explains a little bit about this interview and provides information about your rights as a participant. In addition, the back page asks for your permission to have this session audio recorded. Please ask me any questions you have about this document. Once you have finished reading the document, please sign it.
|
|
Here is a copy of this form for you to keep.
B. IF PARTICIPANT PROVIDES CONSENT TO HAVE THE SESSION AUDIO-TAPED: I will now turn on the tape recorder.
|
TEST TAPE RECORDER AND TURN IT ON AGAIN AFTER TEST |
Now, I would like to confirm on tape that it is fine with you to record this interview.
VERBAL CONSENT MUST BE AUDIO-RECORDED
SECTION II. INTERVIEW OBSERVATIONS |
This is a form called “Language Assistance Guide”. It has been translated in [Chinese/Korean] for the American Community Survey conducted by the Census Bureau.
The Census Bureau counts population in the U.S. and also conducts various kinds of
surveys. They routinely encounter non-English speaking households and therefore would
like to have this form available in other languages beside English. It will be mailed to
people who request them across the U.S.
Researchers at the Census Bureau are interested in knowing your opinions about the translation.
They have hired RTI International to conduct this research. Your participation is very
important because it will help the Census Bureau determine if this form is translated well
and is culturally appropriate.
I’d like you to read through this form and record your answers to the questions. Please try to complete the form on your own, as if I were not here. While you fill out the form, I’ll be taking some notes and then we will discuss the translation together.
This is NOT any kind of test and there are no right or wrong ways to fill it out. We are practicing with as many different kinds of people as possible in different parts of the country to see if everyone fills out the form the same way and to see if it’s easy or difficult for people to fill out.
Here is the form. When you are ready, begin with “Start Here”.
|
INTERVIEWER:
|
A. Introduction and Practice
Now I have some specific questions about how you understood some of the words and phrases that were used in this form. I will first point out these words and phrases to you and let you read them over before I ask about your thoughts. I really want to hear all of your opinions and reactions.
Let’s do a short practice before we begin. Please look at this question:
How many windows are there in the house or apartment where you live?
What would your answer be?
Now, tell me everything you were thinking about as you went about answering this question.
INTERVIEWER: Probe as necessary.
How did you come up with that answer?
What do you think they mean by the term “window”?
What is this question asking in your own words?
Thank you. Don’t hesitate to speak up whenever something seems unclear, is hard to answer, or doesn’t seem to apply to you. Again, this is NOT any kind of test and there are no right or wrong answers. Just tell me what you think of the translation.
Okay, now let’s get to the form that you filled out.
SECTION III. COGNITIVE INTERVIEW |
B. Housing Questions 12-21
Housing Question 12
First, take a look at Question 12. How did you come up with your answer?
|
INTERVIEWER: ADMINISTER PROBES AND RECORD ANSWERS IN TABLE BELOW. |
This question mentions several government programs.
Have you heard of [PHRASE] before? (IF APPLICABLE, SAY: Its English acronym is …, have you hard of it before?)
What do you think they mean by [PHRASE] here?
[IF YES] Is there a better way to describe this program?
Phrase |
Heard before? (y/n) |
(If heard before) Better translation |
What it is |
Food Stamps |
|
|
|
Food Stamp benefit card |
|
|
|
Supplemental Nutrition Assistance Program (SNAP) |
|
|
|
WIC |
|
|
|
National School Lunch Program |
|
|
|
Housing Question 13
In Question 13, what do you think they mean by “condominium” here? Have you heard of the English word “condo” or “condominium”?
What do you think they mean by “condominium fee”?
The “condominium fee” is assessed by the condominium owners’ association for the purpose of improving and maintaining the common areas. Are there better ways to describe it?
Now I’d like you to read the instruction here [POINT TO ITALIC TEXT] aloud to me then I will ask some questions. “For renters, answer only if you pay the condominium fee in addition to your rent; otherwise, mark the “None” box.”
|
INTERVIEWER: WHILE R IS READING, DOCUMENT DIFFICULTIES OBSERVED.
|
PROBE IF APPROPRIATE: “I noticed when you read those instructions…, you were (laughing/ hesitating/pausing), what were you thinking at that point?”).
Do those instructions sound natural to you in <language>?
What is the instruction saying in your own words?
|
INTERVIEWER: DISCUSS DIFFICULTIES WITH QUESTION LAYOUT IN QUESTION 13.
|
Routing Instruction B
Now please read the phrase right above Question 15 to yourself. What do you think these phrases are asking you to do?
Did you notice them while filling out the form?
|
INTERVIEWER: IF R DID NOT FOLLOW THE ROUTING INSTRUCITON, DETERMINE THE CAUSE.
|
Housing Question 15a-b
You are doing great. Let’s take a look at Question 15b. IF R SKIPPED QUESTION 15b, SAY: That’s okay. Let’s look at it now.
What do you think this question is asking here?
Routing Instruction C
Now please read the phrase right above Question 16 to yourself. What do you think these phrases are asking you to do?
Did you notice them while filling out the form?
|
INTERVIEWER: IF R DID NOT FOLLOW THE ROUTING INSTRUCITON, DETERMINE THE CAUSE.
|
Housing Question 16
Let's look at Question 16. IF R SKIPPED QUESTION 16, SAY: That’s okay. Let’s look at it now.
Was there something confusing? What is this question asking in your own words?
What does the term “lot” mean to you here?
Housing Questions 17
Please read Questions 17 to yourself. IF R SKIPPED IT, SAY: That’s okay. Let’s look at it now.
What do you think they mean here in Question 17?
How did you come up with the amount?
When you read the phrase “annual real estate taxes” in this question, what period of time were you thinking about?
What does the term “property” mean to you here?
|
INTERVIEWER: DISCUSS DIFFICULTIES WITH QUESTION LAYOUT IN QUESTION 17.
|
Housing Questions 18
Please read Questions 18 to yourself. IF R SKIPPED IT, SAY: That’s okay. Let’s look at it now.
What do you think they mean here in Question 18?
How did you figure the amount?
When you read the phrase “annual payment for fire, hazard, and flood insurance” in this question, what period of time were you thinking about?
What does the term “none” mean to you here?
Housing Question 19a and 19b
Okay, let's look at Question 19a. How did you come up with your answer?
IF R SKIPPED QUESTION 19a, SAY: That’s okay. Let’s look at it now. What do you think this question is asking?
Have you heard of the term “deed of trust”?
A deed of trust is used in some areas instead of a mortgage and involves a third party, known as the trustee, who is given technical title to the property. Can you think of a better way to say this phrase in your language?
Have you heard of the term “contract to purchase”?
With a contract to purchase, the original owner has the title to the property, while the borrower has the right to occupy or otherwise use the property so long as he or she makes the required payments to the original owner towards the purchase of the property. Can you think of a better way to say this phrase in your language?
Now, please read Question 19b aloud. What do you think this question is asking about?
How did you come up with your answer?
In this question, what period of time were you thinking about?
What do you think they mean by “regular mortgage payment?”
What do you think they mean by “Include payment only on FIRST mortgage or contract to purchase”.
Housing Question 20a-b
Okay, let's look at Question 20a and 20b. How did you come up with your answer?
IF R SKIPPED QUESTION 20a-b, SAY: That’s okay. Let’s look at it now. What do you think this question is asking?
Have you heard of the term “second or junior mortgages”?
A second or junior mortgage is secured by real estate but has been made by the homeowner in addition to the first mortgage. It gives the lender a claim against the property which is second to the claim of the holder of the first mortgage. Can you think of a better way to say this phrase in your language?
Housing Question 21
Good, now Question 21. IF R SKIPPED QUESTION 21, SAY: That’s okay. Let’s look at it now. What do you think this question is asking?
There are several terms in Question 21. Let’s discuss them. First, could you figure out what the phrase “personal property taxes” mean here?
This question is about a mobile home. It is a home that can be moved around. It is parked on a piece of land and there may be rent for using the land. The translation here says “site rent”. Does it sound natural to you for describing this land?
Routing Instruction E
[POINT TO INSTRUCTION E] Now please read these phrases. Did you notice them while filling out the form?
What do you think these phrases are asking you to do?
C. Debriefing and Closing
Finally, I have just a few more questions.
Have you ever participated in any cognitive interviewing like the one we did today?
1 Yes Tell me more about it. By what organization?
2 No
Have you filled out a survey questionnaire in your country of origin?
1 Yes Tell me more about it. Was it for market research, demographic surveys, or something else?
2 No
How about in the U.S.? Have you filled out a survey questionnaire in the U.S.?
1 Yes Tell me more about it. What kind? Was it for market research, demographic surveys, or something else?
2 No
Have you ever participated in any survey interview before? For example, instead of your filling out the answers to questions on a survey questionnaire like today, the survey questions were read to you by an interviewer? This can be done in person or over the phone.
1 Yes Tell me more about it. Was it by phone or in person? What was it like? By what organization?
2 No
Think about your family members, friends and neighbors who speak [Chinese/Korean] primarily. Have you heard that they have ever participated in any survey interview before?
1 Yes
2 No
INTERVIEWER: MAKE ADDITIONAL NOTES HERE.
INTERVIEWER: PROBE FOR THE OBSERVED DIFFICULTIES IN SECTION II THAT
YOU HAVE NOT DISCUSSED WITH THE RESPONDENTS. I noticed when you were filling out the questionnaire, you ….., what were you thinking at that point?
Are there any phrases or words in this form that did not sound natural to you but we have not discussed? What about things that would not be appropriate for you or someone else who speaks [Chinese/Korean]? For example, offensive or strange wording, etc.
[TIME PERMITTING] Let’s look at the Start Here box again. I’m going to read a brief description of a Mr. Yang’s situation: Mr. Yang’s daughter goes to school in another state. She lives in the dormitory during the academic year but spends summer vacation with Mr. Yang. According to the DO NOT INCLUDE instructions, should Mr. Yang count his daughter in this question if he is filling it out during the summer? Why or why not?
Do you have any final comments or questions?
I want to thank you very much for your participation. I will now give you $40 and I will ask you to sign a receipt form verifying that you received the money.
INTERVIEWER: TURN OFF THE TAPE RECORDER. HAND THE CASH INCENTIVE TO THE PARTICIPANT.
End Time: ____________ AM / PM
File Type | application/msword |
File Title | Cognitive Interview Protocol Guide |
Author | RTI Staff |
Last Modified By | mom |
File Modified | 2010-01-29 |
File Created | 2010-01-29 |