Protocol

Attachment D:

OY3 ACS Respondent Burden Testing

Cognitive Interview Protocol

Introduction

Hello, my name is ______________ and I work for Westat, a survey research company in Rockville, Maryland. Thank you for taking the time to participate in this research study.

Westat is working to develop new questions for the American Community Survey, which is sponsored by the U.S. Census Bureau. Before surveys are conducted, it’s important to try out questions with the help of people such as yourself. The survey asks questions about your household, yourself, and other people who live with you.

It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way. If you agree to take part in this study, I will hand you a questionnaire to answer. Afterwards, I will ask you some questions about the answers you gave. There are no right or wrong answers. Our purpose is not to compile information about you. Instead, your interview along with those of others will show us how to improve these questions for a later survey.

The information collected in this interview is covered under OMB # 0607-0725.

Informed Consent

Before we get started, there are a few things I should mention. This is a research project, and your participation is voluntary. If you prefer not to answer any questions just say so, and you can go on to the next one. It’s also okay if you change your mind after starting and would rather not participate.

All your answers, everything you say, will be kept confidential. We will not use your name in any reports. We may use quotes in our report that you provide during the interview, but never in a way that would identify you. The interview will take about 60 minutes and you will receive $40. We will also would like to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking; it will also help when we write up a summary of this interview. Only project staff will have access to the recording and other project materials, and those materials will be stored according to Title 13 requirements for protecting the identity of individual respondents. {Finally, some of the researchers developing the questions are here today observing our interview to learn if there are things that might need to be changed.}

HAND CONSENT FORMS TO RESPONDENT, This form contains all of the things I just told you about your rights in this interview. Please read it over and sign both copies if you are willing to take part in the study.

HAVE R SIGN TWO CONSENT FORMS, KEEP ONE AND GIVE ONE TO RESPONDENT.

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again, is it okay with you if we record this interview?

Protocol

In a moment, I am going to hand you the questionnaire. I’d like you to fill it out as if you received it in the mail at home. If you had received this form at home, your address would be located on the front of the questionnaire in the white area. Some questions ask about “this building.” For these questions, fill them out as if you’re filling the survey out at home.

After you have finished answering the questions, we will talk about some of the answers you gave. Before we begin, do you have any questions about the process?

HAND QUESTIONNAIRE TO RESPONDENT: Please go ahead and start filling out the questionnaire.

PAUSE RESPONDENT AFTER COMPLETING ROSTER. BASED ON ROSTER RESPONSES, DETERMINE WHO PERSONS 2 AND 3 SHOULD BE, AND INFORM RESPONDENT WHO TO THINK ABOUT WHEN THEY GET TO PERSONS 2 AND 3 IN THE PAPER QUESTIONNAIRE.

SELECTION OF PERSONS TO PROBE ON:

WRITE RESPONDENT’S FIRST NAME AS PERSON #1 BELOW.

ASK: Please tell me which adults ages 18 + in the household worked for pay last week, either full-time or part-time.

IF NONE, ASK IF ANY TEENS AGES 15+ WORKED FOR PAY LAST WEEK.

SELECT PERSONS 2-3 FROM AMONG THOSE WHO WERE WORKING LAST WEEK. IF NONE, DO NOT SELECT ANYONE FOR P2 OR P3.

Person 1 First Name _________________________

Person 2 First Name _________________________

Person 3 First Name _________________________

HAVE R COMPLETE PAPER QUESTIONNAIRE.

condominium/hoa fees: Q15, PAGE X

Research Questions:

Do respondents know to report only the required costs of an HOA and not voluntary/optional costs?

Are respondents able to report the monthly cost of the HOA fees (not quarterly or yearly)?

Does probing using a multipart question with more Yes/No choices help?

PERSON 1 PROBES

Thank you for filling out the survey. Before we talk about some of your answers to the survey, I have a few additional questions for you.

Now let’s look at question 15 about condominium or homeowner’s association fees. How did you decide to answer [Yes/No] here?

Was it easy or difficult to answer this question? What made it easy or difficult?

IF ANSWERED YES TO Q15: Tell me how you arrived at your answer of [X] dollars/why you selected None.

IF NEEDED: Tell me more about what you included in your answer.

Is the fee required or voluntary?

Was there anything you thought about including in your answer then decided not to? IF YES, Tell me about that.

IF ENTERED DOLLAR AMOUNT: How often do you pay this amount?

CHECK Q16, IF R IS RENTER: Is the amount you wrote in here included in your rent, or do you pay it separately?

Do you pay any [other] voluntary fees to a condominium or homeowner’s association? IF YES: What made you include/not include these in your answer?

I’d like to present you with another way this question might be asked. Please go ahead and answer these questions. (HAVE R ANSWER ALTERNATE Q15).

How would you answer each of these questions?

How did you come up with your answers to these questions?

[SHOW R BOTH VERSIONS OF QUESTION 15]. Does one of these seem easier to answer than the other? What makes you say that?

Place of work: q31, PAGE x

Research Questions:

How easy or difficult is this question for respondents and why?

Are respondents clear about whether or not their workplace location is inside the limits of a city or town?

For respondents with no obvious fixed workplace location, what do they consider to be their primary workplace location?

For respondents who have a fixed location, but do not know their workplace address, how do they identify the location of their workplace? Does adding ‘landmark’ to the instructions provide a useful additional option for respondents?

Does the work address given in response to Question 31 match the work location the respondent identifies in later probing?

PERSON 1 PROBES

Now let’s look at your answers for question 31. How easy or difficult was it to answer this series of questions? What made it easy/difficult?

Question 31f asked whether the work address is inside the city limits of [NAME OF CITY]. What made you answer [YES/NO]? How easy or difficult was it to answer this question?

How confident are you about your answer about city limits?

IF NEEDED: Do you know what the city limits for [NAME OF CITY] are?

CHECK RESPONDENT DATA SHEETFOR RESPONDENT’S WORK LOCATION, ASK: You reported during screening that you work at [one/more than one] location, is that correct? IF MORE THAN ONE: How did you decide which location to answer about?

IF R DOES NOT KNOW FULL ADDRESS: Did you notice this instruction at 31a? [INTERVIEWER: POINT TO INSTRUCTION] In your own words, what is that instruction asking you to do?

Do you think these instructions are helpful or not helpful? What makes you think that?

What do you think is meant by the word “landmark” in this instruction? Is this something you would be able to identify?

IF WROTE PARTIAL ADDRESS: How did you decide what to write in? Is there anything else you thought about writing instead?

INTERVIEWER: USE GOOGLE MAPS TO IDENTIFY ACTUAL STREET ADDRESS FOR THE LOCATION THE RESPONDENT ANSWERED ABOUT IN Q31.

Now, I’d like to see if we can find your work address on a map. (IF NEEDED Even if you don’t know the exact address, let’s see if we can find it on a map.) This is another way to help us assess how well question 31 is working. I can type the information in myself, or you can do it. Which do you prefer?

IF NEEDED, OR IF RESPONDENT HAS INCOMPLETE ADDRESS INFORMATION, WORK WITH THE RESPONDENT TO IDENTIFY THE ADDRESS USING SUGGESTIONS SUCH AS NAME OF PLACE OF WORK, LANDMARKS, STREET NAMES, OR NEARBY STREETS.

WHEN LOCATED, RECORD ADDRESS PROVIDED BY GOOGLE MAPS ON RESPONDENT DATA SHEET (OR FOR PARTIAL ADDRESSES, ANY AVAILABLE ADDRESS INFORMATION).

Is this the building where you work?

IF NEEDED: How accurate is this address for the building where you work?

Would you look up the answer yourself, if you were answering the questionnaire at home?

PERSON 2 PROBES

How easy or difficult was this for you to answer about (PERSON 2)’s work address? What made it easy/difficult?

Question 31f asked whether the work address is inside the city limits of [NAME OF CITY]. What made you answer [YES/NO]? How easy or difficult was it to answer this question?

IF LEFT ANY FIELDS BLANK: Tell me more about why you left (question) blank.

How confident are you in the answers you provided for (PERSON 2)?

INTERVIEWER: USE GOOGLE MAPS TO IDENTIFY ACTUAL STREET ADDRESS FOR THE LOCATION THE RESPONDENT ANSWERED ABOUT IN Q31. IF R COULD NOT ANSWER ANYTHING ABOUT P2’S WORK ADDRESS, SKIP THIS STEP AND GO ON TO P3.

Now, I’d like to see if we can find (PERSON 2)’s address on a map. (IF NEEDED Even if you don’t know the exact address, let’s see if we can find it on a map.) I can type the information in myself, or you can do it. Which do you prefer?

IF NEEDED, OR IF RESPONDENT HAS INCOMPLETE ADDRESS INFORMATION, WORK WITH THE RESPONDENT TO IDENTIFY THE ADDRESS USING SUGGESTIONS SUCH AS NAME OF PLACE OF WORK, LANDMARKS, STREET NAMES, OR NEARBY STREETS.

WHEN LOCATED, RECORD ADDRESS PROVIDED BY GOOGLE MAPS ON RESPONDENT DATA SHEET (OR FOR PARTIAL ADDRESSES, ANY AVAILABLE ADDRESS INFORMATION).

How accurate is this address for the building where [PERSON 2] works?

Would you find the answer for [PERSON 2] yourself, if you were answering the questionnaire at home?

PERSON 3 PROBES

How easy or difficult was this for you to answer about (PERSON 3)’s work address? What made it easy/difficult?

Question 31f asked whether the work address is inside the city limits of [NAME OF CITY]. What made you answer [YES/NO]? How easy or difficult was it to answer this question?

IF LEFT ANY FIELDS BLANK: Tell me more about why you left (question) blank.

How confident are you in the answers you provided for (PERSON 3)?

INTERVIEWER: USE GOOGLE MAPS TO IDENTIFY ACTUAL STREET ADDRESS FOR THE LOCATION THE RESPONDENT ANSWERED ABOUT IN Q31. IF R COULD NOT ANSWER ANYTHING ABOUT P2’S WORK ADDRESS, SKIP THIS STEP.

Now, I’d like to see if we can find (PERSON 3)’s address on a map. (IF NEEDED Even if you don’t know the exact address, let’s see if we can find it on a map.) I can type the information in myself, or you can do it. Which do you prefer?

IF NEEDED, OR IF RESPONDENT HAS INCOMPLETE ADDRESS INFORMATION, WORK WITH THE RESPONDENT TO IDENTIFY THE ADDRESS USING SUGGESTIONS SUCH AS NAME OF PLACE OF WORK, LANDMARKS, STREET NAMES, OR NEARBY STREETS.

WHEN LOCATED, RECORD ADDRESS PROVIDED BY GOOGLE MAPS, (OR FOR PARTIAL ADDRESSES, ANY AVAILABLE ADDRESS INFORMATION) ON RESPONDENT DATA SHEET.

How accurate is this address for the building where [PERSON 3] works?

Would you find the answer for [PERSON 3] yourself, if you were answering the questionnaire at home?

Closing and Incentive

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.
GIVE INCENTIVE AND HAVE RESPONDENT SIGN RECEIPT.

OY3 ACS Respondent Burden Testing – Cognitive Interview Protocol 11