Cognitive Interview Guide

Draft FDA Ad Exposure Frequency Questionnaire
Version 11.17.14

Cognitive interview guide

Round 1—stimuli testing

Thank you for agreeing to participate in this study today. During today’s interview, we will show you two ½ hour television programs with some advertisements and ask you to complete a survey afterwards. Your feedback will help us to strengthen the survey.

First, I’d like to provide an informed consent form that describes the study—and your responsibilities—in more detail.

[REVIEW INFORMED CONSENT]

[Note: Participants will be blind to FDA’s sponsorship]

SCREENING / INTRODUCTION / CONSENT

Thank you again for agreeing to participate.

[INSERT OMB LANGAUGE AND NUMBER]

[INSERT CONSENT LANGUAGE-OMB control number to appear on every page]

This survey is being conducted by RTI International (RTI), an independent nonprofit research organization, on behalf of a public health agency.

[INSERT LINK]

SHOW STIMULI

Now we would like you to view the television programs and ads. Please make sure you are comfortable and can read the screen from where you sit.  For the next hour, you will watch the programs and ads and then answer questions about them.

[SHOW PARTICIPANT STIMULUS]

SURVEY INSTRUCTIONS

Now that you have viewed the television programs and ads, I would like to ask you some questions.

Questions regarding television programs

• What did you think of the television program(s) you watched?

• Were they interesting?

  • What about them made them interesting/not interesting?

  • Was one more interesting than the other?

• Were the topics relevant to you?

  • What, specifically, made them seem relevant/not relevant?

• How much attention did you pay to the television programs?

  • If little attention paid:

    • What kept you from paying more attention to the TV programs?

• Did the television programs keep your attention for the full hour?

  • If not:

    • At what point did you lose focus?

Questions regarding clutter ads

• What did you think of the advertisements you saw during these television programs?

• Did any stand out? (which ones?)

• Were there any that seemed outdated or irrelevant? (which ones?)

• Were any more attention grabbing than others? (which ones?)

• For those who saw our ad more than once:

  • Were there any ads that seemed to be repeated more than once? (which one/ones?)

  • How many times did you see that ad/those ads?

Questions regarding TRINASE ad

• One of the ads you saw was for a seasonal allergy medication. Do you recall the name of that medication?

• How many times did you see that ad? (OVERLAPS WITH #19)

  • How easy or difficult was it to remember how many times you saw that ad?

  • What did you think when you saw it more than once?

  • Was it annoying, frustrating, informative etc.?

• How much attention did you pay to that ad? (OVERLAPS WITH #20)

  • The first time? (If little attention paid, what kept you from paying more attention?)

  • All subsequent times? (If little attention paid, what kept you from paying more attention?)

• What were your general impressions of that ad?

• How did it compare to other ads you have seen for prescription medications?

  • Was it similar or different?

  • Is the ad believable?

  • Is there anything about the ad that you would change?

Testing of Survey items

Thank you for answering my questions about the television programs and ads. Now, I would like you to answer a few survey questions.

COGNITIVE TESTING INSTRUCTIONS

As you review the survey, I’d like you to read the instructions and questions aloud and then “think aloud” as you answer each question. This may feel a little unnatural, but it will help us to understand how you think about and answer each question. Here’s an example:

Question: How many times did you go to a doctor’s office for a scheduled appointment in the past three months?

Answer: Well, I see my heart doctor every month, so that’s three visits. I see my primary care doctor twice a year, but I didn’t go in the last three months. I also had to go to urgent care last week for a sinus infection…but that wasn’t a scheduled appointment. I guess my answer is three visits.

After each survey question, I also may ask you some follow-up questions. We are interested in your initial impressions and honest opinions. There are no right or wrong answers, and it is ok to have strong opinions. Please feel free to use the entire range of response options.

Let’s begin…

[Recall-brand name]

1. One of the ads you saw was for a medication for seasonal allergies. Do you recall the name of that drug? Was it:

• Rheutopia

• Trinase

• Coravaz

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• What made it difficult?

• How did you choose your answer?

[Counter arguing (# of negative thoughts)]

2. The name of the seasonal allergy drug you saw advertised was Trinase. Please think of the Trinase ad you watched. List all thoughts (including unrelated thoughts) that came to your mind as you watched it.

[INCLUDE 5 TEXT BOXES WITH CHARACTER COUNTERS]

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• Was this enough space to fill in all of the thoughts you had?

• What did you think of the character counter?

• The TV programs included several ads. How easy or difficult was it for you to recall this particular ad?

2b. Now that you have listed your thoughts, please rate each one as either positive (+), negative (-) or neutral (0):

[INCLUDE SMALL BOX NEXT TO EACH THOUGHT FOR RATING]

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• Did the instructions make sense to you?

[RANDOMIZE ORDER OF QUESTION BLOCK 3-15 and QUESTION 16]

[RANDOMIZE ORDER OF Q3 and Q4]

[Recall-Risks (unaided)]

3. What are the side effects or risks of TRINASE? (Please list as many side effects or risks as you can remember.) [open ended]

[Include same number of text boxes as number of risks in ad-(12)]

Cognitive Testing Probes:

• How easy or difficult were these questions to answer?

• Was there enough space to fill in all of the thoughts you had?

• What do you think of when you hear the word “side effects”? What about “risks”? Do those mean the same thing to you?

• What do you think of when you hear the words “negative effects”? What about “warnings”? Do those mean the same thing to you as “side effects” or “risks”?

[Recall-Benefits (unaided)]

4. What are the benefits of TRINASE? (Please list as many benefits as you can remember.) [open ended] [Include same number of text boxes as number of benefits in ad-(5)]

Cognitive Testing Probes:

• How easy or difficult were these questions to answer?

• Was there enough space to fill in all of the thoughts you had?

• What do you think of when you hear the word “benefits”?

5. What else do you remember from the Trinase ad? [open-ended]

[INCLUDE TEXT BOX WITH CHARACTER COUNTER]

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• Was this enough space to fill in all of the thoughts you had?

• What kinds of things were you thinking of?

[RANDOMIZE ORDER OF Q6 and Q7]

[Recognition-Risks]

6. Please check which of the following statements were mentioned in the ad as side effects or risks of taking TRINASE. Select “Mentioned in the Ad” if the side effects or risks are mentioned in the ad, even if the statement does not match word for word what you recall from the ad. Even if you think a statement is true, please select it only if it was mentioned in the ad. Check all that apply.

[RANDOMIZE ORDER]	Mentioned In Ad	Not Mentioned In Ad
a. The most common side effects of TRINASE include headache, viral infection, sore throat, coughing and nosebleeds.	X
b.  TRINASE can cause nausea in some people.		X
c.  Some people may experience eye problems from TRINASE such as glaucoma or cataracts.	X
d.  TRINASE may cause slow wound healing.		X
e.  In rare cases, TRINASE can cause severe allergic reactions.	X
f.  TRINASE can cause extreme dizziness in some people.	X
g.  Contact your doctor if you experience sudden changes in hearing.		X

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• How clear or unclear were the instructions, particularly the 2^nd and 3^rd sentences in the instructions?

• Were there any of the statements more difficult than others to choose an answer for?

• Did you guess on any of the statements? If so, which ones? (i.e. were there any that you were unsure of?)

[INTERVIEWER: NOTE ANY SIGNS OF FATIGUE OR FRUSTRATION.]

[Recognition-Benefits]

7. Please check which of the following statements were mentioned in the ad as benefits of taking TRINASE. Select “Mentioned in the Ad” if the benefits are mentioned in the ad, even if the statement does not match word for word what you recall from the ad. Even if you think a statement is true, please select it only if it was mentioned in the ad. Check all that apply.

[RANDOMIZE ORDER]	Mentioned In Ad	Not Mentioned In Ad
a.  TRINASE is taken once a day.	X
b.  TRINASE is non-drowsy.		X
c.  TRINASE is available in pill form.		X
d.  TRINASE can treat runny nose.	X
e.  TRINASE is non-habit forming.	X
f. TRINASE can also be used to treat the common cold.		X
g.  TRINASE can treat nasal congestion.	X

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• How clear or unclear were the instructions?

• Were any of the statements more difficult than others to choose an answer for?

• Did you guess on any of the statements? If so, which ones? (i.e. were there any that you were unsure of?)

[INTERVIEWER: NOTE ANY SIGNS OF FATIGUE OR FRUSTRATION.]

[RANDOMIZE ORDER OF QUESTION BLOCK 8-10 and QUESTION BLOCK 11-13]

[Perceived Risk (Likelihood)]

8. In your opinion, If 100 people take TRINASE, how many will have any side effects or risks?

Please enter a number between 0 and 100 in the box below.

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• How did you choose your answer? Was it based on any specific words from the ad?

• What would make this question easier to answer?

• Which answer would you choose if the answer options were “most,” “some” or “all”?

9. In your opinion, if you were to take TRINASE, how likely would you be to have side effects or risks?

• Not at all likely

• Slightly likely

• Moderately likely

• Very likely

• Extremely likely

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• What did you think of the answer options?

• What type of side effects come to mind? (Note whether respondent mentions life-threatening side effects or milder side effects.)

• What did you consider when thinking about your answer? (e.g. current health condition, past experience with other prescription drugs for seasonal allergies)

[Perceived Risk (magnitude)]

10. If you did have side effects or risks, how serious do you expect they would be?

• Not at all serious

• Somewhat serious

• Moderately serious

• Very serious

• Extremely Serious

• Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• What would make it easier?

• What does the word “serious” mean to you? (i.e. What did you think when you see the word “serious”?)

[Perceived Efficacy (Likelihood)]

RANDOMIZE ORDER

11. In your opinion, if 100 people take TRINASE, for how many will the drug work?

Please enter a number between 0 and 100 in the box below.

Cognitive Testing Probes:

• How easy or difficult was this question to answer?

• What would make it easier?

• What does “the drug working” mean to you? Do any other words come to mind when you see the words “the drug working”?

• Which answer would you choose if the answer options were “most,” “some” or “all”?

12. In your opinion, if you were to take TRINASE, how likely is it that the drug would work for you?

• Not at all likely

• Slightly likely

• Moderately likely

• Very likely

• Extremely likely

Cognitive Testing Probes:

• Which question is easier to answer, number 11 or 12?

• What is the main difference between these two questions?

• What does “the drug working” mean to you? Do any other words come to mind when you see the words “the drug working”?

[Perceived Efficacy (Magnitude)]

13. In your opinion, how effective would TRINASE be in helping your seasonal allergies?

• Not at all effective

• Slightly effective

• Moderately effective

• Very effective

• Extremely effective

Cognitive Testing Probes:

• How easy or difficult is this question to answer?

• How did you select your answer?

• What is meant by “how effective” would TRINASE be in helping your seasonal allergies?

• What other words come to mind when you see the words “how effective”?

[Risk/benefit balance]

14. Think about the risks and benefits of TRINASE. How would you rate the drug overall?

• Risks completely outweigh benefits

• Risks mostly outweigh benefits

• Risks slightly outweigh benefits

• Risks and benefits are equal

• Benefits slightly outweigh risks

• Benefits mostly outweigh risks

• Benefits completely outweigh risks

Cognitive Testing Probes:

• How easy or difficult is this question to answer?

• What do you think of the number of response options? (too few, too many, just right?)

Closing

This concludes the survey. Our goal was to gather patient reactions to important information about prescription drugs. To get your true reaction to this information, we used a fake brand of drug in this project.

TRINASE is not a real drug and it is not available for use or sale. Please contact your healthcare provider for any questions about seasonal allergies.

Thank you very much for your time.