Cogniitive Interview Guide

OMB Control No. 0910-0695

Expiration date: 3/31/2024

DSC Landing Page and Format

Cognitive Interview Guide

Color Key:

Purple – Section headings and titles

Black –Text for interviewer to read

Red – Interviewer instructions (not to be read aloud)

Green – Research questions (for interviewer’s information, not to be read aloud)

Introduction

Hello, my name is [INSERT INTERVIEWER NAME]. I work for Westat, a research company in Rockville, Maryland. Thank you for taking the time to be a part of this study.

Westat is conducting this study, which is sponsored by the U.S. Food and Drug Administration (or FDA), to see what people think about health information related to medicines.

You should have gotten an email for this study. Do you have the email we sent you available right now?

IF YES, CONTINUE

IF NO BUT EMAIL IS NEARBY, ALLOW PARTICIPANT TIME TO GET IT

IF NO AND NOT NEARBY, ASK IF IT’S OKAY TO RESEND THE EMAIL TO PARTICIPANT. IF PARTICIPANT IS NOT ABLE TO RECEIVE OR SCREENSHARE AND VIEW THE DOCUMENT, RESCHEDULE INTERVIEW.

Please wait to open the email attachment until I tell you it is time to start. After you look at the material inside the attachment, I will ask you some questions about your thoughts and feelings about the information. Any ideas you have are okay.

Informed Consent

Before we get started, there are a few things I need to tell you. Your participation in this interview is voluntary, and you don’t have to answer any questions you prefer not to. It is also okay if you want to stop after we start. Just let me know.

All of your answers and everything you say will be kept secure to the extent permitted by law. This means we will not share your name or contact information. The interview will take about 45 minutes and you will receive $50 as a token of appreciation. We would like to record this conversation to make sure we hear everything you say correctly. Only the research team working on this project will be able to listen to the recording and see our notes. Your name will not be linked to any of your responses, though we may include quotes that you provide in our reports. The recordings and our notes will be destroyed after we finish the project.

IF FDA STAFF ARE ON THE PHONE: I also want you to know that a couple members of the FDA research team are listening to this interview so they can hear directly from you.

Any questions? Is it okay if I record the interview? IF NO, TERMINATE.

TURN ON RECORDER. The date and time is ____________. Now that I am recording, I want to ask again, is it okay if I record this interview?

Research Questions

Landing Page

1. How well does the landing page meet participants’ expectations?

2. Is the landing page text clear and understandable? How well does the text describe how new safety issues for medicines surface and what DSCs are?

3. What is the main message that participants get from the landing page? How well does this align with FDA’s goals of communicating that a) new safety issues may be uncovered with approved medicines, b) DSCs provide important information about these safety issues, and c) the public can subscribe to receive email alerts about DSCs?

4. How well do participants think the landing page is organized?

5. How would participants attempt to find a DSC for a specific medicine? How well does this align with their expectations for how to find a DSC for a specific medicine?

6. How intuitive do participants find the “See more” button under Current Drug Safety Communications?

7. Are the instructions for subscribing to DSCs clear and easy to find?

8. What improvements do participants suggest for the landing page?

DSC Q&A Format

9. Are the questions listed under the sample DSC the ones that participants want answered?

10. How intuitive do participants find the drop-down format to see answers to each question? What would make it more intuitive?

11. What improvements do participants suggest for the DSC formatting?

Background Questions

To start, I have a few general questions.

You told us when you volunteered for this interview that you’ve used a prescription medicine in the past 30 days. Is this correct? IF NO, How long has it been since you last used a prescription medicine? IF MORE THAN 5 WEEKS (35 DAYS), TERMINATE.

• How many different prescription medicines have you used in the past 30 days?

Have you ever heard about a new safety concern or risk for a medicine that you or a loved one take? This could be for a medicine prescribed by a healthcare provider or bought over-the-counter.

• IF YES, Where did you first learn about this new safety concern?

• What did you do as a result of hearing about this new safety concern? IF NEEDED:

  • Did you look for additional information about this new safety concern? IF YES, Where did you look? What types of information were you looking for, and did you find it?

Why do you think new safety concerns or risks might come up for medicines that are already available over-the-counter or by prescription?

IF TIME ALLOWS, If there was a website about new safety concerns or risks that have come up for medicines, what is the most important information you would want to find there? RECORD RESPONSES SO YOU CAN REFER BACK TO THEM LATER. IF NEEDED, For example, data related to the new safety concern, or something else.

The FDA does release information about new safety issues with medicines and now I’d like to ask you some questions about that. Please open the attachment in the email I just sent you. Do you see a document titled “Drug Safety Communications?” This is a mock-up of a draft FDA webpage about new safety issues with medicines. You won’t be able to click on everything in this mock-up the way you would on the real webpage, but it gives you a sense for how the page might look. Please take a few minutes to read only the first page without clicking on any of the links that take you to different pages, which we’ll look at later. When you finish, I will ask you some questions about the information on the first page.

CHECK IN AFTER 2 MINUTES. ALLOW 1 MORE MINUTE IF NEEDED. IF PARTICIPANTS NAVIGATE AWAY FROM PAGE 1, DIRECT THEM BACK TO THE FIRST PAGE FOR NOW AND INFORM THEM THAT THEY WILL LOOK AT OTHER PAGES LATER.

CONCURRENT OBSERVATIONS/INSTRUCTIONS.

NOTE ANY OF THE FOLLOWING BEHAVIORS TO RECORD IN NOTES OR PROBE ON RETROSPECTIVELY:

• Any verbal reaction to statements in the information.

• Any verbal expressions of confusion, surprise, discomfort, offense. Note which statements evoked any of these reactions.

RETROSPECTIVE GENERAL PROBES

TO BE ADMINISTERED AFTER RESPONDENT COMPLETES HIS/HER REVIEW.

LANDING PAGE

Messaging and Layout

In your own words, what is the overall, main message that this first page is trying to tell you?

• Why do you think FDA is telling you this information?

What are your overall thoughts about what you see here on the first page?

What do you like about it and why?

What don’t you like and why?

Is there too little or too much information and why?

How easy or difficult is it to understand the information on this first page?

Are there any words or phrases that confuse you or that you aren’t sure of?

ONLY IF EARLIER QUESTION ASKED: Earlier you said [INSERT EARLIER RESPONSES] would be the most important information you would want to see on a website about new safety concerns or risks that have come up for medicines. How does what you see on this page compare with what you said previously? IF NEEDED: Is the information you want to see here on this page? If not, what’s missing? Is the information presented or explained in a way that makes sense to you? Why or why not?

What new information have you learned from this page about how medicines are researched, reviewed, and approved? About how new safety issues may be uncovered?

Is there any information that could be stated more clearly? IF YES, What could be stated more clearly?

What information is missing? Where on the site should this information go?

What caught your attention? Remembering this is just a mock-up of the website and most of the links don’t work, what would you click on first if you could? Why?

E-mail Alerts

The text indicates that you can sign up for email alerts. How would you sign up for those alerts? RECORD OPTION(S) MENTIONED.

• IF ONLY BY CLICKING LINK IN TEXT, Please click that link and you should see a grey box at the bottom of the page. Did you notice any other links or buttons above that would bring you down to this grey box? IF NO, There is also a “Subscribe to Email Updates” button at the top of the page that takes you to the grey box. What would help that button catch your attention?

• IF ONLY BY SUBSCRIBE BUTTON AT TOP, Please click that button and you should see a grey box at the bottom of the page. Did you notice any other links or buttons above that would bring you down to this grey box? IF NO, The words “email alerts” in the 4^th paragraph are in blue and clicking them takes you to the grey box. What would make it clearer that you can click on these words?

• IF ONLY BY SUBSCRIBE BOX AT BOTTOM, Did you notice any links or buttons above that would bring you down to this grey box? IF NO, There is a “Subscribe to Email Updates” button at the top of the page and the words “email alerts” in the 4^th paragraph are blue, and clicking either of them takes you to the grey box. What would help these catch your attention or make it clearer that you can click on them?

• IF UNSURE, There is a “Subscribe to Email Updates” button at the top of the page and the words “email alerts” in the 4th paragraph are blue. Please click on either of them and you should see a grey box at the bottom of the page. What would help these catch your attention or make it clearer how to sign up for email alerts? IF NEEDED, What additional instructions would you like to see?

• How clear are the instructions in the grey box at the very bottom of the page? Would you say anything differently?

• What do you think will be included in the email alerts? IF NEEDED, Do you think the emails will include Drug Safety Communications or something else?

• Would you sign up for email alerts? Why or why not? How else would you prefer to get alerts about any safety issues about medicines you are taking?

Current and Previous Drug Safety Communications

What are your thoughts on the way this page is organized?

• What do you think about the different sections? IF NEEDED: Do the section titles make sense? IF NO, Why not? Is it clear what’s being included under each section? IF NO, What’s not clear and why?

• Please don’t click anything yet, but do you see the “See More” button? LISTEN FOR PARTICIPANTS SAYING THEY HAVE A HARD TIME FINDING IT. What do you expect you’ll see if you click on the “See More” button under the last “Current Drug Safety Communication”?

  • Go ahead and click “See More.” What page did it take you to? PARTICIPANT SHOULD BE ON PAGE 4 TITLED “2020 Drug Safety Communications.” How does this page compare with what you expected to see? IF NEEDED, Is the information you expected to see here? If not, what’s missing? Is the information presented or explained in a way that makes sense to you? Why or why not?

• Please scroll back up to page 1. What do you expect to see if you click on any of the years listed under “Previous Drug Safety Communications?”

  • Go ahead and click “2020 Drug Safety Communications.” PARTICIPANT SHOULD BE ON PAGE 4 TITLED “2020 Drug Safety Communications.” How does this page compare with what you expected to see? IF NEEDED, Is this the information you expected to see? If not, what’s missing? Is the information presented or explained in a way that makes sense to you? Why or why not?

  • Does it make sense that both the “See More” button and the link to “2020 Drug Safety Communications” navigate you to the same information? Why or why not?

Please scroll back up to page 1. If you wanted to find a Drug Safety Communication about a specific medicine, how would you look for it? IF NEEDED, What if it wasn’t listed under Current Drug Safety Communications and you were unsure what year it was communicated?

• IF UNSURE, The blue banner at the top of the page includes a “Search” button. Did you notice that button? IF NO, What would help this button catch your attention?

• ASK ALL, What would you expect to see if you clicked the Search button and typed the name of the medicine? IF NEEDED, Would you expect the Search result to provide only Drug Safety Communications, or would you expect it to provide other types of information about the medicine?

  • How would you feel if the Search result provided a list of links to various types of information about the medicine and the Drug Safety Communication was somewhere among them? What would you do if that happened? IF NEEDED, Would you look through the list of links until you found the Drug Safety Communication?

  • What would you think if you scrolled through the list and there wasn’t a Drug Safety Communication about the medicine? How would that make you feel? What would you do if that happened?

DSC Q&A FORMAT

Now I’d like to ask you a few questions about one of the Drug Safety Communications listed on this page. Have you ever used the medicine Singulair?

• IF YES, Are you currently using Singulair? IF NOT CURRENLY USING, When would you say was the last time you used it?

• IF NO, It’s alright if you haven’t used Singulair. I only asked because we’re going to talk about the Drug Safety Communication for that medicine today as an example.

ASK PRIOR TO VIEWING, What would you expect to see if you clicked the communication dated 3/4/2020 under Current Drug Safety Communications? RECORD RESPONSES SO YOU CAN REFER TO THEM LATER

Please go ahead and click the link to that communication. PARTICPANT SHOULD BE ON PAGE 2. You should now see a page that reads, “FDA requires Boxed Warning about serious mental health side effects” at the top. Suppose you were using Singulair and heard something on the news about a new safety issue with it. You wanted to learn more so you Googled “Singulair,” and it brought you to this page. Please take a minute to look at this page without clicking on anything. When you finish, I will ask you some questions about the information.

CHECK IN AFTER 1 MINUTE. ALLOW 1 MORE MINUTE IF NEEDED.

How does what you see on this page compare with what you expected to see? IF NEEDED, Is this the information you expected to see? If not, what’s missing? Is the information presented or explained in a way that makes sense to you? Why or why not?

What are your thoughts on the way this page is set up?

• Are these the questions you would want to know about this medicine if you were taking it and heard about a new safety issue? IF NO, What would you ask and why?

  • Which question would you click on first? Why?

  • Are there any questions that you think are NOT needed or can be removed? IF YES, Which questions and why do you think they are unnecessary?

What would you do if you wanted to see answers to these questions? IF NEEDED, What would you click?

IF UNSURE: What would make it easier for you to see how to get the answers to these questions?

Now please click on the first question, “What safety concern is FDA announcing?” or the drop-down arrow on the right. PARTICIPANT SHOULD BE ON PAGE 3.

This is just a mock-up so not all of the questions are linked to their answers, but on the actual website you would be able to view and hide answers to each of these questions by clicking on each question, the drop-down arrows, or the gray bars containing the questions. What do you think about that? IF NEEDED: Is that clear? Is there a way to make it clearer?

WRAP-UP

Please scroll back up to page 1. Now that you’ve seen the example Drug Safety Communication for Singulair, do you have any other suggestions for improving this website?

• How well do you think this page explains how new safety issues come up for medicines that are already available over-the-counter or by prescription? What needs to be explained better?

• How well do you think the information on this page describes what Drug Safety Communications are? What needs to be explained better?

• Is there information in the description that is NOT needed or can be removed? IF YES, What information and why do you think it is unnecessary?

• How user friendly would you say this webpage is? Would you change or move anything to make it more user friendly?

PROBE ON ANY OUTSTANDING ISSUES FROM OBSERVATION (INFORMATION THAT RESPONDENT SEEMED CONFUSED ABOUT).

Closing

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything else you would like the FDA to know about this webpage, the Drug Safety Communication format, or communicating about new drug safety issues?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.