Focus Groups of US Adults KAP of Foodborne Disease Outbreaks and Illness Prevention

Form Approved

OMB Control No.: 0920-1154

Expiration date: 01/31/2020

Attachment F: Focus Group Approach – Message Testing – Focus Group Moderator Guide

[100 minutes]

Moderator instructions:

This guide will be used to steer discussion to the topic areas to be covered, and the specific questions of interest within each topic area. For the purposes of qualitative research, it is not necessarily intended that these questions be asked exactly as they are worded here. Focus group discussions are more like a conversation than a set of structured questions. The discussion should be informal and participants should be encouraged to speak openly and freely. The Moderator will also ensure that all participants in the group have an opportunity to express their opinions. Because each group of participants may be different, a responsive and flexible approach should be used for the focus group discussion.

1. Welcome/Introduction

Good [morning/afternoon]. My name is _______________ and I work at EurekaFacts, a research company in Rockville, Md. We are conducting a project on behalf of the Centers for Disease Control and Prevention, known as the “CDC”.

The purpose of this project is to understand people’s thoughts, reactions, and impressions of messaging used by CDC to inform the public on foodborne disease outbreaks and illness prevention. The information you share here with us today will help us understand the public’s reactions and knowledge on foodborne illnesses, in the hopes of improving messaging and overall awareness. This session will last about 100 minutes.

Public reporting burden of this collection of information is estimated to average 100 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB Control Number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA 0920-1154

Your views and experiences are very helpful, it is important that you provide us with your honest opinions and that you understand that we will not make any judgements of you for your thoughts and opinions. Also, as we are talking about your personal thoughts and opinions, it is not necessary for everyone to agree with each other.

DISCLOSURES

1. Mention mikes, audio, and videotaping.

2. One-way mirror; associates viewing, note takers may come in.

3. The session is being audio-taped so I can write an accurate report, not of “who said what,” but "what got said.” Your name will not be used in the report. All the information discussed in this room this today will remain confidential.

GUIDELINES

In order to get the best information from this session , there are some guidelines to follow:

1. Please talk in a voice at least as loud as mine.

2. Please allow one person to speak at a time and allow everyone a turn to speak.

3. There are no wrong answers – just different points of view.

   1. So, say what you believe – be candid.

   2. There is no need to change anyone’s mind and we are not looking to build a consensus.

   3. Feel free to comment on each other’s remarks, whether you agree, would like to add something, or disagree.

4. I may need to move on to another area of discussion sooner than you are ready, and I may have to cut a conversation short in the interest of time. If I interrupt you, it is not personal; I just want to cover everything on the agenda.

5. You will receive your incentive for your time and opinions at the conclusion of the focus group session.

6. Please silence your cell phones or pagers.

PERMISSIONS

You may excuse yourself at any time to go to the restroom or to get more food or beverages. Do you have any questions before we begin?

2. Respondent Introductions/ Warm Up

Before we get started, I’d like to learn a little more about you. Please share with me:

1. your first name;

2. a little about how you spend your day; and

3. your favorite past-time.

Moderator begins by introducing himself/herself.

Topic 1 – Defining Foodborne Disease Outbreak

1. In the first half of our session we are going to talk about some general issues and in the second half I will ask for your feedback on specific messages. Let’s start with a very general topic, and we will get more specific as we go. What initially comes to mind when I say the words “foodborne disease outbreak”?

Probe:

1. What, if any, other words or phrases would you use to define or describe foodborne illnesses?

Topic 2 – Foodborne Outbreak Messaging

2. Thinking back over the past 3-4 years, approximately how often and where have you heard or seen a message to the general public about a foodborne disease outbreak? (If necessary, that could be per year, month, etc.)

Probe:

1. How did you learn or hear about the outbreaks you’re familiar with? (If necessary, provide some examples: a government agency, private individual or organization, news outlet, social media, or the internet)

3. Please share some of your thoughts about the types of foodborne disease messages you have heard or read.

Probe:

1. What are your impressions of the messages you’ve seen about foodborne disease?

2. What was your reaction to these messages? (If necessary, how concerned are you when you hear a message about foodborne disease?)

3. Who or what was the source of the message (e.g. CDC, local news, friends/family)? How does the source of the message impact how serious the message seems?

4. In general, how clear and understandable are the messages? (If necessary, do they tend to be too technical, full of jargon, etc.?) Do images or video help you understand the messages? Can you give an example of a message that was clear and one that was not clear? What was the difference between them?

5. Did you understand what food items were affected and how to identify them?

6. When you read or hear a message about a foodborne illness outbreak, in general how do determine if it applies to foods you prepare or consume? What about your likelihood of being affected by preparing or consuming the food? (If necessary, how do the specific words, or the tone, of a message make it seem more, or less, relevant to you?) {Ask for examples}

7. When you read or hear a message about a foodborne illness outbreak, in general how do you evaluate how serious a problem it could be for you? (If necessary, how do the specific words, or the tone, of a message make it seem more, or less, likely to make you very sick?) {Ask for examples}

Topic 3 – Message-Based Consumer Behavior

4. What action, if any, do you take upon hearing or reading a message about foodborne disease outbreak?

Probe – Action:

1. Share it with family and friends, and if so, how?

2. Switch to a similar type of food, brand, etc.?

3. Throw away the food mentioned?

4. Consume the food mentioned but perhaps in a different way than usual such as with extra cooking or other precautions?

5. Consume the food with no changes in how you prepare and eat it?

6. Try to return it to place of purchase?

7. Avoid that type of food or that food manufacturer altogether, “just to be safe”?

8. Find more information? If so, where do you look?

9. For those of you who don’t take any action: what are some reasons you might not take action?

Probe – What action depends on:

1. What determines whether you take an action, and what action you take?

2. What might determine your decision to share the information with others? (If necessary, specific language, tone, source, relevance, relationship.)

3. Does your action depend on how well you understand the situation and your choices?

5. Thinking about everything we have talked about, how do you react when you see multiple messages on foodborne illnesses in a short period of time? Do you have any experience related to this that you can describe?

Probe:

1. Do you ever feel that you hear or see too many messages about foodborne disease outbreaks in general?

2. What are your reactions to seeing information about foodborne illnesses frequently? (if needed, probe further on whether they become more or less likely to react to the messaging)

3. What leads to you feeling “information fatigue” on a topic?

(Define information fatigue as a term used to describe the difficulty of understanding an issue and effectively making decisions about an issue if there is an overwhelming amount of information on the topic and it is being discussed repeatedly.)

Topic 4 – High Risk Foods

6. In general, what are high-risk foods in terms of possible foodborne disease and illness? In other words, what foods are more likely to result in foodborne disease?

Probe:

1. What do you know about the risk factors associated with soft cheeses? What do you consider a ‘soft cheese’? Could you provide some examples?

(If necessary, after some discussion, define soft cheese as cheeses that are high in moisture content, such as fresh, surface-ripened or aged cheeses. Soft cheeses have a moist, creamy, or crumbly texture and are often spread on bread or crackers. Examples of soft cheeses include Brie, bleu cheese, gorgonzola and queso fresco.)

2. What do you know about the risk factors associated with leafy greens? What would you consider as leafy greens? Could you provide some examples? (If necessary, after some discussion, define leafy greens as plant leaves eaten as a vegetable, which include lettuce, spinach, arugula, red leaf lettuce, kale, and cabbage. They can be sold loose as heads or individual leaves, in bags of just one type of leafy green or a mixture, or in plastic “clamshell” containers of one type of leafy green or a mixture.)

Topic 5 – Message Testing

As I mentioned earlier, we are going to ask for your feedback on specific messages about foodborne illness outbreaks. Please remember that I did not design any of the materials so your comments do not affect me personally. I am going to show you a series of messages, one at a time, and first ask you to rate them in a quick little survey. Then we’ll discuss your ratings as a group and I’ll ask you a few more questions.

The four ratings you will be asked for are about message tone, how well you understand the message, how serious the message seems, and your likelihood of taking some form of action after seeing the message.

Rating Scales (not to be read aloud, but to be on each survey form)

Message Tone

1	2	3	4	5	6	7
Too mild and gentle			Just the right tone			Too aggressive

{probes: What makes it aggressive or gentle? How would you improve the tone?}

Understand the Message

1	2	3	4	5	6	7
Not at all understandable			Somewhat understandable			Completely understandable

{probes: What words or phrases helped you understand the message? What made the message hard to understand?}

Likelihood of Message Impact

1	2	3	4	5	6	7
Definitely will not affect me			May affect me			Definitely will affect me

{probes: How and why will it affect you?}

Likelihood of taking some form of action

1	2	3	4	5	6	7
Definitely will not take action			May take action			Definitely will take action

{probes: What type(s) of action would you take? What would determine your actions?}

Messages

Consumer Guidance

Message 1: Consumers should be aware that FOOD XXX may be contaminated with germs and consumers should always follow steps to prevent (https://www.cdc.gov/salmonella/reading-07-18/index.html#advice)food poisoning from these products, CDC will update the advice to consumers and retailers if more information comes available, such as a supplier or type of FOOD XXX linked to illness.

Message 2: Do not eat, sell, or serve recalled FOOD XXX from COMPANY XXX; return any recalled FOOD XXX to the store for a refund or throw it away. {If necessary, probe on what it means for a food to be recalled.}

Message 3: If you do not know if the FOOD XXX you purchased was recalled, ask the place where you purchased it or throw it away; CDC is not advising that consumers avoid eating properly cooked FOOD XXX.

Investigation Details

Message 4: XXX people infected with the outbreak strain of GERM have been reported from XX states.

Message 5: This outbreak appears to be over. However, recalled products have a long shelf life and may still be in people’s homes. Consumers unaware of the recall could continue to eat these products and potentially get sick.

Illness/Symptoms

Message 6: Pregnant women and their newborns, children younger than 5 years, adults older than 65 years, and people with weakened immune systems are more likely to get sick with listeriosis or other severe illness.

Specific Foods

Message 7: Do not eat soft cheese. Soft cheeses include brie, camembert, blue-veined cheese, Mexican-style cheeses such as queso fresco, and soft artisanal or farmstead cheeses. Hard cheeses, cottage cheese, and cream cheese can be eaten without concern about the risk of listeriosis.

Message 8: Wash leafy greens under running water before eating, cutting, or cooking; leafy greens labeled “prewashed” do not need to be washed again at home; the safest produce is cooked, the next safest is washed.

Social Media

Message 9: E. coli outbreak: If you have store-bought chopped romaine lettuce at home, including salads and salad mixes with chopped romaine, don’t eat it and throw it away. If you don’t know if the lettuce is romaine, don’t eat it. (https://www.facebook.com/CDC/photos/a.184668026025/10155762222781026/?type=1&theater)

Message 10: Salmonella outbreak: Gravel Ridge Farms cage-free eggs recalled after 14 people sick in 2 states. Do not eat, serve or sell recalled eggs. Learn more: LINK. (https://twitter.com/CDCgov/status/1039251062732021760)

Close Out

I am going to quickly ask my associates if there is anything else we need to cover. I will be back in one moment, but please feel free to continue talking among yourselves.

That is all the material we needed to cover, but is there anything else that any of you would like to add, or are there any questions that I should have asked but did not?

Thank you for your participation and insights in this focus group. Your participation will help improve future communication and messaging about foodborne illness.

CDC Foodborne Outbreak Message Testing

Hybrid Approach – Cognitive Interview Protocol [40 minutes]

Interviewer Instructions:

The interviewer should not read the script word for word but should be familiar enough with its contents to conduct the interview in a natural and conversational manner, paraphrasing or giving further explanation as appropriate.

The protocol provides a list of suggested probes to ask and guidelines for which of the potential areas need to be tested. However, the interview will require a flexible probing approach, selecting relevant probes or modifications of the listed probe formulation depending on the participant think-aloud output. Text written in italics is to be read aloud by the interviewer.

Welcome/Introduction

Good [morning/afternoon]. My name is _______________ and I work at EurekaFacts, a 15-year-old research company here in Rockville. We are conducting a project on behalf of the Centers for Disease Control and Prevention, known as the “CDC”.

The purpose of this project is to get feedback on messages and portions of messages about foodborne disease outbreaks and illness prevention. The information you share here with us today will help the CDC understand the public’s reactions to messages in the hopes of improving messaging and overall awareness. This interview will last about 40 minutes, followed by a 15-minute break, and then a 45-minute focus group with the other people also now doing an individual interview.

Your views and experiences are very helpful, it is important that you provide us with your honest opinions and that you understand that we will not make any judgements of you for your thoughts and opinions.

DISCLOSURES

1. Mention mikes, audio, and videotaping.

2. One-way mirror; associates viewing, note takers may come in.

3. The session is being audio-taped so I can write an accurate report, not of “who said what,” but "what got said.” Your name will not be used in the report. All the information discussed in this room this today will remain confidential.

GUIDELINES

In order to make this session work as a research session, there are some guidelines to follow:

4. Please talk in a voice at least as loud as mine.

5. There are no wrong answers, so, say what you believe – be candid.

6. I may need to move on to another area of discussion sooner than you are ready, and I may have to cut a conversation short in the interest of time. If I interrupt you, it is not personal; I just want to cover everything on the agenda.

7. You will receive your incentive for your time and opinions at the conclusion of the focus group session.

8. Please silence your cell phone or pager.

PERMISSIONS

You may excuse yourself at any time to go to the restroom or to get more food or beverages. Do you have any questions before we begin?

Topic 1 – Defining Foodborne Disease Outbreak

1. Let’s start with a very general topic, and we will get more specific as we go. What initially comes to mind when I say the words “foodborne disease outbreak”?

Probe:

2. What, if any, other words or phrases would you use to define or describe foodborne illnesses?

Topic 2 – Message Testing

As I mentioned earlier, we are going to ask for your feedback on specific messages about foodborne illness outbreaks. I am going to show you a series of messages, one at a time, and ask you to do two things. First, as you read each item in your head, please also do what we call ‘think aloud’ about the message you are reading or specific pieces of it. Examples, but not the only things you might want to say aloud, include: that’s interesting; I didn’t know that; what does that mean; that’s harsh; that’s scary; that doesn’t make sense; that would get me to …, and so on. The second thing you’ll do for each message is rate them in a quick little survey, and while doing that please also ‘think aloud’ about how you are choosing your rating. And, I may ask a few quick follow-up questions along the way. Please remember that I did not design any of the materials we will look at so your comments do not affect me personally.

The four topics you’ll rate the messages on are: message tone, how well you understand the message, what is the likelihood that the message will impact you, and your likelihood of taking some form of action after seeing the message.

{Interviewer probe: think aloud for personal relevance, experience, perceptions, beliefs, fears, attitudes, preferences, as necessary.}

Rating Scales (not to be read aloud, but to be on each survey form)

Message Tone

1	2	3	4	5	6	7
Too mild and gentle			Just the right tone			Too aggressive

{probes: What makes it aggressive or gentle? How would you improve the tone?}

Understand the Message

1	2	3	4	5	6	7
Not at all understandable			Somewhat understandable			Completely understandable

{probes: What words or phrases helped you understand the message? What made the message hard to understand?}

Likelihood of Message Impact

1	2	3	4	5	6	7
Definitely will not affect me			May affect me			Definitely will affect me

{probes: How and why will it affect you?}

Likelihood of taking some form of action

1	2	3	4	5	6	7
Definitely will not take action			May take action			Definitely will take action

{probes: What type(s) of action would you take? What would determine your actions?}

Messages

Consumer Guidance

Message 1: Consumers should be aware that FOOD XXX may be contaminated with germs and consumers should always follow steps to prevent (https://www.cdc.gov/salmonella/reading-07-18/index.html#advice)food poisoning from these products, CDC will update the advice to consumers and retailers if more information comes available, such as a supplier or type of FOOD XXX linked to illness.

Message 2: Do not eat, sell, or serve recalled FOOD XXX from COMPANY XXX; return any recalled FOOD XXX to the store for a refund or throw it away. {If necessary, probe on what it means for a food to be recalled.}

Message 3: If you do not know if the FOOD XXX you purchased was recalled, ask the place where you purchased it or throw it away; CDC is not advising that consumers avoid eating properly cooked FOOD XXX.

Investigation Details

Message 4: XXX people infected with the outbreak strain of GERM have been reported from XX states.

Message 5: This outbreak appears to be over. However, recalled products have a long shelf life and may still be in people’s homes. Consumers unaware of the recall could continue to eat these products and potentially get sick.

Illness/Symptoms

Message 6: Pregnant women and their newborns, children younger than 5 years, adults older than 65 years, and people with weakened immune systems are more likely to get sick with listeriosis or other severe illness.

Specific Foods

Message 7: Do not eat soft cheese. Soft cheeses include brie, camembert, blue-veined cheese, Mexican-style cheeses such as queso fresco, and soft artisanal or farmstead cheeses. Hard cheeses, cottage cheese, and cream cheese can be eaten without concern about the risk of listeriosis.

Message 8: Wash leafy greens under running water before eating, cutting, or cooking; leafy greens labeled “prewashed” do not need to be washed again at home; the safest produce is cooked, the next safest is washed.

Social Media

Message 9: E. coli outbreak: If you have store-bought chopped romaine lettuce at home, including salads and salad mixes with chopped romaine, don’t eat it and throw it away. If you don’t know if the lettuce is romaine, don’t eat it. (https://www.facebook.com/CDC/photos/a.184668026025/10155762222781026/?type=1&theater)

Message 10: Salmonella outbreak: Gravel Ridge Farms cage-free eggs recalled after 14 people sick in 2 states. Do not eat, serve or sell recalled eggs. Learn more: LINK. (https://twitter.com/CDCgov/status/1039251062732021760)

Close Out

That is all the material we needed to cover in this portion of today’s event, but is there anything else that you would like to add, or are there any questions that I should have asked but did not?

It was a great interview and I enjoyed working with you in this format. It is time for a 15-minute break and then the 45-minute focus group about similar topics. Please bring your rating sheets to the focus group portion.

Focus Group Approach – Hybrid Focus Group Moderator Guide [45 minutes]

Moderator instructions:

This guide will be used to steer discussion to the topic areas to be covered, and the specific questions of interest within each topic area. For the purposes of qualitative research, it is not necessarily intended that these questions be asked exactly as they are worded here. Focus group discussions are more like a conversation than a set of structured questions. The discussion should be informal and participants should be encouraged to speak openly and freely. The Moderator will also ensure that all participants in the group have an opportunity to express their opinions. Because each group of participants may be different, a responsive and flexible approach should be used for the focus group discussion.

3. Welcome/Introduction

Thanks everyone for participating in the individual interviews. My name is _______________ and I work at EurekaFacts also. This next section is a focus group to generate more information for CDC about messaging related to foodborne disease outbreaks and illness prevention. This session will last about 45 minutes.

Your views and experiences are very helpful, it is important that you provide us with your honest opinions and that you understand that we will not make any judgements of you for your thoughts and opinions. Also, as we are talking about your personal opinions and experiences, it is not necessary for everyone to agree with each other, and, again, I did not design any of the materials that we will look at.

DISCLOSURES

1. Mention mikes, audio, and videotaping.

2. One-way mirror; associates viewing, note takers may come in.

3. The session is being audio-taped so I can write an accurate report, not of “who said what,” but "what got said.” Your name will not be used in the report. All the information discussed in this room this today will remain confidential.

GUIDELINES

In order to make this session work as a research session, there are some guidelines to follow:

1. Please talk in a voice at least as loud as mine.

2. Please allow one person to speak at a time and allow everyone a turn to speak.

3. There are no wrong answers – just different points of view.

   1. So, say what you believe – be candid.

   2. There is no need to change anyone’s mind and we are not looking to build a consensus.

   3. Feel free to comment on each other’s remarks, whether you agree, would like to add something, or disagree.

4. I may need to move on to another area of discussion sooner than you are ready, and I may have to cut a conversation short in the interest of time. If I interrupt you, it is not personal; I just want to cover everything on the agenda.

5. You will receive your incentive for your time and opinions at the conclusion of the focus group session.

6. Please silence your cell phones or pagers.

PERMISSIONS

You may excuse yourself at any time to go to the restroom or to get more food or beverages. Do you have any questions before we begin?

1. Respondent Introductions/ Warm Up

Before we get started, I’d like to learn a little more about you. Please share with me:

1. your first name;

2. a little about how you spend your day; and

3. your favorite past-time.

Moderator begins by introducing himself/herself.

1. TOPICS AND QUESTIONS FOR DISCUSSION

Topic 1 – Review of Message Testing Feedback from Cognitive Interviews (CIs)

Now I’d like to share some of your thoughts and ratings about the foodborne illness messages you reviewed during the individual interviews. This will help us put more context around your feedback, but again we are not looking to build a consensus or judge anyone.

{Moderator: initiate conversation about each of the 5-6 messages covered in the CIs.}

{Moderator: probe for clarification, elaboration, and synergy as necessary; also try to ensure at least some coverage of each of the four ratings metrics.}

Here is the first message you reviewed in the interviews. Please share any of the four ratings you did for this message, along with how you decided the rating.

Consumer Guidance

Message 1: Consumers should be aware that FOOD XXX may be contaminated with germs and consumers should always follow steps to prevent (https://www.cdc.gov/salmonella/reading-07-18/index.html#advice)food poisoning from these products, CDC will update the advice to consumers and retailers if more information comes available, such as a supplier or type of FOOD XXX linked to illness.

Message 2: Do not eat, sell, or serve recalled FOOD XXX from COMPANY XXX; return any recalled FOOD XXX to the store for a refund or throw it away. {If necessary, probe on what it means for a food to be recalled.}

Message 3: If you do not know if the FOOD XXX you purchased was recalled, ask the place where you purchased it or throw it away; CDC is not advising that consumers avoid eating properly cooked FOOD XXX.

Investigation Details

Message 4: XXX people infected with the outbreak strain of GERM have been reported from XX states.

Message 5: This outbreak appears to be over. However, recalled products have a long shelf life and may still be in people’s homes. Consumers unaware of the recall could continue to eat these products and potentially get sick.

Illness/Symptoms

Message 6: Pregnant women and their newborns, children younger than 5 years, adults older than 65 years, and people with weakened immune systems are more likely to get sick with listeriosis or other severe illness.

Specific Foods

Message 7: Do not eat soft cheese. Soft cheeses include brie, camembert, blue-veined cheese, Mexican-style cheeses such as queso fresco, and soft artisanal or farmstead cheeses. Hard cheeses, cottage cheese, and cream cheese can be eaten without concern about the risk of listeriosis.

Message 8: Wash leafy greens under running water before eating, cutting, or cooking; leafy greens labeled “prewashed” do not need to be washed again at home; the safest produce is cooked, the next safest is washed.

Social Media

Message 9: E. coli outbreak: If you have store-bought chopped romaine lettuce at home, including salads and salad mixes with chopped romaine, don’t eat it and throw it away. If you don’t know if the lettuce is romaine, don’t eat it. (https://www.facebook.com/CDC/photos/a.184668026025/10155762222781026/?type=1&theater)

Message 10: Salmonella outbreak: Gravel Ridge Farms cage-free eggs recalled after 14 people sick in 2 states. Do not eat, serve or sell recalled eggs. Learn more: LINK. (https://twitter.com/CDCgov/status/1039251062732021760)

Topic 2 – Brainstorm Prospective Message Development

We are doing great and are almost done. This next section is what some would call brainstorming, and others would call a ‘lightening round’. In the next 5-7 minutes please say what words, phrases, or information, you think would be most effective for communication messages about foodborne illness. As you can see, we have four flip charts corresponding to four categories of messages:

• Consumer guidance, or what kinds of information you need or want to hear about foods and outbreaks;

• Investigation details, or what CDC, other government agencies, and private companies are doing in response to the outbreak;

• Illness/symptoms, or what might happen to you or your family or friends as a result of the foodborne illness; and

• Information about specific foods, including those considered high risk.

We welcome any and all comments here, and as needed we can work together to decide which category your suggestions fall in, or perhaps they fall into more than one.

Close Out

I am going to quickly ask my associates if there is anything else we need to cover. I will be back in one moment, but please feel free to continue talking among yourselves.

That is all the material we needed to cover, but is there anything else that any of you would like to add, or are there any questions that I should have asked but did not?

Thank you for your participation and insights in this focus group and the individual interviews before this. Your participation will help improve future communication and messaging about foodborne illness.

CDC Message Testing Focus Group Moderator’s Guide – EurekaFacts – 3/14/19 – Page 17 of 17