DCH Message Testing Supplement

8

Message Testing for CDC’s Division of Community Health

Supplement to the HMTS Expedited Review Form

This document provides information about Page

• The concepts and messages to be tested 2

• Modes for message testing 2

• Incentives 3

• Duration of discussion groups 4

• Remote testing procedures 4-7

• Materials that respondents will see

  • On-line discussion groups (incl. screen shots) 8-16

  • In-person testing procedures 17-23

Concept	Message
Community health	This program seeks to improve community health.
Healthier communities	Investing in healthier communities across the country.
Making healthy living easier and more affordable	The goal of this effort is making healthy living easier and more affordable.
Environmental change(s)	Environmental changes are at the heart of making healthy living easier.
Environments where we live, work, learn and play	The way that environments where we live, work, learn, and play are designed and built—for example, whether there are sidewalks or bike lanes—influences how physically active we are.
Choice(s)/Healthy choice(s)	Making the healthy choice the easy choice.

Modes for message testing data collection

CDC’s Division of Community Health is proposing two modes of data collection: in-person focus groups and remote triads using a telephone and a computer.

The focus groups with the general public will be completed in-person, with individual messages presented on separate sheets of paper to the participants (see subsequent pages for how each of the messages will be presented to participants).

The triads with community leaders and the engaged public will be conducted remotely via a toll-free telephone line and by computer via the GoToMeeting web conferencing service. Instructions on how to join the discussion by phone and computer will be provided to the participants via e-mail. During the discussion group messages will be presented to the participants similarly to how they are shown to the in-person focus group participants, except that they will be viewing them on their computer screen [See subsequent pages for further details about procedures that will be following for remote discussion groups and for screenshots of how messages will be viewed by remote participants).

Incentives

All participants will receive a $50 incentive. With respect to the level of incentive, Krueger and Casey, in the publication “Focus Group: A Practical Guide for Applied Research” (publication date of 2009), indicate the following: “At the time of this writing, amounts of $50 to $75 usually work for public and nonprofit studies. As the amount approaches $100, an interesting phenomenon begins to occur. If the participant has a last-minute conflict, they are more likely to call the moderator and offer to send a replacement in their absence. When working with elite categories of focus group participants, the amounts may need to be adjusted upward. Focus groups with engineers, physicians, attorneys, upper-level managers, and similar categories may require amounts in the $100 to $500+ range.”¹ Therefore, the proposed $50 incentive represents the minimal level of incentive deemed suitable for focus group participants in the recent past.

All participants will receive the same incentive of $50 even though (1) the duration of the discussion groups will vary based on the type of group/audience (i.e., 90 minutes for general public focus groups, 60 minutes for engaged public, and 45 minutes for community leaders); and (2) the focus groups will be conducted in-person and the triads will be remote. An equal incentive amount is proposed given that the engaged public and community leaders are individuals with competing demands for their time and attention due to their level of engagement/leadership in their community. Therefore, an incentive comparable to the general public focus group participants will help ensure sufficient attendance and participation. Additionally, while the triads with community leaders will be 15 minutes shorter than those with the engaged public, community leaders are expected to take part in the triads during the business day, particularly during their lunch time. Therefore, the $50 incentive for community leaders will also help ensure sufficient attendance and participation.

The $50 incentive will be a cash incentive rather than an alternate form of incentive (e.g., gift card to a store) because guidance on focus group research also suggests that “incentives should be selected that have universal value to participants; what may be valuable to one person may have little value to other. This is one reason money is employed most often.”² The in-person focus group participants (general public) will receive the incentive in cash at the focus group facility immediately after the conclusion of the discussion groups. Given that the engaged public and community leaders will be participating in remote triads, the incentive will be mailed to them upon the completion of the discussion group in the form of a $50 cash card (e.g., Visa Gift Card).

Appropriate incentives are key to the success of research efforts and to preventing over-burdening the public. “Incentives are needed because it takes effort to participate in a focus group… it [also] serves as a stimulus to attend the session. The primary function of the incentive is to get participants to show up for the focus group—and to show up on time.”³ In other words, even when individuals agree to participate, insufficient incentive may result in a greater likelihood of participants not showing up to participate in the discussion (i.e., “no-shows”). This may not only negatively impact the ability to gather the necessary data but, additionally, the time that would have already been spent in the recruitment of these participants would therefore translate into unnecessary burden to the public. Additionally, because there are costs associated with recruiting each participant (even for no-shows), when participants are no-shows it involves expenditures that could otherwise be avoided.

Duration of Discussion Groups

The duration of the discussion groups, by type of group/audience, will be as follows:

• Focus groups with general public: 90 minutes

• Triads with engaged public: 60 minutes

• Triads with community leaders: 45 minutes

The duration varies given factors unique to the type of group and audience. The focus groups with the general public have the longest duration (90 minutes) given the larger number (up to 10) of individuals that will take part in the groups. The allotted time of 90 minutes will allow time for all participants to share their thoughts and opinions. While the nature of the message testing with the engaged public triads will be similar to that of the focus groups, given the smaller number of participants (3 to 4), it is anticipated that the same depth of discussion that can take place in the focus group in 90 minutes will be able to take place in the engaged public triad in 60 minutes. While the community leaders will also participate in triads of 3-4 participants, it is anticipated that these individuals will be participating during a regular business day, in their lunch time. To accommodate for their work schedules, a duration of 45 minutes for the discussion group was deemed most appropriate. While the message testing with the community leaders will be similar in nature to that of the general public and engaged public, it is possible that the shorter duration of the discussion may not allow for as much in-depth discussion of certain messages/topics.

Remote testing procedures

Once participants who will be taking part in the discussion groups remotely (i.e., engaged public and community leaders) are scheduled, they will receive via e-mail instructions for joining the discussion on their scheduled date and time. The instructions they will receive will look as follows:

The e-mailed instructions will contain:

1. A toll-free telephone number for the voice/audio portion of the discussion.

2. Directions for joining the web conference via GotoMeeting.

GoToMeeting is an online web conferencing service that will allow the moderator to project on her computer screen, during the discussion group, the messages being tested. Participants will be able to see on their own computer screen the messages as they are being projected by the moderator on her own computer screen. In other words, it is a way for all participants to be looking at the same material at the same time during the discussion. While participants will be able to see what the moderator is projecting on her computer screen, the moderator will not be able to see or access in any way the participants’ computer screen.

Although GoToMeeting allows for video capabilities, they will not be used for these discussion groups. The moderator nor the participants will be able to see each other.

The e-mailed instructions will also include directions for the consenting procedures. It will include the consent form as an attachment. It will also request for participants to read, sign and return the attached consent form prior to the date of the interview. Participants will have the option of returning the signed consent form by fax or by e-mail (scanned). While it will not be indicated in the e-mail, if any participants were to indicate that they are unable to return the form by either fax or e-mail, arrangements will be made to send them a pre-addressed, stamped envelope so they can return it by regular mail prior to the discussion group.

On the day of the interview, the participants will first call the toll-free telephone line. They will then follow the procedures to join the web conference via GotoMeeting. The screenshots on the following pages illustrate the interface/procedures that participants will follow to join the web conference.

Step 1: From the GotoMeeting homepage, they will click on the “Join Meeting” button at the top of the page.

Step 2: They will enter the unique meeting ID # that was provided to them in the instructions.

[Messages for in-person

Step 3: Once they enter the Meeting ID #, the system will lead them to the online meeting space. In other words, at that point they will be able to see what the moderator has on her computer screen, which will be a welcome message that also includes the 1) three-line OMB control number, 2) the project/form title, and 3) the OMB burden statement (that corresponds to each respondent group).

[Version Community Leaders will see]

[Version Community Leaders will see]

[Version Engaged Public will see]

Once participants view the entry screen, the discussion will begin. Throughout the discussion, the moderator will show on the screen the messages being tested. These will be shown and discussed individually. Once the group is finished discussing a given message, the next message will be shown on the screen. See the screenshots below for an illustration of how remote participants will see the messages.

Once the discussion group has concluded, participants will be able to simply close their Internet browser window to exit GoToMeeting and the discussion group.

[Messages for remote triads as they will appear to participants via GoToMeeting follow]

In-person testing procedures

General public participants will take part in in-person focus groups. When they arrive at the focus group facility, they will be asked to read and sign the consent form prior to the commencement of the discussion group.

Throughout the discussion, the moderator will show on a printed piece of paper the messages being tested. These will be shown and discussed individually. Once the group is finished discussing a given message, the next message will be shown.

In the pages that follow, the messages (as they will be shown to in-person participants) are presented.

[Messages for in-person focus groups follow]

This program seeks to improve community health.

Investing in healthier communities across the country.

The goal of this effort is making healthy living easier and more affordable.

Environmental changes are at the heart of making healthy living easier.

The way that environments where we live, work, learn, and play are designed and built—for example, whether there are sidewalks or bike lanes—influences how physically active we are.

Making the healthy choice the easy choice.

1 Krueger, R.A. (2009). Focus Groups: A Practical Guide for Applied Research (4th ed.). Thousand Oaks, CA: Sage Publications.

2 Stewart, D.W. & Shamdasani, P.N. (1990). Focus Groups: Theory and Practice. Newbury Park, CA: Sage Publications.

3 Krueger, R.A. (2009). Focus Groups: A Practical Guide for Applied Research (4th ed.). Thousand Oaks, CA: Sage Publications.