OADC Material Feedback Survey - Protocol

OADC Message and Material Feedback Surveys — Protocol

Table of Contents

Objectives 3

Participants 3

Recruitment 4

Methods 5

Objectives

The Centers for Disease Control and Prevention (CDC), Office of the Associate Director for Communication (OADC) is collecting feedback on 4 sample health communication materials that are used to communicate with the public about urgent and emerging public health issues (such as a flu epidemic). The goal of the study is to learn how to create more effective messages and materials to communicate to the public about urgent public health issues in the future.

On behalf of OADC, CommunicateHealth will administer surveys (Attachment B), via a 3^rd party vendor, to a diverse mix of the general public to evaluate the following aspects of the health messages and materials:

• Clarity of the material’s target audience, main message, and intended behavioral recommendation

• Comprehensibility — how easy are messages in the material to understand

• Utility — how useful are messages in the material

• Design quality

Table 1 summarizes the number of surveys, health materials, and number of participants.

Table 1. Material and Participants by Survey

Survey	Material	Participants
Survey 1	Survey #1 Material	15
Survey 2	Survey #2 Material	15
Survey 3	Survey #3 Material	15
Survey 4	Survey #4 Material	15
		Total: 60

Specifically, OADC wants to learn:

• How do participants rate the materials and messages in terms of being easy to understand, useful, and well designed?

Based on findings, CommunicateHealth will make recommendations for improving messages and materials on urgent and emerging public health issues in the future.

Participants

On behalf of OADC, CommunicateHealth will conduct online surveys with 60 participants using Qualtrics, a secure survey platform¹. IT security clearance for the use of the Qualtrics online survey has been granted by the CDC, NCEH Information System Security Officer (ISSO). Our desired participants are a diverse mix of the general public in education levels, races and ethnicities, and income levels.

The screener questions and response quotas will yield a sample with the following characteristics:

• Age: Mixed, 18 and over

• Gender: Mixed

• Race/Ethnicity: Mixed

• Education Level: Mixed, at least half of the participants cite some college education completed or less

• Income: Mixed, at least half of participants cite an income of $50,000 or less

• Geography: Mixed

• Health Literacy Level: Mixed

The screener questions will exclude from participation people who:

• Are under the age of 18

• Have participated in any research within the past 6 months

• Conduct market research or work in advertising or public relations

• Are unable to speak and read English well enough to complete the study

Recruitment

Qualtrics will invite participants from their nation-wide participant panel to complete the online surveys. The panel is a large, national database of diverse participants who have volunteered to participate in their online surveys. Qualtrics volunteers are anonymous to both CDC and CommunicateHealth.

Qualtrics will email potential participants a survey invitation (Attachment C). The process for screening and completing the surveys will include the following steps:

• If the panel member volunteers to participate, they will click on a survey link in the email invitation.

• The participant will answer a set of screener questions, which will ask about the participant’s age, gender, race/ethnicity, education level, income, geographic location, and health literacy level (estimated completion time is 3 minutes) (Attachments D [text] and F [online]).

• To ensure a diverse mix of participants, the number of eligible participants will be capped using quotas on each screener question (e.g., Quota on education level will be 50% of participants with some college education completed or less).

• If the participant qualifies and consents (Attachments F [text] and G [online]), they’ll complete survey questions about one material (estimated completion time is 12 minutes) (Attachments H [text] and I [online]).

Participants will answer all screener and material-related survey questions anonymously. OADC and CommunicateHealth will not have access to any personally identifiable information about participants. CommunicateHealth will access participant responses through the Qualtrics reporting dashboard where participants are identified only by a unique ID number.

Incentives

Qualtrics will offer participants reward points (small, non-monetary incentives) as a token of appreciation for their participation. The email invitation includes the number of reward points offered for the survey (Attachment C).

This small non-monetary incentive:

• Is consistent with industry standards for quantitative data collection efforts.

• Is given by Qualtrics (not CommunicateHealth or OADC) to panel members for surveys they take through Qualtrics, in accordance with their panel membership.

• Will accumulate for panel participants as they take additional surveys (unrelated to this study) through Qualtrics, eventually being redeemed for items such as gift cards and subscriptions (e.g., magazines).

• Is required by Qualtrics in order to conduct recruitment from their diverse national panel of participants.

• Is needed to receive a sufficient response rate, based on CommunicateHealth’s and Qualtrics’ previous experiences to ensure recruitment and retention for this hard-to-reach population (diverse, low income, and limited education) in similar studies. For example, CommunicateHealth, on behalf of ATSDR, received OMB clearance for a similar study involving the use of Qualtrics reward points under NCEH/ATSDR Generic Clearance for Routine Customer Feedback (OMB# 0923-0047), approved October 2015.

Qualtrics will provide reward points to participants who qualify, consent, and complete the survey. Qualtrics will not provide reward points to non-eligible participants.

Methods

On behalf of OADC, CommunicateHealth will collect feedback on 4 health materials via 4 separate brief online Qualtrics surveys. Each survey (screener plus survey) will take approximately 15 minutes to complete.

All surveys contain the same set of questions.

The survey will include:

• Screener questions (3 minutes) (Attachments D [text] and E [online])

  • Note: no personally identifiable information will be collected

• Consent (Attachments F [text] and G [online])

• Health material questions (12 minutes) (Attachments H [text] and I [online])

  • Review a 1-2 page CDC health material

  • Answer questions about main target audience, message, and intended behavioral recommendation

  • Answer questions adapted from the Consumer Information Rating Form (CIRF) developed by Krass et al. (2002) on comprehensibility, utility, and design quality

• Thank you message

Participants will complete screener questions, consent, and health material questions via 1 survey link.

Handling of Data and Records

Neither OADC nor CommunicateHealth will collect individually identifiable information from participants in this study. The proposed data collection will have little or no effect on the participants’ privacy. CommunicateHealth will use only comments, quotes, and quantitative responses from participants to inform improvements to future material revisions.

IT security clearance for the use of the Qualtrics online survey has been granted by the CDC, NCEH Information System Security Officer (ISSO).

IRB review

On February 3, 2016, the OADC IRB officer, Dr. Dogan Eroglu, determined this project is non-research. Natalie Brown, OADS, agreed with this determination.

HMTS review

This project will undergo HMTS review.

Data analysis plan

CommunicateHealth will:

• Report participant characteristics in the aggregate

• Analyze and report descriptive statistics only for close-ended survey responses

• Review and code open-ended responses for themes

Final Report and Recommendations

CommunicateHealth will analyze feedback from the surveys to identify how participants rated each of the 4 materials on dimensions of message comprehension, utility, and attractiveness. Findings will be summarized in a report and used to improve public health communication materials in the future.

1 http://www.qualtrics.com/security-statement/