FDA DOCUMENTATION FOR THE GENERIC CLEARANCE,
“TESTING
COMMUNICATIONS ON DRUGS PRODUCTS”
(0910-0695)
.
TITLE OF INFORMATION COLLECTION: Studies to Enhance FDA Communications Addressing Medical Countermeasures Prescription Drug Products
DESCRIPTION OF THIS SPECIFIC COLLECTION
Statement of need:
The Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) has an ongoing responsibility to communicate about the medical products it approves or authorizes for use in medical emergencies (Sections 564, 564A, and 564B of the Federal Food, Drug, and Cosmetic Act [FD&C Act] as amended or added to by the Pandemic and All-Hazards Preparedness Reauthorization Act of 2013 [PAHPRA]). For more information, please click on the following link:
In order to ensure the messages, materials and information CDER is providing shall be easily understood and acted upon, provide a basis for informed decision-making, and promote trust in the Agency, its communications must be evidence-based so they are consistent with the best science, and they must be evaluated to determine how well they are accomplishing their objectives and how they may be improved.
The provisions in PAHPRA, described in section II of the Guidance, include key legal authorities to sustain and strengthen national preparedness for public health, military, and domestic emergencies involving chemical, biological, radiological, and nuclear (CBRN) agents, including emerging infectious disease threats such as influenza pandemic. PAHPRA clarifies and enhances FDA’s authority to support emergency preparedness and response, and fosters the development and availability of medical products for use in these emergencies. These medical products also referred to as “medical countermeasures” or “MCMs,” include drugs, biological products (e.g., vaccines, blood products, and biological therapeutics), and devices (e.g., in vitro diagnostics and personal protective equipment). This guidance, when finalized, will replace the current Guidance, Emergency Use Authorization of Medical Products (July 2007) and Emergency Use Authorization Questions and Answers (April 2009)1.
CDER communications provide the public with the most current and reliable information concerning these medical products in order to help them make more informed treatment choices. Thus, it is critical that CDER communicate clearly and effectively with the public.
For many health threats, medical countermeasures medicines represent a lifeline to the affected public, including members of vulnerable and special populations. However, some of these medicines are not widely used or available. Furthermore, these medicines may carry unique or undocumented risks. For this reason, it is imperative that the FDA communicate rapidly and effectively with the public so they understand the risks associated with these medicines and will be able to make decisions that maximize benefits and minimize risks.
In an effort to most efficiently and effectively communicate about various medical countermeasures that could be used to treat a variety of health threats associated with biological, chemical and radiological terrorism exposures, CDER’s Office of Communications (OCOMM) will conduct survey testing of a series of medical countermeasures-related risk communication messages and information that can be used in public health emergencies. The messages and information to be tested in the survey were developed based on prior qualitative formative research, and the survey builds on those findings and will provide generalizable and practical findings.
This work is critical to strengthening the Agency’s ability to fulfill its public health mission by raising awareness of and educating various audiences/stakeholders about medical countermeasures medications that may be used during a public health emergency.
Intended use of information:
This data gathered from this survey will include several outcome measures that could be affected by exposure to the messages being tested through this survey, including message comprehension, emotional reactions, risk perceptions, crisis efficacy, trust in government, response efficacy, behavioral intentions, and message effectiveness. The data will be analyzed, and the tested messages revised based on the resultant findings. The revised messages will be available for posting on the FDA website in the event of a terrorist threat or other public health emergency.
Description of respondents:
Respondents for this data collection are adults 18 years of age and older, including those from special needs groups such as racial and ethnic minorities and those with low health literacy/education. Our objective is to gather evidence from a broad cross-section of respondents with different backgrounds, concerns, and information needs to help ensure the effectiveness of the messages and information that may be posted on FDA’s website during a public health emergency.
The study will include a main sample of 2,000 respondents plus oversamples sufficient to bring three vulnerable populations – African Americans, Hispanics, and persons will less than a high school education – to 375 each. Because there is overlap between these vulnerable groups, it is expected that only 469 additional respondents will be required to meet this goal, bringing the total number of survey respondents to 2,469.
Date(s) to be Conducted:
Data collection will begin as soon as possible after OMB and FDA RIHSC approval and is expected to be completed within two weeks.
How the Information is being collected:
FDA will conduct an experimental survey lasting 20 minutes on average. The questionnaire does not include any sensitive questions, and the questions are designed to be as straightforward as possible to understand and respond to. This information collection will be carried out online using GfK Knowledge Networks and their KnowledgePanel, the largest probability-based online panel, designed to be statistically representative of the U.S. population. Responses are submitted online so no paperwork is required of respondents. Participation is entirely voluntary.
KnowledgePanel was chosen because (1) it is it the largest probability-based online panel designed to be statistically representative of the U.S. population, (2) it utilizes innovative question formats that engage respondents and reduce burden, and (3) it produces high cooperation rates. Because all KnowledgePanel households were selected randomly with a known probability of selection, KnowledgePanel estimates can be used with statistical confidence. This study will be posted online and be accessible to respondents for 7 to 10 days.
KnowledgePanel was initially created using random-digit-dialing (RDD) and is now continuously maintained using the United States Postal Service's Delivery Sequence File. This file is essentially a complete list of all U.S. residential households, including households that are cell phone-only and often missed in RDD sampling. This probability-based sampling methodology improves population coverage, particularly for hard-to-reach individuals such as young adults and minority subgroups. Persons in selected households are then invited to participate in the Web-enabled panel. Those who agree to participate but do not already have home access to the Internet are sent a laptop computer, and they are provided an Internet service connection paid for by GfK Knowledge Networks. People who already have computers and Internet service are permitted to participate using their own equipment. Once on KnowledgePanel, panelists receive unique login information for accessing surveys online.
In order to achieve the high standard for a large probability-based online panel designed to be statistically representative of the U.S. population, GfK Knowledge Networks undertakes a multi-step process. As part of the initial KnowledgePanel recruitment process, randomly sampled addresses are invited to join KnowledgePanel through a series of mailings, including an initial invitation letter, a reminder postcard, and a subsequent follow-up letter. Approximately 48% of the physical addresses selected for the sample can be matched to a corresponding valid telephone number. About five weeks after the initial mailing, experienced interviewers start making calls to households that have not responded to the invitation to become a KnowledgePanel member and for whom a telephone number was matched to the sampled address, and the telephone numbers are dialed for up to 90 days with at least 10 dial attempts. During the initial panel recruitment phase, households that initially refused membership but were not hostile are called and, if reached, offered another opportunity to join. All non-responders receive a letter asking them to become members of KnowledgePanel.
Invited households can join the KnowledgePanel by: (1) completing and mailing back a paper form in a postage-paid envelope, (2) calling a toll-free hotline phone number maintained by GfK, or (3) going to a designated website and completing the recruitment form at the website. By doing so, the household becomes eligible to participate in KnowledgePanel surveys. Following recruitment to the KnowledgePanel, each household receives a unique password to access the member portal. It contains links to their surveys, options to change contact information, links to the point redemption site, links to privacy and other panel policies, and other panel relevant information. As members of KnowledgePanel, members agree to: (1) check their mail for new surveys at least once a week, (2) report any technical problems that might prevent response, (3) answer all survey questions truthfully, unless they feel uncomfortable answering the question (note that respondents will not be forced to answer questions in the current survey; we will provide “no opinion,” “neutral,” or “decline to answer” options for all questions), (4) keep confidential the details of KnowledgePanel surveys, and (5) refrain from taking advantage of their KnowledgePanel membership in any way. They also complete a demographic profile survey, which includes questions on gender, age, race/ethnicity, income, education, and prior computer and Internet use. We will append this demographic data to each respondent’s data file so respondents will not have to answer these questions again, reducing respondent burden.
Selection for specific surveys is typically random within the strata (e.g., specified age groups) relevant to the survey. Once assigned to a survey, potential respondents are notified by email that a survey is available and are given a password-protected link to the survey that enables them to participate by simply clicking the link. Appendix 1 shows the standard email invitation, which will be sent to potential participants of this survey. The length of time the survey is held open depends on the client’s needs and can range from a few hours to several weeks. It is expected that this survey will remain open for one to two weeks, depending on the pattern of response.
For selection of general population samples, GfK Knowledge Networks has a patented methodology that has been developed to ensure the resulting samples behave as EPSEM (equal probability selection method). This methodology starts by weighting the entire KnowledgePanel to the detailed geo-demographic benchmarks of U.S. adults from the latest March supplement of the Current Population Survey, which is conducted by the U.S. Census Bureau. This ensures that the weighted distribution of KnowledgePanel perfectly matches that of the U.S. adult population. Using the weights as the measure of size (MOS) for each panel member, a probability proportional to size (PPS) procedure is used to select study-specific samples. The application of this PPS methodology with the MOS values produces fully self-weighting samples, which means that it is not necessary to apply a weight. In instances where the study design requires any oversampling of specific subgroups, as will be the case here, departures from an EPSEM design are corrected by adjusting the corresponding design weights, with Current Population Survey benchmarks serving as reference distributions.
Knowledge Panel surveys average a cooperation rate of about 50% for studies that are online and accessible to respondents for 7 to 10 days using the following process:
Invitations. Selection of members for a particular survey is typically random within strata relevant to the study. When members join KnowledgePanel, they receive a unique password to access the member portal, and they agree to check for new surveys at least once a week. In addition, once assigned to a study sample, they are notified by email that a survey is available for them to complete. Each notification includes a password-protected link that directs them to the survey questionnaire and can be used for one completion only. All information is included in the link so the respondent does not have to enter a login field, password, or any other information in order to access the survey. All initial notifications are done by email, which is less intrusive than telephone calls. Please see the standard mail invitation, which will also be used for this study, in Appendix 1.
Respondents for this survey will be asked to sign an online informed consent form (see Appendix 2) notifying them that participation is voluntary, and that the respondent may discontinue participation at any time without penalty or loss of benefits to which they are otherwise entitled. The online informed consent form will be embedded at the beginning of the survey with consent given by clicking an “accept” or “decline” button. The survey will not open unless the respondent clicks “accept.”
Reminders. Members who do not respond to the first email notification will be sent an email reminder after three days. After another three to four days, respondents receive an automated telephone call reminding them to take the survey.
KnowledgePanel processes also make it easy for sampled members to participate. For example:
The survey will be open and accessible to respondents for 7 to 10 days.
Respondents can choose the day and time for completing the survey.
KnowledgePanel respondents participate in profile surveys, which capture socio-demographic, household, and behavioral information, once a year. Since these questions can be added to the databases for client survey, these types of questions do not need to be asked for every client survey.
The survey will be media device independent, thus allowing respondents to answer using the device of their choice, including computers, tablets, or smart phones.
The questionnaire will be formatted so that respondents can see the questions and information without having to scroll.
When a respondent stops a survey without completing it, GfK systems return the respondent to the exact location in the questionnaire where they left off; they do not need to review or re-enter any information previously entered.
The following additional survey programming capabilities will be included for this survey:
automated questionnaire versions
individual question routing
rotating response option lists
auto fill of question language based on previous responses
Prior to this submission to OMB, Lake Research Partners, the contractor undertaking the survey on behalf of FDA, conducted an online pretest of nine members of its staff who are not part of the project team to test the length of the survey, and to assure that all questions are understood as intended and working on the online platform. The results showed that the questionnaire averaged 19 minutes. After OMB approval is obtained, a second pretest or “soft start” with about 50 actual respondents will be conducted to verify average timing remains at 20 minutes or less and to make sure that the survey questions and platform are working as intended.
Confidentiality of Respondents:
All study participants will be informed as part of the written consent that no reports or other information will identify participants by name, that all information will be anonymized and reported in aggregate, and that their information will be kept private to the extent possible.
The KnowledgePanel recruitment and empanelment process is designed to comply with the law that sets out rules for commercial email and the guidelines of the Council of American Survey Research Organizations. Further, KnowledgePanel policies conform to participant treatment protocols outlined by OMB. Survey responses are confidential; personally identifying information is never revealed to clients or other external parties without explicit respondent approval and a client-signed nondisclosure agreement. When surveys are assigned to KnowledgePanel panel members, they are notified in their password-protected email account that a survey is available for completion. Surveys are self-administered and accessible any time of day for a designated period. Participants can complete a password-protected survey only once. Members may withdraw from the survey at any time, and continued provision of the web-enabled device (e.g., laptop or netbook) and Internet service is not contingent on completion of any particular survey.
All KnowledgePanel panel members are given a link to access the privacy terms electronically at all times via the Panel Member website and also are able to review it at any time on the Members Page and in links contained in survey invitations. The Privacy and Terms of Use Policy is posted at: http://www.knpanel.com/participate/privacy2.html and is also shown in Appendix 3.
All data and information collected will be kept in a secured fashion that will permit access only by authorized project staff. All personally identifiable information will be removed from all materials before LRP provides them to CDER/OCOMM. All files will be stored on password-protected computers at LRP and FDA. We will not collect any information until receiving approval from FDA’s Research Involving Human Subjects Committee (RIHSC).
Amount and justification for any proposed incentive
KnowledgePanel’s participant panel members will receive the standard incentive of either 1,000 thousand points (equivalent to $1) or the use of a GfK-provided Internet device and an unlimited internet service provider (ISP) (for non-internet households only) for participating in the panel and the completion of a survey. These are the same incentives that respondents agree to when they choose to participate as a member of the KnowledgePanel. Members can redeem points they’ve been awarded for participating in a survey(s) for cash, merchandise, gift cards, or game entries.
They may also be entered into special sweepstakes drawings, from which a winner(s) is randomly drawn. Generally, members are invited to complete one survey per week, and on average, they complete two to three 10-minute to 15-minute surveys per month. Survey-specific incentives are provided whenever: (1) the survey is expected to take more than 15 minutes or (2) the survey includes an unusual request, such as viewing a television program or completing a daily diary. If a survey requires more than 15 minutes, respondents are entered into a sweepstakes. If the survey takes more than 25 minutes, which will not be the case for this project, participants receive additional points.
The proposed survey for this project is expected to take 20 minutes and thus respondents will be entered into a sweepstakes for their participation. There is one grand prize (50” flat screen television), five first prizes (Nook eReader or Apple TV), 50 second prizes (noise reducing head phones, $50 hotel card, or two movie tickets), and 400 third prizes ($25 dining certificate). The odds of winning a prize are 1:657.9.
Questions of a Sensitive Nature
There survey contains no questions of a sensitive nature, although the overall topic may be unpleasant as it relates to messages that may be released in the event of a public health emergency. The questions will relate to the messages, including comprehension, emotional reactions, risk perceptions, crisis efficacy, trust in government, response efficacy, behavioral intentions, and message effectiveness. Before they can start the survey, respondents will “sign” an online informed consent form (see Appendix 2) notifying them that participation is voluntary, and that they may discontinue participation at any time without penalty or loss of benefits to which they are otherwise entitled. The informed consent form will be embedded at the beginning of the survey with consent given by clicking an “accept” or “decline” button. The survey will not open unless the respondent clicks “accept.”
Description of Statistical Methods
This study is based on three experiments designed to test the effects of being exposed to one of three messages that describe medical countermeasures to biological threats. Each experiment will include an experimental group and a control group for a total of six groups. The 2,469 respondents will be randomly assigned to these groups as shown in Table 1. The survey is designed to produce valid and reliable results that can be generalized to U.S. adults.
Table 1. Allocation of Respondents to Experimental and Control Groups in the Message Conditions |
||||
|
Message Condition 1 FDA-Approved MCM |
Message Condition 2 Unapproved MCM allowed under Emergency Use Authorization Only |
Message Condition 3 No MCM |
Total |
Experimental Group |
412 |
412 |
412 |
1,236 |
Control Group |
411 |
411 |
411 |
1,233 |
Total |
823 |
823 |
823 |
2,469 |
Cross Tabulations - The cross tabulations will include separate tables for each message that compare the results across groups of the several outcome variables for the experimental group against the results for the control group, with some also tested within groups from pre-test to post-test. The outcome measures include comprehension, emotional reactions, risk perceptions, crisis efficacy, trust in government, behavioral intentions, and message effectiveness.
We will use appropriate statistics (t-tests for means and z-tests for proportions) to identify statistically significant differences between experimental and control conditions. All of the comparisons described above are binary. Thus, we can use the t-test for means and the z-test for proportions. All of the groups to be compared are of equivalent size, with an average base of 411.5.
Table 2. Base Sizes, Sample Precision and Statistical Power |
||||
Total Base |
Sample Precision |
# of Groups* |
Avg. Base Size of Groups |
Difference Need for Significance*** |
2,469 |
2.0 Pct. Pts. |
6 |
411.5 |
6.9 Pct. Pts. |
*Margin of Sampling Error for percentages around 50% at the 95% Confidence Level
** Experimental plus control groups
***For percentages around 50% at the 95% Confidence Level (2-tailed Z-test for proportions).
We will also use cross tabulations to document any demographic differences in the effects of each message for vulnerable populations, that is, low education individuals, Hispanics, and Blacks.
For the control groups only, there will also be a highlighting exercise as part of the survey. To avoid potential bias and reduce respondent burden, the highlighting exercise will not be administered to the experimental groups. This highlighting exercise will show the specific words, phrases, and sentences that evoked positive and negative reactions for each message. We will analyze the overall results and then cross tabulate the results of this exercise by demographics, particularly those demographics that identify vulnerable populations, in order to make specific recommendations for communicating with different population subgroups.
Advanced Analytics – A more advanced analysis of the communication messages tested, with the independent variable being exposure to a particular message will also be conducted using Path Analysis, a multi-stage, multiple regression method that reveals the communication process, i.e. how thoughts, feelings, crisis efficacy, response efficacy, and other perceptual factors affect intended action. In doing so, Path Analysis shows the strengths and weaknesses of each message and provides implications for revision of the messages.
The analytic process will result in three separate models, one for each message. It will involve the following steps:
Create a causal model that sets out the elements (concepts) as predictors, mediating variables, and criterion variables. In this case, the predictor variable is exposure (or non-exposure) to the FDA message in the experiment. The intervening variables would include – but not necessarily limited to – message comprehension, emotional reactions to the message, crisis efficacy, risk perceptions, and trust in the federal government. The criterion variable is behavioral intentions. We will discuss expected relationships in the model based on communication and marketing theory and research, particularly Fishbein’s Theory of Reasoned Action2 and the Lavidge and Steiner’s Hierarchy of Effects Model3. However, the third step will enable us to test other possibilities.
Create indices for the multi-item concepts using Factor Analysis to test inter-item consistency (how well the items “hang together” as a unified concept). We will also check correlations between items in each index and the behavioral intentions index to assess predictive power and recommend exclusion of any items that would dampen the predictive power of the index.
Analyze the model using SPSS multiple regression and SPSS macros developed by Hayes4 and Muthen, Muthen, and Asparouhov 5. The macros will enable us to cross-validate our theoretical assumptions about causal sequence of variables in the model, as well as possible relationships between concepts that were not originally specified.
Correct coefficients based on indices for unreliability.
Estimate total effects of concepts based on both direct and indirect effects.
Report the output of the analyses, including statistically significant path coefficients for direct, indirect, and total effects for each concept in each equation in the model. R2 statistics will be provided also.
Create graphic depictions of each model based on the most plausible paths. The directionality of some paths may be reciprocal.
BURDEN HOUR COMPUTATION (Number of responses (X) estimated response or participation time in minutes (/60) = annual burden hours):
NOTE: Some revisions have been made to the questionnaire since the initial pre-testing, so as noted above, additional testing for timing will occur, and FDA will ensure that the final instrument will take no more than 20 minutes on average to complete by removing questions if needed.
Type/Category of Respondent |
No. of Respondents |
Participation Time (minutes) |
Burden (hours) |
Survey/ Nationally representative sample of adults 18+ |
2,469 |
20 |
823.0 |
REQUESTED APPROVAL DATE: April, 2017.
NAME OF PRA ANALYST & PROGRAM CONTACT: Ila S. Mizrachi; Ila.Mizrachi@fda.hhs.gov; 301-796-7726.
FDA CENTER: Center for Drug Evaluation and Research
Program Contact:
Paula Rausch, PhD, RN
CDER Office of Communications
301-796-3121
Paula.rausch@fda.hhs.gov
11www.federalregister.gov/documents/2016/04/04/2016-07478/emergency-use-authorization-of-medical-products-and-related-authorities-draft-guidance-for-industry
2 Icek Ajzen and Martin Fishbein. Understanding Attitudes and Predicting Social Behavior. Prentice-Hall, Inc., 1980.
3 Lavidge, Robert J. and Gary A. Steiner. 1961. A model for predictive measurements of advertising effectiveness. Journal of Marketing. 25(October), 59–62. Reprinted with the permission of the American Marketing Association. Reprinted with the permission of the American Marketing Association.
4 Andrew F. Hayes. Introduction to Mediation, Moderation, and Conditional Process Analysis. The Guilford Press (e-book) 2013.
5 Bengt Muthen, Linda Muthen, Tihomir Asparouhov. Regression and Mediation Analysis Using M Plus: Version 7 M Plus Users’ Guide. Muthen & Muthen, 2016.
| File Type | application/msword |
| File Title | OMBMemoMERCPtP |
| Subject | MERC OMB MEP |
| Author | Hillabrant |
| Last Modified By | Mizrachi, Ila |
| File Modified | 2017-03-29 |
| File Created | 2017-03-14 |