Supporting Statement PART A RAAL 2018

U.S. Food and Drug Administration

Experimental Study of Risk Information Amount and Location in Direct-to-Consumer Print Ads

OMB Control No. 0910-NEW

SUPPORTING STATEMENT Part A. Justification

1. Circumstances Making the Collection of Information Necessary

Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 300u(a)(4)) authorizes FDA to conduct research relating to health information. Section 1003(d)(2)(C) of the Federal Food, Drug, and Cosmetic Act (the FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act.

Section 502(n) of the FD&C Act (21 U.S.C.352(n)) specifies that advertisements (ads) for prescription drugs and biological products must provide a true statement of information "in brief summary" describing the advertised product's "side effects, contraindications and effectiveness." This is clarified further in the prescription drug advertising regulations. The brief summary shall include a true statement of information relating to side effects, contraindications, warnings, precautions, and any such information under such headings as cautions, special considerations, important notes, etc., as well as effectiveness (§ 202.1(e)(1)). The prescription drug advertising regulations also specify that the phrase side effect and contraindication refers to all of the categories of risk information contained in the required, approved, or permitted product labeling written for health professionals, including the side effects, warnings, precautions, and contraindications (§ 202.1(e)(3)(iii)). Ads must also "present a fair balance between information relating to side effects and contraindications and effectiveness…." An ad must present true information relating to side effects and contraindications in comparable depth and detail with the claims for effectiveness or safety (§ 202.1(e)(5)(ii)).

To fulfill the regulatory requirements for fair balance and the brief summary, sponsors have typically included risk information about the product in direct-to-consumer (DTC) print ads both in the main part of the ad where the product claims appear, and in a separate brief summary page. The section of the main ad where the risks appear is often referred to as the "Important Safety Information" (ISI). Including risks in both the ISI and the brief summary may have advantages. Some research has found that repetition of information improves recall, especially for older adults (Ref. 1). This might result in improved recall for risks that appear both in the ISI and brief summary. However, it is possible that risks appearing on the main page in the ISI may be more likely to be read than risks appearing in the brief summary. Based on FDA survey research, about 27 percent of consumers surveyed in 2002 reported reading half or more of the brief summary in DTC print ads (Ref. 2). In comparison, when asked how much of the "main" ad they read, about 78 percent reported reading "all" or "almost all" of the main body portion of the ad.

One potential downside to including the same warnings in both the ISI and again in the brief summary is the potential to overwarn consumers. Overwarning is the concept that individuals are exposed to so many warnings in the course of daily life that they are less likely to pay attention to any one particular warning (Ref. 3). In terms of presenting risk information, detailing too many risks may lead consumers to discount all risks, or miss the most important risk information. Similarly, habituation follows when readers see the same warning repeatedly. Upon seeing a particular warning repeatedly, consumers may cease to pay attention to it (Refs. 4-6). Even if a warning has features that make it noticeable, it still has the potential for habituation with repeated exposure (Ref. 5). Although researchers caution against habituation and overwarning, there appears to be limited empirical research in the area of DTC advertising for prescription drugs for the logical supposition that seeing repeated warnings will lead to increased selectivity and reduced attention by recipients over time. Of note, the Office of Prescription Drug Promotion (OPDP) is studying the presentation of risk information in the context of DTC TV ads ("Disclosure Regarding Additional Risks in Direct-to-Consumer Prescription Drug Television Advertisements," OMB control number 0910-0785).

2. Purpose and Use of the Information Collection

OPDP plans to investigate, through empirical research, various combinations of the ISIand brief summary. We propose to test two levels of the ISI (short versus long) and the presence of a consumer brief summary (absent versus present) in two different medical conditions (overactive bladder (OAB) and rheumatoid arthritis). The consumer brief summary will follow the draft recommendations for language, readability, content, and format described in "Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs: Guidance for Industry, Revised Draft Guidance" (Ref. 7)). The "long" ISI is a selection of risks from the brief summary and is typical of what would appear in current DTC ads for each condition. The "short" ISI was created by applying the ideas from recent FDA work on the major statement in broadcast ads (see Refs. 8 and 9). The study design is described in part B. This will be investigated in DTC print ads for prescription drugs. FDA intends to use the results of this study to inform its approach to the communication of important risk information in DTC prescription drug print advertising.

3. Use of Improved Information Technology and Burden Reduction

Automated information technology will be used in the collection of information for this study. One hundred percent (100%) of participants will self-administer the survey via a computer, which will record responses and provide appropriate probes when needed. Eye-tracking data will be recorded by computer. In addition to its use in data collection, automated technology will be used in data reduction and analysis. Burden will be reduced by recording data on a one-time basis for each participant.

4. Efforts to Identify Duplication and Use of Similar Information

The literature revealed a good background on which to base the current research. However, we found no studies that have examined the specific issues we propose to study in DTC prescription drug print ads.

OPDP is studying the presentation of risk information in the context of DTC TV ads ("Disclosure Regarding Additional Risks in Direct-to-Consumer Prescription Drug Television Advertisements," OMB Control No. 0910-0785). We believe these studies will collectively provide FDA with valuable information about amount of risk information to be included in DTC promotion in order to ensure safe and effective use of prescription drugs.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

6. Consequences of Collecting the Information Less Frequently

The proposed data collection is one-time only. There are no plans for successive data collections.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice for public comment in the FEDERAL REGISTER of June 19, 2017 (82 FR 27842). Five public comments were received. Comments received along with our responses to the comments are provided below. For brevity, some public comments are paraphrased and therefore may not reflect the exact language used by the commenter. We assure commenters that the entirety of their comments was considered even if not fully captured by our paraphrasing in this document. The following acronyms are used here: FRN = Federal Register Notice; DTC = direct-to-consumer; FDA and the Agency = Food and Drug Administration; OPDP = FDA's Office of Prescription Drug Promotion.

Comment 1a, regulations.gov tracking number 1k1-8xet-419m (verbatim)) The research methodology that is outlined here, does not take into consideration prior exposure to ads and the fact that it is known to take about seven exposures to anything before the information sticks. Exposing the respondents to an hour-long eye-tracking research study does not take this into consideration.

(Response) We are not testing long-term retention of information. We are recruiting participants who have the medical condition of interest and may currently be under treatment. Also, Question 21 asks about familiarity with treatments for the targeted condition, which can be used as a covariate in analyses. We do not expect participants to have prior exposure to advertising for the product in the study because the ad is for a fictional product.

(Comment 1b (verbatim)) A sample of 400 is what is considered robust for comparative analysis. Although you will have enough to do some comparison with 200 respondents in each group, it would be better to increase to 400 per group.

(Response) Analysis will be conducted within medical condition. This yields a sample size within each study of 200, which will be used to examine the main effect of length of ISI, the main effect of the presence of a brief summary, and the interaction effects of the two. The sample size of 200 was determined through a power analysis using an alpha level of 0.05, a power of 0.90 and a medium effect size (f = 0.25). The power to detect a medium effect size (f = 0.25) is 0.999 given an alpha of 0.05 if the sample size for each study was increased to 400. The increase in sample size would not substantially improve our ability to detect differences.

(Comment 1c (verbatim, edited for length)) It seems like the research is front loaded to give the answer that the FDA is looking for--give less information to consumers so that they think less about the side effects of the product and buy more product. Consumers should be given all the information to make an informed choice by themselves not determined by what the FDA or other governmental organization feels is what they can handle.

(Response) Please see our responses to Comments 2i and 5a. This research is intended to develop scientific evidence to help inform policy decisions and ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. OPDP seeks to ensure that prescription drug promotional materials provide truthful, balanced and accurately communicated information that helps patients make informed decisions about their treatment options. In each study, the ads will all include the same risk concepts and we will measure comprehension of these risks. We will vary the amount of detail about each risk concept in the ISI section of the ad and we will test the effects of repeating information across the ISI and the consumer brief summary.

(Comment 2a, regulations.gov tracking number 1k1-8xz7-z732 (verbatim)) Do the exclusion criteria adequately account for all potential subjects that have vision impairments that can affect how their eyes move as they read? Additional exclusions may be needed to address these (e.g. blindness in one eye, artificial eye, etc.).

(Response) The study design currently calls for excluding potential participants with vision impairments that interfere with the capabilities of the eye-tracking glasses. This includes wearing regular glasses, bifocals, trifocals, progressive lenses, hard contact lenses, and colored contact lenses. We will also add exclusion criteria for potential participants who have cataracts, amblyopia (lazy eye/blind in one eye), strabismus (cross-eyed), mydriasis (permanent pupil dilation), nystagmus (involuntary eye movements), an ocular prosthesis (glass eye), and who are designated as legally blind.

(Comment 2b (verbatim)) Consider adding an arm to the design that shows an ad without any specific risk content or a brief summary, but alternatively consists of a statement that informs a potential patient that the drug in question has risks, including serious risks, associated with its use, and that it is very important that a patient talk with his/her doctor about these risks, prior to use, to determine if the drug is appropriate for the patient. It would be interesting to see what type of recall and what type of eye movement data would occur for this type of statement.

(Response) FDA regulations state that prescription drug advertisements must contain "a true statement of information in brief summary relating to side effects, contraindications (…[to] include side effects, warnings, precautions, and contraindications and include any such information under such headings as cautions, special considerations, important notes, etc.) and effectiveness" (§ 202.1(e)(1)). Additionally, advertisements must also "present a fair balance between information relating to side effects and contraindications and . . . effectiveness…." (§ 202.1(e)(5)(ii)). We decline the suggestion to test the proposed statement at this time.

(Comment 2c (verbatim)) Question 1: The relevance of asking a subject to assess how many risks are presented in comparison to how many benefits is not apparent. We recommend that FDA consider deleting the question or alternatively rewording it to get data on how many risks the subjects think are presented in the ad. Response options should be quantitative, such as: No risks, 1-3 risks, 4-6 risks, > 6 risks.

(Response) The purpose of Question 1 is to assess participants' initial impressions of balance of risks versus benefits in the ad. Additionally, Question 4 has been revised based on the results of cognitive testing to collect risks that participants can recall. This provides both a quantitative measure and an accuracy evaluation. We believe this approach will yield richer data as far as how many risks the participant recalls from the ad.

(Comment 2d (verbatim)) Question 4: If subjects are going to be asked to recall, using free text, the risks presented in the ad, it would similarly be interesting to add a similar question to recall, using free text, which benefits were presented in the ad.

(Response) The questionnaire contains several questions about benefit/efficacy (Questions 3, 10, and 11). We also have questions that measure the perceived risk/benefit tradeoff (Questions 1, 18, and 19). Although it would be interesting from a conceptual standpoint to include an open-ended recall question about product benefits, our focus in this study is on the risk information. Further, we are concerned about adding length to the questionnaire as we have worked to minimize the burden of the collection of information on respondents.

(Comment 2e (verbatim)) Questions 8, 10, and 11: Suggest rewording the questions so that they describe the likelihood that a person taking the drug experiences a side effect or a benefit.

(Response) The items used in this section were developed through scale validation research. Thus, we prefer to retain them in their original form.

(Comment 2f (verbatim)): Questions 12-15: It may be confusing for the reader to discern differences between the terms "main ad", "page following the main ad", and "advertisement". These terms might need to be accompanied by further explanatory text.

(Response) Cognitive testing revealed participants did have difficulty discerning the differences in the ad components based on the descriptive terms provided. To address this problem and help with data quality, thumbnail images will be provided next to Questions 13-15, so that participants will have a visual cue of what portion of the ad the question is asking about without allowing them to re-read the ad stimulus.

(Comment 2g (verbatim)) Questions 16 and 17: Randomize the order in which the personal involvement adjectives/tasks are presented to minimize bias.

(Response) Question 16 is The Personal Involvement Inventory, a validated measure with high internal consistency (coefficient α = .88) and has been used in prior studies to provide useful information about personal relevance (Refs. 10 and 11). The author of the inventory confirmed that it was developed and has been administered without randomization of these items. For the current study, values across items will be averaged in order to produce an overall personal involvement score for comparison across participants. Since this question is a validated measure and will be used only as a moderator variable, the item order will not change. Question 17 is a measure of self-efficacy, which will serve as an additional outcome of interest. We will randomize Question 17.

(Comment 2h (verbatim)) Question 18: It is not clear what the term "leave" means. It may mean "take time off from work." Please clarify.

(Response) Question 18 was developed through scale validation research. "Leave" does in fact mean "take time off from work." We did not encounter any confusion on the part of respondents during cognitive testing of the questionnaire. We prefer to retain this question in its original form.

(Comment 2i (verbatim)) Question 19: A consumer should not be expected to make a risk/benefit assessment of a drug simply by reading an ad. Such an assessment can occur only after a patient has had a discussion with his/her healthcare provider. Thus, we suggest deletion of this question.

(Response) An important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in their health care decision-making. FDA seeks to improve our understanding of what baseline judgements about product risks and benefits individuals make on the basis of advertising. Question 19 does not indicate that FDA expects that the advertisement will be the sole basis for individuals to assess benefit and risk or make ultimate healthcare decisions. Rather, Question 19, which was developed through scale validation research, measures one aspect of the consumer's perception of the drug's risk-benefit tradeoff. Further, we did not encounter any confusion on the part of respondents during cognitive testing of the questionnaire.

(Comment 2j (verbatim)) Questions 28-33: We note these questions assess the ability of the respondent to answer questions using an ice cream nutrition facts label. We assume the inclusion of these questions is to assess how well respondents are capable of comprehending complex numeric information. However, we note that some respondents may not be able to comprehend and apply numeric information or be motivated to do so, regardless of how it appears. The format may not matter when this is the case. Therefore, we suggest that FDA consider analyzing results based on those who can vs. cannot answer the ice cream questions. Alternatively, the ice cream questions could be used at the start of the survey to screen out those who are unable to answer the questions, thereby further focusing the sample on persons who are able to comprehend numeric presentations likely to be found in drug promotion.

(Response) Questions 28-33 make up the Newest Vital Sign (NVS), developed by Pfizer. (See https://www.pfizer.com/health/literacy/public-policy-researchers/nvs-toolkit). The NVS is a valid and reliable measure of health literacy and numeracy that was used and recommended by two studies (Refs. 12 and 13). In this study, the NVS will be used as a covariate that measures risk of low health literacy/numeracy. It is important that potential participants of various health literacy levels are included, because level of health literacy/numeracy of the individual has been shown to play a particularly strong role in viewing and processing health information (Ref. 14). For the stated reasons, no change to the analysis or use of the questions to filter the sample of participants is planned.

(Comment 3a, regulations.gov tracking number 1k1-8y5u-ecif (verbatim)) One omitted variable in the study design is recall after viewing the ad and ISI/brief summary. It would seem potential negative effects of overwarning and habituation would be even more apparent after a lapse of time. The commenter suggests incorporating a parameter to capture this, for example, including a re-contact option to test recall and interpretation after a period of 2-4 days. For this recall option, we suggest that a quota of -30 respondents per cell in order to ensure a robust sample for statistical testing.

(Response) Question 4 captures open-ended recall of risks and negative effects. The comment proposes an interesting research idea. However, testing long-term retention of information is beyond the scope of this study.

(Comment 3b (summarized)) The commenter suggests ensuring a representative sample of respondents with the conditions of interest is collected (~ 30 per cell). Analysis of these respondents compared to those without the conditions would act as a control.

(Response) The study design calls for only including individuals who have the medical condition targeted for each study. This is based on the rationale that, relative to the general population, individuals who suffer from a specific medical condition pay more attention to DTC ads related to that medical condition (Refs. 15 -17). Thus, we do not plan to add a general population sample.

(Comment 3c (verbatim)) Neither the full stimuli nor specific examples of the disclosure language were provided. The lack of access to these makes full interpretation of the study objectives difficult as well as leaves us unable to provide suggestions or comments on the stimuli to be tested.

(Response) We have described the purpose of the study, the design, the population of interest, and have provided the questionnaire to numerous individuals upon request. The brief summary for each ad contains a summary of the product risks, side effects, and contraindications. The "long" ISI is a selection of risks from the brief summary and is typical of what would appear in current DTC ads for each condition. The "short" ISI was created by applying the ideas from recent FDA work on the major statement in broadcast ads (see Refs. 16 and 17). Our full stimuli are under development during the PRA process. We do not make draft stimuli public during this time because of concerns that this may contaminate our participant pool and compromise the research.

(Comment 3d (summarized)) The commenter suggests that the data and information collected with eye-tracking be used as secondary evidence of attention. This is due to both difficulty of interpretation inherent in eye-tracking data along with subjectivity introduced by the ad copy stimuli under examination, as stimuli can be manipulated to increase/decrease attractiveness to a respondents' eye. The commenter believes these limitations make use of this data to direct policy difficult. Additionally, the briefing document does not expand upon exactly how the eye-tracking data will be analyzed other than tracking attention. There are various ways to analyze eye-tracking data, such as order of attention, number of multiple viewings, and possibly pupil dilation as a measure of attention. The commenter has traditionally added qualitative elements to its use of eye-tracking technology in research, by discussing what the respondent saw after viewing the stimuli and even reviewing a respondents' eye-tracking map with them to get further insights.

(Response) To clarify, two types of data will be collected in each study. Both data types are considered useful evidence. Self-report measures will be collected via a web-based questionnaire, and physical measures of attention will be collected via eye-tracking glasses. Existing research has relied on self-report measures to determine how much and what parts of the risk and benefit information consumers are reading. Because of the known unreliability of self-report measures (Ref. 18), research is needed to accurately determine what and how much consumers are reading when they see risk and benefit statements in prescription drug ads.

During the debriefings for the pilot study, respondents will be shown their eye-gaze data and asked to comment on the elements of the stimuli they attended to, the elements they did not attend to, and why. These data in aggregate form will be reviewed to determine whether to modify the stimuli prior to the main studies. Eye-tracking data (both heat maps and gaze plots) will be used in the analyses to identify general patterns across participants and to investigate how those relate to questionnaire measures.

(Comment 3e (verbatim)) The FRN states the location of risk information is also an objective of the study. The commenter assumes this "location" testing will be via testing risk information communicated in stimuli having the ISI plus the Brief Summary against stimuli having the ISI only. If this is inaccurate, then we are not sure the study design as described in the FRN adequately tests for a variable of "location." If varying location of risk information beyond ISI versus ISI + Brief Summary is desired, the commenter suggests this be tested in a subsequent study or that the proposed study better specify variation of "location."

(Response) The commenter has correctly interpreted the study design. We are not manipulating where the information appears on the page. Location, as used here, refers to the presence of information in both the brief summary and the ISI, or just the ISI. Within each medical condition, we have endeavored to maintain consistency of where the information appears on the page, and the order of the information, across experimental conditions.

(Comment 3f (summarized)) Through the survey, the commenter suggests maintaining a single scale for all rating questions. For example, the commenter generally employs a 5-point scale, which includes a midpoint, and is defined at each point. In the current questionnaire, the scales switch from 5-point to 6-point scales which could cause confusion among some respondents. If the 6-point scales are included explicitly to omit a neutral mid-point, the commenter suggests that each of the points are defined to ensure that respondents know what the point on the scale they are choosing means (similarly to what is provided in Question 20 onwards).

(Response) Many of the items used in the survey were developed through scale validation research (i.e., Questions 8-11, 18, and 19). These items utilize a six-point scale, so we have attempted to use six-point scales where possible. In other cases, however, we are using items that have been used in prior FDA studies (i.e., Questions 1 and 24) or are established measurement inventories (Question 16 is the Personal Involvement Inventory; Ref. 11). Changing the scale range or altering the scale to add definitions to each scale point would preclude comparison with prior study results. Thus, we prefer to maintain the scale ranges currently in use.

(Comment 3g (summarized)) For Questions 8-11, the commenter suggests adding a "Don't know" option as respondents might not be able to assess likelihood of side effects, seriousness of side effects, efficacy, and potential improvement based on the information presented in the ad. The current range of answer choices may force inaccurate or speculative responses; a "Don't Know" answer would be a legitimate choice and informative for the study. The commenter's standard practice is to provide a "Don't Know" option whenever it could be a valid answer.

(Response) The items used in this section were developed through scale validation research. Thus, we prefer to retain them in their original form, for this study, though we will consider this for future measurement studies.

(Comment 3h (verbatim)) For Q[uestion]12, without ability to review the stimuli, it is unclear what content will appear in Area A, B, C and D. It is also unclear whether the content will be the same across all 4 stimuli ads or whether content will change location in the ad.

(Response) We have endeavored to maintain consistency of information location across conditions. Area A is the part of the ad with a picture. Areas B, C, and D are all sections of the ISI.

(Comment 3i (summarized)) The commenter wonders what the utility of asking Question 16 is as the question appears to be out of scope with the objectives of the study. Whether or not the ad is important, boring, or relevant to the respondent seems irrelevant to the stated goals. We suggest removing the question.

(Response) Please see our response to Comment 2g.

(Comment 3j (summarized) In Question 18, the inclusion of "… outweigh all the things I have to do to obtain it (appointments, prescriptions, leave)" seems out of scope when considering the objectives of the study. The commenter suggests removing the question.

(Response) This question measures one aspect of product benefits, the benefit-inconvenience tradeoff, which is an important component of drug product perceptions. Additionally, please see our response to Comment 2h.

(Comment 3k (summarized)) For Question 19, the commenter suggests a minor adjustment to the wording. Instead of saying "The benefits of [DRUG NAME] outweigh any side effects it may have", the commenter suggests saying "… any side effects it is described/indicated as having". "May have" could be interpreted subjectively by respondents to include side effects not in the ISI and brief summary.

(Response) Question 19 is a validated question so it will be retained as is. Cognitive testing revealed no comprehension or reporting issues for this question.

(Comment 3l (verbatim)) For Q[uestions]22-23 pertaining to respondent perception of condition. There does not appear to be any skip logic to ensure that only those with one of the specified conditions can answer those questions. These questions should not be asked of those who do not suffer from one of the specified conditions.

(Response) We intend to recruit individuals who self-identify as having either OAB or rheumatoid arthritis. Those individuals will be assigned to view an ad that treats their medical condition. The questionnaire will contain questions relevant to that medical condition only.

(Comment 4a, regulations.gov tracking number 1k1-8y4d-os71 (summarized)) The commenter recommends that greater emphasis be placed on the recall/questionnaire metric rather than the eye-tracking metric. The eye-tracking data will determine if there is indeed a direct correlation between the length (amount) of the risk information and length of time spent looking at that information; however, it will not differentiate between what content and format is more effective for communicating that risk information. The commenter suggests that FDA include in the questionnaire (and/or debriefing interview) specific inquiries regarding the repetitiveness of the risk information in order to further explore the link between the amount and placement of risk information and the ultimate recall of this information.

(Response) Please see our response to Comment 3d. In addition, we will add a question regarding repetitiveness to the questionnaire.

(Comment 4b (summarized)) The commenter believes it is important that the fictitious drugs in this study have safety profiles reflecting the complex safety profiles of actual, currently-approved and promoted products.

(Response) The DTC ads to be used in this research were developed using actual ads for these medical conditions. Additionally, we consulted with expert reviewers in OPDP on content and format to ensure the stimuli are realistic.

(Comment 4c (summarized)) The "short" versus "long" ISI should be defined explicitly. The commenter believes it is critical to know the specific ISI content ("short" and "long") in order to fully understand the study results. Additionally, OPDP examples of adequate "short" and "long" ISIs used in the context of print ads would be valuable templates for industry, especially given the lack of consensus in acceptable utilization of "short" iterations of ISI as observed in past OPDP advisory comments and Warning Letters.

(Response) Please see our responses to comments 1c and 3c. This study is not intended to provide specific guidelines on what content should be included in the ISI.

(Comment 4d (summarized)) The commenter proposes the content of the brief summary be stated as well so as to understand what risk information is repeated from the ISI and what impact this may have on the study results.

(Response) We have described how the consumer brief summary will be constructed in the Background section. Please see our responses to Comment 1c and 3c.

(Comment 4e (summarized)) The commenter questions the utility of including the control, consumer product ad if the eye-tracking data is not utilized. FDA should clarify if the questionnaire will assess the recall of the control ad. The commenter recommends FDA fully evaluate the data from the control ad in order to provide appropriate context for the results obtained from the study, health-related ads.

(Response) The purpose of participants viewing the consumer product ad, otherwise known as the warm-up ad, is to orient them to the ad-viewing task. In addition, the warm-up ad permits the research team to do an initial review and adjustment of the eye-tracking equipment as needed before the study task begins. Therefore, there is no plan to analyze the warm-up ad data as it is not relevant to the focus of the study, and is mainly a procedure to orient the participant to the eye-tracking task.

(Comment 5a, regulations.gov tracking number 1k1-8y60-6g3m (summarized)) The commenter is concerned with the Agency's recent approaches to studies in this area. FDA has proposed to undertake projects in a variety of disparate topics without articulating a clear, overarching research agenda or adequate rationales on how the proposed research related to the goal of further protecting public health. Within the last year, the Agency has increased such efforts at an exponential pace. At times, FDA proposes new studies seemingly without fully appreciating its own previous research published on the OPDP website. Proposed studies are often unnecessary in light of existing data. The commenter suggests that the Agency publish a comprehensive list of its prescription drug advertising and promotion studies from the past 5 years and articulate a clear vision for its research priorities for the near future. Going forward, FDA should use such priorities to explain the necessity and utility of its proposed research and should provide a reasonable rationale for the proposed research.

(Response) OPDP's mission is to protect the public health by helping to ensure that prescription drug information is truthful, balanced, and accurately communicated, so that patients and health care providers can make informed decisions about treatment options. OPDP's research program supports this mission by providing scientific evidence to help ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. Toward that end, we have consistently conducted research to evaluate the aspects of prescription drug promotion that we believe are most central to our mission, focusing in particular on three main topic areas: advertising features, including content and format; target populations; and research quality. Through the evaluation of advertising features we assess how elements such as graphics, format, and disease and product characteristics impact the communication and understanding of prescription drug risks and benefits; focusing on target populations allows us to evaluate how understanding of prescription drug risks and benefits may vary as a function of audience; and our focus on research quality aims at maximizing the quality of research data through analytical methodology development and investigation of sampling and response issues.

Because we recognize the strength of data and the confidence in the robust nature of the findings is improved through the results of multiple converging studies, we continue to develop evidence to inform our thinking. We evaluate the results from our studies within the broader context of research and findings from other sources, and this larger body of knowledge collectively informs our policies as well as our research program. Our research is documented on our homepage, which can be found at: https://www.fda.gov/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cder/ucm090276.htm. The website includes links to the latest FRNs and peer-reviewed publications produced by our office. The website maintains information on studies we have conducted, dating back to a survey of DTC attitudes and behaviors conducted in 1999.

(Comment 5b (The commenter provided a summary of their comments followed by a more detailed description of the same comments. For brevity, the summary of comments has been omitted and only the specific comments [5b through 5t] are provided below. The commenter’s full comments may be accessed at regulations.gov via tracking number 1k1-8y13-m7td) (summarized)) The PRA Notice states there has been little empirical research for the logical supposition that seeing repeated warnings will lead to increased selectivity and reduced attention. This is not correct. As some authors have commented, "[h]abituation has been found in a variety [of] contexts and domains." The commenter is aware of at least three empirical research studies, none cited in the PRA Notice, that demonstrate the "habituation effect is a robust phenomenon." This effect has been documented in "studies involving different contexts and response measures."

(Response) We thank the commenter for pointing out this mischaracterization. We have revised our introduction to clarify that whereas there is an overall body of research relating to habituation, there is limited, if any, research on habituation in the specific context of DTC print advertising for prescription drugs.

(Comment 5c (summarized)) FDA should clarify whether the proposed study will adopt the brief summary format outlined in "Guidance for Industry--Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs" (Draft Guidance).

(Response) We plan to utilize the Question and Answer consumer-friendly format described in the referenced draft guidance.

(Comment 5d (summarized)) The commenter requests that the Agency make available for public comment the study stimuli, including the non-study ad for a consumer product unrelated to health. In particular, the commenter wishes to provide comments on: (1) what constitutes "short" and "long" length for the ISI and (2) the content, format, and design of the Brief Summary.

(Response) Please see our responses to Comments 1c, 3c, 4c and 4e.

(Comment 5e (summarized)) The Agency proposes to use eye tracking technology "to determine how risk presentations in DTC print ads are perceived." The commenter encourages the Agency to use this technology in conjunction with other inputs (for example, qualitative research) to understand why subjects are looking at a portion of the proposed materials, rather than to draw conclusions that such portions were viewed. Additionally, an explanation of the use of eye tracking technology should also be included during the subject enrollment process.

(Response) FDA plans to collect and analyze eye-tracking (physical measures of attention) data in conjunction with other measures, including self-report measures of attention, recall, and comprehension. The recall measures will be collected via qualitative (open-ended) questions. To avoid the potential for priming effects, the goals of the eye-tracking component of the study will not be explained to recruited individuals before they report for their in-person sessions. However, participants will be made aware of the eye-tracking component during the informed consent process. Please also see our response to Comment 3d.

(Comment 5f (summarized)) Recall Questions. FDA should capture whether subjects comprehend that there are side effects and negative outcomes, even if the subject does not recall information on the specifics. The commenter suggests adding a question concerning whether subjects were aided in the recall of information by the "short" or "long" ISI format.

(Response) Questions 4a-c capture recall of risk in an open-ended format. Our approach involves random assignment to experimental conditions; each participant will see only one version of the stimuli. Because participants will not be aware there is another, different format, asking them their impressions of the long versus the short format is not feasible.

(Comment 5g (verbatim)) Recall questions (e.g., Q[uestion]4) ask test subjects to identify specific side effects and negative outcomes of the featured drug products. It is not clear why such questions are necessary for the research purpose of the study.

(Response) An important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in their health care decision-making. In this study, we are investigating how different presentations of risk information impact perception and comprehension of drug risks and benefits. These questions are designed to provide information to help us identify effective ways to communicate risk and benefit information in DTC advertising. See our response to Comment 2b for additional context.

(Comment 5h (verbatim)) The questionnaires do not define certain key terms (e.g., risk, side effect). Subjects may interpret these terms based on different standards. FDA might consider providing user-friendly definitions.

(Response) We appreciate the importance of ensuring uniform interpretation of terms. In cognitive interviews preceding this work, we assessed whether individuals interpret key terms similarly and made revisions where necessary. We have also considered the additional time (burden) that would be required to complete the survey if every term were defined in the pilot and main study. With these factors in mind, we have chosen not to provide additional definitions.

(Comment 5i (summarized)) The commenter recommends that: (1) FDA replace the phrase "negative outcomes" with "risks and warnings" and (2) insert "possible" before the phrase "side effects."

(Response) We have deleted "negative outcomes" from the question wording in Question 2 and Question 4b. Also, please see our response to Comment 3g concerning the proposal to reword the previously validated question.

(Comment 5j (verbatim)) The Agency should consider changing the sliding scale to an odd number system to permit a "neutral" response. Most questions (e.g., Q[uestion]2-Q3, Q8-Q11) provide six choices, not permitting a neutral response.

(Response) Please see our response to Comment 3f.

(Comment 5k (verbatim)) FDA should reconsider the inclusion of the perceived efficacy likelihood (Q[uestion]10) and perceived efficacy magnitude (Q11) questions. It is not apparent what utility these specific questions have in the context of the study.

(Response) We note that this comment is the opposite of Comment 2d, which suggests adding recall questions about product benefits. Although the main focus of this research is on the risk information, an important purpose of communicating the drug's specific risk and benefit information in DTC advertising is to position consumers as active and well-informed participants in health care decision-making. These questions will allow us to assess the impact of our study variables on perception and comprehension of drug benefits.

(Comment 5l (summarized)) The commenter supports a study design that includes an analysis of whether the inclusion of the brief summary, along with a short or long ISI, presents duplicative information to the user, and therefore, introduces overwarning.

(Response) We thank the commenter for their support of research. We reiterate that the purpose of the study is to examine how various means of presenting risk information impact consumer comprehension and perceptions of product information.

(Comment 5m (verbatim)) FDA states that it will conduct the studies in person in at least five different cities across the United States. The Agency should address what efforts it will take to avoid enrichment of the sample population when selecting cities.

(Response) We interpret the commenter's request for FDA to address how it will "avoid enrichment of the sample population when selecting cities" to mean that FDA should address how it will avoid collecting data in cities where the medical conditions are more prevalent than in other cities. This is not the aim of collecting data in five different cities. Rather, the cities have been selected to represent metropolitan areas in various geographic areas of the United States, including the West, Southwest, Midwest, Southeast, and the mid-Atlantic. These locations include Chicago, IL, Tampa, FL, Phoenix, AZ, Houston, TX, and Marlton, NJ. Due to the low population prevalence rate of the two medical conditions and the need to conduct sessions with 40 individuals with the condition in each of five areas, testing in rural areas is not feasible.

(Comment 5n (summarized)) Study participants diagnosed with one of the medical conditions of interest may be more prone to pay attention and read information concerning prescription drugs for these conditions. Additionally, the study setting may prompt participants to pay closer attention to stimuli. FDA should clarify how it plans to limit such response biases.

(Response) The study method randomly assigns each participant to an experimental condition, ensuring that potential pre-existing biases will be evenly distributed across the conditions. The only aspect of the participants' experiences that will be varied in the study will be the manipulations that we have described. Thus, given the experimental design of the study, if we find differences between and among conditions, we can be reasonably sure that the manipulations caused the differences. Similarly, any individual differences in attention or ability should be spread across experimental conditions. We have not found in the past that our participants spend an inordinate amount of time viewing stimuli, but we will be careful to place the research in context when we interpret the data.

(Comment 5o (verbatim)) An "FDA employee" category, similar to S6 and S7, should be added to the Screener Survey. These individuals should also be terminated from the study.

(Response) We have added a category to exclude employees of HHS, which includes employees of FDA.

(Comment 5p (verbatim): S2 and S3 of the Screener Survey should be rewritten as follows:

"Has a doctor or other health care professional ever diagnosed you with overactive bladder (OAB)?"

"Has a doctor or other health care professional ever diagnosed you with rheumatoid arthritis (RA)?"

(Response) We will leave the wording of the screener questions S2 and S3 as is. Cognitive testing results in various contexts have indicated comprehension and reporting errors associated with using the more formal phrase "…diagnosed you with…[condition]." Common practice is to use the wording "…ever told you…."

(Comment 5q (verbatim)) Q[uestion]16 of the Questionnaire and P1 of the Pilot Study should be deleted. Whether a subject considers the study stimuli to be "Exciting/Unexciting" or "Boring/Interesting" or whether the subject "likes" the study stimuli has no apparent relevance to FDA's study goals.

(Response) Please see our response to Comment 2g.

(Comment 5r (verbatim)) Q[uestion]12-Q17 should be the first questions of the Questionnaire. A subject will likely answer these questions most accurately immediately after reviewing the study stimuli and before answering other questions that could influence these answers.

(Response) FDA agrees that it is important to position certain questions where they will be answered in close proximity to the ad-viewing time, which may improve reporting accuracy. However, the decision was to place the questions that assess recall and recognition of risks (Questions 4-7) earliest in the question sequence, so as to minimize memory decay and contamination of responses by exposure to questions covering other constructs (risk likelihood, risk magnitude). The attention (Question 12) and ad reading (Questions 13-15) measures will be retained in their current order (in the first half of the questionnaire).

(Comment 5s (verbatim)) Q[uestion]18 should include considerations for prescription drug access.

(Response) Please see our response to Comment 2h.

(Comment 5t (summarized)) It is unclear how FDA plans to utilize the non-study ad (related to ice cream). Q[uestions]27-32 appear very different in nature, substance, purpose, format, and length than the questions concerning the drug ad. FDA should clearly articulate the purpose of this stimulus and how it will be used in analyzing study results (if at all). If the sole purpose is to "stimulate normal ad viewing," the commenter encourages adding another one to two non-study ads.

(Response) The comment suggests that the nutrition facts label was interpreted as the "non-study ad." That is not the case. The ice cream nutrition facts label and accompanying questions (Questions 27-33) are included in the questionnaire as skills-based measures of health literacy and numeracy and have been adapted for self-administration in these studies. Please see our response to Comment 2j.

External Reviewers

In addition to public comment, OPDP solicited peer-review comments from researchers in fields relevant to the communication of DTC prescription drug information. We received responses and incorporated the thoughts of the following individuals:

Angie Fagerlin, PhD

Professor and Chair

Department of Population Health Sciences

University of Utah

Research Scientist, Salt Lake City VA Center for Informatics Decision Enhancement and Surveillance (IDEAS)

President, Society For Medical Decision Making

Angie.fagerlin@hsc.utah.edu

Ph: 801-587-0049

Christopher B. Mayhorn, Ph.D.

Professor and Head

Department of Psychology   

North Carolina State University 

cbmayhor@ncsu.edu

Ph: 919-513-4856

Michael J. Kalsher, Ph.D.

Associate Professor, Department of Cognitive Science

Associate Dean for Academic Affairs

School of Humanities, Arts, and Social Sciences

Rensselaer Polytechnic Institute

kalshm@rpi.edu

Ph: 518-276-8267

9. Explanation of Any Payment or Gift to Respondents

After completing the sessions, participants will receive an incentive as a token of appreciation for participating. As participants often have competing demands for their time, incentives are used to encourage participation in research. When applied in a reasonable manner, incentives are not coercive but rather they serve as an approach that acknowledges respondents for their participation (Ref. 19). The use of incentives treats participants justly and with respect by recognizing and acknowledging the effort that they expend to participate. In this particular research study, we are asking participants to provide feedback on concepts that require a high level of engagement.

Incentives must be high enough to equalize the burden placed on respondents with respect to their time and cost of participation (Ref. 20), as well as provide enough motivation for them to participate in the study rather than another activity. If the incentive is not adequate, participants may agree to attend a session, but then drop out early (before completing the survey). An additional consideration for use of incentives is the potential increased cost due to low participation. Low participation can cause a difficult and lengthy recruitment process that, in turn, can cause delays in launching the research, which leads to increased costs.

Participants will receive an incentive of $50. These incentives reflect the increased amount of time—including travel time—that participants must devote to the study. In addition to increased time burden, eye-tracking studies place an increased cognitive burden on participants, further justifying the incentive amounts.

10. Assurance of Confidentiality Provided to Respondents

All data will be collected with an assurance that the participants’ responses will remain private to the extent allowable by law. The consent form (see Appendix C) contains a statement emphasizing that no one will be identified by name, and each respondent’s name will not be linked to his/her responses. Researchers will not tie respondents’ personally identifiable information (PII) to their answers. All analyses will be done in the aggregate, and respondent information will not be appended to the data file used. Further, no identifying information will be included in the data files delivered by Westat to FDA.

The following procedures will be used to ensure participant confidentiality before, during, and after fielding: (1) all screening-related information will not be tied to any PII but will be identified and matched by the assigned unique ID; (2) data sets and reports will not contain any PII; and (3) respondents will not be tied to their individual responses, and all analyses will be conducted in the aggregate (i.e., any data used in reporting will not be attributed to specific participants). Video recordings made during the pretest sessions may be reviewed in order to identify any procedural changes to make before the main studies, but only by project staff. Recordings will be destroyed within 6 weeks of the end of the study.

Any data sets and reports delivered to FDA will not include PII. All identifying information will be kept on a separate password-protected computer and/or in locked cabinets for a period of three years after the conclusion of the project and will only be accessible by Westat. After three years, Westat will destroy the information by securely shredding documents or permanently deleting electronic information. In the case of a breach of confidentiality, appropriate steps will be taken to notify participants.

All data will also be maintained in consistency with the FDA Privacy Act System of Records #09-10-0009 (Special Studies and Surveys on FDA Regulated Products).

11. Justification for Sensitive Questions

This data collection will not include sensitive questions. The complete list of questions is available in Appendix B.

12. Estimates of Annualized Burden Hours and Costs

12a. Annualized Hour Burden Estimate

For both the pilot and main study, the questionnaire is expected to last no more than 60 minutes. This will be a one-time (rather than annual) collection of information. FDA estimates the burden of this collection of information as follows:

Table 3: Estimated Annual Reporting Burden¹

Table 1.--Estimated Annual Reporting Burden1
Activity	No. of Respondents	No. of Responses per Respondent	Total Annual Responses	Average Burden per Response	Total Hours
Pilot Screener	120	1	120	.03 (2 minutes)	4
Study 1 Screener	600	1	600	.03 (2 minutes)	18
Study 2 Screener	600	1	600	.03 (2 minutes)	18
Completes, Pilot	40	1	40	1	40
Completes, Study 1	200	1	200	1	200
Completes, Study 2	200	1	200	1	200
Total					480

¹ There are no capital costs or operating and maintenance costs associated with this collection of information.

These estimates are based on FDA’s and the contractor’s experience with previous consumer eye-tracking studies.

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

There are no capital, start-up, operating or maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

The total estimated cost to the Federal Government for the collection of data is $798,027 ($266,009 per year for three years). This includes the costs paid to the contractors to create the stimuli, program the study, draw the sample, collect the data, and create and analyze a database of the results. The contract was awarded as a result of competition. Specific cost information other than the award amount is proprietary to the contractor and is not public information. The cost also includes FDA staff time to design and manage the study, to analyze the resultant data, and to draft a report ($97,000; 8 hours per week for three years).

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Conventional statistical techniques for experimental data, such as descriptive statistics, analysis of variance, and regression models, will be used to analyze the data. See Part B for detailed information on the design, hypotheses, and analysis plan. The Agency anticipates disseminating the results of the study after the final analyses of the data are completed, reviewed, and cleared. The exact timing and nature of any such dissemination has not been determined, but may include presentations at trade and academic conferences, publications, articles, and Internet posting.

Table 2. – Project Time Schedule
Task	Estimated Number of Weeks after OMB Approval
Pilot completed	32 weeks
Main study data collected	80 weeks
Final methods report completed	91 weeks
Final results report completed	95 weeks
Manuscript submitted for internal review	111 weeks
Manuscript submitted for peer-review journal publication	125 weeks

17. Reason(s) Display of OMB Expiration Date is Inappropriate

No exemption is requested.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

References

1. McGuire, L. C., "Remembering What the Doctor Said: Organization and Older Adults' Memory for Medical Information." Experimental Aging Research, vol. 22, pp. 403-428, 1996.

2. Aikin, K. J., J. L. Swasy, and A. C. Braman, "Patient and Physician Attitudes and Behaviors Associated with DTC Promotion of Prescription Drugs: Summary of FDA Survey Research Results" (2004). Available at https://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDER/ucm151498.htm.

3. Warnings and Risk Communication (2005). Wogalter, M. S., D. M. DeJoy, and K. R. Laughery (Eds.). Philadelphia: Taylor & Francis, Inc.

4. Conzola, V. C. and M. S. Wogalter, "A Communication-Human Information Processing (C-HIP) Approach to Warning Effectiveness in the Workplace." Journal of Risk Research, vol. 4(4), pp. 309-322, 2001.

5. Wogalter, M. S. and K. R. Laughery, "Warning! Sign and Label Effectiveness." Current

Directions in Psychological Science, vol. 5(2), pp. 33-37, 1996.

6. Wogalter, M. S., T. L. Smith-Jackson, B. J. Mills, and C. S. Paine, "The Effects of Print Format in Direct-to-Consumer Prescription Drug Advertisements on Risk Knowledge and Preference." Drug Information Journal, vol. 36(3), pp. 693-705, 2002.

7. Brief Summary and Adequate Directions for Use: Disclosing Risk Information in Consumer-Directed Print Advertisements and Promotional Labeling for Prescription Drugs. Revised Draft Guidance. Available at https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM069984.pdf.

8. "Content of Risk Information in the Major Statement in Prescription Drug Direct-to-Consumer Broadcast Advertisements; Establishment of a Public Docket; Request for Information and Comments." August 21, 2017, 82 FR 39598.

9. Betts, Kevin R., et al., "Serious and Actionable Risks, Plus Disclosure: Investigating an Alternative Approach for Presenting Risk Information in Prescription Drug Television Advertisements." Research in Social and Administrative Pharmacy, 2017.

10. Bhutada, N. S., B. L. Rollins, and M. Perri III, "Impact of Animated Spokes-Characters in Print Direct-to-Consumer Prescription Drug Advertising: An Elaboration Likelihood Model Approach." Health Communication, vol. 32, pp. 391-400, 2017.

11. Zaichkowsky, J. L., "The Personal Involvement inventory: Reduction, Revision, and Application to Advertising." Journal of Advertising, vol. 23, pp. 59-70, 1994.

12. Mackert, M., S. E. Champlin, K. E. Pasch, and B. D. Weiss, "Understanding Health Literacy Measurement Through Eye Tracking." Journal of Health Communication, vol. 18, pp. 185-196, 2013.

13. Chiang, K. P. and A. Jackson, "The Impact of Health Literacy on Involvement and Attitude Toward Pharmaceutical Print Ads." International Journal of Healthcare Management, vol. 9(1), pp. 47-57, 2016.

14. An, S. and N. Muturi, "Subjective Health Literacy and Older Adults' Assessment of Direct-to Consumer Prescription Drug Ads." Journal of Health Communication, vol. 16(3), pp. 242-255, 2011.

15. Ball, J. G., D. Manika, and P. Stout, "Consumers Young and Old: Segmenting the Target Markets for Direct-to-Consumer Prescription Drug Advertising." Health Marketing Quarterly, vol. 28(4), pp. 337-353, 2011.

16. Christensen, T. P., F. J. Ascione, and R. P. Bagozzi, "Understanding How Elderly Patients Process Drug Information: A Test of a Theory of Information Processing." Pharmaceutical Research, vol. 14, pp. 1589-1596, 1997.

17. Mehta, A. and S. C. Purvis, "Consumer Response to Print Prescription Drug Advertising." Journal of Advertising Research, vol. 43(2), pp. 194-206, 2003.

18. Paulhus, D. L. and S. Vazire, "The Self-Report Method." Handbook of Research Methods in Personality Psychology, vol. 1, pp. 224-239, 2007.

19. Halpen, S. D., J.H. Karlawish, D. Casarett, J.A. Berlin, and D.A. Asch, "Empirical Assessment of Whether Moderate Payments are Undue or Unjust Inducements for Participation in Clinical Trials." Archives of Internal Medicine, vol. 164, pp. 801–803, 2004.

20. Russell, M. L., D.G. Moralejo, and E.D. Burgess, "Participants’ Perspectives." Journal of Medical Ethics, vol. 26, pp. 126–130, 2000.

18