OMB-FDA Q&A Summer 2011

201101-0910-009: Experimental Study of Patient Information Prototypes

Responses to OMB questions/comments on 6/21/11.

1a. The issues discussed re: the need to improve the documents focus on organization and format to reduce processing demand but we didn’t see much related to comprehension per se and utility.  Assuming that this is the actual goal, please provide more about how these issues were addressed by the advisory group and working group contributions to developing the prototypes.

Comprehension and utility were two of the primary topics of discussion at all public meetings, expert workshops, and internal working group meetings. Documents with greater processing demands lead to lower comprehension, thus the description of processing demands in Section A1. Health literacy issues were discussed at length and are investigated in the currently proposed research, also because comprehension of the document is of primary importance. The ultimate goal of the project is to find a document that people can read and that will provide useful information to them when they have received a prescription for a drug. For an example of the discussion of comprehension that has occurred, please see the presentation by Ruth Day of Duke University, who spoke at the Part 15 hearing on September 27, 2010 (http://www.fda.gov/downloads/Drugs/NewsEvents/UCM229059.pdf). Also, more in-depth information on the expert workshop held at the Brookings Institute on July 21, 2010, can be found at: http://www.brookings.edu/events/2010/0721_CMI.aspx. You will notice the topics converge around the usefulness of the document. Downloads of presentations are available at this site. Finally, comprehension will be measured and is one of our primary dependent variables in the design of Phase II (see Section B2, Analysis Plan, Dependent Variables; also Section B2, in text under “Phase II”).

1b. Please also clarify the overall goal of the research – is it primarily confirmatory?

The goal of the research is to determine whether new prototypes developed on the basis of social science principles provide a more understandable and usable standardized document for patients who have just been prescribed a prescription drug. It is confirmatory in that we expect the new prototypes to result in better comprehension and usability than existing PMI formats. We do not know at this time which of the new formats, if either, will result in better comprehension than the other.

2a. It is not clear whether the ‘assurance of confidentiality’ is being provided by FDA to the Phase I or the Phase II participants.  Is it necessary to collect PII from Phase I participants?  Since KN controls and does not share the its PII, is FDA giving an assurance of confidentiality in this case?

Please clarify this question.

2b. If FDA gives an assurance of confidentiality, it must be linked to specific statutory authority.  What is the statute under which this is being promised here?  If there is not statute, for Phase I, suggest saying ‘be kept private to the extent permitted by law’ and then describe controls (as is done in the remainder of this section). 

The phrasing in Section A.10 was changed to “kept private to the extent permitted by law,” as suggested.

2c. Were consent forms uploaded?  They would need the same language edits.

We have included the same “as permitted by law” language in the consent forms.

3a. Moderator’s Guide

1. Consider not boxing subjects in to making a decision in the first couple of minutes of the study. Why do they have to look at all three documents at the same time?  This might be overwhelming.    How about introducing each one separately? (e.g., quest 1 and 2).

Thank you for your suggestions regarding the first few minutes of the in-depth interviews. Based on past research experience and the input of various communications experts, we believe the initial questions and approach we have proposed the guide is the best way to capture the information we are looking for. We do not want to introduce the formats one at a time because that may induce order effects. Based on past qualitative experience, we do not expect choosing a format to be an overwhelming task for participants.

2. Clarify what FDA want them to make each decision based on (“like” and “confusing” are amorphous concepts). Consider asking them instead, “when you read xxx, what did this mean to you?”  Stay away from asking if something is “confusing” or “good” because these constructions are leading.  (e.g., quest 1 and 15 and 21).

We have structured the interview to include general questions followed by increasingly more focused questions. This strategy will allow us to receive unprompted reactions from participants initially, followed by more specific information about particular issues. We are looking for information about how confusing the information is and we are looking for information about how well the formats work for each individual. Moreover, we need to know what their preferences are. We have worded the questions so that we can maximize the information we gather while still maintaining language that will be meaningful and understandable to participants.

3. What is the relevance of questions 10-13?

Question 10 is relevant because we want to ensure that we have not missed any topic areas that real people feel important enough to include. Questions 11-14 are relevant so that we can capture the way people think about their drugs and what their basic knowledge is between generic and brand names. This will allow us to better tailor future documents toward patients because we will be able to use the brand or generic terms where appropriate to increase understanding.

4. For questions 20 consider asking them “what did you expect under this heading; now that you know what we were looking for, what would have been a better title?”

Assuming this question refers to 21 and not 20, we have adopted this suggested language.

5. Might want to leave Q34 open ended.  Q 35 – does Paper imply in the postal mail or giving the Paper in person?

Assuming this question refers to 35 and not 34, this is an open-ended question. The second question is included as a probe to follow up after participants have provided their own unprompted responses. In question 36, we have changed the language to “in paper form.”

3b. Clarify how Phase I results will be factored into Phase II.

Before embarking on a large scale quantitative study (Phase II), we want to ensure that we have heard from the end users of the document—the people who will receive this information when they obtain a prescription drug product. Specifically, we want to hear from individuals with lower health literacy. Phase I will allow us to make sure the wording in the formats we test in Phase II is appropriate, not unnecessarily difficult to understand, and accessible to people who will read it. Moreover, we will make specific decisions about font style and the order of information based on information in Phase I.

4a. How are the health focused questions at the end related to the research questions  Have they been cognitively tested – that is, do you know what you’re getting when you get a particular answer – how much of the response has to do with health literacy and processing information and what has to do with the questions themselves?

The health focused questions at the end of the questionnaire are the items of a health literacy measure developed by our contractor, RTI (McCormack, 2010).¹ These questions have been extensively tested and are designed to measure health literacy. Health literacy is an important measure for this study because it is likely that it will play an important role in the way participants read and understand PMI.

5. How is this approach similar or different to what was used for the nutrition labeling?

According to Alan Levy, Ph.D., Consumer Scientist, CFSAN, FDA (alan.levy@fda.hhs.gov), the nutritional label testing focused on evaluating formats to present nutritional information. In this testing, FDA defined measures based on task specific information, such as product choice, judgment about the helpfulness of the product, and ratings of the disclosure format. FDA obtained ratings of disclosure format to demonstrate the lack of correlation between performance and preference ratings. Determining whether the information would be useful was the major focus of the testing.

The process to test variations of PMI is somewhat similar to the process used to evaluate nutrition labels, yet there are some key differences. The nutrition labels contain a content list whereas the PMI contains complex risk and benefit information that interacts to form a risk/benefit ratio. Nutrition labeling is provided to assist consumers in making their own wise choices when selecting foods, whereas PMI is provided to a patient after a drug product has already been selected by a healthcare provider in order to aid the patient in using their medication safely and effectively.

1 McCormack, L. (2010, October). A 10-item short form to measure health literacy. Poster session presented at the Health Literacy Annual Research Conference (HARC II), Bethesda, MD.