Completes, Pretest

APPENDIX B

Questionnaire

PROGRAMMING NOTES:

Overall

• All questions should be programmed onto a new page/screen. Other page/screen breaks (e.g., for instruction text) are noted throughout as applicable.

• Program progresses through the questioning in a “one-way” manner; participants should not be able to return to questions after they have answered them.

• Refused=-99; Valid Skip=-100

• Include one additional variable in the data set not shown in this document:

Respondent_ID (a unique identifier).

[INFORMED CONSENT HERE]

Question Type: Single Punch

Consent. If you have read the previous screen and agree to participate, please click Yes. If not, click No.

Variable Label: Consent. If you have read the previous screen and agree to participate, please click Yes. If not, click No.

Value	Value Label
01	Yes, I agree to participate. [Go to ‘Randomization’]
02	No, I do not agree to participate. [TERMINATE, show End Screen]
-99	Refused

[IF CONSENT = 1 ‘Yes’, GO TO ‘Randomization’ All others, TERMINATE and show REFUSAL END SCREEN]

[REFUSAL END SCREEN]

Thank you for taking the time to answer our questions today.

[INSTRUCTIONS]

The following study describes research on a drug product. Please read the study and then answer the questions related to the drug product. These questions will relate to the study and its impact on your prescribing practices.

For this task, please assume that:

• This study description was [published in a medical journal/presented in a promotional piece, like a sales aid].

• The drug is fully covered by your patient’s insurance; assume cost is not a factor.

• TIME PRESSURE VARIABLE: You will have [x seconds] to review the study description before answering questions and will not be allowed to return to the study description. / You will have [as much time as you need] to review the study description before answering questions and will not be allowed to return to the study description.

Note: The ‘Continue’ button will be functional after you have reviewed the instructions for at least 15 seconds.

[SHOW INSTRUCTIONS FOR 15 SECONDS BEFORE MAKING ‘Continue’ BUTTON ACTIVE]

[Next Screen]

[Abstract stimulus here]

[SHOW STIMULUS FOR 30 SECONDS BEFORE MAKING ‘Continue’ BUTTON ACTIVE]

Note: The ‘continue’ button will be functional after you have reviewed the screen for at least 30 seconds.

[PROGRAMMER: TIME PRESSURE VARIBALE ABOVE: FOR THOSE IN TIME PRESSURE CONDITION, LIMIT TIME ON STIMULUS TO 2 MINUTES (NOTE: time to be reevaluated after pretesting). FOR OTHERS, ALLOW UNLIMITED TIME REVIEWING STIMULUS. NO PARTICIPANTS MAY BE PERMITTED TO RETURN TO THE STIMULUS AFTER CLICKING ‘Continue’]

[IF TIME RUNS OUT FOR TIME PRESSURE PARTICIPANTS, PLEASE CHANGE THE SCREEN TO THIS MESSAGE: “The time to read this study description is up. Please click ‘Continue’ to answer questions about what you read.”]

[Next Screen]

Question Type: Single Punch

Recall. Based on the study you reviewed, what is Glutaraz used as a treatment for?

Variable Label: Recall: Based on the study you reviewed, what is Glutaraz used as a treatment for?

Value	Value Label
01	Asthma
02	Diabetes
03	Stroke
04	Arthritis
05	None of these

Question Type: Single Punch

Prescribe. A 60 year old male patient comes to your office with a history of Type 2 diabetes and coronary heart disease (noted on previous coronary catheterization). He has hemoglobin A1c of 8.5% and HDL level of 35 mg/dL despite maximal tolerated treatment with metformin and a sulfonylurea. How likely would you be to prescribe Glutaraz?

Variable Label: Prescribe: How likely would you be to prescribe Glutaraz?

Value	Value Label
01	Very unlikely to prescribe
02
03
04	Completely unsure whether would prescribe
05
06
07	Very likely to prescribe
-99	Refused

Question Type: Single Punch

Confidence. How confident are you in the validity of the conclusion that the study authors draw about Glutaraz?

Variable Label: Confidence: How confident are you in the validity of the conclusion that the study authors draw about Glutaraz?

Value	Value Label
01	Not at all confident
02
03
04	Unsure
05
06
07	Very confident
-99	Refused

Question Type: Single Punch

Rigor. Rate the overall rigor of the study methodology.

Variable Label: Rigor: Rate the overall rigor of the study methodology.

Value	Value Label
01	Not at all rigorous
02
03
04	Unsure
05
06
07	Very rigorous
-99	Refused

Question Type: Single Punch

Importance. Rate the importance of the study.

Variable Label: Importance: Rate the importance of the study.

Value	Value Label
01	Not at all important
02
03
04	Unsure
05
06
07	Very important
-99	Refused

Question Type: Single Punch

Interest. Are you interested in reading more details of the study?

Variable Label: Interest: Are you interested in the reading more details of the study?

Value	Value Label
01	Not at all interested
02
03
04	Unsure
05
06
07	Very interested
-99	Refused

Question Type: Single Punch

Caution. To what extent do you agree or disagree with the following statement:

The data you saw in the study should be interpreted cautiously.

Variable Label: Caution: The data you saw in the study should be interpreted cautiously.

Value	Value Label
01	Completely disagree
02	Disagree
03	Somewhat disagree
04	Somewhat agree
05	Agree
06	Completely agree
-99	Refused

[Present “Caution_open” on same page as “Caution” item]

Question Type: Open End Text

Caution_open. Please explain why you [pipe in answer; bold answer] with this statement.

Variable Label: Caution_open. Explain answer?

Value	Value Label
-99	Refused

Question Type: Multi Punch

Action. If you wanted to get more information about the drug you saw, which of the following would you do? You may choose more than one.

Variable Label: Action: If you wanted to get more information about the drug you saw, which of the following would you do? You may choose more than one.

Value	Value Label
01	Talk to colleagues
02	Seek more information from manufacturer
03	Seek out more publications about this product
04	Something else [SPECIFY] _______
05	Do nothing
-99	Refused

[IF PARTICIPANT SELECTS 05 “Do nothing,” MAKE OTHER OPTIONS UNAVAILABLE TO SELECT]

[FOR 04: PIPE IN TYPE OF SOURCE PARTICIPANT SAW]

Question Type: Single Punch

Decisions. Which one of the following statements best summarizes how you feel about the data you saw?

Variable Label: Decisions: Which one of the following statements best summarizes how you feel about the data you saw?

Value	Value Label
01	I would not use this data when making prescribing decisions.
02	I would consider the data when making prescribing decisions.
03	I would change my prescribing decisions based on the data.
-99	Refused

Question Type: Single Punch

Credible. How credible are the data you saw?

Variable Label: Credible: How credible are the data you saw?

Value	Value Label
01	Not at all credible
02	Somewhat credible
03	Moderately credible
04	Very credible
05	Extremely credible
-99	Refused

Question Type: Single Punch

Bias. How unbiased or biased is the study you saw?

Variable Label: Bias: How biased is the study you saw?

Value	Value Label
01	Very unbiased
02	Somewhat unbiased
03	Neither unbiased nor biased
04	Somewhat biased
05	Very biased
-99	Refused

Question Type: Open End Text

Explainbias. Please explain why you chose [pipe in answer] on this question.

Variable Label: Explainbias. Why did you choose [pipe in answer]?

Value	Value Label
-99	Refused

Question Type: Single Punch

Clinbenefit. To what extent do you

agree or disagree with the following statement: It is clear how the data translate into clinical

benefit.

Variable Label: Clinbenefit: It is clear how the data translate into clinical benefit.

Value	Value Label
01	Completely disagree
02	Disagree
03	Somewhat disagree
04	Somewhat agree
05	Agree
06	Completely agree
-99	Refused

[Counterbalance such that a random half of participants are presented “benefit”, “effectiveness”, and “riskbenefit” items BEFORE “risklikely”, “riskmagnitude” , and “riskduration” items]

Question Type: Single Punch

Benefit. How likely do you think it is that a patient would benefit if s/he took Glutaraz?

Variable Label: Benefit: How likely do you think it is that a patient would benefit if s/he took Glutaraz?

Value	Value Label
01	Extremely unlikely
02	Very unlikely
03	Somewhat unlikely
04	Somewhat likely
05	Very likely
06	Extremely likely
-99	Refused

Question Type: Single Punch

Effectiveness. How effective is Glutaraz in helping with diabetes?

Variable Label: Effectiveness: How effective is Glutaraz in helping with diabetes?

Value	Value Label
01	Extremely ineffective
02	Very ineffective
03	Somewhat ineffective
04	Somewhat effective
05	Very effective
06	Extremely effective
-99	Refused

Question Type: Single Punch

Riskbenefit. To what extent do you agree or disagree with the following statement: Based on my impression of the data, the benefits of Glutaraz are worth the possible side effects.

Variable Label: Riskbenefit: The benefits of Glutaraz are worth the possible side effects.

Value	Value Label
01	Completely disagree
02	Disagree
03	Somewhat disagree
04	Somewhat agree
05	Agree
06	Completely agree
-99	Refused

Question Type: Single Punch

Risklikely. Based on your impression of the data, how likely do you think it is that a patient would experience at least one side effect if s/he took Glutaraz?

Variable Label: Risklikely: How likely do you think it is that a patient would experience at least one side effect if s/he took Glutaraz?

Value	Value Label
01	Extremely unlikely
02	Very unlikely
03	Somewhat unlikely
04	Somewhat likely
05	Very likely
06	Extremely likely
-99	Refused

Question Type: Single Punch

Riskmagnitude. Based on your impression of the data, how serious are Glutaraz’s side effects?

Variable Label: Riskmagnitude: How serious are Glutaraz’s side effects?

Value	Value Label
01	Not at all serious
02	Somewhat serious
03	Moderately serious
04	Very serious
05	Extremely serious
-99	Refused

[GENERAL QUESTIONS]

Question Type: Single Punch

Mancheckpub. The study you read was:

Variable Label: Mancheckpub: The study you read was:

Value	Value Label
01	In a promotional piece
02	Not in a promotional piece
-98	Don’t know
-99	Refused

Question Type: Single Punch

Manchecktime. To what extent did you perceive time pressure while reviewing the data?

Variable Label: Manchecktime: To what extent did you perceive time pressure while reviewing the data?

Value	Value Label
01	No time pressure
02
03
04
05	Very strong time pressure
-98	Don’t know
-99	Refused

Question Type: Open End Text

Approvalopen. What does FDA approval mean to you?

Variable Label: Approvalopen: What does FDA approval mean to you?

Value	Value Label
-99	Refused

Question Type: Multiple Punch

Unapproved. In which of the following outlets have you encountered information about unapproved drugs? [list possible outlets, multiple responses permitted]:

Variable Label: Unapproved: In which of the following outlets have you encountered information about unapproved drugs? [list possible outlets, multiple responses permitted]:

Value	Value Label
01	Journal advertisements
02	Sales aids with product information from pharmaceutical sales representatives
03	Commercial drug websites (e.g., lipitor.com)
04	Sponsored conference sessions
05	Direct-to-consumer advertising (e.g., television advertisements)
06	Dinner talks sponsored by pharmaceutical companies
07	Key experts in the field presenting at conferences
08	Colleagues/peers
09	Online communities of healthcare providers
10	Medical reference websites for professionals (e.g., Epocrates, Prescribers’ Digital Reference [PDR], UpToDate, or Clinical Key)
11	Medical reference websites for consumers (e.g., WebMD)
12	Peer-reviewed medical journal publication
13	CME
14	I have never encountered information about unapproved drugs
15	Other [SPECIFY] __________
99	Refused

Question Type: Open End Numerical

Readfreq. Over the course of a week, about how many medical journal abstracts do read describing trials related to prescription drugs?

Variable Label: Readfreq: Over the course of a week, about how many medical journal abstracts have you read describing trials related to prescription drugs?

Journal abstracts

Value	Value Label
-99	Refused

[RANGE: 0 to 250] [Show “Please provide a number” if incorrect value is entered]

Question Type: Open End

PromoPiece. What does the term “promotional pieces” mean to you?

Variable Label: PromoPiece: What does the term “promotional pieces” mean to you?

Value	Value Label
-99	Refused

Question Type: Open End Numerical

Readsale. Over the course of a week, about how many promotional pieces do you read describing trials related to prescription drugs?

Variable Label: Readsale: Over the course of a week, about how many promotional pieces have you read describing trials related to prescription drugs?

Promotional pieces

Value	Value Label
-99	Refused

[RANGE: 0 to 250] [Show “Please provide a number” if incorrect value is entered]

Question Type: Single Punch

Usualtime. How much time do you typically spend reading materials of the type you saw today?

Variable Label: Usualtime. How much time do you typically spend reading materials of the type you saw today?

Value	Value Label
01	Less than a minute
02	1 to 5 minutes
03	6 to 10 minutes
04	10 to 30 minutes
05	More than 30 minutes
06	Other: Please specify ____ [Open-end]
-99	Refused

Question Type: Open End Numerical

Salesrep. How many drug sales representatives do you speak with in a typical week?

Variable Label: Salesrep: How many drug sales representatives do you speak with in a typical week?

Drug sales representatives

Value	Value Label
-99	Refused

[RANGE: 0 to 250] [Show “Please provide a number” if incorrect value is entered]

[SCRIPT]

We now have just a few more questions about your background and professional experience.

Question Type: Open End Numerical

Years. Since you received your license to treat patients, how many years have you been practicing in your primary specialty?

Variable Label: Years: Since you received your license to treat patients, how many years have you been practicing in your primary specialty?

years

Value	Value Label
-99	Refused

[RANGE: 0 to 80] [Show “Please provide a number” if incorrect value is entered]

Question Type: Multiple Punch

Setting. In the past 12 months, in which of the following practice settings have you seen patients for treatment or evaluation? Select all that apply.

Variable Label: Setting: In the past 12 months, in which of the following practice settings have you seen patients for treatment or evaluation?

[Multi-punch response items should be saved to dataset as binary variables with response options: 1=‘Selected’ OR 0=‘Not selected’.]

Value	Value Label
01	Academic medical center or medical school
02	Medical center not affiliated with a medical school
03	Community hospital
04	Office-based practice
05	HMO or integrated health care system
06	Other: Please specify ____ [Open-end]
-99	Refused

Question Type: Single Punch

TrustPromo. In general, how credible do you find prescription drug promotional pieces designed for health care providers?

Variable Label: TrustPromo: In general, how credible do you find prescription drug promotional pieces designed for health care providers?

Value	Value Label
01	Not at all credible
02	Somewhat credible
03	Moderately credible
04	Very credible
05	Extremely credible
-99	Refused

[Next Screen]

Question Type: Single Punch

TrustJournal. In general, how credible do you find prescription drug information in journal articles?

Variable Label: TrustJournal: In general, how credible do you find prescription drug information in journal articles?

Value	Value Label
01	Not at all credible
02	Somewhat credible
03	Moderately credible
04	Very credible
05	Extremely credible
-99	Refused

[Next Screen]

Question Type: Single Punch

ResultsInterpretation. How comfortable do you feel interpreting results from clinical drug trials?

Variable Label. ResultsInterpretation: How comfortable do you feel interpreting results from clinical drug trials?

Value	Value Label
01	Not at all comfortable
02	Somewhat comfortable
03	Moderately comfortable
04	Very comfortable
05	Extremely comfortable
-99	Refused

[Next Screen]

Question Type. Open-End Text

InterpretationTraining. What type of formal training, if any, do you have on interpreting this type of data? This may have been in medical school or after (e.g., class, on-the-job training).

Variable Label. InterpretationTraining: What type of formal training, if any, do you have on interpreting this type of data?

-99

Refused

[Next Screen]

Question Type: Single Punch

Rxvolume. Over the course of a week, about how many prescriptions do you issue?

Variable Label: Rxvolume: Over the course of a week, about how many prescriptions do you issue?

Value	Value Label
01	Less than 50 per week
02	50 to 99 per week
03	100 to 149 per week
04	More than 150 per week
-99	Refused

[Next Screen]

Question Type: Single Punch

Gender. What is your gender?

Variable Label: Gender: What is your gender?

Value	Value Label
01	Male
02	Female
-99	Refused

[Next Screen]

Question Type: Single Punch

Ethnicity. Which of these best describes your ethnicity?

Variable Label: Ethnicity: Which of these best describes your ethnicity?

Value	Value Label
01	Hispanic or Latino
02	Not Hispanic of Latino
-99	Refused

Question Type: Multiple Punch

Race. Which of these best describes your race? Select all that apply.

Variable Label: Race: Which of these best describes your race?

[Multiple punch response items should be saved to dataset as binary variables with response options: 1=‘Selected’ OR 0=‘Not selected’.]

Value	Value Label
01	American Indian or Alaska Native
02	Asian
03	Black or African American
04	Native Hawaiian or other Pacific Islander
05	White
06	Other: Please specify ____ [Open-end]
07	Prefer not to answer [If selected, no other options can be selected]
-99	Refused

Question Type: Multiple Punch

Doctorate. Please select any other doctoral degrees you hold. Select all that apply.

Variable Label: Doctorate: Please select any other doctoral degrees you hold.

Value	Value Label
01	D.N.Sc.
02	D.P.H.
03	D.Sc.
04	Ed.D.
05	J.D.
06	Ph.D.
07	Other: Please specify ____ [Open-end]
08	None [If selected, no other options can be selected]
-99	Refused

Question Type: Open End Numerical

Zipcode. What is the ZIP code for your primary practice?

Variable Label: Zipcode: What is the ZIP code for your primary practice?

Value	Value Label
-99	Refused

[ALLOW 5-DIGIT ZIP] [Display “Please enter a five digit zip code” if incorrect value is entered]

[Next Screen]

[DEBRIEF]

Your responses have been very helpful. Thank you very much for taking part in this survey! The drug described in this survey is hypothetical and for research purposes only. No such drug exists, nor was this study conducted.