Author Manuscript - Patient Understanding of Nonmedical Terms

NIH Public Access
Author Manuscript
Subst Abus. Author manuscript; available in PMC 2015 January 01.

NIH-PA Author Manuscript

Published in final edited form as:
Subst Abus. 2014 ; 35(1): 12–20. doi:10.1080/08897077.2013.789463.

How patients understand the term ‘nonmedical use’ of
prescription drugs: insights from cognitive interviews
Jennifer McNeely, MD, MS,
Department of Population Health and Department of Medicine, Division of General Internal
Medicine, NYU School of Medicine, 550 1st Avenue, VZ30, 6th Floor, New York, NY 10016, Tel
(212) 263-4975, Fax (646) 501-2706
Perry N. Halkitis, PhD, MPH,
Department of Applied Psychology, NYU Steinhardt School of Culture, Education, and Human
Development and Department of Population Health, NYU School of Medicine
Ariana Horton, MSN, RN, BSN,
NYU College of Nursing

NIH-PA Author Manuscript

Rubina Khan, PAC, MPA, and
Department of Population Health, NYU School of Medicine
Marc N. Gourevitch, MD, MPH
Department of Population Health, NYU School of Medicine
Jennifer McNeely: jennifer.mcneely@nyumc.org; Perry N. Halkitis: perry.halkitis@nyu.edu; Ariana Horton:
aeh399@nyu.edu; Rubina Khan: rubina.khan@nyumc.org; Marc N. Gourevitch: marc.gourevitch@nyumc.org

Abstract
Background—With rising rates of prescription drug abuse and associated overdose deaths, there
is great interest in having accurate and efficient screening tools that identify nonmedical use of
prescription drugs in health care settings. We sought to gain a better understanding of how patients
interpret questions about misuse of prescription drugs, with the goal of improving the accuracy
and acceptability of instruments intended for use in primary care.

NIH-PA Author Manuscript

Methods—A total of 27 English speaking adult patients were recruited from an urban safety net
primary care clinic to complete a cognitive interview about a four-item screening questionnaire for
tobacco, alcohol, illicit drugs, and misuse of prescription drugs. Detailed field notes were analyzed
for overall comprehension of the screening items on illicit drug use and prescription drug misuse,
the accuracy with which participants classified drugs into these categories, and whether the
screening response correctly captured the participant’s substance use behavior.
Results—Based on initial responses to the screening items, 6 (22%) participants screened
positive for past year prescription drug misuse, and 8 (30%) for illicit drug use. The majority
(26/27) of participants correctly interpreted the item on illicit drug use, and appropriately
classified drugs in this category. Eleven (41%) participants had errors in their understanding of the
prescription drug misuse item. The most common error was including use of medications without
Correspondence to: Jennifer McNeely, jennifer.mcneely@nyumc.org.
Statement of Contributions:
JM conceived of the study, conducted the interviews, and led the writing and analysis
PH assisted with study design, advised on the analysis, and contributed to the writing
RK led data collection and participated in interpretation of results
AH contributed to the analysis and presentation of results and contributed to the writing
MG assisted with conception and design of the study and contributed to the writing
All authors have approved of the final manuscript

McNeely et al.

Page 2

abuse potential as misuse. All cases of misunderstanding prescription drug misuse occurred among
participants who screened negative for illicit drug use.

NIH-PA Author Manuscript

Conclusions—Our results suggest that terminology used to describe misuse of prescription
medications may be misunderstood by many primary care patients, particularly those who do not
use illicit drugs. Failure to improve upon the language used to describe prescription drug misuse in
screening questionnaires intended for use in medical settings could potentially lead to high rates of
false positive results.
Keywords
Screening; Prescription drug misuse

INTRODUCTION

NIH-PA Author Manuscript

There is intense interest in identifying and addressing substance use in general healthcare
settings. Considerable federal resources are now devoted to promoting ‘screening and brief
intervention (SBI)’ approaches for alcohol and other drugs, which involve screening and
assessment, followed by brief counseling interventions that can be carried out by medical
providers in the course of a regular office visit.1,2,3,4,5 Having a short and accurate substance
use screening and assessment instrument is essential for the successful implementation of
SBI in time-pressured general medical settings. In response, researchers have moved rapidly
to develop and validate brief screening tools.6,7,8,9,10,11
Recent developments highlight the importance of identifying not only use of alcohol and
illicit drugs, but also misuse of prescription drugs. Prescription drug misuse, and in
particular the misuse of potent prescription opioids, has skyrocketed in the past decade.12
Overdose deaths attributed primarily to prescription opiates are now the leading cause of
accidental deaths among adults,13,14 and emergency room visits due to prescription
painkiller misuse have doubled in the past five years.15 Currently there is no standardized
approach to accurately and efficiently screening adult patients for misuse of prescription
drugs.

NIH-PA Author Manuscript

Many individuals who misuse prescription drugs are not engaged in use of any illicit
drugs. 16 To capture this increasingly prevalent type of drug use, screening tools need to be
able to detect prescription drug misuse independently from illicit drug use. Yet the screening
and assessment instruments that are generally considered for implementation in medical
settings (e.g. DAST-10, ASSIST) were developed at a time when prescription drug misuse
was less prevalent, and little is known about their effectiveness for identifying it. Newer
instruments, such as the NIDA-modified ASSIST and the single item drug screening
question developed by Smith et al. (2010) have included prescription drugs.6,17 But the
NIDA-modified ASSIST has not been validated, and the single item drug screening
question, which does have good sensitivity and specificity for detecting any drug use, has
not been evaluated for its accuracy in specifically detecting prescription drug misuse.
Concisely and clearly communicating the meaning of prescription drug misuse on a short
screening instrument poses considerable challenges. Unlike illicit drugs, which are defined
by their legal status, prescription drugs can be used appropriately or can be misused,
depending on the conditions under which they are used and the intentions of the user. To
draw a distinction between appropriate medical use and misuse of prescription medications,
the language adopted by both the single item drug screening question and the NIDAmodified ASSIST is ‘nonmedical’ use. Yet little is known about how patients interpret this
term.

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 3

NIH-PA Author Manuscript

As a first step in developing a screening instrument that could quickly and accurately
identify both illicit and prescription drug misuse, we sought to gain a better understanding of
how primary care patients interpret and answer screening questions about their use of these
substances. We employed cognitive interviewing, an approach developed in the 1980s by
survey methodologists and psychologists to evaluate sources of response error in
questionnaires. Cognitive interviewing is supported by a large body of methodological
research, and is one of the primary methods used by survey researchers to test the accuracy
with which items are understood and answered by respondents. 18,19,20,21,22 Given the
similarity between surveys and structured screening instruments, we applied cognitive
interviewing techniques to elucidate how primary care patients understand items in a
substance use screening questionnaire.

NIH-PA Author Manuscript

Cognitive interviewing recognizes that the question answering process can be complex,
involving the cognitive steps of comprehending the question, retrieving relevant information
from memory, making a decision about how to answer, and then mapping the response onto
the options given in the survey question. We postulated that answering questions about illicit
and prescription drug use primarily poses difficulty in the areas of comprehension and
deciding on an answer. Comprehending the question requires an understanding of the
language used to describe the drug or behavior, such as ‘nonmedical’ or ‘illegal’ use.
Making a decision about how to answer is influenced by the participant’s comfort with the
way in which the question is asked, and is important to consider when the item includes
stigmatized behavior such as drug use.

METHODS
Participants
The study was conducted in over a 3-month period in early 2012, in the adult primary care
medicine clinics of a large public hospital in New York City. Individuals eligible to
participate were current clinic patients, fluent in English, and 18–65 years old. A purposeful
sampling approach was used to achieve approximately equal numbers of male and female
participants in the predetermined age categories of 18–35 years, 36–50 years and 51–65
years. These categories were chosen to achieve a balanced representation of groups within a
primary care population that would likely be targeted for substance use screening. Potential
participants were approached consecutively in the clinic waiting area and screened for
eligibility. There was no advertisement of the study, and all recruitment was by a single
research assistant. Those expressing interest received a written information sheet, and verbal
consent for participation was obtained. The NYU School of Medicine Institutional Review
Board of reviewed and approved of all study procedures.

NIH-PA Author Manuscript

Data Collection
Participants completed a 30–40 minute interview that sought to ascertain their understanding
of a brief substance use screening questionnaire. The questionnaire consisted of 4 items
assessing past year use of tobacco, unhealthy alcohol consumption, illicit drug use, and
misuse of prescription drugs (Table 1). The questionnaire was adapted from the NIDA
Quick Screen V1.0, which was developed by NIDA for use in healthcare settings but has not
yet been validated.23 This screening tool incorporates elements of the single item screening
questions for alcohol and drugs that were developed and validated in a primary care
population by Smith et al. (2010).6,24 Depending on the version of the questionnaire used,
participants were given response options of entering a number, giving a dichotomous
response (yes or no), or categorizing their frequency of use (never, once or twice, more than
once or twice, or frequently). All subjects were given a one-page handout containing the
screening questionnaire.

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 4

NIH-PA Author Manuscript

For each item in the questionnaire, participants were first asked to answer the item using the
specified response categories, and then to respond to a series of follow-up questions from
the interviewer. Because the goal was to understand the thought processes guiding
participants’ responses, they were asked to ‘think-aloud’ as they initially formulated their
answer. Thinking-aloud was demonstrated by the interviewer, and then practiced by
participants using a warm-up item before beginning the interview. After the participant
responded to each item in the screening questionnaire, the interviewer used a series of
probes and open-ended questions to gather more information on the cognitive process.
Participants were asked to explain what they were thinking about as they answered the item,
to repeat the item in their own words, to describe how they formulated their answer, and to
assess whether the question was easy or difficult to answer. They were also asked to give
examples of the type of substance that each screening question asked about (e.g. to name
substances that belong in the ‘illegal drug’ category for the illicit drug item). Responses to
the screening items in the primary questionnaire were recorded based upon the participant’s
initial answer, and were not changed based on responses later in the interview. All
interviews were conducted by the primary author (JM).

NIH-PA Author Manuscript

Two adaptations were made to the questionnaire after the first 16 interviews were
completed. The first adaptation was to change from an interviewer- to a self-administered
screening form. The first 16 subjects (Study ID# 1–16) were read each item by the
interviewer and answered verbally. The following 11 subjects (Study ID# 17–27) read and
completed the questionnaire on their own (8 on paper, 3 on a touchscreen computer) before
proceeding with the cognitive interview. The self-administered approach was introduced to
assess the feasibility of administering the questionnaire without an interviewer’s assistance.
However, to ensure that reading difficulty did not interfere with the cognitive interviewing
assessment, all participants were read each item aloud by the interviewer before answering
the additional probes and open-ended questions.

NIH-PA Author Manuscript

A second adaptation was altering the term used to describe prescription drug misuse. The
initial interviews (Study ID #1–16) asked participants about use of any prescription drug ‘for
nonmedical reasons,’ while for subsequent participants (Study ID #17–27) the item was
changed to use of any prescription drug ‘recreationally.’ This change was made based on
responses to the initial interviews, which indicated that some participants misinterpreted the
meaning of ‘nonmedical reasons.’ The choice of ‘recreationally’ to describe misuse was
guided by findings from the initial 16 interviews, which explored alternative approaches to
phrasing the prescription drug item. These alternative items were administered and
discussed in a second part of the interview, following completion of the screening
questionnaire and cognitive interviewing portions that are the focus of the present analysis.
Five items were tested, of which each participant received two or three. These items were
adapted from terms that have been used in other substance use questionnaires and surveys,
and described the behavior of prescription drug misuse as taking a prescription medication
a) recreationally, b) only for the experience or feeling it caused,25 c) in a way the doctor
didn’t intend,26 d) to get high,27 and e) have you abused any prescription drug.28 The item
describing prescription drug misuse as using medications ‘recreationally’ was not initially
included, but was added after being suggested by participants during the early interviews.
Because the ‘recreationally’ version was better accepted and more clearly understood by
participants than the other alternative items, it was selected for inclusion in the screening
questionnaire for the last 11 interviews.
Analysis
During each interview, notes were taken on paper by two members of the research team, (the
interviewer and an observer). These interview notes were incorporated into detailed field

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 5

NIH-PA Author Manuscript

notes that were written by each of the two researchers at the end of every interview day. To
facilitate the accurate recording of field notes, a maximum of two interviews were
conducted per day. Notes included participants’ verbatim responses when feasible, which
were designated with quotation marks. The field notes, as well as responses to the screening
questionnaire, were the data used for this analysis.
This analysis focuses primarily on the screening question for misuse of prescription drugs,
and secondarily on the screening question for illicit drugs. On the screening questionnaire,
any response greater than ‘never’ or ‘zero’ was considered a positive screen. We examined
the field notes for participants’ freely given definitions of nonmedical or recreational use of
prescription medications, comprehension of screening questions intended to describe misuse
of prescription medications, and examples of illegal drugs and of drugs that could be used
nonmedically or recreationally. Notes were analyzed by the interviewer and by a second
investigator who was not present during the interviews.

NIH-PA Author Manuscript

The first step in our analysis was to create a matrix that listed individual responses to each
questionnaire item, whether the participant seemed to have a correct understanding of the
question, and a summary of the participant’s explanation for why they gave this response.29
In a second step, the field notes were entered into Atlas.ti (version 6.2.27) and analyzed for
themes related to comprehension of the illicit drug use and prescription drug misuse items.
Codes were developed by the primary author in the course of the analysis using a grounded
theory approach. Codes identified examples of how the questionnaire item was interpreted,
types of correct and incorrect responses to the item, and names of drugs that would be
included in that category.

NIH-PA Author Manuscript

We then sought to determine whether responses to the illicit drug use and prescription drug
misuse screening items were correct versus incorrect, based on the final coded interviews.
For illicit drug use, having a correct answer required naming only illegal drugs, and an
incorrect answer would include only legal drugs. An imprecise answer would include both
illegal drugs and legal substances having abuse potential (e.g. alcohol). For prescription
drug misuse, the minimum criteria for a ‘correct’ response were a) included only
prescription medications having abuse potential (controlled substances), and b) taken other
than as prescribed (which could include taking without a prescription, taking more or for
reasons other than as prescribed, and taking for a euphoric effect). Responses that failed to
meet both of these criteria were classified as ‘incorrect.’ Responses that included the two
elements of a correct response, but also discussed use of prescription medications without
abuse potential, (taken for reasons other than as prescribed), were classified as imprecise.
For example, a participant who described nonmedical use as taking a higher than prescribed
dose of benzodiazepines ‘to get high’ but also said that taking a selective serotonin reuptake
inhibitor (SSRI) without a prescription is nonmedical use, would be classified as imprecise.
The second investigator reviewed the uncoded interviews and made an independent
determination of whether the participant’s response was correct, incorrect, or imprecise.
Discrepancies in classification between the two investigators conducting the analysis were
identified and discussed, until consensus was reached on each participant’s responses.
We examined the frequency of correct, incorrect, and imprecise responses to the illicit and
prescription drug use items by screening response (positive vs. negative screen). We then
looked for respondent characteristics (such as age, gender, education level, other drug use)
that might be associated with misclassification. In a final step, we compared the screening
response to the participant’s reported drug use behavior from the entire interview and
determined that the participant entered a correct response (demonstrating concurrence with
the narrative description of drug use), or an incorrect response (demonstrating lack of
concurrence with the narrative description).

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 6

RESULTS
NIH-PA Author Manuscript

Characteristics of the 27 participants are summarized in Table 2. While all participants were
fluent in English, for 6 English was not their primary language. Eight participants were born
outside the US. Participants’ initial responses to the 4 items included in the screening
questionnaire are displayed in Table 3.
Definitions of prescription drug misuse

NIH-PA Author Manuscript

After answering each screening item, participants were asked to give their own description
or definition of its meaning. For the prescription drug item, which was phrased as either
nonmedical use or recreational use, 5 of the 27 participants described a full range of
behaviors that could be considered misuse of prescription drugs. These behaviors included
taking a medication only for its euphoric effects, and taking a medication having abuse
potential that was either not prescribed, or was prescribed but then taken either more
frequently or at higher doses than directed. For example, this participant described
recreational use as: “[n]ot taking it when you’re supposed to, or over taking it…taking too
many pills…taking it too often…abusing medication.” (Study ID #18: Female, 51 years)
More frequently, participants (n=9) gave a limited definition, with the dominant theme being
taking a medication ‘to get high’ or for pleasure. Participant answers included: “[w]hen you
take something because you wanna get a high or a good feeling… a feeling of being happy”
(Study ID #26: Female, 62 years), and “[f]or example with a pain medicine prescribed for
pain, if you were still taking it because it gives you a high or something extra…that’s not
what the medication is for.” (Study ID #13: Male, 52 years)
Two participants did not emphasize the euphorigenic aspects of prescription drug misuse,
and instead described addictive behaviors in their definitions. One of these participants
focused on illegitimate medication seeking, while the other discussed the risks of addiction
and overdose. No participants included taking a prescription medication for a condition
other than that for which it was prescribed in their definitions of prescription drug misuse.
Differences in interpretation of ‘nonmedical’ versus ‘recreational’ use
There was considerable overlap in responses among those who were administered the
screening question phrased as using prescription drugs ‘for nonmedical reasons’ (n=16)
versus ‘recreationally’ (n=11). However, there were some differences between
interpretations of these items that fell into an identifiable pattern. Nonmedical use
definitions generally focused on taking medications that were not prescribed by a medical
provider, while recreational use definitions focused on taking medications for pleasure.

NIH-PA Author Manuscript

Nonmedical use
Nonmedical use interpreted to include medications without abuse potential: Many
participants described nonmedical use in terms of taking any medication without a
prescription, including medications that are not considered to have abuse potential. 5 of the
16 participants who received the ‘nonmedical use’ version of the screening question
interpreted it to include use of over-the-counter drugs. Others (n=3) included as nonmedical
use medications that may be prescribed but are not controlled substances. Examples
included Lipitor, and acetaminophen prescribed by a medical provider (Table 4).
Nonmedical use as self medicating: Nonmedical use was viewed by some participants as
taking medications for the purpose of treating a legitimate medical condition, but without
seeking the care of a medical provider. We classified this behavior as self medicating. Of
those participants who received the nonmedical use version of the screening item, 6
described it as self medicating. For example, “[n]onmedical means it’s something the doctor
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 7

NIH-PA Author Manuscript

didn’t prescribe for you, you decided to take it on your own.” (Study ID #14: Male, 39
years) Another participant described nonmedical use as taking “[m]edicine that could cure
you, but that is not with a doctor’s recommendation,” and went on to explain that “[t]he
difference is that for non-medical, it’s not required to see the doctor.” (Study ID #10, Male,
19 years)
Another participant talked about nonmedical use being an illicit behavior, but also
something that may be engaged in by individuals who lack access to medical care: Her
initial response was “[r]ight away I think of drug addicts.” But then she went on to say that
she also thinks of people who have problems with immigration, who can’t get medications.
In this scenario, she said, “[i]t may be justified, but it’s still nonmedical.” (Study ID #9:
Female, 30 years)
In other cases, it was less clear that the drug would be medically indicated, but the intention
is still to treat what the patient believes is a legitimate medical problem. One participant
responded that “medical use is what the doctor tells you”, while “nonmedical use is playing
doctor, taking it into my own hands, which I might do if the doctor wasn’t giving me what I
need.” (Study ID #8: Female, 28 years)
Recreational use

NIH-PA Author Manuscript

Recreational use as taking medications to get high: In contrast, for the 11 participants
who received the ‘recreational use’ version of the screening question, most definitions (n=7)
focused on taking medications for their euphoric effects. These participants indicated that
using recreationally means taking medication “to have fun, get high, party with.” (Study ID
#20: Female, 56 years)
Recreational use and addiction: Two participants, both of whom had used illicit drugs in
the past, noted a discrepancy between recreational use and addiction. As one pointed out,
individuals who are addicted to a drug may not consider their use to be fully voluntary or
recreational: “There’s a difference between doing drugs recreationally and being addicted…
[r]ecreationally is more of a want, and addiction is a need.” (Study ID #19: Male, 65 years)
Despite this distinction, the participant believed that someone who was addicted would still
respond affirmatively if they were asked about recreational use on a screening questionnaire.
Misinterpretation of the term ‘recreationally’: Two of the 11 participants who received
this item did not understand the meaning of ‘recreationally’ and instead interpreted it to
mean ‘occasionally,’ as in taking it only as needed for symptoms. None included over-thecounter medications in their examples of drugs that might be used recreationally.

NIH-PA Author Manuscript

Definitions of illicit drug use
Definitions and examples of illicit drug use focused on the illegal status of these substances.
Misclassification of drugs belonging to this category was uncommon (Table 4). One
participant (10C: male, 19 years) included tobacco and alcohol in his examples of ‘illegal
drugs.’ Two participants included prescription medications with abuse potential
(benzodiazepines, opioids) in this category along with illegal drugs such as heroin,
marijuana, and cocaine. In the context of the interview, it was clear that these participants
were listing these prescription medications as examples of drugs that are bought and sold
illicitly.
Overall understanding of screening items and accuracy of screening response
Illicit drug use—Among the 27 participants, 26 (96%) correctly understood the screening
item for illicit drug use (Table 5). One participant had an imprecise understanding, based on
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 8

NIH-PA Author Manuscript

including tobacco and alcohol in this category along with illicit drugs. Because the 3
participants who listed street-purchased medications (pills, sedatives, opioids) as illicit drugs
restricted their classification to drugs that were bought and sold illicitly, they were
considered to have an overall correct understanding of illicit drug use. The single case of an
inaccurate screening response came from a participant who correctly understood the
meaning of ‘illegal drugs,’ but mistakenly reported lifetime use rather than past year use and
thus generated an inaccurate (positive) screening result.
Prescription drug misuse—Of the 16 participants who received the nonmedical use
version of the prescription drug item, 8 (50%) understood it correctly. Five had an incorrect,
and 3 had an imprecise understanding of the screening item. Among the 8 participants with
an incorrect or imprecise understanding of the screening item, 6 screened negative for
prescription drug misuse. The screening result was accurate, in that it reflected the
participant’s narrative account of their drug use, for all individuals with a correct
understanding of the item. The screening result was inaccurate for 2 of the 8 participants
with an incorrect or imprecise understanding of the item: one participant screened positive
based on their Tylenol use, and the other screened negative because they did not consider
taking a drug prescribed to someone else to be nonmedical use.

NIH-PA Author Manuscript

For the 11 participants who received the recreational use version of the screening item, 8
(73%) understood it correctly. Among the 2 participants with an incorrect understanding of
the item, 1 did not give an answer and the other interpreted ‘recreationally’ to mean
‘occasionally.’ The participant with an imprecise understanding included antidepressants
and appetite suppressants, along with pain medications, in their examples of drugs that could
be used recreationally. One participant misread the question as asking how many
prescription medications she took, but had a correct understanding of the screening item
when it was read by the interviewer. As a result, although she had a correct understanding of
recreational use her response on the screening form was inaccurate.
In examining characteristics of the 11 individuals who had an incorrect (n=7) or imprecise
(n=4) response to the prescription drug misuse item, we found that all screened negative for
illicit drug use. Conversely, a correct interpretation of the prescription drug item was given
by all 7 participants who screened positive for illicit drugs. Females appeared to have a
higher frequency of incorrect responses to the prescription drug item (5/14 females versus
2/13 males). We did not observe a clear association between correctness of response to the
prescription drug item and age, race, education level, or primary language.

DISCUSSION
NIH-PA Author Manuscript

Many primary care patients in our sample misunderstood the language used to describe
misuse of prescription medications. The screening item using the term ‘nonmedical’ use
frequently led to misclassification errors, the most common of which was inclusion of drugs
without abuse potential, including over-the-counter medications. Understanding seemed to
improve somewhat when the question was rephrased as using a medication ‘recreationally.’
While the concept of recreational use seemed easier for most participants to grasp, this term
also has potential drawbacks. One participant misinterpreted ‘recreationally’ to mean
‘occasionally,’ and generated a false positive screen as a result. It is also possible that using
a term such as recreational use, which focuses on euphoric effects, may fail to capture use of
prescribed medications for conditions other than that for which they were prescribed. For
example, a patient who had an opioid prescribed for back pain and then took it for help with
sleep may not classify their use as recreational, though this behavior would be considered
misuse of a prescription drug.

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 9

NIH-PA Author Manuscript

Despite being better understood by most participants, the accuracy of the screening response
was not improved by changing the term to ‘recreationally.’ However, interpretation of these
results are confounded by switching to a self-administered version of the questionnaire, at
the same time that the screening item was changed from ‘nonmedical reasons’ to
‘recreationally.’ Among those who received the nonmedical (interviewer-administered)
version, the two inaccurate responses were generated by misinterpreting the intent of the
screening question (to include over-the-counter medication in one case, and to exclude use
of a drug having abuse potential that was not prescribed to her in the other). In the
recreational (self-administered) version, the inaccurate responses were due to language
difficulty. One participant correctly described prescription drug misuse but read the question
as asking how many prescription medications she took, and the other thought the word
‘recreationally’ meant ‘occasionally.’ Rephrasing the question using the words ‘recreational
use’ instead of ‘recreationally,’ or adding a clarification of the term ‘recreationally’ could
potentially improve the performance of this item.

NIH-PA Author Manuscript

Most individuals who misunderstood the meaning of the prescription drug screening item in
fact did not appear to be engaged in misuse. We found a relatively low rate of false positive
screens (3 of 27 participants) when the screening result was compared to narrative
descriptions of actual drug use behavior. Another two participants were unable to give an
answer. This implies that among our sample, 5/27 (18%) would require further assessment
due to either poor understanding or inability to respond to the prescription drug misuse
language. By contrast, for the illicit drug use item there was just one inaccurate screening
response, and the question was answered by each of the 27 participants.
All participants who reported illicit drug use had a correct interpretation of prescription drug
misuse. All those who had an incorrect or imprecise interpretation of prescription drug
misuse screened negative for illicit drugs. This may indicate that among populations with
relatively low rates of illicit drug use, as in primary care, screening instruments to detect
prescription drug misuse are likely to have a higher rate of misclassification.
Limitations

NIH-PA Author Manuscript

The primary limitation of our study is its small sample size, which hinders the
generalizability of our results and precludes the ability to rigorously test for predictors of
correct versus incorrect understanding of the screening items. Our assessment of accuracy of
the screening response relied only on the participant’s narrative description of their actual
substance use behavior during the course of the interview, and did not include additional
questionnaires or a medication review. Additional measures of substance use could have
provided stronger evidence of the screener’s accuracy, and would be required for a
validation study. We did not test the ‘nonmedical reasons’ version of the prescription drug
item in a self-administered questionnaire, and we did not test the ‘recreationally’ version in
an interviewer-administered questionnaire. This limited our ability to compare the accuracy
of the screening response based upon wording alone.
Another potential limitation was the relatively small number of illicit or prescription drug
users, though the prevalence in our sample is equivalent to, or somewhat higher, than what
is found in other primary care settings,3,4,6 Consistent with our diverse urban safety net
patient population, over one-third of our sample had high school degree/GED or less and a
high percentage was foreign born. The language used in the screening questionnaire may be
easier to interpret for a highly educated or less culturally diverse population. Yet given the
goal of developing screening tools that can be used in a wide range of primary care settings,
these attributes of the study population could also be interpreted as a strength of our
approach.

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 10

Conclusions

NIH-PA Author Manuscript

In terms of prevalence as well as morbidity and mortality, prescription drug misuse is
becoming the US’s dominant drug problem. Any broadly recommended screening tool
should be able to identify prescription drug misuse, and to elicit it even among those who
are not engaged in illicit drug use. Yet there is currently no brief screening tool, suitable for
use in medical settings, that has been validated specifically for detection of prescription drug
misuse in a general primary care population. Nonmedical use language has the potential to
lead to a substantial number of false positive or invalid responses on standardized screening
instruments. These responses would trigger further assessment to rule out a substance use
problem; a process that could present a significant burden to health care systems. Our
findings indicate that language that describes the purpose of using these drugs, such as
‘recreational use,’ may be more easily understood by primary care patients, but it could fail
to capture prescription drug misuse that is not for euphoric effects.

NIH-PA Author Manuscript

Finally, it should be noted that many instruments considered for screening and assessment in
primary care patients were validated in populations having a higher prevalence of illicit drug
use, such as in drug treatment or psychiatric patients. We found that individuals who
reported illicit drug use were able to correctly understand the screening questions on
prescription drug misuse, but there was considerable misinterpretation of these questions
among individuals who did not use illicit drugs. Our study thus highlights the potential
hazards of adopting substance use screening instruments in general medical settings without
rigorously evaluating their accuracy in the population in which they are intended to be used.

Acknowledgments
The authors thank Shiela Strauss, John Rotrosen, Scott Sherman, Donna Shelley, Andrew Wallach, Valerie Perel,
Barbara Porter, Derek Nelsen, and Arianne Ramautar for their contributions to this research.
Funding sources:
NIH-NIDA K23DA030395
NIH/NCRR 1UL1RR029893 (NYU-HHC CTSI Pilot Grant)
NIH-NIDA 2R25DA022461
NYU School of Medicine Center of Excellence on Addictions (Seed Grant)
The content is solely the responsibility of the authors and does not necessarily represent the official views of the
National Institute on Drug Abuse, the National Institutes of Health, or the NYU School of Medicine.

NIH-PA Author Manuscript

References
1. Babor TF, McRee BG, Kassebaum PA, Grimaldi PL, Ahmed K, Bray J. Screening, brief
intervention, and referral to treatment (SBIRT): Toward a public health approach to the
management of substance abuse. Substance Abuse. 2007; 28(3):7–30. [PubMed: 18077300]
2. Substance Abuse and Mental Health Services Administration. [Accessed Nov 12, 2011.] White
paper on screening, brief intervention and referral to treatment (SBIRT) in behavioral healthcare.
http://www.samhsa.gov/prevention/SBIRT/SBIRTwhitepaper.pdf
3. Saitz R, Alford DP, Bernstein J, Cheng DM, Samet J, Palfai T. Screening and brief intervention for
unhealthy drug use in primary care settings: randomized clinical trials are needed. J Addict Med.
2010; 4:123–130. [PubMed: 20936079]
4. Madras BK, Compton WM, Avula D, Stegbauer T, Stein JB, Clark HW. Screening, brief
interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare
sites: Comparison at intake and 6 months later. Drug Alc Depend. 2009; 99:280–95.

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 11

NIH-PA Author Manuscript
NIH-PA Author Manuscript
NIH-PA Author Manuscript

5. Substance Abuse and Mental Health Services Administration. Treatment Improvement Protocol
(TIP) Series 24. Rockville, MD: Center for Substance Abuse Treatment; 1997. A guide to substance
abuse services for primary care clinicians.
6. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug
use in primary care. Arch Intern Med. 2010; 170:1155–1160. [PubMed: 20625025]
7. Brown RL, Rounds LA. Conjoint screening questionnaires for alcohol and other drug abuse:
criterion validity in a primary care practice. Wis Med J. 1995; 94:135–40. [PubMed: 7778330]
8. Skinner HA. The Drug Abuse Screening Test. Addict Behav. 1982; 7:363–67. [PubMed: 7183189]
9. National Institute on Drug Abuse (NIDA). [Accessed May 14, 2011.] NIDA Quick Screen:
Clinician’s screening tool for drug use in general medical settings. http://ww1.drugabuse.gov/
nmassist/
10. Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, Shaffer HJ. A new brief screen for
adolescent substance abuse. Arch Pediatr Adolesc Med. 1999; 153:591–96. [PubMed: 10357299]
11. Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use Disorders
Identification Test (DUDIT) in Criminal Justice and Detoxification Settings and in a Swedish
Population Sample. European Addiction Research. 2005; 11:22–31. [PubMed: 15608468]
12. Substance Abuse and Mental Health Services Administration (SAMHSA). Results from the 2010
National Household Survey on Drug Use and Health: Detailed Tables. Rockville, MD: Substance
Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and
Quality; 2011. http://oas.samhsa.gov/NSDUH/2k10NSDUH/tabs/Sect1peTabs1to10.pdf [Accessed
Oct 3, 2012.]
13. Warner, M.; Chen, LH.; Makuc, DM.; Anderson, RN.; Minino, AM. [Accessed Oct 3, 2012.] Drug
poisoning deaths in the United States, 1980–2008. NCHS Data Brief Dec. 2011. (DHHS
Publication No. (PHS) 2012–1209). http://www.cdc.gov/nchs/data/databriefs/db81.pdf
14. Centers for Disease Control and Prevention. [Accessed Oct 3, 2012.] Policy Impact: Prescription
painkiller overdoses. Dec. 2011 http://www.cdc.gov/homeandrecreationalsafety/rxbrief/
15. Substance Abuse and Mental Health Services Administration. Drug Abuse Warning Network:
selected tables of national estimates of drug-related emergency department visits. Rockville, MD:
Center for Behavioral Health Statistics and Quality, SAMHSA; 2010.
16. Becker WC, Sullivan LE, Tetrault JM, Desai RA, Fiellin DA. Non-medical use, abuse and
dependence on prescription opioids among U.S. adults: psychiatric, medical and substance use
correlates. Drug Alc Dep. 2008; 94:38–47.
17. National Institute on Drug Abuse (NIDA). [Accessed May 14, 2011.] Screening for drug use in
general medical settings: A resource guide for providers. http://www.drugabuse.gov/nidamed/
resguide/resourceguide.pdf
18. Campanelli P. Testing survey questions: New directions in cognitive interviewing. Bulletin de
Methodologie Sociologique. 1997; 55:5–17.
19. Campanelli P, Martin E, Rothgeb JM. use of respondent and interviewer debriefing studies as a
way to study response error in survey data. The Statistician. 1991; 40:253–264.
20. Willis GB, Royston P, Bercini D. The use of verbal report methods in the development and testing
of survey questionnaires. Applied Cognitive Psychology. 1991; 5:251–267.
21. Willis GB, Schechter S. Evaluation of cognitive interviewing techniques: do the results generalize
to the field? Bulletin de Methodologie Sociologique. 1997; 55:40–66.
22. Jabine, T.; Straf, M.; Tanur, J.; Tourangeau, R., editors. Cognitive Aspects of Survey
Methodology: Building a Bridge Between Disciplines. Washington, DC: National Academy Press;
1984. Tourangeau, R. Cognitive sciences and survey methods; p. 73
23. National Institute on Drug Abuse (NIDA). [Accessed May 14, 2011] NIDA Quick Screen:
Clinician’s screening tool for drug use in general medical settings. at http://ww1.drugabuse.gov/
nmassist/
24. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. Primary care validation of a singlequestion alcohol screening test. J Gen Intern Med. 2009; 24(7):783–88. [PubMed: 19247718]
25. Substance Abuse and Mental Health Services Administration (SAMHSA). Results from the 2009
National Household Survey on Drug Use and Health: Volume II, Technical appendices and
selected prevalence tables. Rockville, MD: Substance Abuse and Mental Health Services
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 12

NIH-PA Author Manuscript

Administration, Center for Behavioral Health Statistics and Quality; 2010. http://oas.samhsa.gov/
NSDUH/2k9NSDUH/2k9ResultsApps.htm#AppC [Accessed March 1, 2013.]
26. Huang B, Dawson DA, Stinson FS, et al. Prevalence, correlates, and comorbidity of nonmedical
prescription drug use and drug use disorders in the United States: results of the National
Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2007; 67:1062–73.
[PubMed: 16889449]
27. Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, Shaffer HJ. A new brief screen for
adolescent substance abuse. Arch Pediatr Adolesc med. 1999; 153:591–6. [PubMed: 10357299]
28. Skinner HA. The Drug Abuse Screening Test. Addict Behav. 1982; 7:363–7. [PubMed: 7183189]
29. Aicken CR, Gray MG, Clifton S, et al. Improving questions on sexual partnerships: lessons learned
from cognitive interviews for Britain’s Third National Survey of Sexual Attitudes and Lifestyles
(“Natsal-3”). Arch Sex Behav. 2013; 42:173–85. [PubMed: 22695641]

NIH-PA Author Manuscript
NIH-PA Author Manuscript
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 13

Table 1

Items included in the screening questionnaire

NIH-PA Author Manuscript

Item

Administered to participants

1. In the past year, how often have you used tobacco?

#1–27

2. In the past year, how often have you used alcohol, X or more drinks in a day? (X = 5 for men and = 4
for women)

#1–27

3. In the past year, how often have you used any illegal drug?

#1–27

4(a). In the past year, how often have you used any prescription drug for non-medical reasons?
4(b). In the past year, how often have you used any prescription drug recreationally?

#1–16 (interviewer administered)
#17–27 (self administered)

NIH-PA Author Manuscript
NIH-PA Author Manuscript
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 14

Table 2

Characteristics of the 27 participants

NIH-PA Author Manuscript

Characteristic

N (%)

Age(years)
Mean, SD

44, 14

Median

48

Range

19–65

Interquartile range

25

Gender
Female

14 (52)

Male

13 (48)

Race/Ethnicity

NIH-PA Author Manuscript

Black non-Hispanic

9 (33)

White non-Hispanic

8 (30)

Hispanic

7 (26)

Asian

1 (4)

Other

2 (7)

Foreign Born

8 (30)

Primary language
English

21 (78)

Spanish

2 (7)

Other

4 (15)

Education
Less than HS

3 (11)

HS grad or GED

7 (26)

Some college or trade school

7 (26)

College degree or higher

10 (37)

Income

NIH-PA Author Manuscript

<$5,000

6 (22)

$5,000 to $14,999

10 (37)

$15,000 to $74,999

9 (33)

$75,000 or greater

2 (7)

Employment
Employed full-time

4 (15)

Employed part-time or occasional work

7 (26)

Unemployed

6 (22)

Disability

6 (22)

Student/Other

4 (15)

Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 15

Table 3

Initial responses to the screening questionnaire(N=27)

NIH-PA Author Manuscript

Substance

Positive
N (%)

Negative
N (%)

No answer
N

Tobacco

8 (30)

18 (67)

1a

Alcohol

12 (44)

15 (56)

0

Illicit drugs

8(30)

19 (70)

0

Prescription drugs

6 (22)

19 (70)

2b

a

ID#3: Interviewer administeredquestionnaire; participant did not comprehend the question.

b

ID#4: Interviewer administeredquestionnaire; participant refused to answer

ID#17: Self administeredquestionnaire; participant wrote in answer box ‘only what the doctors give’

NIH-PA Author Manuscript
NIH-PA Author Manuscript
Subst Abus. Author manuscript; available in PMC 2015 January 01.

McNeely et al.

Page 16

Table 4

NIH-PA Author Manuscript

Examples given by participants of substances belonging in ‘illegal drugs’ and ‘nonmedical (or recreational)
use of prescription medications’

NIH-PA Author Manuscript

Drug Class

Correctly classified (number of times mentioned)

Incorrectly classified (number of times mentioned)

Illegal drugs

Marijuana /weed/pot/haze/Thai stick (26)
Cocaine /coke (19)
Heroin /dope (12)
Ecstasy/MDMA (9)
Crack (5)
Crystal meth (2)
Acid (2)
Angel dust (1)
Mushrooms (1)
Opium (1)
Glue (1)
PCP (1)
Recreational drugs (1)
‘That date rape drug’ (1)

Tobacco (1)
Alcohol (1)
Prescription drug misuseclassified as ‘illegal’:
Barbiturates (1)
Methadone (1)
Percocet (1)
Xanax (1)
Klonopin (1)
Pills–‘ups or downers’ (1)

Prescription drugs
(nonmedical use or
recreational use)

‘Painkillers’ (7)
Percocet (5)
Vicodin (4)
Amphetamines/Adderall (3)
Xanax (3)
OxyContin (2)
Valium (2)
Morphine (2)
Codeine (1)
Tylenol with codeine (1)
Sleep aids (1)
Barbiturates (1)
Muscle relaxants (1)
Methadone (1)
Appetite suppressant (1)
‘Pills’ (1)

Tylenol (6)
Advil (3)
Antidepressants (3)
Cholesterol medications (2)
Cold medicine (2)
Blood pressure medications (1)
Naprosyn (1)
Cough syrup (1)
[P]enicilin (1)
Excedrin (1)
Allegra (1)

NIH-PA Author Manuscript
Subst Abus. Author manuscript; available in PMC 2015 January 01.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

7

0

Positive

No answer

25

1

0

Accurate

Inaccurate

No answer

Accuracy of screening response

19

Negative

Screening response

0

0

0

0

0

0

0/27 (0%)

26/27 (96%)

0

0

1

0

0

1

1/27 (4%)

0/27 (0%)

1/27 (4%)

26/27 (96%)

0/27 (0%)

7/27 (26%)

20/27 (74%)

27/27 (100%)

TOTAL
N (%)

Incorrect
N

Correct
N

Overall
understanding of
screening item

Imprecise
N

In the past 12 months, how many times have you used
any illegal drugs?
N=27

Questionnaire Item

Illicit Drug Use

0

0

8

0

2

6

8/16 (50%)

Correct
N

1

1

3

1

1

3

5/16 (31%)

Incorrect
N

0

1

2

0

0

3

3/16 (19%)

Imprecise
N

1/16 (6%)

2/16 (13%)

13/16 (81%)

1/16 (6%)

3/16 (19%)

12/16 (75%)

16/16 (100%)

TOTAL
N (%)

In the past 12 months, how many times have you used
any prescription drug for nonmedical reasons?
N=16

0

1

7

0

2

6

8/11 (73%)

Correct
N

1

1

0

1

1

0

2/11 (18%)

Incorrect
N

0

0

1

0

0

1

1/11 (9%)

Imprecise
N

1/11 (9%)

2/11 (18%)

8/11 (73%)

1/11 (9%)

3/11 (27%)

7/11 (64%)

11/11 (100%)

TOTAL
N (%)

In the past 12 months, how many times have you used
any prescription drug recreationally?
N=11

Prescription Drug Misuse

Screening results and accuracy of screening response for participants having correct versus incorrect or imprecise understanding of illicit (illegal) drug
use or prescription drug misuse (nonmedical or recreational use)

NIH-PA Author Manuscript

Table 5
McNeely et al.
Page 17

Subst Abus. Author manuscript; available in PMC 2015 January 01.