OMB NDTS 2011 Survey Testing Question Protocol and Summary

U. S. Department of Justice
National Drug Intelligence Center
National Drug Threat Survey 2011 (DRAFT)
Survey Question Testing Protocol and Summary

Overview
In response to input from Office of Management and Budget staff, the
National Drug Intelligence Center (NDIC), a component of the U.S. Department of
Justice, pre-tested several questions that are contained in the National Drug Threat
Survey (NDTS) 2011 data collection form. A test plan and testing protocol were
developed and a test group of nine potential NDTS respondents was selected for the
testing. The results of the testing are summarized below.

Testing Plan
A testing plan was developed to obtain feedback from a sample of potential
NDTS respondents. The plan included the following:


Develop a testing protocol to obtain feedback on various aspects of the NDTS
2011 survey instrument



Select a group of potential NDTS respondents to review the testing protocol
and provide feedback



Have each tester review the test question protocol and provide either written
or verbal responses to the questions listed for each survey item



Contact each tester for a follow-up telephone interview to elicit any additional
feedback



Summarize tester feedback



Modify NDTS 2011 survey instrument based on tester feedback

Question Testing Protocol
To lessen the time burden on the test participants, a subsample of NDTS 2011
questions was chosen for evaluation by the test sample members. The Office of
National Drug Control Policy (ONDCP) recommended additional questions on
drugged driving be added to the NDTS 2011 questionnaire and that a “Don’t Know”
response category be added to all applicable existing and new survey questions.
Further, OMB recommended that NDTS items that contained Likert-type response
categories (e.g., “Low”, “Moderate”, “High”) have defined categories. NDIC was
able to provide category definitions for the existing NDTS item on drug availability
(Question 2). Thus Question 2 and Question 12 that contains several sub-items on
drugged driving were chosen for review by the test group.


Question 2-Drug Availability – This question was selected for testing to elicit
feedback on three aspects of this survey item:
o Indicate if the definitions for “Low”, “Moderate”, “High” response
categories that were added to this question were understandable and
enabled the respondent to provide consistent responses for each of the
drugs listed in this item
o Indicate if the “Don’t Know” response category added to the question
is a valid response that can be used by respondents when they do not
know the availability of a specific drug in their jurisdiction
o Ensure that respondents are familiar with the drugs listed in the item,
especially “Controlled Prescription Drugs,” previously listed as
“Pharmaceuticals”



Questions 12a, b, c-Drugged Driving – This three-part question was selected
for testing to elicit feedback on three aspects of this survey item:
o Assess respondent familiarity with the term “drugged driving”
o Determine if respondents can provide drugged driving data in response
to the question and the timeliness of providing such data
o Determine the appropriate ranges for response categories for Question
12c
A copy of the testing protocol is appended to this report.

Question Testing Participants
A sample of nine potential NDTS respondents was chosen for the testing with
one respondent from each of the nine Organized Crime Drug Enforcement Task Force
(OCDETF) regions in the United States. NDTS results (statistical estimates) are
presented at the OCDETF region level as well as state and national levels. The table
below lists the participants, their law enforcement agency, type of agency, and the
date of the follow-up interview with NDIC.

NDTS 2011 Question Testing Participants
OCDETF Region

Test Participant

Agency Type

Interview
Date

Florida/Caribbean

Lieutenant Whitney Burnett
Alachua County, Florida, Sheriff’s Office and
Gainesville-Alachua County Drug Task Force

Metropolitan County
Sheriff

7/16/10

Great Lakes

Jeff Capretto, Special Agent in Charge
Westshore Enforcement Bureau (Ohio)

Suburban City
Police Departments

7/14/10

Mid-Atlantic

Sergeant Scott Haymaker
Johnstown, Pennsylvania, Police Department

Rural City
Police Department

7/15/10

New England

Roy McKinney, Director
Maine Drug Enforcement Agency

State
Drug Enforcement Agency

7/26/10

New York/New Jersey

Detective Anthony Sickinger
Union Township, New Jersey, Police Department

Suburban City
Police Department

7/29/10

Pacific

Captain Frank Romanaggi
Regional Organized Crime Narcotics Task Force

Metropolitan County
Sheriff/Police Department

8/3/10

Southeast

Sergeant Frank Young
New Orleans, Louisiana, Police Department

Metropolitan City
Police Department

7/13/10

Metropolitan City
Police Department

8/2/10

Rural County
Sheriff

7/26/10

Southwest

West Central

Sergeant Cole Lester
Houston, Texas, Police Department
Targeted Narcotics Enforcement Team
Lieutenant Pat Downs
La Plata County, Colorado, Sheriff’s Office
Southwest Drug Task Force

Summary of Question Testing Results
The NDTS 2011 Testing Protocol was emailed to eight of the nine testers –
Sergeant Scott Haymaker of the Johnstown, Pennsylvania, Police Department is on
temporary duty at NDIC through the National Guard and was able to complete the
protocol in person. Additionally, Sergeant Haymaker reviewed the entire NDTS 2011
data collection form and provided feedback on all survey items and the overall
accuracy and readability of the instrument.
Question 2 – Availability
The written protocol and follow-up interview focused on five specific
aspects of this survey item that had been included in the NDTS since 2003,
although several modifications were made for 2011 – definitions of the low,
moderate, and high response categories were added and a “Don’t Know”
response category was added to the item.
Responses to the five tester questions are summarized below:


Are you familiar with all of the drugs listed?
All respondents were familiar with the 12 drugs listed for each of the sub-items
of the question.



When assessing the availability of a drug in your jurisdiction, what specific drug
trafficking activities come to mind?
Responses focused on the accessibility of drugs in community, the volume of
street sales of drugs, the number of drug dealers in the community, the ease and
frequency of “street buys” by law enforcement, and the pattern of drug
distribution by drug trafficking organizations – all recognized components of the
broader term of “drug availability.”



Do the definitions of low, moderate, and high availability make sense to you?
All respondents agreed that newly-added definitions of the “Low”, “Moderate”,
“High” response categories made sense to them as a scale to rate the availability
of the 12 drugs listed for each of the sub-items of the question.



Do the definitions of low, moderate, and high availability allow you to accurately
indicate the availability of each drug in your jurisdiction?
All respondents agreed that newly-added definitions of the “Low”, “Moderate”,
“High” response categories allowed them to accurately rate the availability of
the 12 drugs listed for each of the sub-items of the question. Three of the nine

respondents further commented that the category definitions were relative to
their jurisdictions.


Are the “Not Available” and “Don’t Know” response categories self explanatory?
All respondents understood the “Not Available” and “Don’t Know” response
categories. One respondent added that the “Don’t Know” categories is
particularly useful as drug trends change within a jurisdiction and respondents
might not be able to rate the availability for a specific period of time.

Question 12 – Drugged Driving
The written protocol and follow-up interview focused on three specific
aspects of this survey item that was added to the NDTS for 2011 at the request
of the Office of National Drug Control Policy. The tester questions were
designed to obtain information from potential NDTS respondents concerning
their awareness of the issue of drugged driving and their ability to provide
drugged driving incident data for their jurisdictions.
Responses to the three tester questions are summarized below:


Are you familiar with the term “drugged driving?”

All respondents were familiar with the term drugged driving with one
commenting the he is in the process of becoming more familiar with it in terms
of a specific definition. He further added that as a result of being asked to review
this question, he plans address the issue with the law enforcement agencies
within his task force in terms of awareness and officer training and will discuss
the subject with courts within his agency’s jurisdiction.


Were you able to provide the number of violations in you jurisdiction easily, or
did it require research to obtain an answer?

All respondents generally indicated that they would not be able to respond to
this question, citing a number of reasons:






Driving under the influence of drugs or alcohol is not designated on
the agency’s DUI citation
Drug task forces (that provide drug enforcement activities for
member agencies) are not responsible for DUI enforcement
This information would be tracked by another unit within the agency
(e.g., vehicular crimes unit)
It would require extensive manual research of all DUI citations to
determine those that are drug-related
State-level drug agencies do not track such data



Are the response categories for question 12(c) too narrow or too broad?

Four of the six respondents who provided feedback for this item
indicated that the response categories were too narrow as their
responses would be based on estimates of the number of incidents
due to the fact that their jurisdictions do not collect or report this
information routinely. One respondent thought the categories were
acceptable and one respondent thought that incidents that involve
driving under the influence of drugs and alcohol need to be
distinguished from incidents of driving under the influence of drugs
only. The three respondents who did not provide feedback included
two task forces that do not have drugged driving enforcement
responsibility and an agency that has yet to address the issue of
drugged driving.

NDTS Modifications Based on Tester Feedback
The feedback received from the sample of testers was used to modify the
NDTS 2011 survey data collection form to enhance the data received through the
instrument. Specifically several modifications were made to the drugged driving
questions requested by ONDCP. The modifications discussed below have been
approved by ONDCP and are incorporated in the draft NDTS 2001 quesionnaire.


Adding the parenthetical qualifying sentence (highlighted below) to Question
12 to address feedback that many NDTS respondents may not be able to
respond to the sub-items in Question 12, especially sub-item 12c. The
qualifying sentence was added to encourage respondents to use the “Don’t
Know” response category, if applicable, thereby providing important feedback
for ONDCP for this first attempt to gather information on drugged driving.

Drugged Driving
12. Drugged driving is a serious consequence of illicit drug use. For purposes of this survey, drugged driving
is defined as driving after recent use of illegal drugs, including the non-medical use of controlled
prescription drugs. (Your responses to this question, even in the negative, will assist NDIC and federal
policymakers in guiding future research regarding drugged driving.)



Reducing the number of response categories for Question 12c and providing
wider numeric categories that, based on tester feedback, will enable
respondent to provide an “approximate” response to the number of drugged
driving violations based on available data.

Please indicate the approximate number of drugged driving violations that occurred in your jurisdiction in
the past year:
None

O

1–100

O

101–500

O

501–1,000

O

1,001 or more

O

Don’t Know

O

Appendix – Question Testing Protocol
U. S. Department of Justice
National Drug Intelligence Center
National Drug Threat Survey 2011 (DRAFT)
Survey Question Testing

The National Drug Intelligence Center (NDIC), a component of the U.S.
Department of Justice, conducts the National Drug Threat Survey (NDTS) annually.
The NDTS is designed to elicit information from state and local law enforcement
agencies related to the threats posed by the trafficking and distribution of various
illicit drugs in their jurisdictions. To ensure that NDIC collects relevant and accurate
information through the NDTS, we are asking for your assistance in evaluating
several new questions for the NDTS 2011 survey instrument (questionnaire). As a
member of the NDTS target audience, your insights and feedback are critical to our
efforts to continually improve the NDTS.
Participation in the NDTS 2011 Testing consists of two distinct phases:
1. Complete the two attached NDTS 2011 survey questions as if you were
actually responding to the survey. Review the Tester Questions for each
question and formulate any thoughts or suggestions to improve the questions
that you may have. If you prefer, you can type your thoughts and suggestions
into the test question document or you can provide your feedback verbally.
2. Participate in a follow-up interview with the NDTS administrator to provide
feedback and any additional information or suggestions for improvement.

The feedback obtained through the NDTS Testing Protocol will be used to
refine the NDTS 2011 survey instrument that is scheduled for dissemination in
December 2010.

Thank you for your assistance.
Phil Ponzurick
NDTS Administrator
National Drug Intelligence Center
Collection Management Group
(814) 532-4086

Drug Availability
2. Indicate the level of availability of the following drugs in your jurisdiction using the following definitions:
Low availability – drug is difficult to obtain most of the time; Moderate availability – drug is easily obtained most of the time;
High availability – drug is easily obtained at any time.
Not
Don’t
Not
Don’t
Low Moderate High Available Know
Low Moderate High Available Know
Powder cocaine
O
O
O
O
O
MDMA (ecstasy)
O
O
O
O
O
Crack cocaine
O
O
O
O
O
GHB
O
O
O
O
O
Heroin
O
O
O
O
O
LSD
O
O
O
O
O
Powder methamphetamine O
O
O
O
O
PCP
O
O
O
O
O
Ice methamphetamine
O
O
O
O
O
GBL
O
O
O
O
O
Marijuana
O
O
O
O
O
Controlled prescription drugs O
O
O
O
O

Tester Questions


Are you familiar with all of the drugs listed?



When assessing the availability of a drug in your jurisdiction, what specific drug
trafficking activities come to mind?



Do the definitions of low, moderate, and high availability make sense to you?



Do the definitions of low, moderate, and high availability allow you to accurately
indicate the availability of each drug in your jurisdiction?



Are the “Not Available” and “Don’t Know” response categories self explanatory?

Consequences of Illicit Drug Use
13. Drugged driving is a serious consequence of illicit drug use. For purposes of this survey, drugged driving
is defined as driving after recent use of illegal drugs, including the non-medical use of controlled
prescription drugs.
a.

Do you believe that drugged driving poses a safety threat in your jurisdiction?
Yes

O

No

O

Don’t Know

O

b. Does your agency provide drugged driving recognition training for your officers?
Yes

c.

O

No

O

Don’t Know

O

Please indicate the approximate number of drugged driving violations that occurred in your
jurisdiction in the past year:
None
1–25
26–50
51–75
76–100 
101–500
501–1,000
1,001 or more
Don’t Know

O
O
O
O
O 
O
O
O
O

Tester Questions


Are you familiar with the term “drugged driving”?



Were you able to provide the number of violations in you jurisdiction easily, or did it
require research to obtain an answer?



Are the response categories for question 12(c) too narrow or too broad?