pdf version of instrument

NIST Human Factors in DNA
Survey_TEST MODE_FINAL
Start of Block: Introduction
OMB Statement
OMB Control #0693-0043
Expiration Date: 03/31/2022
NIST Generic Clearance for Usability Data Collections
A Federal agency may not conduct or sponsor, and a person is not required to respond to, nor
shall a person be subject to a penalty for failure to comply with an information collection subject
to the requirements of the Paperwork Reduction Act of 1995 unless the information collection
has a currently valid OMB Control Number. The approved OMB Control Number for this
information collection is 0693-0043. Without this approval, we could not conduct this
survey/information collection. Public reporting for this information collection is estimated to be
approximately 45 minutes per response, including the time for reviewing instructions, searching
existing data sources, gathering and maintaining the data needed, and completing and
reviewing the information collection. All responses to this information collection are voluntary.
Send comments regarding this burden estimate or any other aspect of this information
collection, including suggestions for reducing this burden to the National Institute of Standards
and Technology (NIST) point of contact: Melissa Taylor, melissa.taylor@nist.gov.

Page Break

Page 1 of 70

Thank you for taking part in this survey. The survey comprises questions relating to laboratory
management, DNA interpretation, cognitive bias, training and research, testimony and reporting
practices, and stakeholder engagement opportunities. Data from this survey will provide insight
to inform best practice recommendations for forensic DNA interpretation.
This survey should be completed by the DNA laboratory's TECHNICAL LEADER or
equivalent. This is the individual who is responsible for the technical oversight of the DNA
laboratory, which may include (but is not limited to) day-to-day quality assurance and
accreditation compliance, design and implementation of methods development, verification of
analytical instrumentation function, and validation of new technologies.

Page Break

Page 2 of 70

This survey is best taken on a computer rather than mobile device.

Your responses will be saved as you work through the survey. If you stop and wish to complete
at a different time, you will need to use the same device in order to resume at the last saved
point. Please complete the survey within two weeks of starting it or your response will be
recorded as incomplete.
End of Block: Introduction
Start of Block: Laboratory Information
Page Break

Page 3 of 70

Q2.1 This survey should be completed by the laboratory's technical leader or equivalent. Are
you a technical leader or equivalent?

o Yes
o No
o Not sure
Display This Question:
If This survey should be completed by the laboratory's technical leader or equivalent. Are you a tec...
!= Yes

Q2.2 If you answered "no" or "not sure" to the previous question, please end the survey here
and forward the invitation link to the technical leader or equivalent, within your laboratory.

o End survey now
o I am a technical leader or equivalent
Skip To: End of Survey If If you answered "no" or "not sure" to the previous question, please end the
survey here and forwa... = End survey now

Q2.3 What type of crime laboratory or forensic science service provider (FSSP) do you
represent?

o Publicly-funded local crime laboratory (to include city or town)
o Publicly-funded county crime laboratory
o Publicly-funded state crime laboratory
o Publicly-funded federal crime laboratory
o Private laboratory
o Consultant
o Other (please specify) ________________________________________________
Page 4 of 70

Q2.4 What region is your organization located in?

o New England (CT, ME, MA, NH, RI, VT)
o Mid-Atlantic (NJ, NY, PA)
o West North Central (IA, KS, MN, MO, NE, ND, SD)
o East North Central (IL, IN, MI, OH, WI)
o South Atlantic (DE, FL, GA, MD, NC, SC, VA, DC, WV)
o East South Central (AL, KY, MS, TN)
o West South Central (AR, LA, OK, TX)
o Mountain (AZ, CO, ID, MT, NV, NM, UT, WY)
o Pacific (AK, CA, HI, OR, WA)
o
Non-U.S. (please specify Country)
________________________________________________
Display This Question:
If What region is your organization located in? = Non-U.S. (please specify Country)

Q2.5 What continent is your laboratory in?

o Asia
o Europe
o South America
o Oceania (Australia, New Zealand)

Page 5 of 70

Q2.6 Who is (are) your primary customer(s)? (Select all that apply)

▢
▢
▢
▢
▢

Law enforcement (local, county, state, federal)
Prosecutor
Defense attorney
Private client

Other (please specify)
________________________________________________

Page 6 of 70

Q2.7 What Forensic DNA services are you providing to your primary customer(s)? (Select all
that apply)

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

Autosomal STR
Mitochondrial
Y-STR
Next Generation Sequencing
Mixture Interpretation
Probabilistic Genotyping
CODIS upload and search
Familial Searching
Forensic Genetic Genealogy
Paternity/parentage (criminal)
Paternity/parentage (non-criminal)
Phenotyping

Other (please specify)
________________________________________________

Page Break

Page 7 of 70

Q2.8 Does your laboratory track the TYPE of DNA samples that you routinely analyze? (e.g.,
track whether a sample is liquid blood, saliva stains, dried semen stains, touch DNA, etc.)

o Yes
o No
o Not sure
o Not applicable
Display This Question:
If Does your laboratory track the TYPE of DNA samples that you routinely analyze? (e.g., track
wheth... = Yes
Or Does your laboratory track the TYPE of DNA samples that you routinely analyze? (e.g., track
wheth... = Not sure

Q2.9 What are the categories that your laboratory uses to track DNA samples? (Select all that
apply)

o Bodily fluid type
o Case scenario
o Crime type
o Number of contributors
o Template amount
o Evidence item type (e.g., gun, clothing)
o Other (please list) ________________________________________________
o Not applicable
Page Break

Page 8 of 70

Q2.10 How many DNA analysts does your FSSP employ?

For the purpose of this survey, a DNA analyst is defined as: an employee or contract employee,
that successfully completed the laboratory's training requirements for casework sample
analysis, passed a competency test, and has entered into a proficiency testing program
according to these standards. This individual can conduct and/or direct the analysis of forensic
samples, interpret data, reach conclusions, and generate reports.

This definition includes both persons who process the DNA samples and those who perform the
statistical analysis and interpretation of the DNA results (for laboratories who separate these
functions).

o0
o 1-5
o 6-10
o 11-30
o 31-50
o >50
Q2.11 In your laboratory, do the same analysts perform both the analytical/instrument
processing and the interpretation of DNA results, or are these functions separated?

o Analysts perform all aspects of the analysis and interpretation
o These functions are separated
o Combination of both
o Other (please specify) ________________________________________________

Page 9 of 70

Q2.12 Does your laboratory or agency employ a human subjects officer (or similar)?

A human subjects officer (or similar) is a generally a person responsible for reviewing and
approving (or seeking appropriate approvals for) human-subjects research in the
laboratory/agency. They will likely coordinate and manage institutional review board (IRB)
activities and other compliance activities.

o Yes
o No
o Don't know
End of Block: Laboratory Information
Start of Block: Tasks that the lab performs
Q3.1 How often does your laboratory perform the following tasks?
Note: Direct-to-DNA is a DNA casework approach in which serology is removed from the
workflow as the initial screening of a Sexual Assault Kit sample, and instead, the initial

Page 10 of 70

screening is completed during the DNA quantification step to determine the level of male DNA
among female DNA to inform downstream processing.
Rarely
Presumptive
test for
semen

Sometimes

Often

Always

Not
applicable

o

o

o

o

o

Presumptive
test for saliva

o
o

o
o

o
o

o
o

o
o

Microscopic
search for
sperm

o

o

o

o

o

o
o
o

o
o
o

o
o
o

o
o
o

o
o
o

o

o

o

o

o

Presumptive
test for blood

Confirmatory
test for blood
Confirmatory
test for saliva
Y-STR typing
Direct-toDNA
approach

Display This Question:
If How often does your laboratory perform the following tasks? Note: Direct-to-DNA is a DNA
casewo... != Y-STR typing [ Rarely ]

Page 11 of 70

Q3.2 At what point in the workflow is Y-STR typing incorporated (Select all that apply)

▢
▢
▢
▢
▢
▢

At the biological screening stage
After quantitation
After initial autosomal STR results are obtained and evaluated.
When specifically requested by the client.
When specifically requested for court purposes.

Other (please specify)
________________________________________________

Display This Question:
If How often does your laboratory perform the following tasks? Note: Direct-to-DNA is a DNA
casewo... != Y-STR typing [ Rarely ]

Q3.3 Which criteria are used to inform the incorporation of Y-STR typing? (Select all that apply)

▢
▢
▢
▢
▢
▢

Based on screening results.
Based on male DNA/ratio results.
Based on autosomal STR results.
If specifically requested by the client.
If specifically requested for court purposes.

Other (please specify)
________________________________________________
End of Block: Tasks that the lab performs
Start of Block: Communication of results and testimony

Page 12 of 70

Page Break

Page 13 of 70

The following questions relate to reporting and testimony.

Q4.1 How are your laboratory's reports formatted?

o
Narrative (written explanations or paragraphs that describe evidence/items tested and
the DNA results and opinions)
o Tabular (lists and tables of the evidence/items tested and the DNA results and opinions)
o Combination
o Not sure
Q4.2 How and why did you select that format? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢

Clarity
Brevity
Aesthetic
Simplicity
It has always been formatted that way
Not sure

Other (please specify)
________________________________________________

Page 14 of 70

Q4.3 How/why did you select the specific terminology language that you use in your reports?
(Select all that apply)

▢
▢

Based on published research pertaining to effective communication

Based on best-practice recommendations from guidance bodies (e.g., ISFG,
ISO, NAS. etc.)

▢
▢
▢
▢
▢

Based on feedback from stakeholders (e.g., lawyers, investigators)
Based on internal research (e.g., in consultation with DNA analysts)
The language we use now is the language we've always used
Not sure

Other (please specify)
________________________________________________

Q4.4 Is your DNA laboratory reporting a quantitative value only or a combination of quantitative
and qualitative statements?

o Quantitative only (Likelihood Ratio or other numerical value)
o Qualitative only (verbal equivalent or written explanation)
o Quantitative and qualitative
o Not sure

Page 15 of 70

Q4.5 Does your laboratory have a standard operating procedure for testimony (to include
recommendations on how to testify to specific results)?

o Yes
o No
o Not sure
o Not applicable
Q4.6 Does your laboratory have a procedure to monitor testimony?

o Yes
o No
o Not sure
o Not applicable

Page 16 of 70

Q4.7 How often do analysts within your laboratory:
Never

Sometimes

About half
the time

Most of
the time

Always

Not
applicable

Solicit (and
receive)
feedback from
your customers
specific to the
comprehension
of your
reports?

o

o

o

o

o

o

Attend a pretrial conference
with the
prosecution?

o

o

o

o

o

o

Attend a pretrial conference
with the
defense?

o

o

o

o

o

o

Offer forensic
reports as
evidence
exhibits in
court? (rather
than simply
referring to
them during
testimony)

o

o

o

o

o

o

Use visual aids
during
testimony?

o

o

o

o

o

o

Page Break

Page 17 of 70

End of Block: Communication of results and testimony
Start of Block: Contextual information management / bias management / QA / QC

Page 18 of 70

Q5.1 Which of the following terms does your laboratory regularly use as part of your quality
management system? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

Analyst Error
Conflict
Deviation from protocol
Disagreement
Error
Incident
Instrument Error
Lapse
Mistake
Non-conformity
Quality Issue
Slip
Systematic Error
Technological Error
Unexpected finding

Other (please specify)
________________________________________________

Page 19 of 70

Q5.2 How does your laboratory define "error", "disagreement", "conflict", or any other related
terms that it regularly uses? Please include the term(s) and definition(s) here or write N/A.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page Break

Page 20 of 70

Q5.3 If consensus is not reached following a technical review, which of the following steps could
be engaged? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

Conversation between reviewer and analyst
Mediation by supervisor
Mediation by technical leader
Involve the quality manager
Re-amplification of sample by original analyst
Re-amplification of sample by second analyst
Independent re-interpretation by third party
Send to independent laboratory for complete re-analysis
Report the most conservative opinion
Report an inconclusive opinion
Report both opinions
Not reporting the case
No action

Other (please specify)
________________________________________________

Page 21 of 70

Q5.4 If any results or opinions are changed as a result of the review processes, how are the
disagreement/non-consensus and action documented? (Select all that apply)

▢
▢
▢
▢
▢

Report
Case file
Personnel file
Not documented

Other (please specify)
________________________________________________

▢

⊗Review process would not change results or opinions

Page 22 of 70

Q5.5 If any results or opinions are changed as a result of review processes, to whom are these
disclosed to? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢

Quality Manager
Technical Leader
Prosecution
Defense
Client (if not prosecution or defense)
Law Enforcement

Other (please specify)
________________________________________________

▢
▢

⊗No one (it is not disclosed)
⊗Review process would not change results or opinions

Page 23 of 70

Q5.6 If a change of result or opinion is disclosed, how is it disclosed? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢
▢
▢
▢
▢

⊗Not disclosed
Phone call to client
Email to client
Within report
Within routinely disclosed case file
Within case file disclosed upon request
Upon request
Oral testimony
Pre-trial case conference with prosecution
Pre-trial case conference with defense

Other (please specify)
________________________________________________

Page Break

Page 24 of 70

Q5.7 What type of risk assessment do you perform as part of your Quality Management System
(QMS)?

▢

Matrix-based (based on intersecting factors; for example, the likelihood that the
risk event will occur, and the potential impact that the risk event could have)

▢

Level-based (based on categories of risk tolerance, for example: acceptable
level, tolerable level, and intolerable level)

▢

Other (please specify)
________________________________________________

▢

⊗Not applicable

Display This Question:
If What type of risk assessment do you perform as part of your Quality Management System (QMS)?
!= Not applicable

Q5.8 What factors do you consider in your risk assessment? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢

Frequency
Effect on reported opinion
Likelihood of occurrence
Likelihood of detection
Cost
Severity

Other (please specify)
________________________________________________

Page Break

Page 25 of 70

Q5.9 How do you monitor DNA analysts' abilities to perform complex tasks (excluding routine
open proficiency testing), and how often?
Monthl
y

Quarterl
y

Biannuall
y

Yearl
y

Bienniall
y

When
require
d

Neve
r

Not
sur
e

In-house
testing/researc
h

o

o

o

o

o

o

o o

Internal
collaborative
exercises

o

o

o

o

o

o

o o

Training
exercises

o
o

o
o

o
o

o
o

o
o

o
o

o o
o o

Blind
proficiency
tests

o

o

o

o

o

o

o o

Other (please
specify or
select "never")

o

o

o

o

o

o

o o

Inter-laboratory
exchange

Page 26 of 70

Q5.10 Does your laboratory use any sort of blinding during casework?

Page 27 of 70

Yes, formalized
in SOPs

Yes, but not
formalized in
SOPs

No

Not sure

Sequential
unmasking (taskrelevant
information
presented in
sequential order,
for example
evidence/item
sample analyzed
before reference
sample)

o

o

o

o

Context
manager
(someone who
filters taskrelevant and
task-irrelevant
information and
only passes on
to the analyst
that which is
deemed to be
task-relevant)

o

o

o

o

Blind technical
review (reviewer
does not know
original analyst's
interpretation
opinion)

o

o

o

o

Blind reanalysis
of samples (reanalyze samples
without knowing
original result)

o

o

o

o

Blind review of
Number-ofContributors
assessments
(assess NoC
without knowing
original analyst's
opinion)

o

o

o

o

Page 28 of 70

Blind data review
(review data
without knowing
original analyst's
opinion)

o

o

o

o

Blind
interpretation of
Probabilistic
Genotyping
Software outputs
(interpretation of
PGS outputs
without knowing
original analyst's
interpretation)

o

o

o

o

Blind
genotyping/EPG
assessment
(assessment of
genotyping/EPG
without knowing
original analyst's
assessment)

o

o

o

o

Page Break

Page 29 of 70

Q5.11 Please indicate your level of agreement with the following statements.

Page 30 of 70

Note: Depending on the device that you are viewing this on, you may need to scroll across or
down to see all options.

Page 31 of 70

Strongly
disagree

Somewhat
disagree

Neither
agree nor
disagree

Somewhat
agree

Strongly
agree

The
interpretation
of DNA data
can be
influenced by
the
information
given to the
DNA analyst.

o

o

o

o

o

Bias can still
occur when
using
Probabilistic
Genotyping
Software.

o

o

o

o

o

The research
community
should
conduct more
studies about
contextual
bias in
forensic
biology
before our
laboratory will
change or
implement
contextual
information
management
policies.

o

o

o

o

o

We already
know enough
about
cognitive bias
in forensic
biology to
start making
policy
changes in
our
laboratory.

o

o

o

o

o

Page 32 of 70

Cognitive
bias can
affect
forensic
biologists'
interpretation
of DNA data.

o

o

o

o

o

All forensic
analysts
should be
aware of
cognitive
bias.

o

o

o

o

o

Page 33 of 70

Q5.12 Please indicate your level of agreement with the following statements.

Page 34 of 70

Strongly
disagree

Somewhat
disagree

Neither
agree nor
disagree

Somewhat
agree

Strongly
agree

Cognitive bias
is a bigger
issue for
analysts in
other forensic
disciplines
than those in
forensic
biology.

o

o

o

o

o

Knowing
about the
reference
profile before
examining a
complex DNA
mixture can
affect how a
DNA analyst
interprets the
mixture.

o

o

o

o

o

Knowing
about a
confession
before
examining a
complex DNA
mixture can
affect how a
DNA analyst
interprets the
mixture.

o

o

o

o

o

Knowing one
DNA analyst's
Number-ofContributor
determination
can affect
another DNA
analyst's
Number-ofContributor
determination.

o

o

o

o

o

Page 35 of 70

Likelihood
ratios prevent
contextual
bias in
forensic
biology.

o

o

o

o

o

Probabilistic
Genotyping
Software
prevents bias
in forensic
biology.

o

o

o

o

o

End of Block: Contextual information management / bias management / QA / QC
Start of Block: Probabilistic Genotyping
Page Break

Page 36 of 70

Q6.1 Here, we are interested in your laboratory's data interpretation methods, with an emphasis
on Probabilistic Genotyping Software (PGS). If your laboratory is not using PGS, please select
"not applicable" where appropriate.

Q6.2 Which of the following statistical analysis methods does your laboratory use for autosomal
DNA interpretation? (Select all that apply)

▢
▢
▢
▢

Combined Probability of Inclusion / Random Man Not Excluded
Likelihood Ratio
Random Match Probability

Other (please specify)
________________________________________________

Q6.3 Has your laboratory implemented, or is it in the process of implementing, PGS?

o Not using PGS and have no plans to
o Not using PGS but may in the future
o In the process of validating, but not using in casework
o Validated and online
o Validated, online, and in the process of validating an updated version
Display This Question:
If Has your laboratory implemented, or is it in the process of implementing, PGS? = Not using PGS
and have no plans to
And Has your laboratory implemented, or is it in the process of implementing, PGS? = Not using
PGS but may in the future

Page 37 of 70

Q6.4 What tool(s) is your laboratory using to calculate a statistic for DNA opinions? (Select all
that apply)

▢
▢
▢
▢

Spreadsheet
Manual
Not calculating a statistic

Other (please specify)
________________________________________________

Display This Question:
If Has your laboratory implemented, or is it in the process of implementing, PGS? = In the process of
validating, but not using in casework
Or Has your laboratory implemented, or is it in the process of implementing, PGS? = Validated and
online
Or Has your laboratory implemented, or is it in the process of implementing, PGS? = Validated,
online, and in the process of validating an updated version

Page 38 of 70

Q6.5 Which PGS is your laboratory using or in the process of implementing? (Select all that
apply)

▢

STRmix (include version number)
________________________________________________

▢

TrueAllele (include version number)
________________________________________________

▢

EuroForMix (include version number)
________________________________________________

▢

MixCal6 (include version number)
________________________________________________

▢

LiRa (include version number)
________________________________________________

▢

Other (please specify developer and version number)
________________________________________________

Display This Question:
If Has your laboratory implemented, or is it in the process of implementing, PGS? = In the process of
validating, but not using in casework
Or Has your laboratory implemented, or is it in the process of implementing, PGS? = Validated and
online
Or Has your laboratory implemented, or is it in the process of implementing, PGS? = Validated,
online, and in the process of validating an updated version

Q6.6 Does (or will) your laboratory train ALL DNA casework analysts to use and report PGS
outputs?

o Yes
o No
o Not applicable

Page 39 of 70

Display This Question:
If Does (or will) your laboratory train ALL DNA casework analysts to use and report PGS outputs? =
No

Q6.7 Approximately what percentage of fully trained DNA casework analysts are also trained to
use and report PGS outputs?
0 10 20 30 40 50 60 70 80 90 100
Please move slider to indicate the
percentage of DNA analysts in your
laboratory who are trained on PGS.

Page Break

Page 40 of 70

Q6.8 Does your laboratory routinely perform replicate amplifications?

▢
▢
▢

Yes
No

Case/sample dependent (please describe)
________________________________________________

▢

⊗Not applicable

Q6.9 Does your laboratory have a minimum amplification threshold based on quantification
results? (Select all that apply)

▢
▢
▢
▢

⊗No minimum threshold
Total amount of DNA
M:F ratio

Other (please specify)
________________________________________________

Page Break

Page 41 of 70

Q6.10 An idea being discussed in the DNA community is to create a central repository of
validation summaries that multiple laboratories could contribute data to and use. This repository
could be accessible to all stakeholders/interested parties (including attorneys and researchers),
or it could be password-protected and only available to other DNA laboratories (i.e., private).

Please read the following statements and select the one that best applies to your laboratory:

o Our laboratory would use a central repository, regardless of who can access it.
o Our laboratory would only use a central repository if it was private.
o I do not know if our laboratory would use a central repository.
o Our laboratory would not use a central repository.
o Validation summaries are not applicable to our laboratory.
Q6.11 Please read the following statements and select the one that best applies to your
laboratory:

▢

Our laboratory would contribute data to a central repository, regardless of who
can access it.

▢

Our laboratory would contribute data to a central repository, but only if it was

private.

▢
▢
▢

I do not know if our laboratory would contribute data to a central repository.
Our laboratory would not contribute data to a central repository.
Validation summaries are not applicable to our laboratory.

Q6.12 Please comment on why your laboratory would or would not be able to contribute to or
use such a repository.
________________________________________________________________

Page 42 of 70

________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page Break

Page 43 of 70

Q6.13 Some laboratories use internally-collected DNA samples for their validation studies (e.g.,
from staff members). Collecting samples in this way may restrict sharing data outside of the
laboratory due to privacy concerns.

Would your laboratory benefit from access to appropriately consented, externally-collected DNA
samples to use in your validation studies?

o We already obtain external DNA samples
o We do not currently obtain external DNA samples but would benefit from such samples
o No, we would not benefit
o Not sure
o Not applicable
Q6.14 Has your laboratory encountered any barriers to creating complex DNA mixture samples
for your internal validation exercises? Please discuss or type "not applicable".
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page Break

Page 44 of 70

Q6.15 Does your laboratory print out electropherograms for the case file?

o Yes
o No
o Not sure
o Not applicable
Q6.16 Does your laboratory save electronic data for all PGS runs even if some runs are not
used to render the final report conclusion (e.g., an alternate contributor number was evaluated
and rejected)?

o Yes, save all runs
o No, do not save all runs
o Not sure
o Not applicable
Q6.17 Does your laboratory have a method in place (other than routine network back-ups) for
tracking and maintaining the integrity of all saved electronic files related to PGS?

o Yes
o No
o Not sure
o Not applicable
End of Block: Probabilistic Genotyping
Start of Block: Internal Training Opportunities
Page Break

Page 45 of 70

The following questions relate to training.

Q7.1 To what level are DNA analysts within your laboratory trained? (Select all that apply)

▢
▢
▢
▢
▢
▢

To minimum accreditation standards
To perform relevant laboratory techniques
To explain case file content to stakeholders (separate from court testimony)
To participate in an admissibility hearing
To testify in court

Other (please specify)
________________________________________________

Carry Forward Selected Choices from "To what level are DNA analysts within your laboratory trained?
(Select all that apply)"

Q7.2
Please indicate all ways that performance is assessed for each task.

Note: Depending on the device that you are viewing this on, you may need to scroll across or

Page 46 of 70

down to see all options.

Oral Exam

Written Exam

Moot Court

We do not
assess
performance
on this task

Other

To minimum
accreditation
standards

▢

▢

▢

▢

▢

To perform
relevant
laboratory
techniques

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

▢

To testify in
court

▢

▢

▢

▢

▢

Other (please
specify)

▢

▢

▢

▢

▢

To explain
case file
content to
stakeholders
(separate
from court
testimony)
To participate
in an
admissibility
hearing

Q7.3 Does your laboratory provide in-house training for DNA analysts?

o Yes
o No
o No, but our laboratory does provide external training opportunities
o Not sure
Page 47 of 70

Skip To: End of Block If Does your laboratory provide in-house training for DNA analysts? != Yes

Q7.4
In choosing content for your laboratory's DNA analyst training program, did your laboratory
follow recommendations from any of the following groups? (Select all that apply)

▢
▢
▢
▢
▢
▢

FBI Quality Assurance Standards (QAS)
International Organization for Standardization (ISO)
International Society for Forensic Genetics (ISFG)
Scientific Working Group on DNA Analysis Methods (SWGDAM)
The Organization of Scientific Area Committees for Forensic Science (OSAC)

Other (please specify)
________________________________________________

▢
▢

⊗Not sure
⊗None of the above

Display This Question:
If In choosing content for your laboratory's DNA analyst training program, did your laboratory follo...
!= Not sure
And In choosing content for your laboratory's DNA analyst training program, did your laboratory
follo... != None of the above
Carry Forward Selected Choices from "In choosing content for your laboratory's DNA analyst training
program, did your laboratory follow recommendations from any of the following groups? (Select all that
apply)"

Page 48 of 70

Q7.5 How adequate are the documents provided by the groups you selected previously for
guiding training?
Extremely
inadequat
e

Somewha
t
inadequat
e

Neither
adequate
nor
inadequat
e

Somewh
at
adequate

Extremel
y
adequat
e

Not
sur
e

Not
applicabl
e

FBI Quality
Assurance
Standards
(QAS)

o

o

o

o

o

o o

International
Organization
for
Standardizati
on (ISO)

o

o

o

o

o

o o

International
Society for
Forensic
Genetics
(ISFG)

o

o

o

o

o

o o

Scientific
Working
Group on
DNA Analysis
Methods
(SWGDAM)

o

o

o

o

o

o o

The
Organization
of Scientific
Area
Committees
for Forensic
Science
(OSAC)

o

o

o

o

o

o o

o
o

o
o

o
o

o
o

o
o

o o
o o

o

o

o

o

o

o o

Other (please
specify)

⊗Not sure
⊗None of
the above

Page 49 of 70

Q7.6 Please elaborate on your views of the adequacies (or not) of the training documents that
your laboratory used to guide training.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________

Page Break

Page 50 of 70

Q7.7 How long does it usually take a DNA analyst to complete their training at your agency?

o 0-3 months
o 4-6 months
o 7-9 months
o 10-12 months
o >12 months
o Not sure
Q7.8 Would you like to see more national training efforts similar to programs offered by the
National Forensic Science Technology Center (NFSTC)?

o Yes
o No
o Not sure
o Do not know what the NFSTC offers

Page 51 of 70

Q7.9 Who provides training and continuing education to DNA analysts at your agency? (Select
all that apply)

▢
▢
▢
▢

Designated training coordinator (internal to agency)
Training team (internal to agency)
Occasional trainers (e.g., external speakers or vendors)

Other (please specify)
________________________________________________

▢
▢

⊗Not sure
⊗Not applicable

Display This Question:
If Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... = Designated training coordinator (internal to agency)

Q7.10 What percent of your designated training coordinator's time is dedicated to training
activities?
None
All
0

10 20 30 40 50 60 70 80 90 100

% of time spent on training activities

Display This Question:
If Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... = Training team (internal to agency)

Q7.11
Excluding any designated training coordinator, approximately how many Full Time Equivalent
positions in your training team are dedicated to supporting training and education?
For the purpose of this survey, a Full Time Equivalent (FTE) position is defined as equal to

Page 52 of 70

a 40 hour full-time working week. So, if you have two team members working as a training team
to provide training and education, and each team member spends half of their time on training
and education activities, you have 1 FTE team member dedicated to training and education
activities.

o < 0.5
o 0.5
o1
o 1.5
o2
o 2.5
o3
o 3.5
o4
o 4.5
o5+
Display This Question:
If Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... = Occasional trainers (e.g., external speakers or vendors)

Page 53 of 70

Q7.12
On average, how many times per year does an occasional trainer come to your agency to
provide training to DNA analysts?

o 1 or 2 times
o 3-5 times
o 6-10 times
o >10 times
o Not sure
o Not applicable
Display This Question:
If Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... != Not sure
And Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... != Not applicable
Carry Forward Selected Choices from "Who provides training and continuing education to DNA analysts
at your agency? (Select all that apply)"

Page 54 of 70

Q7.13 How were those who provide training and continuing education for DNA analysts in your
agency selected for their role? (Select all that apply for each factor contributing to trainer
selection)

Opt in/
volunte
er

Designat
ed
training
coordina
tor
(internal
to
agency)
Training
team
(internal
to
agency)
Occasio
nal
trainers
(e.g.,
external
speaker
s or
vendors)
Other
(please
specify)

⊗Not
sure

⊗Not
applicabl
e

Word of
mouth/
recommenda
tion

Relevant
professio
nal/
technical
experienc
e

Relevan
t training
experien
ce

Capacit
y/
availabil
ity

Best
fitting
role
for
the
perso
n
within
the
agen
cy

No
t
sur
e

Not
applica
ble

▢ ▢

▢

▢

▢

▢ ▢▢

▢ ▢

▢

▢

▢

▢ ▢▢

▢ ▢

▢

▢

▢

▢ ▢▢

▢ ▢

▢

▢

▢

▢ ▢▢

▢ ▢

▢

▢

▢

▢ ▢▢

▢ ▢

▢

▢

▢

▢ ▢▢
Page 55 of 70

Display This Question:
If Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... != Not sure
And Who provides training and continuing education to DNA analysts at your agency? (Select all that
a... != Not applicable
Carry Forward Selected Choices from "Who provides training and continuing education to DNA analysts
at your agency? (Select all that apply)"

Q7.14 Have those who provide training and continuing education for DNA analysts in your
agency completed any training on how to train others?
Yes

No

Not sure

Not applicable

Designated
training
coordinator
(internal to
agency)

o

o

o

o

Training team
(internal to
agency)

o

o

o

o

Occasional
trainers (e.g.,
external
speakers or
vendors)

o

o

o

o

o
o

o
o

o
o

o
o

o

o

o

o

Other (please
specify)

⊗Not sure
⊗Not
applicable

End of Block: Internal Training Opportunities
Start of Block: External Training Opportunities
Page Break

Page 56 of 70

Q8.1 Does your agency provide a budget to offer DNA analysts training from outside your
organization (e.g., to attend conferences, workshops, meetings, etc.)?

o Yes
o No
o Not sure
Display This Question:
If Does your agency provide a budget to offer DNA analysts training from outside your organization
(... != Not sure

Q8.2 How adequate is the budgeted amount (even if $0)?

o Extremely adequate
o Somewhat adequate
o Somewhat inadequate
o Extremely inadequate
o Not sure
Page Break

Page 57 of 70

Q8.3 Does your agency rely on external grants to provide DNA analysts training from outside
your organization?

o Yes
o No
o Not sure
Display This Question:
If Does your agency rely on external grants to provide DNA analysts training from outside your
organ... = Yes

Q8.4 Please specify the source(s) of the external grants that your agency relies on. (Select all
that apply)

▢
▢
▢
▢
▢

State Coverdell Funds
National Institute of Justice (NIJ) / Bureau of Justice Assistance (BJA)
National Institute of Standards and Technology (NIST)
Capacity Enhancement for Backlog Reduction (CEBR) grants

Other (please specify)
________________________________________________

Page Break

Page 58 of 70

Q8.5 What percentage of DNA analysts in your laboratory attend training external to your
organization each year?
0 10 20 30 40 50 60 70 80 90 100
% of DNA analysts who attend external
training each year

Q8.6 How is information from external training opportunities shared with others in your agency?

Report to
supervisor
Presentation to
colleagues
Written
summary
Other (please
specify)

Always

Often

o
o
o
o

o
o
o
o

Sometimes

o
o
o
o

Never

o
o
o
o

Page Break

Page 59 of 70

Q8.7 Do DNA analysts at your agency have access to peer-reviewed publications (e.g.,
journals, conference proceedings)?

o Yes
o No
o Not sure
Q8.8 How adequate is the budget to provide DNA analysts with access to peer-reviewed
publications (even if $0)?

o Extremely adequate
o Somewhat adequate
o Somewhat inadequate
o Extremely inadequate
o Not sure
Q8.9 Does your agency run a "journal club" or similar for DNA analysts?

Note: a journal club is typically an informal meeting to discuss peer-reviewed journal articles,
book chapters, or conference proceedings.

o Yes
o No
o Not sure

Page 60 of 70

Q8.10 Are there any incentives or requirements for DNA analysts in your agency to read peerreviewed publications? (Select all that apply)

▢
▢
▢
▢
▢

Yes - Accreditation
Yes - Promotion
Yes - Remuneration
No

Other (please specify)
________________________________________________

▢
▢

⊗Not sure
⊗Not applicable

Page 61 of 70

Q8.11 If your agency had unlimited resources to enhance the training and continuing education
of DNA analysts in your laboratory, what would be your TOP TWO priorities?

▢
▢
▢
▢
▢
▢
▢
▢

Funding for external conferences
Access to peer-reviewed publications
A dedicated training coordinator
An expanded training team
Funding for occasional training (e.g., from vendors)
Better training and education materials
My agency already has all the training and education resources needed

Other (please specify)
________________________________________________
End of Block: External Training Opportunities
Start of Block: Stakeholder engagement
Page Break

Page 62 of 70

Q9.1 The final section of this survey relates to stakeholder engagement.

Q9.2 Which stakeholder groups does your laboratory engage with throughout the process of a
DNA examination? (Select all groups and stages that apply)
Submission
of evidence

Processing
of
evidence

Law
enforcement

▢

▢

▢

▢ ▢ ▢ ▢

Prosecution

▢

▢

▢

▢ ▢ ▢ ▢

Defense

▢

▢

▢

▢ ▢ ▢ ▢

Judge

▢

▢

▢

▢ ▢ ▢ ▢

Defendant

▢

▢

▢

▢ ▢ ▢ ▢

Complainant

▢

▢

▢

▢ ▢ ▢ ▢

Other
FSSPs

▢

▢

▢

▢ ▢ ▢ ▢

Reporting
results

Pretrial

Trial

Posttrial

Never

Page 63 of 70

Q9.3 How important is communication between your laboratory and the stakeholders:
Extremely
important

Very
important

Moderately
important

Slightly
important

Not at all
important

Not
applicable

When
evidence/items
are received by
the laboratory
for DNA
processing?

o

o

o

o

o

o

During the
processing of
evidence/items?

o

o

o

o

o

o

When reporting
results?

o

o

o

o

o

o

In preparing for
trial (i.e., pretrial)?

o

o

o

o

o

o

During the trial?

o

o

o

o

o

o

At the
conclusion of a
trial? (i.e., posttrial)

o

o

o

o

o

o

Page Break

Page 64 of 70

Q9.4 How important is communication between your laboratory and stakeholders about the
following types of information when evidence/items are received by the laboratory for DNA
processing? Please rank in order of importance with 1 being the most important.
Note: click and drag each option to move into order of importance.
______ Information about probative items
______ Information about the number of items
______ Information about case prioritization (rush, trial status, etc.)
______ Information about the case scenario
______ Information about the crime scene

Q9.5 How important is communication between your laboratory and stakeholders about the
following types of information during the processing of DNA evidence? Please rank in order of
importance with 1 being the most important.

Note: click and drag each option to move into order of importance.
______ Information about DNA standards
______ Information about CODIS eligibility
______ Information about additional items and/or rounds of DNA testing

Q9.6 How important is communication between your laboratory and stakeholders about the
following types of information after the processing of DNA evidence? Please rank in order of
importance with 1 being the most important.

Note: click and drag each option to move into order of importance.
______ Information about CODIS confirmation standards
______ Information about case discovery
______ Information about pre-trial hearings
______ Information about pre-trial interviews
______ Information about the trial

Page 65 of 70

Page Break

Page 66 of 70

Q9.7 How satisfied are you with the communication between your laboratory and the
stakeholders at each phase of the case?
Extremely
satisfied

Slightly
satisfied

Neither
satisfied
nor
dissatisfied

Slightly
dissatisfied

Extremely
dissatisfied

Not
applicable

When
evidence/items
are received
by the
laboratory for
DNA
processing

o

o

o

o

o

o

During the
processing of
evidence

o

o

o

o

o

o

When
reporting of
results

o

o

o

o

o

o

In preparing
for trial (i.e.,
pre-trial)

o

o

o

o

o

o

During the trial

o

o

o

o

o

o

At the
conclusion of a
trial (i.e., posttrial)

o

o

o

o

o

o

Page 67 of 70

Q9.8 What mechanisms are in place for stakeholders to access information from your
laboratory? (Select all that apply)

▢
▢
▢
▢
▢
▢
▢
▢
▢

Phone
Email
Website
Laboratory director
DNA technical leader
Case manager
Report
Site visit

Other (please specify)
________________________________________________

Q9.9 How important is it for your laboratory to communicate with stakeholders regarding quality
incidents such as contamination, sample loss, analyst error, changed procedure, misconduct, or
negligence?

o Extremely important
o Very important
o Moderately important
o Slightly important
o Not at all important
o Not applicable
Page 68 of 70

Q9.10 How important is it for your laboratory to communicate with stakeholders regarding the
following quality incidents? Please rank in order of importance with 1 being the most important.

Note: click and drag each option to move into order of importance.
______ Contamination
______ Sample loss
______ Analyst error
______ Changed procedure
______ Misconduct
______ Negligence

Q9.11 What resources would improve communication between your laboratory and the
stakeholder? (Select all that apply)

▢
▢
▢

Glossary
Training

Technology solutions, software packages, RMS communication
pathways/interfaces

▢
▢
▢

Web resources/how-to guides
Informational bulletins

⊗Other (please specify)

________________________________________________

Page Break

Page 69 of 70

End of Block: Stakeholder engagement
Start of Block: Feedback
Q10.1 If you wish to provide any feedback or additional comments about this survey, please use
the box below or email nikola.osborne@nist.gov

Click next to complete the survey.
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
________________________________________________________________
End of Block: Feedback

Page 70 of 70