Bullet Black Box Participant Background Survey

7/27/22, 1:16 PM

Background Survey

Bullet Pattern Analysis Black Box Study — Background Survey
ExaminerID: P7269
The purpose of this study is to evaluate the accuracy, repeatability, and reproducibility of bullet comparisons by firearms examiners. The study is being
conducted by Noblis and NIST. Participation will be open to US firearms examiners who have conducted operational casework in the past year. Participants
are required to complete this on-line background questionnaire.
Questions regarding the study should be directed to firearms@noblis.org .

OMB Control Number: 0693-0033

Expiration Date: 07/31/2022
Training and Experience
1. Highest completed education level  

1A: High school diploma (or equivalent)

1B: Associate degree

1C: Bachelor's degree

1D: Master's degree

1D: Doctorate

2.   Number of years' experience conducting firearms examination casework employing microscopic comparisons (after training).

2A: Less than 1 year

2B: 1-4 years

2C: 5-14 years

2D: 15 or more years
3

What percentage of time have you spent over the last year conducting firearms evidence comparisons?

https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

1/8

7/27/22, 1:16 PM

Background Survey

3.   What percentage of time have you spent over the last year conducting firearms evidence comparisons?

3A: Less than 10%

3B: 10-25%

3C: 25-50%

3D: 50-75%

3E: 75-100%
4.   Are you certified as a firearms examiner (Select all that apply. Leave blank if not certified.)
4A: Association of Firearm and Tool Mark Examiners(AFTE)
4B: Agency certified or qualified as a firearms examiner by a current or previous employer
4C: Other
5.  Type of firearms examination training received

5A: Formal program of instruction for 1 year or more

5B: Formal program of instruction for 6 months to 1 year

5C: Formal program of instruction for less than 6 months

5D: Limited formal training (courses, workshops) for less than 6 months

5E: Other

6.   Have you ever testified in court as an expert in firearms examination?

6A: Yes, more than 20 times

6B: Yes, 10-20 times

6C: Yes, less than 10 times

6D: No
7.  When did you last complete a proficiency test in firearms examination? (Exclude testing completed as part of your training or as competency
https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

2/8

7/27/22, 1:16 PM

Background Survey

tests)

7A: Within the last year

7B: Within the last 2 years

7C: Within the last 5 years

7D: More than 5 years

7E: Never
8.  Who prepared your most recent proficiency test?

8A: Not Applicable (have never completed a proficiency test)

8B: Collaborative Testing Services (CTS)

8C: Forensic Assurance

8D: Internally produced by my employer

8E: Produced by external partner agency

8F: Other

Your Agency and its Policies
9.  Current Employment

(If retired within the last year use your most recent employer. Note that this study is limited to US firearms examiners.)

9A: U.S. Federal Government

9B: U.S. State Government

9C: U.S. City/County Government

9D: U.S. Private sector (non-government)
https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

3/8

7/27/22, 1:16 PM

Background Survey

9E: Individual Examiner (sole practitioner)
10.  Has your agency received accreditation in firearms examination?

10A: Yes

10B: No (my agency is not accredited)

10C: I don't know if my agency is accredited
11.  Who accredited your agency in firearms examination?

(Select all that apply. Leave blank if unknown or not accredited.)

11A: American Society of Crime Laboratory Directors / Laboratory Accreditation Board (ASCLD/LAB)

11B: American Association for Laboratory Accreditation (A2LA)

11C: ANSI-ASQ National Accreditation Board (ANAB)

11D: Forensic Quality Services, Inc (FQS)
11E: Other

12.  What conclusions does your agency use to report comparison findings in firearm evidence casework?

12A: Identification, Elimination, Unsuitable, and three subcategories of Inconclusive ( AFTE Range of Conclusions (https://afte.org/about-us/what-isafte/afte-range-of-conclusions))

12B: Identification, Inconclusive, Elimination, Unsuitable (AFTE range of conclusions, but without subcategories of Inconclusive)

12C: Other

13.  Does your agency require blind verification of firearm evidence casework conclusions?

(Blind verification is a comparison performed by a second examiner who does NOT know the primary examiner's conclusion.)

13A: Yes: blind verification is always required (for all conclusions)

13B: Yes: blind verification is required, but only for some conclusions
13C: Yes: blind verification is performed on a randomly-selected subset of cases (or a percentage of cases)
https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

4/8

7/27/22, 1:16 PM

Background Survey

13C: Yes: blind verification is performed on a randomly selected subset of cases (or a percentage of cases)

13D: Yes: blind verification is performed for some cases depending on case characteristics

13E: No

14.  Does your agency require (non-blind) verification of firearm evidence casework conclusions?

(Verification is a comparison performed by a second examiner who DOES know the primary examiner's conclusion.)

14A: Yes: verification is always required (for all conclusions)

14B: Yes: verification is required, but only for some conclusions

14C: Yes: verification is performed on a randomly-selected subset of cases (or a percentage of cases)

14D: Yes: verification is performed for some cases depending on case characteristics

14E: No
15.   Does your agency have a minimum requirement for making an individualization (or identification) decision (Select all that apply)
15A: AFTE range of conclusions definition of identification ("Agreement of a combination of individual characteristics and all discernible class
characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the
agreement demonstrated by toolmarks known to have been produced by the same tool.")
15B: Quantitative Consecutive Matching Striae (qCMS)
15C: No specified minimum requirement:
15D: Other
16.  When conducting a comparison of bullets, does your agency allow making identification conclusions if the firearm is not available?

(In other words: are you allowed to make an identification of crime scene bullets without a recovered firearm?)

16A: Yes

16B: No

16C: Not specified by agency policies or procedures
17.  When conducting a comparison of bullets, does your agency allow making elimination (exclusion) conclusions when all class characteristics
agree — when a recovered firearm is available to collect exemplar (test fired) bullets?
(I th
d
h
i
b ll t th t
ll l
h
t i ti
ith
l b ll t f
d fi
b t diff
https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

5/8

7/27/22, 1:16 PM

Background Survey

(In other words: you have one or more crime scene bullets that agree on all class characteristics with exemplar bullets from a recovered firearm, but differ on
individual characteristics, does your agency allow you to make an elimination conclusion?)

17A: Yes

17B: No

17C: Not specified by agency policies or procedures
18.  If the firearm is not available when conducting a comparison of bullets, does your agency allow making elimination (exclusion) conclusions
when all class characteristics agree?
(In other words: if you do not have a recovered firearm, and you have two crime scene bullets that agree on all class characteristics, but differ on individual
characteristics, does your agency allow you to make an elimination conclusion?)

18A: Yes

18B: No

18C: Not specified by agency policies or procedures
19.  When making comparisons of multiple known bullets (Ks) to an unknown bullet (Q), do you intercompare Ks before you compare the Ks to the
Q?

19A: Always intercompare Ks first

19B: Always compare the Ks to the Q first

19C: Varies from examination to examination

Your Bullet Comparison Casework
20.  In your casework, approximately what proportion of examinations involve bullet comparisons (as opposed to cartridge case comparisons)?

(Select closest category)

20A: No bullet comparisons (100% of comparisons are cartridge cases)

20B: 10% bullets (90% cartridge cases)

20C: 25% bullets (75% cartridge cases)
20D 50% b ll t (50%

t id

)

https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

6/8

7/27/22, 1:16 PM

Background Survey

20D: 50% bullets (50% cartridge cases)

20E: 75% bullets (25% cartridge cases)

20F: 90% bullets (10% cartridge cases)

20G: 100% bullets (no cartridge cases)
21.   Approximately what percentage of the evidence bullets that you receive in casework do you find suitable for comparison?

21A: Less than 10%

21B: 10-25%

21C: 25-50%

21D: 50-75%

21E: 75-100%
22.   Of the fired bullets collected from crime scenes that you receive in casework, approximately what percentage are similar to the quality of an
exemplar fired in a water tank (i.e. pristine or very good)?

22A: None

22B: Less than 5%

22C: 5-10%

22D: 10-25%

22E: 25-50%

22F: 50-100%
23.   Of the bullet comparisons that you perform in casework, approximately what percentage have the recovered firearm(s) available to collect
exemplar (test fired) bullets?

23A: Less than 10%

23B: 10-25%
https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

7/8

7/27/22, 1:16 PM

Background Survey

23C: 25-50%

23D: 50-75%

23E: 75-100%
24.  Of the bullet comparisons that you perform in casework, approximately what percentage involve the comparison of a fully separated jacket
that is no longer attached to the bullet core?

24A: Less than 10%

24B: 10-25%

24C: 25-50%

24D: 50-75%

24E: 75-100%

Thank you for your interest in the Bullet Black Box study. We will notify you when we ship packets of bullets for comparison. If you have any questions, please
contact firearms@noblis.org .

This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB).
Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a
collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number. Public
reporting burden for this collection is estimated to be 10 minutes per response, including the time for reviewing instructions, searching existing data sources,
gathering and maintaining the data needed and completing and reviewing the collection of information.

Send comments regarding this burden estimate or any aspect of this collection of information, including suggestions for reducing this burden, to the National
Institute of Standards and Technology, Attn: Melissa Taylor melissa.taylor@nist.gov.
SUBMIT

https://script.google.com/macros/s/AKfycbz48NN8TiiRgcwGz_cgEJgoGidXpLtUbUrpffTgxtMH4FOBv6Y/exec?page=survey

8/8