Word version of survey

Bullet Black Box Participant Background Survey

Note the actual survey will be available online using Google forms

The purpose of this study is to evaluate the accuracy, repeatability, and reproducibility of bullet comparisons by

firearms examiners. The study is being conducted by Noblis and NIST.

Participation will be open to US firearms examiners who have conducted operational casework in the past year. Participants are required to complete this on-line background questionnaire.

Questions regarding the study should be directed to firearms@noblis.org .

OMB Control # 0693-0033

Expiration Date: 7/31/2022

1. Enter the Participant ID number you were provided when you completed your registration.

2. Are you a firearms examiner who has conducted comparisons of bullets as part of casework in the United States in the past year? (for use in the U.S. criminal and/or civil justice system)

• Yes

• No

Training and Experience

3. Highest completed education level

• High school diploma (or equivalent)

• Associate degree

• Bachelor’s degree

• Master’s degree

• Doctorate

4. Number of years’ experience conducting firearms examination casework employing microscopic comparisons (after training)

• Less than 1 year

• 1-4 years

• 5-14 years

• 15 or more years

5. What percentage of time have you spent over the last year conducting firearm evidence comparisons?

• Less than 10%

• 10-25%

• 25-50%

• 50-75%

• 75-100%

6. Are you certified as a firearms examiner? (Check all that apply. Leave blank if not certified)

• Association of Firearm and Tool Mark Examiners (AFTE)

• Agency certified or qualified as a firearms examiner by a current or previous employer

• Other [TEXT REQUIRED]

7. Type of firearms examination training received

• Formal program of instruction for 1 year or more

• Formal program of instruction for 6 months to 1 year

• Formal program of instruction for less than 6 months

• Limited formal training (courses, workshops) for less than 6 months

• Other [TEXT REQUIRED]

8. Have you ever testified in court as an expert in firearms examination?

• Yes, more than 20 times

• Yes, 10- 20 times

• Yes, less than 10 times

• No

9. When did you last complete a proficiency test in firearms examination? (Exclude testing completed as part of your training or as competency tests)

• Within the last year

• Within the last 2 years

• Within the last 5 years

• More than 5 years

• Never

10. Who prepared your most recent proficiency test?

• Not applicable (have never completed a proficiency test)

• Collaborative Testing Services (CTS)

• Forensic Assurance

• Internally produced by my employer

• Produced by external partner agency

• Other [TEXT REQUIRED]

Your Agency and its Policies

11. Current Employment (If retired within the last year, use your most recent employer. Note that this study is limited to US firearms examiners.)

• U.S. Federal government

• U.S. State government

• U.S. City/County government

• U.S. Private sector (non-government)

• Individual examiner (sole practitioner)

12. Has your agency received accreditation in firearms examination?

• Yes

• No (my agency is not accredited)

• I don't know if my agency is accredited

13. Who accredited your agency in firearms examination? (Select all that apply. Leave blank if unknown or not accredited.)

• American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB)

• American Association for Laboratory Accreditation (A2LA)

• ANSI-ASQ National Accreditation Board (ANAB)

• Forensic Quality Services, Inc (FQS)

• Other [TEXT REQUIRED]

14. What conclusions does your agency use to report comparison findings in firearm evidence casework?

• Identification, Elimination, Unsuitable, and three subcategories of inconclusive (AFTE Range of Conclusions: https://afte.org/about-us/what-is-afte/afte-range-of-conclusions)

• Identification, Inconclusive, Elimination, Unsuitable (AFTE Range of Conclusions, but without subcategories of inconclusive)

• Other [TEXT REQUIRED]

15. Does your agency require blind verification of firearm evidence casework conclusions? (Blind verification is a comparison performed by a second examiner who does NOT know the primary examiner’s conclusion.)

• Yes: blind verification is always required (for all conclusions)

• Yes: blind verification is required, but only for some conclusions

• Yes: blind verification is performed on a randomly-selected subset of cases (or a percentage of cases)

• Yes: blind verification is performed for some cases depending on case characteristics

• No

16. Does your agency require (non-blind) verification of firearm evidence casework conclusions? (Verification is a comparison performed by a second examiner who DOES know the primary examiner’s conclusion.)

• Yes: verification is always required (for all conclusions)

• Yes: verification is required, but only for some conclusions

• Yes: verification is performed on a randomly-selected subset of cases (or a percentage of cases)

• Yes: verification is performed for some cases depending on case characteristics

• No

17. Does your agency have a minimum requirement for making an individualization (or identification) decision? (Check all that apply.)

• AFTE Range of Conclusions definition of Identification ("Agreement of a combination of individual characteristics and all discernible class characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool.")

• Quantitative Consecutive Matching Striae (qCMS)

• No specified minimum requirement

• Other [TEXT REQUIRED]

18. When conducting a comparison of bullets, does your agency allow making identification conclusions if the firearm is not available? (In other words: are you allowed to make an identification of crime scene bullets without a recovered firearm?)

• Yes

• No

• Not specified by agency policies or procedures

19. When conducting a comparison of bullets, does your agency allow making elimination (exclusion) conclusions when all class characteristics agree — when a recovered firearm is available to collect exemplar (test fired) bullets? (In other words: you have one or more crime scene bullets that agree on all class characteristics with exemplar bullets from a recovered firearm, but differ on individual characteristics, does your agency allow you to make an elimination conclusion?)

• Yes

• No

• Not specified by agency policies or procedures

20. If the firearm is not available when conducting a comparison of bullets, does your agency allow making elimination (exclusion) conclusions when all class characteristics agree? (In other words: if you do not have a recovered firearm, and you have two crime scene bullets that agree on all class characteristics, but differ on individual characteristics, does your agency allow you to make an elimination conclusion?)

• Yes

• No

• Not specified by agency policies or procedures

21. When making comparisons of multiple known bullets (Ks) to an unknown bullet (Q), do you intercompare Ks before you compare the Ks to the Q?

• Always intercompare Ks first

• Always compare the Ks to the Q first

• Varies from examination to examination

Your Bullet Comparison Casework

22. In your casework, approximately what proportion of examinations involve bullet comparisons (as opposed to cartridge case comparisons)? (select closest category)

• No bullet comparisons (100% of comparisons are cartridge cases) [SKIP TO END]

• 10% bullets (90% cartridge cases)

• 25% bullets (75% cartridge cases)

• 50% bullets (50% cartridge cases)

• 75% bullets (25% cartridge cases)

• 90% bullets (10% cartridge cases)

• 100% bullets (no cartridge cases)

23. Approximately what percentage of the evidence bullets that you receive in casework do you find suitable for comparison?

• Less than 10%

• 10-25%

• 25-50%

• 50-75%

• 75-100%

24. Of the fired bullets collected from crime scenes that you receive in casework, approximately what percentage are similar to the quality of an exemplar fired in a water tank (i.e. pristine or very good)?

• None

• Less than 5%

• 5-10%

• 10-25%

• 25-50%

• 50-100%

25. Of the bullet comparisons that you perform in casework, approximately what percentage have the recovered firearm(s) available to collect exemplar (test fired) bullets?

• Less than 10%

• 10-25%

• 25-50%

• 50-75%

• 75-100%

26. Of the bullet comparisons that you perform in casework, approximately what percentage involve the comparison of a fully separated jacket that is no longer attached to the bullet core?

• Less than 10%

• 10-25%

• 25-50%

• 50-75%

• 75-100%

End of survey

Thank you for your interest in the Bullet Black Box study. We will notify you when we ship packets of bullets for comparison. If you have any questions, please contact firearms@noblis.org .


This collection of information contains Paperwork Reduction Act (PRA) requirements approved by the Office of Management and Budget (OMB).  Notwithstanding any other provisions of the law, no person is required to respond to, nor shall any person be subject to a penalty for failure to comply with, a collection of information subject to the requirements of the PRA unless that collection of information displays a currently valid OMB control number.  Public reporting burden for this collection is estimated to be 10 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any aspect of this collection of information, including suggestions for reducing this burden, to the National Institute of Standards and Technology, Attn: Melissa Taylor <melissa.taylor@nist.gov>.