Measuring Accuracy Facial Forensics Comparisons - Informed Consent

Attachment A – ITL-0021

Informed Consent to Participate in Research

Principal Investigator: P. Jonathon Phillips, Ph.D.

Study Title: Measuring the accuracy of facial forensics comparisons

Study Site(s): Laboratory or office location of the participants.

Introduction

You are being asked to take part in a research study. Research studies include only people who choose to take part. This document is called an informed consent form. Please read this information carefully and take your time making your decision. Ask the researcher or study staff to discuss this consent form with you, please ask him/her to explain any words or information you do not clearly understand. The nature of the study, risks, inconveniences, discomforts, and other important information about the study are listed below.

The person who is in charge of this research study is P. Jonathon Phillips, Ph.D. This person is called the Principal Investigator. However, other NIST research staff may be involved and can act on behalf of the person in charge.

This research is being sponsored by the Federal Bureau of Investigation (FBI).

Purpose of the study

The primary purpose of this study is to measure the accuracy of facial forensic examiners when they compare faces in two different photos. Facial forensic examiners will use their laboratory tools and methods and compare the two face images. The goal of comparing the faces is to determine if the two faces are of the same person. From the answers provided by the forensics examiners, we can measure their accuracy. A second purpose of this study is to determine if the accuracy of facial forensic examiners is superior to other groups of people with expertise similar to facial examiners. We make this assessment by measuring the accuracy of face super-recognizers, fingerprint examiners, and non-examiner face experts at comparing faces.

A goal of biometrics is to develop face algorithms at least as accurate as humans, and the accuracy of facial forensic examiners establishes benchmarks for algorithms. The data from this study will be used in establishing benchmarks for algorithm performance; developing face recognition algorithms; and validating algorithms including their accuracy. Data from this study can help develop and validate different models of how humans recognize and match faces, process visual images, and make cognitive decisions. The data may be used to assist in developing benchmark tests for examiners and non-examiners.

To assist in understanding which factors affect accuracy, we ask participants to complete an online questionnaire of demographic information and their forensic training and their job experience. We measure the accuracy of these four groups by giving all participants 20 to 30 pairs of face images. Participants are asked to compare the faces in the two images and rate how similar the two faces are on a seven-point scale. From the ratings, we calculate the accuracy of forensic examiners.

Why are you being asked to take part?

We are asking you to take part in this research study because you might fit into one of the four categories: facial forensic examiner, a face super-recognizer, a fingerprint examiner without experience or training in facial forensic examination, or a face expert without experience or training in facial forensic examination. To be eligible to participate you need to have access to email, the World Wide Web (the Web), and ability to download images that are to be compared. If you are using your employer’s computers, email, laboratory tools, methods, or procedures, as applicable, to participate, you must have your employer’s permission. You must also have your employer’s permission if you will participate in study activities during your normal working hours. The access to email, the Web, and downloading images is required for communicating with participants, taking the questionnaire and submitting test results, and obtaining the images.

Study Procedures:

If you take part in this study, you will be asked to:

• Schedule an interview with a NIST researcher

• Confirm you have your employer’s permission to use your employer’s computers, email, laboratory tools, methods, or procedures, if applicable, to perform facial comparisons.

• Review, prior to the interview, a copy of the consent form we will send to you by email or fax.

• Participate in a phone interview in which we will:

  • Review the goals of the project.

  • Explain the consent form.

  • Answer any questions you may have.

  • Ask you if you want to sign the consent form. You will choose to sign or not to sign the form. Then the form is either scanned and securely emailed or faxed to NIST. The NIST researcher who explained the consent form to you signs the form and sends the form to you by email or fax. If the form is faxed, the arrangements will be made so that you are present when the form is faxed.

  • Ask you a series of screening questions to determine if you fit into one of the three categories of people who are eligible to participate in this study. If you do not fit into one the three categories, your participation in the study will end at this step and your data will be destroyed.

• Fill out a background questionnaire online, through a secure survey service, Survey Gizmo, which will ask demographic questions as well as questions concerning your work experience.

• Download 20-30 pairs of face images that you are to compare. You will be emailed instructions for downloading the face image pairs from NIST’s secure fileserver. You may only use the images for the purposes of this study.

• Compare 20-30 pairs of face images.

• Rate the similarity between each pair on the following scale:

  • +3: The observations strongly support that it is the same person

  • +2: The observations support that it is the same person

  • +1: The observations support to some extent that it is the same person

  • 0: The observations support neither that it is the same person nor that it is different persons

  • -1: The observations support to some extent that it is not the same person

  • -2: The observations support that it is not the same person

  • -3: The observations strongly support that it is not the same person

• Rate the easiness or difficulty on making each comparison on the following scale:

  • 1 – Easy: The comparison was easier than most facial comparisons

  • 2 – Moderate: The comparison was a typical facial comparison

  • 3 – Difficult: The comparison was more difficult than most facial comparisons

  • 4 – Very difficult: The comparison was unusually difficult, involving significant photometric, illumination, or pose changes, other red flags

  • 5 – Not possible: The comparison was virtually impossible, due to a lack of detail in the image(s)

• Submit your ratings online via a secure online survey service, Survey Gizmo. Complete the comparisons in the order listed in the online survey. Enter the ratings for a face-pair prior to starting the next comparison. Please do not consult or discuss this study with friends, family, or colleagues from your institution or other institutions.

• You will have three months to complete the background survey and all comparisons, and finalize your submissions. If you do not finalize your submissions within three months, we will withdraw you from the study.

• You will receive an email from the PI once a month reminding you about the study. In the third month, you will receive a reminder once a week. Once you have completed the study, you will not receive any reminder emails. Once you have completed the experiment, you will receive an email reminding you can request your test score if you wish to do so.

• After ratings for all image-pairs are entered, you will be asked if your entries are final. If yes, you will be asked to click the submit button. Prior to clicking the submit button, you can change your answers. The survey can be accessed in multiple sessions during the course of the experiment. You may withdraw from this study until you click the final submit button.

We will record the results of your ratings of similarity between faces; your ratings of easiness or difficulty of a comparison; and the answers to the screening and background questionnaires. If you agree to participate, your answers to the screening questionnaire and secure online background questionnaire will be included as part of the research data. We will keep this data and your ratings data to perform detailed analysis and for re-analysis and meta-analysis at a later date.

Total Number of Participants

The study is being administered by NIST and up to 300 individuals will take part in this study.

Voluntary Participation / Withdrawal

You should only take part in this study if you want to volunteer. You should not feel that there is any pressure to take part in the study. You are free to participate in this research or withdraw at any time. There will be no penalty if you stop taking part in this study. A decision not to participate will not affect your job status.

You may withdraw from this study until you click the final submit button on the questionnaire for entering ratings from comparing the face images. If you wish to withdraw from the study, you will need to email or phone the PI. If you choose to withdraw, all of your data, including questionnaire answers and face-image comparisons, will be destroyed. If you withdraw, we will not be able to compute your accuracy, and you will not be able to request your score.

Benefits

The potential benefits of participating in this research study include receiving your score. You can request your score after you complete the study. You have to request your score within four months of signing the consent form. If requested, we will provide your score on the comparison portion of the study only after the distribution of accuracies is known and the study has been published as a report or paper. The score will be provided to only you via the postal address you may provide for this purpose. We will not give your score or contact information to anyone else.

Participants electing to request their test score are advised to consider first consulting with their agency’s counsel or counsel of their choice. In some legal systems, knowledge of your test score may create an obligation for the participant or the participant’s employer to disclose the test score in a criminal, civil or regulatory proceeding for which they are called to testify or provide evidence. All requests for a participant’s test score must be made in writing by email or postal mail. When the results for the experiment are published, the key linking participant’s name with their test results will be destroyed. This will make it impossible to recover a participant’s test score after the results are published.

Risks or Discomfort

This research is considered minimal risk. That means that the risks associated with this study are the same as what you may encounter every day (e.g., working on an office computer, or taking an online survey). There are no known additional risks to those who take part in this study.

Compensation

You will receive no payment or other compensation for taking part in this study.

Costs

It will not cost you anything to take part in the study.

Privacy and Confidentiality

We will keep your study records private and confidential. Certain people may need to see your study records. Anyone who looks at your records must keep them confidential. These individuals include:

• The NIST research team, including the Principal Investigator, study coordinator, and all other NIST research staff.

• Certain government people who need to know more about the study, and individuals who provide oversight to ensure that we are doing the study in the right way.

• Any agency of the federal, state, or local government that regulates this research such as the FBI and Office for Human Research Protections (OHRP) within the Department of Health and Human Services (DHHS).   

To communicate with you for administering the experiment and providing test scores, we will collect your name, email, and postal address. At the time of enrollment, we will assign you a study number. Concurrently with publishing the results and mailing requested test scores, the key linking study numbers to subjects’ identities (name, email, postal address) will be destroyed.

A master list linking your personally identifiable information to the study number will be kept in an encrypted disk on a government computer located in a locked office at NIST or on a government encrypted disk in a locked cabinet at NIST. The master list will link the background surveys to the appropriate scores; the master list will not contain any names, emails, or postal addresses. The master list will be kept since there is the potential for this data to be useful many years beyond the point of study completion. The file that includes your study code and other study information is called the de-identified data set. The de-identified data set will be made available to other researchers including Prof. Alice J. O’Toole of the University of Texas at Dallas and Dr. David White of the University of New South Wales, Australia. 

Future research: We will keep all data, including identifiable data, for future research. Future research will be reviewed by an IRB that has the responsibility to make sure research studies are conducted in an ethical manner that protects the rights and welfare of subjects. For future research, only the de-identified data set will be shared outside NIST; no one outside the study team will receive the master list.

Confidentiality of your records will be protected to the extent possible under existing regulations and laws including the Freedom of Information Act, but cannot be guaranteed. We may publish what we learn from this study. If we do, we will not include your name. We will not publish anything that would let people know who you are. 

You can get the answers to your questions, concerns, or complaints

If you have any questions, concerns or complaints about this study, or experience an unanticipated problem, or research-related injury, call P. Jonathon Phillips, Ph.D., at 301-975-5348.

If you have questions about your rights as a participant in this study, or have complaints, concerns or issues you want to discuss with someone outside the research, call the NIST Human Subjects Protection Office at (301) 975-5445.

You will receive a copy of this signed consent form.

Consent to Take Part in this Research Study

I freely give my consent to take part in this study. I understand that by signing this form I am agreeing to take part in research. I also confirm the following by initialing the box next to each item that applies to my participation in the study:

I have my employer’s permission to perform study activities in the course of my

work duties.

I have my employer’s permission to use my work computer and email address to

perform study activities.

I have my employer’s permission to use the employer’s laboratory tools, methods,

and/or procedures, as may be applicable, to perform facial comparisons for this study.

I have received a copy of this form to take with me.

_____________________________________________ ____________

Signature of Person Taking Part in Study Date

_____________________________________________

Printed Name of Person Taking Part in Study

Statement of Person Obtaining Informed Consent

I have carefully explained to the person taking part in the study what he or she can expect from their participation. I confirm that this research subject demonstrated proficiency in English, the language that was used to explain this research, and is receiving an informed consent form in English. This research subject has provided legally effective informed consent.

______________________________________________ ____________

Signature of Person obtaining Informed Consent Date

______________________________________________

Printed Name of Person Obtaining Informed Consent

Version # Page 6 of 6