Memo to OMB for Unmoderated Cognitive Test of SSV instrument

MEMORANDUM TO: Robert Sivinski

Office of Statistical Policy and Planning

Office of Management and Budget

THROUGH: Alexis R. Piquero, PhD

Director, Bureau of Justice Statistics (BJS)

Kevin M. Scott

Principal Deputy Director, BJS

FROM: Rich Kluckow, Corrections Unit Chief, BJS

Emily Buehler, Statistician, BJS

SUBJECT: Generic Information Collection Request: Unmoderated cognitive testing for the 2023 Survey of Sexual Victimization incident forms through the generic clearance agreement granted to BJS (OMB Number 1121-0339).

DATE: May 4, 2023

Request: BJS requests approval to use unmoderated cognitive testing for revisions to incident forms for the Survey of Sexual Victimization (SSV) under its generic clearance (OMB number 1121-0339) and in conjunction with cognitive testing already approved by OMB (ICR reference number 202202-1121-002).

The Survey of Sexual Victimization collects administrative data annually on the incidence of sexual victimization in adult correctional and juvenile facilities. The SSV is primarily collected via a self-administered questionnaire. Respondents are mailed a letter informing them of the requirement to complete the survey and providing them with access information. A preview of the questionnaire is available for respondents to download to review the survey questions and instructions.

Purpose: The SSV is an administrative data collection designed to collect information on allegations of sexual victimization by other inmates or staff that are reported to correctional and juvenile justice authorities. Additional information is collected on the victim(s), perpetrator(s), characteristics of the incident, and outcomes of substantiated incidents.

In 2022, staff from the Data Collections Methodology & Research Branch (DCMRB) of the Census Bureau met with the SSV sponsors from BJS and the Economic Reimbursable Surveys Division (ERSD) of the Census Bureau to discuss findings from an expert review. These findings and recommendations where then used to develop a protocol for early-stage scoping interviews. A participant count of (9) early-stage scoping interviews were conducted in summer of 2022 for the 2020 Survey of Sexual Victimization instrument forms SSV-1, SSV-2, SSV-3, SSV-4, SSV-5, SSV-6, SSV-IA, & SSV-IJ. These scoping interviews and a data quality review of previous years of SSV data were used to modify the data collection instruments. The data quality review included analysis of write-in responses and logic checks to explore if respondents were selecting conflicting response options in cases of multiple victims and perpetrators. Current best practices in survey design were also considered when revising the instruments.

In spring of 2023, the DCMRB, ERSD, and BJS began cognitive interviews to test modifications/revisions to existing questions and definitions as well as newly added survey items. These cognitive testing protocols are ongoing. The purpose of the unmoderated cognitive testing request is to collect more information on key items from a larger group of participants.

Because the SSV contains respondents from a variety of different adult correctional and juvenile justice facilities, contacting more respondents to provide feedback will better ensure saturation of these facility types. Additionally, the items to be tested could potentially be seen as sensitive topics (sex and gender identity, race/ethnicity, and disability) that some respondents may feel uncomfortable discussing with an interviewer and observers present. There is also increased interest among federal statistical agency stakeholders to collect more information about how administrative data responders interpret items related to sexual orientation and gender identity (SOGI) items and race and ethnicity items.

Respondents will be asked to review the definitions for sex and gender identity categories that will be included on the SSV incident forms. These are definitions for the following terms: intersex, gender nonconforming, and transgender.

The following items from the SSV incident forms will be included in the unmoderated cognitive testing:

• Victim sex and separate gender identity questions

  • There is a two-part question that asks about sex of victim first followed by gender identity of victim.

  • A response option “gender nonconforming,” which is a direct term from the PREA Standards for inmates or youth whose “whose appearance or manner does not conform to traditional societal gender expectations,” was added.

  • “Unknown” was also added as a gender identity response option.

• Victim race and ethnicity question revised

  • This question includes revisions that are in line with the current OMB public proposal to amend race and ethnicity data collection on federal statistical surveys

  • The response option of “Middle Eastern or North African” has been added.

  • The “other racial category in your information system- specify” was removed.

  • Response options were reordered slightly.

• Victim disability

  • A new item that asks whether the victim had been identified at the time of the incident as having a serious and persistent mental illness, a cognitive impairment or intellectual disability, or a physical impairment or disability.

• Staff sex and gender

  • The previous form asked only about the sex of staff. The revised form now asks both about sex and gender identity of the staff.

  • Response options of “transgender”, “gender nonconforming” and “unknown” were added to the gender identity question.

• Staff perpetrator race and ethnicity revised

  • This question includes revisions that are in line with the current OMB public proposal to amend race and ethnicity data collection on federal statistical surveys

  • The response option of “Middle Eastern or North African” has been added.

  • The “other racial category in your information system- specify” was removed.

  • Response options were reordered slightly.

The results from the unmoderated cognitive testing will be recorded, combined with results from the cognitive interviews, and a report will be produced that outlines the findings of the early-stage scoping testing and recommendations for improvement to questions. Data collected will include:

• Understanding how respondents comprehend specific questions

• Identifying respondents’ use of records and/or estimation strategies for answering specific questions

• Assessing respondents’ ability to answer specific questions

• Identifying difficulties in completing the questionnaires

• Soliciting feedback on how respondents approach questions on sex and gender identity

• Recommended changes to questions and response options to be implemented in the SSV

Population of Interest: State prison systems; state juvenile correctional systems; the federal prison system; U.S. Immigration and Customs Enforcement (ICE); the U.S. military; and a sample of jail jurisdictions, privately operated adult prisons and jails, facilities in Indian country, and locally and privately operated juvenile justice facilities.

Timeline: Unmoderated cognitive testing will be conducted from May-June 2023.

Language: Web-based testing instruments will be offered in English only.

Method: Outreach via email with a link to the survey instrument items being tested. Respondents will complete and submit their responses online through Qualtrics.

Sample: BJS and the Census Bureau team intend to contact no more than 500 prospective respondents with the goal of getting 100 completed responses. Contacts will be sent an email invitation to complete a short questionnaire online. Respondents will be asked to provide feedback on a select number of items that will be added to incident forms. These responses will be recorded in Qualtrics and later analyzed in NVivo by researchers from DCMRB.

All participants will be informed that their response is voluntary and that the information they provide is confidential and will be seen only by Census Bureau employees and those with special sworn status.

Table of respondent types and survey instruments to be tested:

Respondent affiliation	Recruitment goal number of responders	Incident form type
State prison system	15	SSV-IA
Local jails	30	SSV-IA
Private prisons, ICE facilities, U.S. military facilities, tribal jails	15	SSV-IA
State juvenile system	15	SSV-IJ
Local or private juvenile facilities	25	SSV-IJ

Recruitment: Respondents will be recruited via email. Respondents will be informed that their participation is voluntary and will receive a form for consent via Qualtrics. They will view test items, answer probed questions about these items, and submit responses in Qualtrics.

Protocol: The protocol for the study is enclosed (see Attachment A). BJS and DCMRB anticipate that each unmoderated cognitive test will take 15 minutes.

Use of Incentive: Monetary incentives for participation will not be offered.

Length of questionnaire: BJS and DCMRB plan to reach out to 500 potential respondents with a goal of receiving completed questionnaires from 100 representatives from adult and juvenile facilities. Each unmoderated cognitive test will last no more than 15 minutes (100 cases x 15 minutes per case = 25 hours). The recruiting emails are expected to take on average 1 minute to read. Thus, the estimated burden for the unmoderated cognitive testing of this project is (8 hours and 30 minutes for recruiting + 25 hours for questionnaire). The total burden for all testing is 33 hours and 30 minutes.

Based on the total burden hours (33 hours and 30 minutes) at an average labor cost for correctional officers of about $28.32 per hour in 2023 (based on the 2021 Bureau of Labor Statistics data https://www.bls.gov/oes/current/oes333012.htm), BJS estimated that the total labor cost to 500 respondents is approximately $948.

Summary of burden hours for SSV Unmoderated Cognitive Testing
Reporting mode	Purpose of contact	Number of respondents	Average reporting time	Total burden hours	Hourly Rate	Monetized value of respondent time
Email	Participant recruitment	500	1 min	8 hours, 30 minutes	$28.32	$241
Online questionnaire	Participate in Unmoderated Cognitive Test	100	15 mins	25 hours	$28.32	$708
Total				33 hours, 30 minutes		$948

Estimated Cost: The total estimated cost for the unmoderated cognitive testing for responding staff will be $948.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Emily D. Buehler, Ph.D.

Statistician

Bureau of Justice Statistics

202-598-1036

Emily.buehler@usdoj.gov

Krysten Mesner

Data Collection Methodology & Research Branch

Economic Statistics and Methodology Division

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-9852

Krysten.Mesner@census.gov

cc:
Nick Orsini (ADEP)

Amy Newman Smith (ESMD)

Amy Anderson Riemer  (ESMD)

Kimberley Moore (ESMD)

Magdalena Ramos (ESMD)

Aneta Erdie (ERD)

Naomi Blackman (ERD)

Greta Clark (ERD)

Kevin Scott (BJS)

Rich Kluckow (BJS)

Appendices:

Attachment A: Instrument protocol containing survey items to be tested and accompanying questions about interpretation of these items.

Attachment B: Consent form to obtain participant consent for participation and recording of the cognitive interview session

Attachment C. Recruitment email sent to participants