2021_SCLEA_OMB_Part B_v5

SUPPORTING STATEMENT – Part B

Collection of Information Employing Statistical Methods

2021 Survey of Campus Law Enforcement Agencies (SCLEA)

1. Universe and Respondent Selection

The 2021 Survey of Campus Law Enforcement Agencies (SCLEA) is planned as a census of all eligible agencies. The universe is defined as all law enforcement agencies (LEAs) serving any postsecondary educational institution which offers in-person instruction and has a full-time enrollment of at least 1,000 students.

The primary reason for the minimum enrollment threshold of 1,000 students is to maximize cost efficiency. According to fall 2018 IPEDS data, institutions with 1,000 or more students accounted for 97% percent of all 4-year and 2-year students enrolled nationwide. The remaining 3% of students were enrolled in smaller schools (less than 1,000 students), of which there were nearly 1,600. Including these smallest schools would be costly and add little to the body of knowledge about campus law enforcement.

Frame Design and Response Rates

The SCLEA frame is based on the National Center for Education Statistics’ (NCES) 2019 Integrated Postsecondary Education Data System (IPEDS; OMB 1850-0582), BJS’s Law Enforcement Agency Roster (LEAR), and BJS’s 2018 Census of State and Local Law Enforcement Agencies (CSLLEA; OMB 1121-0346). IPEDS provides information on postsecondary institutions, while the LEAR and CSLLEA provide information on the LEAs serving these institutions.

IPEDS is the most comprehensive data collection effort in U.S. postsecondary education. BJS used the most recent IPEDS (2019) to develop a complete list of postsecondary institutions. The 2019 IPEDS frame was subset to identify postsecondary institutions offering in-person instruction with a full-time student enrollment of more than 1,000 students. This subset serves as the basis for the SCLEA frame. The IPEDS frame was further subset to identify postsecondary institutions by other characteristics of interest, such as size of campus served (based on full-time student enrollment), private/public designation, and 2- or 4- year designation.

The LEAR is a list of all publicly funded law enforcement agencies operating in the United States. Most recently, data from the 2018 CSLLEA were merged into the LEAR to ensure that the LEAR contains the most accurate and timely data available, including contact information for the campus LEAs that responded to the 2018 CSLLEA. CSLLEA is the most comprehensive data collection for identifying active LEAs in the U.S. and serves as a frame for numerous other law enforcement collections. Within CSLLEA, campus LEAs are a specifically-targeted population and available to easily identify. In the 2018 CSLLEA, campus LEAs had a response rate of 95.6%. BJS appended campus LEA information from the LEAR to the IPEDS frame. The appended information included campus LEA point of contact (POC) with email address, campus LEA physical address, and campus LEA phone number.

For campus LEAs missing contact information in the 2018 CSLLEA, BJS first conducted online research in an effort to find the name of the head of the agency and their contact information. Many of the campus LEAs that were missing this information serve private institutions, as agencies serving private institutions are not included in the LEAR or CSLLEA; private institutions were identified through IPEDS. This online research included reviewing institutions’ Clery Act reports. The Clery Act requires colleges and universities that receive federal funding to disseminate a public annual security report each fall. The report typically includes contact information for the LEA serving the institution.

For those LEAs without contact information following the online research effort, BJS conducted a verification calling effort under BJS’ generic clearance (OMB 1121-0339) through which 325 campus LEAs were called and asked to provide a POC. As a result of these calls, contact information was obtained for 321 of these LEAs. Based on the data sources noted above, BJS estimates that 2,067 postsecondary institutions meet SCLEA’s criteria. The SCLEA universe is defined as all LEAs serving any postsecondary educational institution which offers in-person instruction and has a full-time enrollment of at least 1,000 students. To maximize the utility of SCLEA data, RTI and BJS plan to conduct a census of all agencies in the SCLEA universe.

The universe of SCLEA-eligible agencies is further stratified on three factors representing groups of substantive interest for estimates. All stratification is based on data from the IPEDS frame. The first factor separates postsecondary institutions with an enrollment of 1,000-2,499 full-time students from postsecondary institutions with an enrollment of 2,500 or more full-time students. The second factor is institution type, with separate strata for publicly and privately funded institutions. The third factor is whether the institution is a 4-year or a 2-year institution. Table 1 provides the SCLEA universe distribution among these strata.

This frame is expected to yield a response rate of 90% (1,860 completes). In the 2018 CSLLEA, campus LEAs had a response rate of 95.6%. Given the increased burden of SCLEA (60 minutes) as compared to the 2018 CSLLEA (30 minutes), BJS anticipates challenges with obtaining responses from private and smaller campus agencies due to limited time and resources in these types of agencies.

Table 1. SCLEA Postsecondary Institution Universe Distribution among Design Strata

2. Procedures for Collecting Information

Data collection procedures. SCLEA data collection is designed as a multi-mode method using a web survey as the primary mode, a hard copy survey as an alternative for respondents, and telephone nonresponse follow-up. The SCLEA data collection and nonresponse follow-up period will last approximately eight months. SCLEA data collection materials will include a pre-notification letter, mail invitation package, seven reminders, telephone nonresponse follow-up, and an end-of-study notification letter. A brief description of each contact method for campus LEAs is provided below and a complete timeline of SCLEA data collection is shown in Table 2.

• SCLEA pre-notification letter. The letter (Attachment 5), on BJS letterhead and signed by the BJS director, will be mailed via USPS to the chief executive of the campus LEA. This pre-notification will highlight the importance of SCLEA and encourage participation. It will also provide contact information that can be used to obtain additional information about SCLEA.

• SCLEA invitation package. Two weeks after the pre-notification letter, the invitation package will be mailed via USPS to all SCLEA POCs and will include three items: an invitation letter; SCLEA flyer; and a letter of support from the International Association of Campus Law Enforcement Administrators (IACLEA). The invitation letter (Attachment 6), on BJS letterhead and signed by the BJS program manager, will highlight the importance of SCLEA and encourage participation. The invitation letter will also provide instructions for accessing and completing the web survey questionnaire (including the web address, username, and password), how to update the agency POC (if required), contact information for obtaining additional information about SCLEA, and the data collection end date. The SCLEA flyer (Attachment 7) will highlight the uses and benefits of SCLEA data. The IACLEA letter of support (Attachment 8) will further emphasize the importance of SCLEA and provide contacts for additional information.

• SCLEA invitation email. One week after the invitation package is mailed, an email invitation will be sent to all SCLEA POCs with an email address. The email invitation (Attachment 9) will be identical to the invitation letter sent the week before, highlighting the importance of SCLEA, encouraging participation, and providing instructions for completion and contacts for additional information. Any email bounce backs will be reviewed on a case-by-case basis. If the bounce back indicates the email address is no longer active or invalid, the case will be flagged for removal from future emails. If the bounce back indicates a full inbox, the case will be included in future email distributions. For those that are flagged as invalid or no longer active from the first email, IACLEA membership files will be reviewed for new email addresses.

• SCLEA mail and email reminders. Two weeks after the invitation email is sent, the first hardcopy reminder letter will be mailed to nonrespondents (Attachment 11). One week later, that same letter will be emailed to nonrespondents (Attachment 11). The reminder letter and email will express the importance of the SCLEA and encourage response via the online survey (or paper copy, if preferred). Two weeks later, a reminder postcard will be sent to POCs (Attachment 12) that will include the website and their login information. A third reminder will be mailed two weeks later and will include a reminder letter (Attachment 13), a paper copy of the SCLEA survey (Attachment 2), and a business reply envelope. Three weeks later, a fourth reminder will be sent via email (Attachment 14). Two weeks later, a fifth reminder will be sent via mail (Attachment 15). Five weeks later, a sixth reminder will be sent via mail (Attachment 17). Each reminder will emphasize the importance of SCLEA, provide instructions for accessing and completing the survey (via web or mail), contact information for obtaining additional information about SCLEA, and the data collection end date. The contact schedule is summarized in Table 2.

Table 2. SCLEA Survey Contact Schedule

Week	Stage	Attachment Number
1	SCLEA pre-notification letter (mail)	5
3	SCLEA invitation package (mail)	6, 7, 8
4	SCLEA invitation (email)	9
6	SCLEA reminder letter #1 (mail)	11
7	SCLEA reminder letter #1 (email)	11
9	SCLEA reminder #2 (postcard)	12
11	SCLEA reminder #3 (mail reminder package)	13, 2
14	SCLEA reminder #4 (email)	14
16	SCLEA reminder #5 (mail)	15
19	SCLEA telephone nonresponse follow-up	16
21	SCLEA reminder #6 (mail)	17
25	SCLEA end-of-study notification letter (mail and email)	18
Ongoing	SCLEA thank you letter	10

• SCLEA telephone nonresponse follow-up. Follow-up with nonrespondents by telephone (Attachment 16) will begin three weeks after the fifth reminder and 19 weeks into the survey. As direct telephone contact can be labor-intensive, this process starts late in the survey time frame to limit the number of nonrespondents being contacted and will be concurrent with other mail/email reminders. Up to five call attempts will be made for each LEA before the case receives a “maximum call attempts reached” code. An attempt is defined as a call where an interviewer talks to the POC at the LEA or leaves a message on the POC’s answering machine or voicemail. During each attempt, the POC will be reminded of the purpose and importance of the survey and informed of the goal of receiving a completed survey from each LEA. The telephone interviewer will reference the most recent communication in the introduction of the phone call to determine if the POC has received any of the previous communications. Those who did not receive any of the letters, reminders, messages, or the invitation packet will be assisted by the interviewer in getting the information they need to complete the survey. For those who received the previous communications and/or the invitation packet, the interviewer will determine why they have not yet completed the survey, offer assistance, and try to gain cooperation. Given the telephone follow-up is taking place late in the survey timeframe, POCs who agree to complete the full survey will be asked to submit the survey online but will be sent another hard copy version of the survey if requested.

• SCLEA end-of-study notification. Six weeks after the start of telephone nonresponse follow-up, we will mail an end-of-study letter to LEA nonrespondents. The letter (Attachment 18), on BJS letterhead, will notify nonrespondents that the study is coming to an end and that their response is needed within two weeks. Data collection will continue for approximately three more weeks to allow for receipt of any remaining surveys. This letter will again provide instructions for accessing and completing the survey (via web or mail) and contact information for obtaining additional information. An end-of-study email will be sent concurrently with the end-of-study letter.

• SCLEA thank you correspondence. After SCLEA POCs complete their survey, a thank you will be sent to the POC. If the POC has an email on file, they will receive a thank you e-mail immediately following their submission. If the POC does not have an e-mail address on file, they will receive a thank you letter printed on BJS letterhead. Thank you letters will be mailed in batches on a biweekly basis. The thank you correspondence (Attachment 10) will thank the POC for their time and provide contact information for obtaining additional information about participation.

Data Editing. RTI will attempt to reconcile missing or erroneous data through automated and manual edits of each survey within two weeks of completion. In collaboration with BJS, RTI will develop a set of edit rules that use other completed survey fields to confirm acceptable responses or identify possible errors due to missing or inconsistent data elements. For example, if Q6 of the instrument had the race and ethnicity sworn officer breakdowns provided (Q6, a.-g.) but the total sworn officer summation (Q6, h.) missing, a manual edit would be made to compute a total value in (h.) based on (a.-g.) and cross-checked with Q5 to validate the total full-time sworn count. Through this type of process, RTI can quickly identify which cases require follow-up and indicate the items that need clarification or retrieval from the respondent.

Data Entry. Respondents completing the SCLEA survey via the web instrument will enter their responses directly into the online instrument. For those respondents returning their survey via hardcopy (mail), the survey will be scanned upon receipt and the data will be extracted from the TeleForm. RTI will perform a quality control check on randomly selected survey forms to ensure all data is scanned correctly. To confirm editing rules are being followed, RTI will review frequencies for the entered data and any anomalies, inconsistencies, or unexpected values will be investigated and resolved. Throughout the remainder of the data collection period, RTI staff will conduct regular data frequency reviews to evaluate the quality and completeness of data captured via web and hard copy. RTI will deliver at least three data files to BJS, including two preliminary files (at 50% and 75% response) and a final dataset upon completion of the survey.

Data Retrieval. When errors due to missing or inconsistent data elements are found during data review and editing, attempts to verify or collect the correct information with the respondent will occur. When it is determined that additional data retrieval is needed, an Agency Liaison (AL) will contact the respondent for clarification. Throughout the data retrieval process, RTI will document the questions needing retrieval (e.g. missing or inconsistent data elements), request clarification on the provided information, obtain values for missing data elements, and examine any other issues related to the respondent’s submission. Any issues will be investigated and resolved within two weeks.

Data Quality Review. RTI staff will conduct regular data quality reviews to evaluate the quality and completeness of data captured in both the web and paper copy modes. To confirm that editing rules are being followed, RTI will review frequencies for the entered data within one week of submission. Any issues will be investigated and resolved within two weeks. RTI will provide BJS with data quality updates as part of their weekly data collection status reports and include a summary in the final program report to be delivered to BJS once the dataset is cleaned and verified.

3. Methods to Maximize Response Rates

The 2011 SCLEA achieved a response rate of 90% amongst the core agencies and the 2018 CSLLEA had a response rate of approximately 96% from campus agencies. BJS and RTI will undertake various activities to ensure that high response rates are achieved for the 2021 SCLEA.

The SCLEA survey instrument was reviewed to ensure the collection of the most pertinent information, removing any unnecessary questions to reduce burden. BJS convened an expert panel to provide feedback on the 2011 questionnaire, noting items that were no longer relevant and identifying new topics of interest. The questionnaires were also reviewed by BJS and RTI staff for ease of use, flow, and compliance with questionnaire design best practices to ensure ease of administration. In addition, cognitive interviews were conducted on the SCLEA survey. More details are included in B.4, Testing of Procedures.

BJS will use a web-based instrument supported by several online help functions to maximize the response rate. The web survey interface is user-friendly, which encourages response and ensures more accurate responses. Because online submission is such an important response method, close attention will be paid to the formatting of the web survey instrument. The online application will be flexible so it can adapt to meet the needs of multiple device types (e.g., desktop computer, tablet, and phone), browser types (e.g., Microsoft Edge and Google Chrome), and screen sizes. Other features of the web instrument will include the following:

• Respondents’ answers will be saved automatically, and they will have the option to leave the survey partway through and return later to finish.

• The online instrument will be programmed with data consistency checks and automatic prompts to ensure inter-item consistency and reduce the likelihood of “don’t know” and out-of-range responses, thereby eliminating the need for follow-up with the respondent after survey submission. For example, the total number of full-time sworn officers by race or ethnicity and sex in question 6 will be checked against the total number of full-time sworn officers reported in question 5.

• The online instrument will also have a version of the survey that respondents can print out and mail back.

At all stages of the survey, a Help Desk will be available to provide both substantive and technical assistance. BJS will supply the Help Desk with answers to frequently asked questions and guidance on additional questions that may arise.

In the case that agencies may prefer to complete a paper survey, a hard copy of the SCLEA questionnaire will be sent to nonrespondents several weeks into the survey period. Additionally, an agency may contact the Help Desk at any time to request that a hard copy be mailed to them.

Multi-stage survey administration and follow-up procedures have been incorporated into BJS’s response plans to obtain higher response rates and to ensure unbiased estimates. Ensuring adequate response (not just unit/agency response rates, but also item responses) begins with introducing campus LEA POCs to SCLEA. This will be accomplished initially through the SCLEA pre-notification letter, followed by the invitation package, which will include a flyer describing the importance of the data collection and a letter of support from IACLEA. These introductory mailings will be followed by various reminders encouraging agencies to participate. Resources available to help SCLEA respondents complete the survey (e.g. telephone- or e-mail-based Help Desk support) will be described in each communication.

Nonresponse Adjustments

Despite a purposeful design and best efforts to collect complete data from all campus LEAs, some campus LEAs will not respond to the SCLEA survey request (i.e., unit nonresponse); among those agencies that do respond to the survey, some will not respond to particular questions (i.e., item nonresponse). In order to ensure all agencies in the SCLEA universe (as captured in the 2019 IPEDS and 2018 CSLLEA) are represented by SCLEA respondents, and to mitigate against any potential bias introduced by differential nonresponse, weight calibration will be used to finalize the dataset and during analysis. Throughout SCLEA data collection, RTI and BJS will monitor unit response rate goals and develop a weighting plan based on unit response rates and current weighting best practices for census data collections.

Campus LEA characteristics predictive of response propensity will be retained in final nonresponse adjustment models, stratified by the institution type and size categories. These nonresponse-adjusted weights will be used for estimation of agency characteristics measured in the SCLEA questionnaire.

Nonresponse bias analysis. In the event that SCLEA unit response falls below 80%, a nonresponse bias analysis will be used to assess the potential for bias resulting from differential nonresponse. For campus LEAs, agency and institution characteristics available for both respondents and nonrespondents (from 2019 IPEDS and 2018 CSLLEA) will be compared across response groups. BJS will use:

• Institution type (private or public; 2-year or 4-year);

• Institution size (measured by full-time enrollment), and;

• Geographic location (measured by Census region).

If any of these characteristics is found to differ meaningfully (as measured by Cohen’s d > 0.5) across response groups, then there is a potential indication of bias. Confirming nonresponse bias depends on how strongly each of these identified characteristics correlates to characteristics measured on the SCLEA survey. Survey measurements strongly associated with identified frame characteristics among SCLEA respondents are those most at risk for nonresponse bias. This analysis will take place both at the stratum level and overall, and frame characteristics associated both with response propensity and survey measurements among respondents will be incorporated in nonresponse weight adjustment models to mitigate against nonresponse bias. Additionally, since hard-to-reach agencies/institutions have lower relative response propensities, incorporating time to respond into nonresponse weighting models may help to ameliorate bias. To assess whether or not this is necessary, survey estimates will be compared across groups comprised of early and late responders. If meaningful differences are observed, time to respond will be included as part of nonresponse weight calibration.

3. Testing of Procedures

The proposed questions in the 2021 SCLEA instrument were developed through a multi-stage effort to update the 2011 SCLEA instrument. Those processes are described below.

• Expert Panel. BJS, RTI, and IACLEA hosted an Expert Panel meeting in July 2020 with representatives and experts in the area of campus policing. As a result of that meeting, BJS compiled feedback from the panel and was able to identify key measures to collect from campus law enforcement agencies and update the types of department technologies, policies, and trainings that are currently relevant to campus policing (Attachment 3).

• Cognitive Interviews. One round of cognitive testing was conducted with SCLEA POCs. The cognitive interviews focused on (1) the clarity of the instructions and question wording; (2) respondents’ ability and willingness to apply the study definitions when answering the questions; (3) the availability of data needed to provide accurate responses; (4) the estimated burden associated with participation, and (5) the SCLEA POC’s thoughts on efficient and effective data collection methodology. SCLEA cognitive testing included 14 respondents and led to minor changes, including clarifying instructions and replacing “nonsworn officers” with “nonsworn personnel”. RTI provided a report on the cognitive testing effort describing all findings and recommendations (Attachment 4).

Prior to the national implementation of the 2021 SCLEA, BJS and RTI will (1) conduct thorough testing of the web-based survey administration system through systematic user testing, including testing skip patterns, ensuring seamless reporting of data, and back-end data checks on entered responses, and (2) use respondent recruitment and support procedures informed by the above tests of procedures, which in many respects are the same as used on other successful BJS agency collections (e.g., LEMAS, CSLLEA), which have been field tested and successfully employed. These include mailing a pre-notification letter, letter of support, and offering several help functions to respondents.

Additionally, RTI has developed and utilized web-based survey instruments that are substantially similar to the format and design for the 2021 SCLEA. The web-based survey administration procedures successfully employed in similar BJS collections, such as LEMAS and Annual Surveys of Probation and Parole (ASPP; OMB 1121-0064), will be followed to ensure the successful administration of the 2021 SCLEA.

3. Contacts for Statistical Aspects and Data Collection

1. BJS contacts include:

Elizabeth Davis

202-305-2667

Elizabeth.Davis@usdoj.gov

Kevin M. Scott, Ph.D.

202-616-3615

Kevin.M.Scott@usdoj.gov

2. Persons consulted on statistical methodology:

Nicole Mack

RTI International

3. Persons consulted on data collection and analysis:

Dustin Williams Tim Smith

RTI International RTI International

Christian Genesky Tim Flanigan

RTI International RTI International