2020_LEMAS_Part_B_05272020_omb_BJS_clean

Part B. Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The target population for the 2020 Law Enforcement Management and Administrative Statistics (LEMAS) survey is all local and county police, sheriffs’ offices, and primary state police agencies that employ the equivalent of at least one full-time sworn officer (FTE).¹ The 2020 LEMAS will use the Law Enforcement Agency Roster (LEAR), a list of all publicly funded law enforcement agencies operating in the United States. The LEAR is routinely updated to reflect the changing landscape of law enforcement agency (LEA) operation. Most recently, data from the 2018 Census of State and Local Law Enforcement Agencies (CSLLEA) were merged into the LEAR to ensure that the LEAR contains the most accurate and timely data available. A sample of agencies will be drawn from the LEAR to serve as the sampling frame for the 2020 LEMAS.

Sampling Frame

The LEAR lists all LEAs known to BJS. It includes all general purpose (i.e., local/county/regional police departments, sheriffs’ offices, and the 49 primary state and highway patrol agencies) and special purpose LEAs (e.g., park police, transit, tribal, campus, and independent school districts) in the United States of which BJS is aware. In 2016, the LEAR was used as the universe from which the sample for the 2016 LEMAS was drawn. To produce the LEMAS sampling frame, the LEAR is first filtered to relevant agencies. This removes cases that are out of service, special purpose, not publicly funded, provide law enforcement services under a contract with another agency, or do not employ at least one FTE sworn officer. The LEAR is continuously updated when new information is obtained on agencies though data collections or media announcements. This update process is managed through the Agency Records Management System (ARMS), a database where BJS manages agency information collected through LEMAS and CSLLEA administrations as well as from periodic reviews of state Peace Officer Standards and Training (POST) commissions or offices, state chief and sheriffs’ association lists, agency websites, and FBI Police Employee Data. In addition to including eligible agencies, ARMS includes agencies that are out of service, covered by other law enforcement agencies, fee for service, duplicate entries, were found to not engage in law enforcement, and do not have at least one full-time equivalent. This information is necessary to maintain due to the evolving nation of law enforcement in the U.S. The LEAR is a frame of eligible agencies that can be in the LEMAS. New information is committed to the LEAR after proper vetting. Data on eligibility, staffing, and contact information from the most recent law enforcement data collection, the 2018 CSLLEA, have been pushed into the LEAR through ARMS.

A majority of current case information for LEMAS-eligible agencies comes from the 2018 CSLLEA, which achieved a 93% response rate. For non-respondents to the 2018 CSLLEA, additional information was used to establish in-service status, eligibility, and agency staffing size. Additional data sources integrated into the LEAR included: the 2018 FBI’s Police Employee Data, data from state POST commissions or offices, state chief and sheriffs’ association lists, the 2016 LEMAS, and the 2008 and 2014 CSLLEA. Information from these sources was merged and vetted, resulting in the most comprehensive and up-to-date list of law enforcement agencies known.

LEMAS Sampling Designs and Response Rates

The LEMAS has been administered 10 times from 1987 to 2016. This historical experience has led to the development of a well-refined sampling strategy that facilities a LEMAS sample that is representative of the broader population of LEAs. The LEMAS uses a stratified simple random sample design in which LEAs are stratified by agency type and agency size. Agency type has three categories: (1) local police, (2) sheriff’s offices, and (3) state police. To obtain a representative sample of all agency sizes, agency type is further stratified by agency size. Agency size is split into seven categories: (1) 1-1.5 FTE, (2) 2 – 4.5 FTEs (3) 5 – 9.5 FTEs, (4) 10 – 24.5 FTEs, (5) 25 – 49.5 FTEs, (6) 50 – 99.5 FTEs, and (7) 100 or more FTEs. In the LEMAS, LEAs with 100 or more FTEs are sampled with certainty making these agencies self-representing (SR). For agencies with less than 100 FTEs (non-self-representing), past administrations of the LEMAS have employed various allocation methods depending on the analytic goals of that particular administration.

The LEMAS has traditionally experienced a high response rate; the 2016 LEMAS had an overall response rate of 80.1 percent. However, as seen in Table 1, the response rate varied by agency type and agency size. As a result, we will assume a response rate that differs by agency type and size. Lower response rates are assumed for sheriffs and smaller agencies.

Table 1: 2016 LEMAS survey response rates, by agency type and size

Agency Type	Agency Sizea	Sample Size	Response Rate
Local Police	100+	640	88.4
	50-99.5	132	81.8
	25-49.5	272	85.3
	10-24.5	542	84.5
	5-9.5	504	80.4
	2-4.5	402	72.4
	1-1.5	148	64.0
Sheriff’s Office	100+	358	75.7
	50-99.5	62	79.0
	25-49.5	99	79.8
	10-24.5	159	66.7
	5-9.5	90	76.7
	2-4.5	38	63.2
	1-1.5	4	50.0
State	All	49	89.8

^a Number of full-time equivalent sworn officers.

Past experience with the administration of the LEMAS suggests that response rates will be lower among smaller agencies and sheriff’s offices. Lower response rates and differential response rates are recognized as potential threats to the reliability of the 2020 LEMAS; several steps will be taken to identify and correct any issues caused by such an occurrence. First, tests for potential bias in response will be performed by comparing agencies on known characteristics, including agency size, and type. Second, the likelihood of potential bias will be reduced through nonresponse adjustments (discussed in Section 3). The goal of this survey is to have unbiased estimates that can be obtained with lower response rates with the proper adjustments. Table 2 presents the assumed response rates for the 2020 LEMAS.

Table 2: Assumed response rates for the 2020 LEMAS, by agency type and self-representation status

Agency Type	Self-Representation Status	Response Rate
Local Police	Self-representing	90%
	Non-self-representinga	80%
Sheriff’s Office	Self-representing	80%
	Non-self-representinga	75%
State Police	Self-representing	90%
All agencies		81%

^a Non-self-representing agencies are comprised of agencies with less than 100 FTEs.

Sample Allocation and Sample Size

As was done in prior LEMAS surveys, the 2020 LEMAS will include all large law enforcement agencies with 100 or more full-time equivalent sworn officers; these SR agencies are found in cells numbered 7, 14 and 15 in Table 3. This ensures the ability to measure change overtime within a consistent set of large agencies. The 2020 LEMAS will sample within the other cells in Table 3; these are the non-self-representing (NSR) agencies with less than 100 FTEs.

Table 3: Sampling Strata for the 2020 LEMAS

Agency Size Categories	Local Police	Sheriff Offices	State Police
One FTE Sworn Personnel	1	8
2 to 4.5 FTE	2	9
5 to 9.5 FTE	3	10
10 to 24.5 FTE	4	11
25 to 49.5 FTE	5	12
50 to 99.5 FTE	6	13
100 or More FTE	7	14	15

The NSR agencies have traditionally been subdivided into six strata based on the number of FTE. These groupings have been used historically by BJS for reporting key characteristics of officers and agencies. They also serve as a stratification factor since independent samples would be drawn from each agency-type-by-size stratum. Such stratification will increase the statistical precision of estimates at both officer and agency levels. The size cut points have been used in prior LEMAS sampling designs; these cut points are retained in the 2020 LEMAS for consistency between current and past data collections.

The 2020 LEMAS survey addresses a wide array of topics. Six statistics have been identified as appropriate to consider:

• Agency-level statistics— (1) average annual operating budget, (2) percent of agencies using body-worn cameras (BWC), and (3) percent of agencies with community policing in their mission statement

• Personnel-level statistics— (4) number of full-time sworn officers (FTS), (5) number of full-time non-sworn (FTNS) officers, and (6) percent of sworn officers that are female

Historic data from the 2016 LEMAS were used to estimate the mean, variance, and standard deviation of the six statistics in each stratum (see Table 4 and Table 5).

Table 4: Estimates of means/percentages of selected statistics, by stratum, 2016 LEMAS

Agency Type	Agency Sizea	Budget ($1,000s)	BWC Usage (%)	Community Policing (%)	Number FTS Officers	Number FTNS Staff	Female Officers (%)
Local Police	100+	74,711	44.6	89.6	462.4	137.1	11.7
	50-99.5	11,001	45.2	90.0	68.0	19.3	8.8
	25-49.5	4,993	29.3	86.9	34.5	8.5	9.2
	10-24.5	1,722	42.3	85.1	14.0	3.1	8.1
	5-9.5	519	49.2	84.2	5.3	0.6	6.1
	2-4.5	179	47.6	80.3	2.1	0.2	5.0
	1-1.5	68	38.1	66.4	0.8	0.1	4.6
Sheriff Office	100+	85,949	33.7	73.4	337.6	247.8	13.9
	50-99.5	13,286	30.7	74.4	64.5	44.2	11.6
	25-49.5	5,776	42.3	69.9	31.9	25.0	8.1
	10-24.5	2,310	44.0	61.4	14.6	9.3	6.0
	5-9.5	996	43.2	61.2	6.5	4.9	6.4
	2-4.5	431	29.9	71.1	3.0	1.4	7.7
	1-1.5	94	0.0	42.2	1.0	0.0	0.0
State Police	All	242,407	13.6	45.5	1,217.2	626.9	6.2

^a Number of full-time equivalent sworn officers.

Table 5: Estimates of standard deviations of selected statistics, by stratum, 2016 LEMAS

Agency Type	Agency Sizea	Budget ($1,000s)	BWC Usage (%)	Community Policing (%)	Number FTS Officers	Number FTNS Staff	Female Officers (%)
Local Police	100+	266,716	49.7	30.5	1,738.4	679.6	5.3
	50-99.5	5,622	49.8	30.0	14.2	15.0	5.0
	25-49.5	2,265	45.5	33.7	7.3	5.9	6.4
	10-24.5	1,053	49.4	35.6	5.1	3.2	10.3
	5-9.5	276	50.0	36.5	2.0	1.3	11.2
	2-4.5	116	49.9	39.7	1.1	0.7	16.8
	1-1.5	44	48.6	47.2	0.4	0.4	20.7
Sheriff’s Office	100+	222,910	47.3	44.2	653.2	604.0	7.9
	50-99.5	8,281	46.1	43.6	13.0	48.7	7.6
	25-49.5	4,283	49.4	45.9	7.0	22.0	6.6
	10-24.5	1,716	49.6	48.7	4.5	8.5	7.9
	5-9.5	612	49.5	48.7	1.6	4.8	9.6
	2-4.5	267	45.8	45.3	0.9	1.8	15.9
	1-1.5	12	0.0	49.4	0.0	0.0	0.0
State Police	All	245,320	34.3	49.8	1,335.3	854.4	2.7

^a Number of full-time equivalent sworn officers.

To optimize the sampling allocation, multiple objectives of estimation have been considered. We considered several allocation methods including proportional to number of agencies (Number of Agencies), proportional to number of FTE (Percent FTE), and proportional to square root of FTE (Percent Square Root FTE). We compared the allocations by evaluating the precision of estimates from each of the allocations. Using the LEAR, frame counts for each stratum are included in Table 6 as well as the distribution of agencies, FTE, and square root of FTE.

Table 6: Distribution of agencies by stratum based on sample allocation, LEAR

Agency Type	Agency Sizea	Number of Agencies	Percent Agencies	Percent FTE	Percent Square Root FTE
Local Police	100+	669	4.5	38.7	20.4
	50-99.5	841	5.6	7.8	9.2
	25-49.5	1659	11.1	7.8	9.1
	10-24.5	3269	21.8	7.0	8.7
	5-9.5	2870	19.1	2.7	5.4
	2-4.5	1972	13.2	0.9	3.0
	1-1.5	612	4.1	0.1	1.0
Sheriff’s Office	100+	409	2.7	17.5	13.7
	50-99.5	393	2.6	3.7	6.3
	25-49.5	645	4.3	3.0	5.7
	10-24.5	947	6.3	2.1	4.7
	5-9.5	473	3.2	0.5	2.2
	2-4.5	166	1.1	0.1	0.9
	1-1.5	19	0.1	0.0	0.2
State Police	All	49	0.3	8.2	9.4

^a Number of full-time equivalent officers.

The sample allocation for the three methods is calculated under the constraint that each stratum must have a respondent size of at least 2 agencies and is only applied to the NSR strata. Table 7 shows the number of expected responses with a sample size of 3,500 and assuming the response rates from Table 2.

Table 7: Estimated final response size by stratum based on allocation method, LEAR

Agency Type	Agency Sizea	Percent Agencies	Percent FTE	Percent Square Root FTE
Local Police	100+	602	602	602
	50-99.5	115	416	308
	25-49.5	227	414	307
	10-24.5	448	371	290
	5-9.5	393	146	182
	2-4.5	270	46	102
	1-1.5	84	6	34
Sheriff’s Office	100+	327	327	327
	50-99.5	50	184	198
	25-49.5	82	152	180
	10-24.5	122	104	149
	5-9.5	61	23	70
	2-4.5	21	4	28
	1-1.5	2	2	6
State Police	All	44	44	44

^a Number of full-time equivalent officers.

To compare the allocations, the relative standard error (RSE) is computed for each estimate. RSE is the ratio of a measure and its standard error (Table 8). RSE is a standardized measure of precision regardless of estimate value. While none of the allocations is best for all measures, the allocation proportional to the number of agencies is chosen, as it is the best when there are the largest differences – namely for percent female officers and body worn camera usage.

Table 8: RSE of estimates for each sample allocation, 2020 LEMAS

Allocation	Budget	BWC Usage	Community Policing	Number FTS Officers	Number FTNS Officers	Female Officers
Agencies	2.51%	2.30%	0.97%	2.24%	2.99%	3.00%
FTE	2.47%	3.94%	1.74%	2.23%	2.86%	7.00%
Square Root FTE	2.47%	2.94%	1.21%	2.23%	2.86%	4.41%

The sample size allocation obtained through the proportional number of agencies is presented in Table 9. Based on the response rate assumptions, the design calls for a sample size of 3,500 with 2,848 complete questionnaires expected.

Table 9: Sample size allocation based on the proportion to number of agencies by stratum, 2020 LEMAS

Agency Type	Agency Sizea	Sample Size	Expected Respondents
Local Police	100+	669	602
	50-99.5	144	115
	25-49.5	284	227
	10-24.5	560	448
	5-9.5	491	393
	2-4.5	338	270
	1-1.5	105	84
Sheriff’s Office	100+	409	327
	50-99.5	67	50
	25-49.5	110	82
	10-24.5	162	122
	5-9.5	81	61
	2-4.5	28	21
	1-1.5	3	2
State Police	All	49	44
Total		3500	2848

^a Number of full-time equivalent officers.

Sampling Error

Although the allocation of sample size will be made with consideration for the overall national estimates, it is not the only domain of interest. Other domains of interest include:

• Local police departments – all sizes

• Sheriff’s offices – all sizes

• State police departments – all sizes

• Local police departments – non-self-representing

• Sheriff’s offices – non-self-representing

The estimated RSEs for each domain are presented in Table 10. The domain with the highest RSEs is the state agencies domain. Since all agencies in this group are sampled with certainty, no allocation could improve the RSEs for this domain. Because the prevalence of female officers is lower than the other proportions of interest, the associated RSE is higher than those for the other selected statistics.

Table 10: RSEs, by selected statistic and domain, 2020 LEMAS

Domain	Budget	BWC Usage	Community Policing	Number of FTS Officers	Number FTNS Staff	Female Officers
National	2.51%	2.30%	0.97%	2.24%	2.99%	3.00%
Local Police	3.14%	2.55%	0.98%	3.08%	4.48%	3.66%
Sheriff’s Office	4.95%	5.45%	3.10%	3.31%	4.66%	4.53%
State Police	4.87%	12.12%	5.28%	5.28%	6.56%	2.07%
Non-Self-Representing	1.99%	2.48%	1.04%	0.72%	3.77%	3.42%
Self-Representing	3.38%	1.46%	0.58%	3.29%	3.89%	0.67%
Local Police – Non-Self-Representing	1.98%	2.70%	1.04%	0.83%	3.16%	4.02%
Sheriff’s Office – Non-Self-Representing	4.58%	6.13%	3.63%	1.42%	6.72%	5.81%

Final Sampling Design

The final design for the 2020 LEMAS will mirror the methods used in the previous LEMAS administrations: (1) strata will be based on agency type and number of FTE sworn officer as shown in Table 2 and (2) sample size allocation will be based on the proportional number of agencies within in each strata. The LEAR will serve as the sampling frame with and the sample size will be 3,500 agencies for the 2020 LEMAS. Sample allocation by strata for the 2020 LEMAS is shown in Table 9.

2. Procedures for Collecting Information

Data Collection Procedures. The 2020 LEMAS will involve a comprehensive series of mailings and non-response follow-up activities to facilitate high response rates and data quality. Data collection will begin with a prenotification letter (mailed via USPS), on BJS letterhead and signed by the BJS director to LEA agency heads announcing the survey and describing the purpose of the data collection (Attachment 8).

Two weeks after the prenotification mailing, a survey invitation letter (mailed via USPS) and an email to the POC for each LEA will be sent to inform him or her about the survey. This letter will explain the purpose and significance of the survey. It will include the survey web address and agency-specific log-in credentials (Attachment 9). The survey invitation letter will provide a toll-free telephone number and project-specific email address for the survey Help Desk should the POC have any questions. Accompanying this letter will be a letter of support from the Police Executive Research Forum (PERF) and other law enforcement professional organizations; letters will be customized to the type of agency, but all letters will include all supporting organizations (Attachments 10 and 11). An enclosed flyer will explain the new design of the LEMAS—use of a shortened questionnaire and distinct versions for police departments and sheriff’s offices (Attachment 12). Also accompanying this letter will be a POC Update Form (Attachment 13) that the recipient can use to fax contact information for a newly designated POC and a list of survey topics to inform respondents about the types of information they will need to collect prior to completing the survey (Attachment 14). Approximately 1 week after sending the survey invitation letter, RTI will send an email message that is identical to the survey invitation letter to those recipients for whom an email address is available to confirm receipt of the study materials (Attachment 15). Within three weeks of receiving a survey, the respondent will receive a thank you email or letter depending on the mode of completion (Attachment 16). The thank you will acknowledge receipt of the survey and state that the agency may be contacted for clarification once their survey responses are processed.

Two weeks after sending the survey invitation letters, a reminder mailing will be sent via USPS to nonrespondents (Attachment 17); this mailing will include a paper copy of the 2020 LEMAS instrument (Attachment 18 or Attachment 19) and a business reply return envelope. One week later, a reminder email will be sent to all nonrespondent POCs, including those who are newly identified (Attachment 20). The reminder letter and email will express the importance of the LEMAS to the LEA community and encourage response via the online survey (or paper copy, if preferred). Two weeks after those reminders are sent, RTI will mail a reminder postcard to POCs (Attachment 21) that will include the website information and instructions on how to download another paper copy of the questionnaire if needed. Two weeks after sending the reminder postcard, we will send a fourth reminder via email (Attachment 22). A fifth reminder (Attachment 23), again via USPS and including a second copy of the paper questionnaire, will be sent three weeks after the fourth reminder. Telephone follow-up with all non-responding LEAs will begin one week after the fifth reminder is sent. The procedures for this activity and the final three nonresponse follow-up contacts are discussed in Section 3.

Upon receipt of a completed survey (web or paper copy), data will be reviewed and edited, and if needed, the respondent will be contacted to clarify answers or provide missing information. RTI will scan all paper copy questionnaires and perform the same data quality review procedures that are applied to web survey data. Prior to contacting the respondent, RTI staff will aim to address data inconsistencies via BJS-approved editing specifications. RTI also will ensure that responses fall within the proper coding schemes specified by BJS. The following is a summary of the data quality assurance steps that RTI will observe during the data collection and processing period:

Data Editing. RTI will attempt to reconcile missing or erroneous data through automated and manual edits of each questionnaire. In collaboration with BJS, RTI will develop a list of edits that can be completed by referring to other data provided by the respondent on the survey instrument. Through this process, RTI can quickly identify which cases require follow-up and indicate the items that need clarification or retrieval from the respondent.

Data Retrieval. When it is determined that data retrieval is needed, an Agency Liaison (AL) will contact the respondent for clarification. Throughout the data retrieval process, RTI will document the questions needing retrieval (e.g. missing or inconsistent data elements), request clarification on the provided information, obtain values for missing data elements, and examine any other issues related to the respondent’s submission.

Data Quality Review. To confirm that editing rules are being followed, RTI will review frequencies for the entered data after the first 10 percent of cases are received. Any issues will be investigated and resolved. Throughout the remainder of the data collection period, RTI staff will conduct regular data frequency reviews to evaluate the quality and completeness of data captured in both the web and paper copy modes.

3. Methods to Maximize Response Rates

The previous waves of the LEMAS survey have achieved high rates of survey response, typically meeting or exceeding 80 percent. BJS and RTI will undertake various procedures to ensure that response rates for the LEMAS are as high as possible.

BJS will use a web-based instrument supported by several online help functions to maximize response rates. For convenience, respondents will receive the survey link in an email invitation and a mailed paper copy invitation.² A Help Desk will be available to provide survey participants with both substantive and technical assistance. In addition, the web survey interface is user-friendly, which encourages response and ensures more accurate responses. Because online submission is such an important response method, close attention will be paid to the formatting of the web survey instrument. The online application will be flexible so it can adapt to meet the needs of multiple device types (e.g., desktop computer and tablet), browser types (e.g., Internet Explorer and Google Chrome), and screen sizes. Other features in the instrument will include the following:

• Respondents’ answers will be saved automatically, and they will have the option to leave the survey partway through and return later to finish.

• The online instrument will be programmed with data consistency checks and automatic prompts to ensure inter-item consistency and reduce the likelihood of “don’t know” and out-of-range responses, thereby reducing the need for follow-up with the respondent after survey submission.

• Respondents will encounter a main menu when they enter the web survey that allows them to complete the survey module-by-module. This will be useful so that respondents can delegate specific modules to other individuals within their organization.

• Upon submission, respondents will receive a message that confirms receipt of their survey.

• LEAs may also download and print a paper version of the survey from the website.

In order to obtain higher response rates and to ensure unbiased estimates, multi-stage survey administration and follow-up procedures have been incorporated into BJS’s response plans. Ensuring adequate response (not just agency response rates, but also item responses) begins with introducing agencies to the survey. This will be accomplished initially through the prenotification mailing to agency heads and then through an invitation letter and accompanying documents (Attachments 8-14). Resources available to help the respondent complete the survey (e.g. phone- or email-based Help Desk support) will be described in detail. We will provide LEAs with online and fax methods to identify respondents and change the POC assignment if needed. POCs will also be able to delegate specific modules of the survey to others within their organization. We will also provide a paper copy of the 2020 LEMAS instrument to inform respondents about the types of information they will need to collect prior to completing the survey

Approximately 1 week after sending the survey invitation letter, RTI will send an email message that is identical to the survey invitation letter to those recipients for whom an email address is available to confirm receipt of the study materials. Within three weeks of receiving a survey, the respondent will receive a thank you email or letter depending on the completion mode (Attachment 16). The thank you will acknowledge receipt of the survey and state that the agency may be contacted for clarification once their survey responses are processed.

The data collection schedule is designed to include several follow-up communications to allow the LEA to complete the survey at a time most convenient for them. Two weeks after the initial invitation, we will send a letter (Attachment 17) and paper questionnaire (Attachment 18 or Attachment 19) to all nonrespondent LEAs. This communication will serve as a reminder for those agencies who have not yet submitted their information. A follow-up email (Attachment 20) will be sent one week later. Following this reminder message, a postcard reminder will be sent to any to-date nonrespondents once again asking them to complete the web survey (Attachment 21). If no survey response is received after the postcard mailing, a fourth nonresponse communication will be sent via email to to-date nonrespondents (Attachment 22). Like previous communications, it will provide information on how to complete the web survey, including the URL and the LEA’s unique survey access code. A fifth reminder, a letter (Attachment 23) will be sent along with a second copy of the LEMAS questionnaire.

Phone follow-up for survey nonresponse will occur with the to-date nonrespondents (see Attachment 24 for sample script) starting approximately 2 weeks after the fifth reminder is sent. In preparation for this outreach, Agency Liaisons (ALs) will be trained on the study protocol and procedures for contacting nonresponding agencies. Most notably, ALs will receive training on how to ask agencies to complete the web survey, arranging for LEAs to receive paper copy questionnaires, and tracking cases (including contact attempts). After nonresponse phone calls, ALs will make targeted attempts with nonresponding agencies to capture critical items.

Finally, four more nonresponse communications will be sent to any to-date nonrespondents. First, 4 weeks after phone follow-up begins, we will send a reminder letter to LEAs (Attachment 25); another reminder email will be sent 4 weeks later (Attachment 26). Next, 4 weeks before data collection ends, a final letter will be sent to the POC which will include an end-of-study notification (Attachment 27). This letter will go to any to-date nonrespondents to announce the forthcoming closure of the study and make a final appeal to participate. The final attempt to encourage response will be an end-of-study email sent concurrent with the end-of-study letter (Attachment 28).

Nonresponse Adjustments

Unit nonresponse. With any survey, it is typically the case that some of the selected subjects will not respond to the survey request (i.e., unit nonresponse) and some will not respond to particular questions (i.e., item nonresponse), despite best efforts made to collect all the data. Using agency data from the LEAR, weighting will be used to adjust for unit nonresponse in the 2020 LEMAS. To determine which factors to use in the facility nonresponse weight adjustments, a procedure available in RTI’s SUDAAN software based on the Generalized Exponential Model (GEM) will be used to model the response propensity based on information from the sampling frame (e.g., agency characteristics such as geography, agency type, agency size) within sampling strata (Folsom, 2000). Ideally, only variables highly correlated with the outcomes of interest will be included in the model in order to reduce the potential for bias. As described above, given the expected differential response rates by agency type and size, the weighting adjustment procedures will attempt to minimize the bias in the estimates within these domains.

Nonresponse bias analysis. As previously stated, and based on the 2016 LEMAS responding, an overall response rate of approximately 81% is expected (Table 2). In order to ensure those agencies that do not participate in the study are not fundamentally different than those that do, a nonresponse bias analysis will be conducted if the agency-level response rate obtained in the 2020 LEMAS falls below 80 percent. BJS will use agency type and agency size in the nonresponse bias analysis.

For each agency characteristic, BJS will compare the distribution of the respondents to the nonrespondents. A Cohen’s Effect Size statistic will be calculated for each characteristic. If any characteristic has an effect size that falls into the “medium” or “high” category, as defined by Cohen, then there is a potential for bias in the estimates. Each estimate will be included in a nonresponse model to adjust weights to minimize the potential for bias in the estimates.

In addition to estimating effect sizes, an examination of early and late responders will be conducted. If late responders (i.e., those that take more contact attempts before responding) are significantly different on the key outcomes of interest, that is an indication of potential bias. This comparison will be made within each strata to determine if the potential for bias varies by strata.

4. Final Testing of Procedures

The 2020 LEMAS survey instrument is built upon previous waves of the LEMAS survey to ensure data comparability. As a result, improvements to the questionnaire were done with caution. BJS shared a copy of the draft 2020 LEMAS survey instrument with research scholars with a known interest in law enforcement issues and law enforcement professionals. The expert reviewers were given an electronic draft of the survey instrument and asked to comment on question wording, response categories, as well as overall structure and layout. Responses were primarily received as written annotations within the document. Further information on the results of the expert panel review are provided in Part A. Only five new items were added to the 51-item instrument. For three other items, additional response categories were added. These adjustments were added to address recommendations from the President’s Task Force on 21^st Century Policing (see Attachment 2). We received expert feedback on these new items and where possible, question wording was consistent with other previously tested items.

In addition to the expert review, BJS conducted cognitive interviews with 11 local police departments and 9 sheriffs’ offices. Based on these interviews, questions and instructions were clarified and two items were dropped or consolidated to reduce confusion or burden. Further information on cognitive interviewing is provided in Part A and Attachment 4. Additionally, we will conduct post-processing assessment of these new items.

5. Contacts for Statistical Aspects and Data Collection

1. BJS contacts include:

Shelley Hyland, PhD

202-305-5552

Shelley.Hyland@usdoj.gov

2. Persons consulted on statistical methodology:

Stephanie Zimmer, PhD

RTI International

3. Persons consulted on data collection and analysis:

Tim Smith

RTI International

Travis Taniguchi, PhD

RTI International

References

Aiello, Michael F. 2019 Influential Women? Policing Styles in Online Recruitment Materials. Police Quarterly, 23 (1), 3-24.

Barrick, K.,  Hickman, M. J., & Strom, K. J. (2014). Representative policing and violence towards the police. Policing.

Bies, K. J.,  Deporto, I. M.,  Long, D. G.,  McKoy, M. S.,  Mukamal, D. A. &,  Sklansky, D. A. (2015). Stuck in the '70s: The demographics of California prosecutors. Stanford, CA: Stanford Law School, Stanford Criminal Justice Center

Bromley, M. & Reaves, B. (1998). Comparing campus and city police operational practices. Journal of Security Administration, 21(2), 41-54.

Cave, B., Telep, C. W., & Grieco, J. (2015). Rigorous evaluation research among US police departments: Special cases or a representative sample? Police Practice and Research, 16(3), 254-15. doi:10.1080/15614263.2014.888348

Carter, J and Fox, B (2019). Community policing and intelligence-led policing: An examination of convergent or discriminant validity. Policing: An International Journal, 42 (1), 43-58.

Chalfin, A., & McCrary, J. (2013). The effect of police on crime: New evidence from U.S. cities, 1960-2010. NBER working paper series. 18815, Cambridge, MA: National Bureau of Economic Research.

DeCarlo, J., & Jenkins, M. J. (2015). Labor unions, management innovation and organizational change in police departments. Springer Briefs in Criminology. Switzerland: Springer International Publishing doi:10.1007/978-3-319-21584-6

Farrell, A. (2014). Environmental and institutional influences on police agency responses to human trafficking. Police Quarterly. 17(1), 3-29.

Folsom, R.E., & Singh, A.C. (2000). The generalized model for sampling weight calibration for extreme values, nonresponse, and poststratification. In Proceedings of the American Statistical Association’s Survey Research Methods Section, 598-603.

Groves, R. & Cork, D. (2009). Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, D.C. National Academies Press.

Gustafson, J. (2013). Diversity in municipal police agencies: A national examination of minority hiring and promotion. Policing. 36(4), 719-736.

Hickman, M. & Piquero, A. (2009). Organizational, administrative, and environmental correlates of complaints about police use of force: Does minority representation matter? Crime and Delinquency, 15(1), 3-27.

Hur, Y. (2013). Racial diversity, is it a blessing to an organization? Examining its organizational consequences in municipal police departments. International review of administrative sciences. 79(1), 149-164.

Jurek, Alicia L.; King, William R. (2020) Structural responses to gendered social problems: Police agency adaptations to human trafficking. Police Quarterly, 23 (1), 25-54

Langton, L. (2010). Women in Law Enforcement, 1987 – 2008. Washington, D.C.: Bureau of Justice Statistics.

Matusiak, M. C., Campbell, B. A., & King, W. R. (2014). The legacy of LEMAS: Effects on police scholarship of a federally administered, multi-wave establishment survey. Policing. 37(3), 630-648.

Melekian, B. K. (2012). Policing in the new economy: A new report on the emerging trends from the Office of Community Oriented Policing Services. Police Chief, 79, 16–19.

Mughan, Sian; Li, Danyao; Nicholson-Crotty, Sean (2020). When law enforcement pays: Costs and benefits for elected versus appointed administrators engaged in asset forfeiture. American Review of Public Administration.

Perez, N. M., & Bromley, M. (2015). Comparing campus and city police human resource and select community outreach policies and practices: An update. Policing: An International Journal of Police Strategies & Management, 38(4), 664-674.

President’s Task Force on 21st Century Policing. (2015). Final Report of the President’s Task Force on 21st Century Policing. Washington, DC: Office of Community Oriented Policing Services.

Randol, Blake M. (2013). An exploratory analysis of terrorism prevention and response preparedness efforts in municipal police departments in the United States: Which agencies participate in terrorism prevention and why? Police Journal. 86(2), 158-181.

Reaves, B. (2011). Census of State and Local Law Enforcement Agencies, 2008. Washington, D.C., Bureau of Justice Statistics.

Reaves, B. (2015). Local police departments, 2013: Equipment and technology. Washington, D.C.: Bureau of Justice Statistics.

Roberts, A. & Roberts, Jr., J. M. (2015). Crime and temporal variation in police investigative workload: Evidence from National Incident-Based Reporting System (NIBRS) data. Journal of Quantitative Criminology, 1-24. doi:10.1007/s10940-015-9270-9

Schuck, A. M. (2014). Female representation in law enforcement: The influence of screening, unions, incentives, community policing, CALEA, and size. Police Quarterly. 17(1), 54-78.

Schuck, A. M., & Rabe-Hemp, C. (2014). Citizen complaints and gender diversity in police organisations. Policing and Society.

Sharp, E. B. (2014). Minority representation and order maintenance policing: Toward a contingent view. Social Science Quarterly. 95 (4), 1155-1171.

Shjarback, J. A. (2015). Emerging early intervention systems: An agency-specific pre-post comparison of formal citizen complaints of use of force. Policing. doi:10.1093/police/pav006

Shjarback, J. A., & White, M. D. (2015). Departmental professionalism and its impact on indicators of violence in police-citizen encounters. Police Quarterly. doi:10.1177/1098611115604449

Smith, B. W.,  Wareham, J., & Lambert, E. G. (2014). Community and organizational influences on voluntary turnover in law enforcement. Journal of Criminal Justice. 37(3), 377-398.

Walfield, S. M. (2015). When a cleared rape is not cleared: A multilevel study of arrest and exceptional clearance. Journal of Interpersonal Violence. doi:10.1177/0886260515569062

Wareham, J.,  Smith, B.W.,  & Lambert, E. G. (2015). Rates and patterns of law enforcement turnover: A research note. Criminal Justice Policy Review. 26(4), 345-370. doi:10.1177/0887403413514439

Willits, D. W. (2014). The organisational structure of police departments and assaults on police officers. International Journal of Police Science and Management. 16(2), 140-154.

1 For reporting purposes, two part-time officers are treated as equivalent to one full-time officer.

2 Remaining nonrespondents will receive a total of two copies of the paper questionnaire during the course of data collection.

17