2012 LEMAS Supporting Statement - Part B v2

Part B. Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The 2012 LEMAS Survey will utilize procedures successfully employed in the prior eight waves of the LEMAS program to identify the universe of eligible respondents.

• These procedures begin with a review of the agencies included in the 2008 Census of State and Local Law Enforcement Agencies. This list will be supplemented with reviews of automated and printed lists of law enforcement agencies available from a variety of public and private sources. BJS will review the most contemporary lists of law enforcement agencies providing data to the FBI’s Uniform Crime Reporting Program.

• BJS will contact the state directors of officer standards and training offices either directly or through their national association, the International Association of Directors of State Law Enforcement Standards; most of these agencies have state authority to certify law enforcement agencies or law enforcement officers and, therefore, know about newly formed and newly disbanded agencies.

• BJS will contact the BJS-supported state-level Statistical Analysis Centers. These agencies are familiar with the status of law enforcement agencies in their states and will be asked to review lists provided by BJS. BJS will continue the past practice of contacting the two major national law enforcement professional associations (i.e., the International Association of Chiefs of Police and the National Sheriffs Association) and request assistance in identifying the creation of new agencies, the consolidation of agencies or agencies that are no longer operational. BJS will also obtain and review lists of agencies that submit requests for funding or for technical assistance from the COPS office.

• Lastly, BJS will purchase an electronic copy of the most recent listing of law enforcement agencies produced by National Public Safety Information Bureau.

This information will be combined to produce a universe list of general purpose law enforcement agencies. The sample of agencies to be surveyed will be drawn from this list based upon the sampling design described below.

2. Procedures for Collecting Information

The administration of the survey will commence with the transmittal of an introductory letter to each agency’s primary point of contact. This communication will explain the purpose and importance of the survey, the burden and benefits for each agency and the extent to which information about surveyed organizations will become public information. A mailing-list revision form will be included to change, if necessary, the agency’s primary respondent. A postage paid envelope will be included, as will instructions for e-mail or telephone communications to change respondents or contact information. A one week verification deadline is planned, after which the survey team will conduct telephone follow up verifications with non-responding departments (see attached draft scripts for contacts with respondents).

Following respondent verification, a personalized survey packet will be mailed to each designated agency’s point of contact for the survey. This packet will include a hard copy of the survey and explanations of the methods by which the survey can be completed. Each respondent will be strongly encouraged to utilize the web-based survey approach; instructions on how to access the website will be provided, including a URL, username and password. Should the respondent opt for a non-web-based alternative, a paper version of the instrument with a return envelope mailer will be provided. Help desk contact information, including a toll free number, will be provided to support all modes of data submission and any questions that may arise about the questionnaire.

Prior Sampling Designs and Response Rates

The basic sampling design of the LEMAS surveys has been maintained over all eight waves—selecting large agencies with certainty and sampling a sharply declining proportion of agencies as the number of sworn personnel of the agency declines. Some aspects of the LEMAS sampling design have changed over time. The criteria for certainty sampling changed from agencies with at least 130 sworn officers in 1987 to agencies with at least 100 sworn officers in 1990. In the late 1990’s, a revised sampling rationale for the LEMAS program was developed and implemented for BJS by the Demographic Statistical Methods Division of the Census Bureau (Ellis, 1997). The exclusion of special purpose law enforcement agencies (e, g., campus, natural resources, and transit police) in the 2000 wave resulted in the LEMAS program becoming a survey limited to general purpose agencies.

The LEMAS surveys have obtained consistently high overall response rates with better than 90% of the agencies responding in each wave (see Table 5). However, the response rate in some waves was below 90% for some components of the sample (e.g., those sheriff offices with fewer than 100 sworn deputies and those police departments with less than 13 sworn officers). The LEMAS sampling design included another facet. In seven of the eight waves, BJS used a short and a long form of the survey instrument, with the larger agencies receiving additional survey items not considered appropriate for smaller agencies (e.g., number of specialized units). For the 1999 LEMAS survey, the same instrument was sent to all sampled agencies and was an abbreviated version of the 1997 short form instrument supplemented with a new bank of questions about the adoption of community policing principles, policies and practices.

Distribution of Law Enforcement Agencies and Personnel

As with prior LEMAS sampling, the 2012 LEMAS sample design needs to consider the highly skewed distribution of the number of law enforcement agencies and personnel in each stratum. As displayed in Table 6 and using the universe of agencies in the 2008 Census of State and Local Law Enforcement Agencies as a base, 706 police agencies with 100 or more sworn law enforcement officers¹ constitute 5.6% of all agencies but employ 344,442 (64.3%) of the sworn officers. At the other extreme, 48.1% of all general purpose police departments employ fewer than 10 sworn personnel. These 6,045 agencies employ 28,927 (5.28%) of all state and local sworn personnel. These skewed distributions are also found among sheriff offices.

Table 5: Law Enforcement Management and Administrative Survey
Number and Percent of Surveys Completed 1987 – 2007
Reference Year	Sheriff Offices	Local Police	State Police	Special Purpose Agencies	Total Completed Surveys	Overall Response Rate
1987	754	1,938	49	166	2,907	95.2%
1990	841	1,831	52	221	2,945	92.0%
1993	918	1,827	49	234	3,028	98.3%
1997	915	2,094	49	354	3,412	94.8%
1999	967	2,052	49	178	3,246	97.8%
2000	961	1,975	49	0	2,985	97.4%
2003	863	1,947	49	0	2,859	90.6%
2007	841	1,989	45	0	2,875	91.8%

Sampling Frame

While the substantive emphasis of the 2012 LEMAS has changed from prior LEMAS surveys, the 2012 LEMAS survey, like the prior LEMAS surveys, is designed to enhance the efficiency and the precision of national estimates of agencies and of personnel for police departments, for sheriff offices and for all general purpose law enforcement agencies combined (see Table 7). This goal creates the need to maximize the quality of six classes of statistics—agency and personnel statistics for police departments, sheriff offices and all law enforcement agencies combined. Attention to these multiple objectives guides BJS’s approach to the sampling plan for the 2012 LEMAS.

As was done in prior LEMAS surveys, the 2012 LEMAS will select all large law enforcement agencies with 100 or more sworn officers; these self-representing (SR) agencies are found in the last row in Table 8. This ensures our ability to measure change overtime within a consistent set of large agencies. The 2012 LEMAS will sample within the other cells in Table 8; these are the non-self-representing (NSR) agencies with less than 100 sworn officers

The NSR agencies have traditionally been subdivided into six strata based on the number of full time equivalent officers (FTE). These groupings have been used historically by BJS for reporting key characteristics of officers and agencies. They also serve as a stratification factor since independent samples would be drawn from each agency-type-by-size stratum. Such stratification will increase the statistical precision of estimates at both officer and agency levels. The size cut points have been used in prior LEMAS sampling designs for police departments and we retained them in this design for consistency between the police and law enforcement strata².

Table 6: Distribution of Law Enforcement Agencies in the United States

Based in the 2008 Census of State and Local Law Enforcement Agencies

	Distribution of Police Departments
Agency Size	Officers		Agencies
One FTE Sworn Officer	542	0.1%	542	4.3%
2 to 4 FTE	6,583	1.2%	2,352	18.7%
5 to 9 FTE	20,902	3.9%	3,151	25.1%
10 to 24 FTE	51,135	9.5%	3,350	26.7%
25 to 49 FTE	54,841	10.2%	1,599	12.7%
49 to 99 FTE	57,288	10.7%	848	6.8%
100 or More FTE	344,442	64.3%	706	5.6%
Total	535,734	100.0%	12,548	100.0%
	Distribution of Sheriff Offices
Agency Size	Officers		Agencies
One FTE Sworn Deputy	18	0.0%	18	0.6%
2 to 4 FTE	702	0.4%	224	7.3%
5 to 9 FTE	3,716	2.0%	540	17.6%
10 to 24 FTE	14,658	7.8%	936	30.6%
25 to 49 FTE	20,576	10.9%	596	19.5%
49 to 99 FTE	24,131	12.8%	351	11.5%
100 or More FTE	124,847	66.2%	398	13.0%
Total	188,646	100.0%	3,063	100.0%
	Distribution of All Law Enforcement Agencies
Agency Size	Officers		Agencies
One FTE Sworn Officer	560	0.1%	560	3.6%
2 to 4 FTE	7,285	1.0%	2,576	16.5%
5 to 9 FTE	24,618	3.4%	3,691	23.6%
10 to 24 FTE	65,793	9.1%	4,286	27.5%
25 to 49 FTE	75,417	10.4%	2,195	14.1%
49 to 99 FTE	81,419	11.2%	1,199	7.7%
100 or More FTE	469,289	64.8%	1,104	7.1%
Total	724,380	100.0%	15,611	100.0%

Table 7: Six Classes of Statistics for the 2012 LEMAS Survey

	Types of Law Enforcement Agencies
Level of Analysis	Police Departments	Sheriff Offices	All Types
Agency	X	X	X
Personnel	X	X	X

Table 8: Sampling Strata for the 2012 LEMAS Survey

Agency Size Categories	Local Police	Sheriff Offices	State Police
One FTE Sworn Personnel	1	8
2 to 4 FTE	2	9
5 to 9 FTE	3	10
10 to 24 FTE	4	11
25 to 49 FTE	5	12
49 to 99 FTE	6	13
100 or More FTE	7	14	15

Design Efficiency

In designing a stratified sampling plan BJS considered alternative ways to 1) select the number of agencies to survey within each of the 12 NSR strata and 2) to select specific agencies within each NSR stratum. BJS then tested these designs for the relative precisions provided for six classes of statistics based on five diverse items from the 2007 LEMAS survey. Each of these steps will be discussed below.

BJS compared four alternative ways to select the number of agencies within the 12 NSR strata³. First, BJS used the actual proportion of agencies in each stratum in the completed 2007 LEMAS survey. Second, BJS created an allocation based on the proportion of all law enforcement agencies in each stratum. A third allocation was based on the proportion of all full-time equivalent sworn personnel in each stratum. The fourth alternative was based on the proportion of the sum of the square roots of the number of full-time equivalent sworn personnel in each stratum. This last option was chosen to blend the agency and personnel distributions. BJS tested the options to see the extent to which each option improved upon the allocation used in the 2007 LEMAS. BJS also wanted to compare the effects of only considering the distribution of agencies and the effects of only considering the distribution of sworn personnel.

Allocation of Agencies between Strata

To evaluate these alternative sampling allocations, BJS constructed a test using a sample size of 3,000. To construct and test these alternatives, BJS factored in the three Self-Representing (SR) Strata; that is, BJS allocated the 645 SR local police departments, the 393 SR sheriff offices and the 50 primary State law enforcement agencies to the three SR strata and then re-allocated the remaining 1,912 non-self-representing (NSR) agencies to the other 12 NSR strata based on the 4 alternative sampling plans outlined above. This approach generated four alternatives to allocating agencies between strata while retaining the 1,088 agencies in our 3 SR strata (see table 9).

Table 9: Alternative Distributions of Agency Counts by Sample Strata

Based on Number of Agencies and Sworn Personnel

Strata	2007 LEMAS	2008 Agencies	2008 Officers	2008 SQRT of Officers
One FTE Sworn Officer	43	71	4	19
2 to 4 FTE Officers	111	310	49	139
5 to 9 FTE Officers	219	415	157	288
10 to 24 FTE Officers	366	442	383	463
25 to 49 FTE Officers	319	211	411	332
49 to 99 FTE Officers	310	112	429	247
100 or More FTE Officers	645	645	645	645
One FTE Sworn Deputy	7	2	0	1
2 to 4 FTE Deputies	48	30	5	14
5 to 9 FTE Deputies	109	71	28	50
10 to 24 FTE Deputies	184	123	110	131
25 to 49 FTE Deputies	116	79	154	124
49 to 99 FTE Deputies	80	46	181	103
100 or More FTE Deputies	393	393	393	393
100 or More FTE State Officers	50	50	50	50
Total	3,000	3,000	3,000	3,000

Selection of Agencies within Strata

BJS also considered two options for selecting the individual agencies within each stratum. The first approach is to give each agency within each stratum an equal probability of being selected (EPS). The second approach to select agencies within each stratum was based on a probability proportionate to the size of that agency (PPS). With three approaches to selecting the number of agencies between strata (see Table 9) and two approaches for selecting agencies within each stratum, BJS considered 8 alternative sampling designs.

Following Kish (1965), BJS computed a design effect⁴ for six alternative designs. There are strengths and weaknesses to each of these approaches for the each of the six classes of statistics essential to our design of the 2012 LEMAS. A sample allocation proportionate to the number of agencies will be more efficient for agency level statistics and a sample allocation proportionate to the number of sworn personnel will favor personnel level statistics. Similarly, weighting the selection within strata by agency or personnel has implications for the efficiency of agency and personnel statistics. As expected, the EPS designs based on the number of agencies generated design effects of 1 for agency statistics and the PPS designs based on the number of personnel generated design effects of 1 for personnel statistics (see Table 10). However, the EPS design effects for personnel statistics in designs based on the number of agencies ranged from 1.84 to 2.32 and the PPS design effects for agency statistics in designs based on the number of personnel ranged from 2.62 to 3.4. Therefore, these 4 designs were dropped from consideration.

Compared to the 2007 LEMAS sample, the two alternative designs based on allocating agencies to strata based on the square root of the number of officers generate reductions in the design effects for some of the six classes of statistics (see Table 10). The design based on allocating agencies between strata by the square root of the number of sworn personnel and within strata on a basis proportionate to agency size (SQRT PPS) generates design effects smaller than the 2007 LEMAS design for all 3 personnel level statistics but larger design effects for all 3 agency level statistics. On average, the reductions are larger than the increases. Moreover, the design effects of the SQRT PPS improved for those statistics where the LEMAS design was weakest.

The alternative design based on allocating agencies between strata based on the square root of the number of sworn personnel and within strata based on an equal probability basis (SQRT EPS) also improved the design effect for 3 classes of statistics and made the design effects for the other 3 classes of statistics deteriorate. This design also reduced the overall design effects on average and reduced the design effect for the single worst type of statistics in the 2008 LEMAS design—personnel level statistics for sheriff agencies.

Table 10: Design Effects for LEMAS and Two Alternatives

Level of Analysis	Analytical Domains	2007 LEMAS EPS	SQRT Officer EPS	SQRT Officer PPS		LEMAS Versus EPS	LEMAS Versus PPS
Agency	Overall Statistics	1.42	1.32	1.53		.10	-.11
	Local Police	1.39	1.32	1.54		.07	-.15
	Sheriff	1.01	1.23	1.43		-.22	-.33
Personnel	Overall Statistics	1.31	1.28	1.20		.03	.11
	Local Police	1.18	1.29	1.21		-.03	.03
	Sheriff	1.74	1.22	1.14		.52	.62

Sampling Error

In addition to the efficiency of our sampling design, we are also concerned about the precision of the estimates obtained for each of our six classes of statistics. Precision estimates are complicated by the different sample sizes for police and sheriff agencies and for all law enforcement agencies and by the type and distribution of items being estimated. To assess the designs under consideration for each of our six classes of statistics, BJS generated margin-of-error estimates under a stratified sample at the 95% confidence interval for percentages at the 50% level (see Table 11).

Table 11: Margin of Error Comparisons

of Four Alternative Designs for 2012 LEMAS Survey

[Cell contents are maximum 95% half-width confidence intervals]

Level of Analysis	Analytical Domains	2007 LEMAS EPS	SQRT Officer EPS	SQRT Officers PPS	Modified SQRT EPS
Agency	Overall Statistics	3.0	2.8	3.0	2.5
	Local Police	3.5	3.2	3.2	3.0
	Sheriff	3.7	5.1	4.3	4.0
Officer	Overall Statistics	1.0	1.0	1.0	1.0
	Local Police	1.2	1.3	1.2	1.3
	Sheriff	2.4	1.9	1.9	2.2

As in the consideration of design effects, none of the designs considered here produced the best precision on all six classes of statistics. While all of the alternatives considered here improve upon the precision levels for agency-level statistics for local police and for all law enforcement agencies, they all do less well than the 2007 LEMAS design with the precision levels for sheriff statistics at the agency level.

We took steps to improve upon the designs produced by the square root function. First, we considered a new design which we call a modified square root design that modifies the allocation of agencies to sample strata that was produced by the square root function.⁵ The new allocation of the number of agencies to each strata, were based on testing many possible allocations and selecting the that resulted in the best balance in design effects and precision estimates for agency level and personnel level statistics for police departments and for sheriffs. This approach, using the equal probability assignment within stratum, results in improvements in the margin of error estimates over the unmodified square root design option, with the largest improvement among the agency-level statistics for sheriffs. This modified SQRT design also compares favorably with the 2007 LEMAS designs, making substantial improvements in the agency-level estimates for police departments and all agencies.

Using this modified allocation of agencies to strata we calculated coefficients of variation⁶ for five survey items from the 2007 LEMAS survey weighing cases for the 2007 allocation and for each of the now four alternative design options. The five items include three agency level statistics—the average annual operating budget, the average annual entry level salary for sworn personnel, and the percent of agencies with collective bargaining. The two personnel level statistics are the number of full-time sworn personnel and percent of sworn personnel that are Hispanic. In Table 12, we compare the results of these comparisons with the 2007 LEMAS design and the modified square root design with equal probability selection, which produced smaller coefficients of variation than the other 6 alternative designs. These tests showed modest improvement in the coefficient of variation for 8 out of 15 tests.

These eight improvements range from a 7.7 % improvement for full-time equivalent sworn deputies in sheriff officers to a 25.0% improvement in entry level salaries for all law enforcement agencies. The seven items for which the LEMAS sample allocation provided better coefficients of variation ranged from 3.5% for percent sworn officers Hispanic for all agencies to a 10.0% improvement for entry level salary for sheriffs.

These comparisons and tests have demonstrated the resilience of the 2007 LEMAS design and the real but modest improvements in efficiency and in precision that can be obtained by utilizing an alternative design. The remaining limitation of the modified square root allocation is the larger margins of error for sheriff statistics relative the local police departments. To address this concern, our second step to improve upon our design is to add cases to the NSR sheriff strata.

We determined that it would require an additional 300 sheriff agencies to make agency level and officer level statistics fully comparable but that most of this improvement occurs with the addition of the first 150 sheriff offices. Table 13 shows the allocations of agencies to strata in the modified square root design with 3,000 and with 3,150 agencies. With the use of a design that empirically allocates agencies to strata and that increases the total sample size by about 5%, the 2012 LEMAS will produce lower confidence intervals for broader spectrum of statistics than the 2007 LEMAS design and outperform other designs considered.

Table 12: Precision Estimates for 2007 LEMAS Design and 2012 LEMAS Sampling Plan

Analytic Domains	Level of Analysis	Five Survey Items	2007 LEMAS		Modified SQRT EPS
			MOE1	CV2	MOE	CV
Local Police	Agency	Average Annual Operating Budget	$180,320	2.1%	$151,981	1.8%
		Average Entry Level Salary	$463	0.7%	$390	0.6%
		Percent of Agencies with Collective Bargaining	2.9%	3.9%	2.0%	3.3%
	Officer	Full-time Sworn Equivalent Sworn Officers	20,188	2.2%	21,939	2.4%
		Percent Sworn Officers Hispanic	2.4%	11.7%	3.0%	12.7%
Sheriffs	Agency	Average Annual Operating Budget	$293,563	1.5%	$319,478	1.6%
		Average Entry Level Salary	$2,561	4.0%	$2,787	4.4%
		Percent of Agencies with Collective Bargaining	6.4%	11.5%	7.0%	12.5%
	Officer	Full-time Equivalent Sworn Deputies	4,790	1.4%	4,277	1.3%
		Percent Sworn Deputies Hispanic	1.3%	8.2%	1.0%	7.3%
All Agencies	Agency	Average Annual Operating Budget	$155,654	1.3%	$132,876	1.1%
		Average Entry Level Salary	$618	1.0%	$527	0.8%
		Percent of Agencies with Collective Bargaining	2.7%	3.8%	2.0%	3.3%
	Officer	Full-time Equivalent Sworn Officers	20,749	1.5%	21,355	1.6%
		Percent Sworn Officers Hispanic	1.6%	8.6%	2.0%	8.9%

¹ Margin of error

² Coefficient of variation

Table 13: Percent Change in Margins of Error Compared to 2007 LEMAS
Level of Analysis	Analytical Domains	Modified Square Root of Personnel Size
		N = 3,000	N = 2,500	N = 3,500
Agency	Overall Statistics	-15	-1	-24
	Local Police	-16	-3	-25
	Sheriff	9	26	-3
Officer	Overall Statistics	3	19	-8
	Local Police	9	25	-3
	Sheriff	-11	3	-20

Change in Precision with a Smaller or Larger Sample

These tests assumed a completed sample of 3,000 agencies. We considered the extent to which the margins of error would deteriorate if we kept all of our other sample requirements but assumed a total sample of 2,500 and the extent to which they would improve if we assumed a total sample of 3,500. Table 13 shows the percent increase or decrease in the margins of error over the 2007 LEMAS sample when we assume a completed sample of 3,000, 2,500 and 3,500. The reduction of 500 cases would eliminate any benefits of the new design among agency-level police and overall statistics and would increase margins of error among three other statistics by greater than 20%. Increasing the sample would create improvements (over the 2007 LEMAS sample) in all six types of statistics but in three of those statistical types the precision increases

are in single digits. These findings suggest little benefit to expanding the sample (and project costs) by 16.6%. Similarly, the substantial reductions in precision resulting from a decrease in sample threatens the capacity of the 2012 LEMAS to perform as well as prior LEMAS surveys.

Final Sampling Design

The designs tested were based on the number and size of agencies reported in the 2008 BJS Census of State and Local Law Enforcement Agencies and the proportion of agencies for which we expect to obtain survey responses. The final sampling design will be based on an updated listing of currently operational law enforcement agencies that identifies new agencies, agencies that have ceased operations and agencies that have been merged within other agencies. The initial sample will be 3,500 agencies. We anticipate a minimum of a 90% response rate will generate 3,150 survey responses. The 2012 LEMAS will select with certainty all agencies with 100 or more sworn officers in the 2008 law enforcement census⁷. Among the remaining non-self-representing local police and sheriff agencies, BJS will allocate agencies to 12 non self-representing strata based on the proportions set out in the last column of Table 14. Agencies will be selected within non-self-representing stratum on an equal probability basis.

Table 14: 2012 LEMAS Sampling Plan

Modified Square Root of Sworn Personnel Designs
Total Sample Size	N = 3,000			N = 3,150		N= 3,500
Self-Representing Strata
Local Police	645	59.3%		645	59.3%	679	60.5%
Sheriffs	393	36.1%		393	36.1%	393	35.0%
State Police	50	4.6%		50	4.6%	50	4.5%
Total Self Representing	1,088	100.0%		1,088	100.0%	1,122	100.0%
Non Self Representing Strata
One FTE Sworn Officer	44		2.3%	44	2.1%	51	2.1%
2 to 4 FTE Officers	153		8.0%	153	7.4%	176	7.4%
5 to 9 FTE Officers	288		15.1%	288	14.0%	332	14.0%
10 to 24 FTE Officers	378		19.8%	378	18.3%	436	18.3%
25 to 49 FTE Officers	306		16.0%	306	14.8%	353	14.8%
49 to 99 FTE Officers	253		13.2%	253	12.3%	292	12.3%
Total Local Police	1,422		74.4%	1,422	69.0%	1,640	69.0%
One FTE Sworn Deputy	5		0.3%	5	0.2%	6	0.2%
2 to 4 FTE Deputies	35		1.8%	35	1.7%	40	1.7%
5 to 9 FTE Deputies	100		5.2%	123	6.0%	142	6.0%
10 to 24 FTE Deputies	140		7.3%	183	8.9%	211	8.9%
25 to 49 FTE Deputies	115		6.0%	162	7.9%	187	7.9%
49 to 99 FTE Deputies	95		5.0%	132	6.4%	152	6.4%
Total Sheriffs	490		25.6%	640	31.0%	738	31.0%
Total Non Self-Representing	1,912		100.0%	2,062	100.0%	2,378	100.0%

Revisions to Survey Instrument

The 2012 LEMAS instrument has been substantially revised to incorporate core and supplement survey items. BJS has identified 17 core questions to be asked in the proposed and future LEMAS surveys and 66 items that address five supplemental topics—agency resources, community policing, information systems, personnel safety and use of force (see Attachment 4).

The 17 core items capture information about each agency’s operating budget, the number and types of agency paid personnel, their salaries and pension benefits, how many were hired and how many terminated in the past year, and each agency’s organizational response to salient issues in contemporary law enforcement. Because similar items have been asked in previous LEMAS instruments, BJS will be able to provide measures of change over time about resources available to law enforcement agencies and about issues agencies do and do not address.

The 17 core items that have been asked in previous LEMAS surveys will provide BJS with the basis for documenting changes in the resources available to law enforcement agencies in 2012. Some of the supplemental items about furloughs and hiring freezes, part-time, seasonal and volunteer personnel, over-time pay and employee benefits, and the number and types of motorized vehicles used by departments have also been asked in prior LEMAS surveys or in the 2008 Census of State and Local Law Enforcement Agencies. In addition, the proposed 2012 LEMAS instrument collects data for several years about some new items (e.g., hiring freezes) that will allow BJS to produce additional measures of change over time.

The proposed 2012 LEMAS instrument retains 9 items about the nature and extent of community policing used in LEMAS surveys since 1999. The use of these items provides a solid basis for measuring change over-time in this major national initiative. The 2012 LEMAS supplement items on information systems builds on some items included in past LEMAS surveys but also includes new items about the extent to which local agencies collect, maintain, and analyze data from incident based reports about crime incidents⁸.

Two new supplemental sections in the 2012 LEMAS capture information relevant to the safety of law enforcement personnel as well the frequency with which law enforcement personnel use force. The 2012 LEMAS asks about agency policies regarding the use of body armor and the extent to which agency policies do and do not restrict motorized and foot pursuits. The survey items about use of force capture which tactics and weapons are authorized for agency personnel. In addition, the 2012 survey also requests information about the number of motorized and foot pursuits and about the number of uses of force recorded.

3. Methods to Maximize Response Rates

The previous eight waves of the LEMAS survey have achieved high rates of survey response, typically meeting or exceeding 90%. BJS and the Urban Institute will undertake various procedures to ensure that high response rates are again achieved for the 2012 LEMAS.

The current draft instrument was designed with the input of law enforcement scholars and practitioners, who expressly provided guidance regarding methods to maximize response rates for the 2012 LEMAS. These methods have been incorporated into the planned data collection procedure and instrument to encourage survey response. BJS will use a web-based instrument supported by various online help functions to maximize response rates. A helpdesk will also be available to provide both substantive and technical assistance.

Prior LEMAS surveys have enjoyed widespread support among national law enforcement professional associations. The International Association of Chiefs of Police has typically placed notices of this survey in their widely read magazine and encouraged participation of its membership in other ways and BJS anticipates their continued support for the 2012 LEMAS.

BJS will use information about which agencies did not respond to the 2007 or the 2003 LEMAS surveys to develop strategies for identifying agencies that may need special attention to ensure their participation in the 2012 LEMAS. These strategies may involve early contact with these agencies by BJS and Urban staff and, if needed, encouragement from law enforcement professionals and professional organizations tailor-made to suit the nature of the nonresponding agencies. In addition, BJS has identified a few survey items, such as information about agency budgets that have had higher than average nonresponse rates in previous LEMAS surveys. Our approach is to give these agencies and these items enhanced attention to maintain or enhance the high response rates of previous LEMAS surveys.

In order to obtain higher response rates, multi-stage survey administration and follow up procedures have been incorporated into BJS’s response plans. Insuring adequate response (not just department response rates, but also item responses) begins with introducing respondent agencies to the survey. This will be accomplished initially through the information package sent to each agency during the respondent verification stage. The packets will include a letter from the Director of BJS describing the survey, its background, and available help resources (see Attachment 5). Help resources will also be described in detail.

Follow up plans to insure a response rate in excess of 90% are designed to give respondents the opportunity to complete the survey at a pace consistent with their own day-to-day workloads. In this way, their personal burden will be minimized. One month following the initial submission of materials to sample agencies, BJS will initiate a series of follow up actions (see Attachment 6). In the first stage, an automated e-mail reminder will be sent to non-respondents on the day of the deadline. Next a member of the survey team will send a personal written reminder to each non-respondent to remind them of the importance of the 2012 LEMAS Survey and to reiterate the assistance that BJS or its data collection agent is willing to provide to them. If a respondent does not initiate survey completion within two weeks after this personal contact, the reminder letter will be followed by a personal telephone call to the non-respondent. As needed, additional personal telephone follow up may be initiated by senior BJS staff or senior personnel at the Urban Institute, the 2012 LEMAS data collection agent. Two LEMAS project consultants, who are widely known and respected in the law enforcement field across the country, may be asked to place personal follow up calls to agencies with which they have personally worked with in the past.

In order to promote 100% item completion, we plan to monitor item responses on all surveys as they are completed and submitted. The Urban Institute will have a management system linked to the web-based data collection system that will be designed to flag non-response items and invalid responses as surveys are completed. We will also flag non-responses on hard copy submissions on a rolling basis. The data collection manager will oversee personal telephone or e-mail contacts with individual respondents to clarify missing or invalid responses and to take corrective action. These changes will also be tracked for follow up, if necessary. Two full time staff at the Urban Institute, in addition to the data collection manager will have primary responsibility for the response follow up. They will be supplemented on an as-needed basis by two senior Urban Institute researchers, two police consultants and the BJS project managers.

4. Final Testing of Procedures

The 2012 LEMAS will maintain similar respondent recruitment and support procedures as previous LEMAS surveys, which have been field tested and successfully employed through the prior eight waves of the LEMAS program. As noted above at B.3, response rates for previous iterations of the LEMAS program have typically exceeded 90%. It is expected that response rates for the 2012 LEMAS will remain similarly high.

In addition, efforts have been undertaken to test the modified procedures in place for the 2012 LEMAS. The Urban Institute has already utilized web-based survey instruments that are substantially similar to the format in design for the 2012 LEMAS in both the BJS 2007 National Census of State Court Prosecutors and the 2009 Census of Publicly Funded Crime Laboratories. The web-based survey administration procedures successfully employed in these previous survey designs will be modified to accommodate the 2012 LEMAS instrument and respondents.

The 2012 LEMAS data collection instrument draft has also been field tested. The Urban Institute conducted pilot surveys with nine agencies at the local and state level. Input regarding the clarity and accuracy of the survey instrument was sought through the completion of respondent interviews with agency representatives. BJS and the Urban Institute reviewed agency feedback, and made appropriate revisions to the final survey instrument prior to data collection.

Prior to full survey roll out, currently anticipated for the fall of 2012, we also plan to thoroughly test the web-based survey administration systems. This will be done in two ways. One will be an internal bugs and crash testing by the Urban Institute survey team as well as members of the BJS project staff. Then we will field test the web-based system with retired law enforcement professionals serving as consultants to BJS in order to identify any confusion with using the system or systematic problems that will need to be corrected before full survey roll out. No data will be collected as part of this effort.

5. Contacts for Statistical Aspects and Data Collection

1. BJS contacts include

• Howard Snyder

• Ron Malega

2. Persons consulted on statistical methodology:

• Rob Santos, The Urban Institute

3. Persons consulted on data collection and analysis:

• David Hayeslip, The Urban Institute

• Allen Beck, Bureau of Justice Statistics

References Cited

Bureau of Justice Statistics. 2011. Law Enforcement Management and Administrative Statistics (LEMAS), 2007 [Computer file]. ICPSR31161-v1. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [producer and distributor].

Bromley, Max and Brian Reaves. 1998. Comparing campus and city police operational practices. Journal of Security Administration 21 (2):41-54.

Ellis, Yukiko. 1997. Documentation of the Sample Selection for the 1997 Sample Survey of Law Enforcement Agencies. U.S. Bureau of the Census. Unpublished memorandum.

Groves, Robert and Daniel Cork. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, D.C. National Academies Press.

Hickman, Matthew and Alex Piquero. 2009. Organizational, administrative, and environmental correlates of complaints about police use of force: Does minority representation matter? Crime and Delinquency 15 (1):3-27.

Kish, Leslie. 1965. Survey Sampling. New York. John Wiley and Sons.

Langton, Lynn. 2010. Women in Law Enforcement, 1987 – 2008. Washington, D.C.: Bureau of Justice Statistics.

Melekian, Bernard. 2012. Policing in the New Economy: A New Report on the Emerging Trends from the Office of Community Oriented Policing Services. Police Chief. 79 (January 2012): 16–19.

Reaves, Brian 2011. Census of State and Local Law Enforcement Agencies, 2008. Washington, D.C., Bureau of Justice Statistics.

Reaves, Brian, 2012. Federal Law Enforcement Officers, 2008. Washington, D.C., Bureau of Justice Statistics.

U.S. Department of Justice, 2011 The Impact of the Economic Downturn on American Police Agencies. Washington, D.C.; Community Oriented Police Service.

Attachments:

1. Title 42

2. Listing of Survey Items Identifying Core, Supplement and Trend Variables

3. Bibliography of Publications from LEMAS Surveys

4. 2012 Law Enforcement Management and Administrative Statistics Survey

5. Draft example of web version of LEMAS survey

6. Intro Letter from the BJS Director.

7. Draft Scripts for Respondent Contacts

8. 60-day ICR notice

9. 30-day ICR notice

1 Agency size categories and reported agency numbers are based on computation of full-time equivalents derived from counting part-time sworn personnel as .5 sworn personnel.

2 We considered adopting different sets of size categories for local police and sheriff but decided against that based on the relative congruence of the distributions of local police and sheriff officers by size categories.

3 These comparisons are based on an N of 3,000 to account for as much as a 15% nonresponse rate in the 2012 LEMAS.

4 Kish’s design effect assesses the increase in variance for a particular type of statistic due to the use a disproportion allocation of agencies between strata and within a stratum. For agency level statistics, the design effect is 1 for the design option that allocates agencies between strata by number of agencies and within strata by weighing each agency equally. For personnel level statistics, the design effect is 1 for the design option that allocates agencies between strata by the number of sworn officers and within strata by weighing each agency proportionate to the number of sworn personnel.

5 The original allocation of agencies is listed in the last column of Table 9 and the modified allocation is listed as the first column in Table 13

6 The coefficient of variation is defined as the standard error of a statistic divided by the statistic. We express this as a percentage. CV% thus standardizes statistical precision so that comparisons can be made across statistics (be they percentages, means, totals, ratios, etc.).

7 In addition, BJS will also include in its self-representing sample 34 agencies that were included in self representing sample in the 2007 LEMAS survey but did not have 100 or more sworn officers in the 2008 law enforcement. This addition will increase the number of agencies for which we can conduct longitudinal analyses.

8 This supplement has been developed in collaboration with staff from the National Institute of Justice (See A4. Efforts to Identify Duplication)

15