CP Stat Supporting Statement B -- CL

Part B. Statistical Methods

Part B. Collections of Information Employing Statistical Methods:

The CP-SAT effort will require the employment of only descriptive statistical methods.

B.1. Respondent Universe and Sampling Methods

The CP-SAT is a purely voluntary effort, so COPS will not be determining sampling methods. We do, however, provide some guidance on sampling in the corresponding Users’ Guide.

There are five groups of individuals who are intended to complete the survey within each participating agency: Officers, Supervisors, Command Staff, Civilian Staff, and Community Partners (specific individuals with whom the agency has a recognized partnership, not the general public). The CP-SAT Users’ Guide recommends that the survey be administered to all personnel and to all key community partners for agencies with 200 line officers or fewer. The Guide introduces the concept of sampling and provides general methodological information on concepts such as stratified random sampling and sample size for consideration by agencies with more than 200 line officers. Ultimately agencies choosing to implement the CP-SAT will choose whether or not a sampling strategy is appropriate, and what strategy to employ.

B2. Procedures for the Collection of Information

The CP-SAT will be administered in an online format using Vovici EFM Community Web-based survey software. Vovici’s survey hosting environment has been designed with security as a foremost consideration, with features such as 128 bit SSL encryption and redundant firewalls. Participant email addresses will be uploaded into the Vovici Community secure Web site and each respondent will be sent a link to the survey via their email address. Although the survey administrators can identify a participant’s survey status (e.g., not started, started, completed), there are no individual identifiers in the data and there is no way to link an individual’s data to their email address. If the respondent does not have an email account, participants can be directed to a URL address via alternate means (e.g., via agency memo with URL address, access to a common computer lab with the site loaded onto each computer). All data exported from the Vovici Community secure Web site will be kept in a secured folder.

Agencies that are interested in participating in the CP-SAT contact the COPS office via email or phone to express interest. A phone meeting between the COPS office, ICF International, and the interested agency is then scheduled to discuss the purpose of the CP-SAT, technological considerations, and details regarding the administration of the CP-SAT. Specifically, the technical considerations discussed are whether agency staff have access to a computer and an email address. In this phone meeting, the following pieces of information are requested from the interested agency to allow ICF International to administer the CP-SAT:

1. Email addresses provided to ICF & CP-SAT form appropriate for each participant

2. Name & email from who the assessment distribution email should be sent

3. Assessment distribution email language

4. Reminder email language

5. Assessment timeline (i.e., data collection begin date, reminder date, end date)

The first two to three sites that contact ICF and commit to administering the CP-SAT will comprise the final test sites.¹ For these test sites and the other agencies covered under this request, the interested agency is instructed to send these materials to ICF International team members, where it will be stored in a secured folder. ICF International will administer the CP-SAT in an online format using Vovici EFM Community Web-based survey software. Vovici’s survey hosting environment has been designed with security as a foremost consideration, with features such as 128 bit SSL encryption and redundant firewalls. Participant email addresses will be uploaded into the Vovici Community secure Web site and each respondent will be sent a link to the survey via their email address.

Emails to participants will include the following statement prominently displayed notifying them of the confidential nature of their surveys:

“Your responses to this survey will be kept confidential. There are no individual identifiers in the data that your agency will receive, and the agency will not be able to link an individual’s responses or data to their email address.”²

This email will look like it came from the name and email address supplied by the agency in #2 above. Although the survey administrators can identify a participant’s survey status (e.g., not started, started, completed), there are no individual identifiers in the data and there is no way to link an individual’s data to their email address. If the respondent does not have an email account, participants can be directed to a URL address via alternate means (e.g., via agency memo with URL address, access to a common computer lab with the site loaded onto each computer). All data exported from the Vovici Community secure Web site will be kept in a secured folder.

            ICF International will prepare a Microsoft Excel report of summary data for each agency that participates in the CP-SAT. The report will use bar charts, pie charts, and tables to report the average percentage of agency staff who selected each answer choice on a group of similar questions (e.g., general problem solving). The report will be individualized for each agency, so that the report only describes that agency’s data. The individualized Microsoft Excel report will be provided to the Chief of Police or Sheriff and it is up to him or her to determine whom he/she makes the data available.

ICF International team members will use the draft user’s guide to help administer the survey, revising as appropriate based on the experiences in following the user’s guide instructions. Participating agencies will be asked to provide feedback on the administration, including additional information they would have liked to know or any improvements they suggest for the administration. The feedback will be used to revise the user’s guide that will be provided publicly for free to agencies along with the survey questions in the future.

At the conclusion of the data collection covered by this request, ICF will make agency-level data available to the government with identifying information removed from the data set (also see description in Section A.10 above).  This data could be useful in identifying areas of need of community policing assistance and resources, and for conducting large-scale research on the state of community policing implementation across different community policing dimensions.

B3. Methods to Maximize Response Rates and Deal with Issues of Non-Response

Because this is a voluntary resource, COPS will not be involved in the collection of data. In the CP-SAT Users’ Guide, however, we provide tips for increasing response rate within an agency. These tips include:

• Chief should stress the importance of the self-assessment process and participation in completing the forms.

• Gain support and “buy in” from each of the groups—officers, supervisors, and command staff—as well as police union leadership or other organized labor bodies. Support from these individuals and groups should be sought at the outset of the project.

• Training/orientation for agency personnel who will be asked to complete the assessment tool form will enhance the overall response rate and reduce the problem of incomplete forms being submitted.

• Respondents should be notified in advance that the assessment tool form will be distributed and that they will be given adequate time to complete the form.

• Respondents should be assured that their responses will be handled in a confidential manner.

• Respondents should be given reminders to complete the assessment tool forms. If necessary, provide new copies of the form instrument each time.

• Offer to provide the respondents a summary report of the results of the self-assessment process.

• Convey how the results will be used to make positive changes, as well as to promote the work being done by the agency and its staff.

B4. Tests of procedures or methods to be undertaken

To date, the researchers have conducted four pilot tests of the content of the survey in law enforcement agencies across the country. Researchers also conducted a usability test of the online format in one law enforcement agency in Gaithersburg, MD. Based on responses and the analysis of findings, improvements were made to both the content and the online format.

B5. Individuals consulted on statistical aspects of the design and organization/persons collecting and analyzing the data.

Rebecca R. Harris Mulvaney, Ph.D.

Manager

Applied Organizational Research

ICF International

(703) 934-3582

Lance Anderson, Ph.D.

Vice President

Applied Organizational Research

ICF International

(703) 934-3674

1 In the final testing phase, the ICF and COPS team will use their experiences to update and finalize the guidebook. For purposes of the test site process, the process of administering the CP-SAT will be the same as for those who take the CP-SAT after the guidebook testing has been completed. However we will be asking for feedback on the process from these final sites, though it will be the ICF/COPS project team’s experiences in applying the guidebook instructions and following the process that will inform any changes and updates to the guidebook.

2 We can only protect the confidentiality of those respondents covered under this collection. The CP-SAT will be more widely distributed and therefore we decided against including any statement on the forms themselves, as those forms may be distributed outside of this survey process.