1110-0050_Supporting Statement B

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

1. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection.

• Estimation procedure.

• Degree of accuracy needed for the purpose described in the justification.

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The FBI NA offers a 10-week training program on a quarterly basis. Approximately 1,000 law enforcement professionals attend the training program each year. These law enforcement professionals represent state and local police and sheriff’s departments, military police organizations and federal law enforcement agencies from the United States and over 150 foreign nations. The FBI NA invites all students to complete two questionnaires; the End-of-Session Student Course Questionnaire and the General Remarks Questionnaire. Two weeks before the end of the program an invitation is e-mailed to all students attending the program. The e-mail message will include the link directing all students to the questionnaires. Each questionnaire includes a description of its aim and the purposes to why the data is being collected. The questionnaires will also explain to respondents that their participation is completely anonymous.

After completing the questionnaires, the respondents will be instructed to select “submit” in order to record their responses. Because all FBI NA students will be offered the opportunity to participate in the questionnaires, the questionnaires do not involve any sampling, methods for stratification, imputations or similar statistical methods.

2. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

To ensure a high response rate, several measures have been put into place. For numerical data collection an electronic application, QuestionMark Perception, will be used to calculate simple totals, percentages, and cumulative percentages. QuestionMark Perception will also capture student comments entered in response to open-ended questions. Next, the questionnaires will focus strictly on those issues of great interest in order to keep it as short as possible while still accomplishing the goals of the questionnaires. In addition, by offering the questionnaires online, we reduce the level of effort needed to complete it, as well as the steps involved in creating the database for analysis. Response rates will also be maximized by e-mail reminders which are sent out automatically through QuestionMark Perception; five days after the questionnaires have been sent. Students are also reminded about the questionnaires during the two–week evaluation period and within their group meetings while at the FBI NA. This face-to-face reminder is done by FBI NA personnel. Finally, by including all students we feel confident that the resulting data will reflect the experience of a wide range of respondents, including the full variety of law enforcement agencies and personnel levels. It is with the understanding of the students that by completing these evaluations they will help the FBI determine the strengths and weaknesses of the existing program.

3. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The questionnaire forms included as part of this package were developed following extensive interviews with FBI NA personnel and instructors. After determining the desired content, items were developed by experienced questionnaire experts. A thorough review of the content centered on several issues, including:

• Clarity of instructions

• Ease with which information requested can be provided

• Length of the instrument, and

• Quality of the resulting data in terms of how it can be used to answer the questions of interest.

When the draft instrument was signed off by FBI NA personnel, programming was undertaken. Following this step, internal tests were run to ensure functionality. After a thorough check, a sample of nine students at the FBI NA was asked to complete the online questionnaire. Each respondent was interviewed afterwards to determine whether they had any problems completing the questionnaire and to get their input on changes that might clarify its content. Only minor wording changes were suggested and implemented.

4. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The following individual was consulted regarding the statistical aspects of the project:

Kevin Furtick

Acting Chief

Evaluation and Assessment Unit

FBI Academy

Quantico, Virginia 22135

The person who will be involved in collecting and analyzing the questionnaire data:

Laleatha B. Goode

Management Program Analyst

Evaluation and Assessment Unit

FBI Academy

Quantico, Virginia 22135

2