Supporting Statement Part B-MFO-3.1.1

Supporting Statement

Survey of Industry’s Response to Safety Alert for Operators (SAFO) 17007

B. Statistical Methods

1. Describe the potential respondent universe.

The universe of potential respondents are employees who lead departments of aircraft operators responsible for Operations and Standards, Training, and Safety in domestic Part 121 and Part 135 operations. We are seeking responses from employees of both operators as the types of flight operations conducted by a Part 121 operator (domestic, flag, supplemental) are different from the types of flight operations conducted by a Part 135 operator (charter on-demand). These differences can influence an operator's policies and procedures, including how a company may respond to SAFO 17007 and its applicability to their operation.

For Part 121 operations, the intent is to conduct a census of the entire population of leaders of the departments listed here¹. This list contains 106 Part 121 operators. This is estimated to consist of up to 6 individuals per operator (although some operators may combine positions, and therefore have fewer than 6 individuals).

For Part 135 operations, the FAA started with a recent list of operators² and limited the universe of respondents to those operating 3 or more turbo-jet powered aircraft to minimize burden on small business operators and operators who may lack the personnel to implement SAFO 17007 recommendations. This filtered the list down to 56 Part 135 operators. The intent is to conduct a census of this population. This is estimated to consist of up to 6 individuals per operator (although some operators may combine positions, and therefore have fewer than 6 individuals).

Based on the 162 cargo and passenger operators identified, there is the potential for up to 972 individual respondents. Based on subject matter expert input from pilots who fly for Part 121 and Part 135 operators it was assumed an organization may have up to six personnel who serve in leadership roles for departments responsible for Operations and Standards, Training, and Safety (e.g. 2 leaders per department). For Part 121, this reflects 636 potential respondents (2 respondents * 3 departments * 106 operators), and for Part 135 this reflects 336 potential respondents (2 respondents * 3 departments * 56 operators). It is possible for an operator to have both a Part 121 and Part 135 certificate. In these rare cases, if a respondent has a leadership role within the department of interest for both Part 121 and Part 135 operations, the duplication would be caught during the initial e-mail to the respondent. It is anticipated the majority of respondents will participate.

2. Describe the procedures for the collection of information.

Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This information collection will be a census of the entire population, so we do not anticipate a statistical method for sampling. MITRE will be reaching out to industry representatives through existing networks and asking representatives to identify personnel in their Part 121 or Part 135 operation who would best be able to provide a response for their organization on operations and standards, safety, or training.

3. Describe methods to maximize response rates.

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Operator representatives at high levels of management have expressed an interest in supporting the FAA’s efforts to improve safety. This commitment will likely lead to the high levels of organizations encouraging participation in the survey.

The researchers will direct each survey solicitation to an individual and the email will describe the value of the information being requested. In addition, the survey software allows tracking of which respondents have accessed the survey, partially filled it out, or completed the survey. Those who have not completed the survey will receive up to 3 reminders to complete the survey prior to the end of the initial 30-day survey period (at 7, 14, and 21 days). Participants have up to 60-days to respond.

4. Describe tests of procedures and methods to be undertaken.

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

MITRE will conduct testing of the electronic survey with not less than 10 MITRE and/or FAA test participants to ensure the survey is accessible, contains clear questions, and the data collection functions as expected. Once testing is completed, the test data will be deleted and the survey opened to the targeted population.

5. Provide the names of consultants and the person who will collect and analyze the information.

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The survey development, distribution, collection, and analysis will be conducted under the direction of Dr. Beth Lyall-Wilson, the Project Leader for this initiative and a Lead Aviation Safety Research and Data Analyst employed by the MITRE Corporation (703)-983-6252 (elyallwilson@mitre.org). Programmatic oversight is provided by Victor Quach, the Scientific and Technical Advisor employed by the FAA at 800 Independence Avenue, SW Washington, DC 20591. (202)-267-3585 (victor.k.quach@faa.gov)

1 https://www.transportation.gov/sites/dot.gov/files/2020-02/Cert Carrier 2020_1.pdf

2 https://www.faa.gov/about/office_org/headquarters_offices/avs/offices/afx/afs/afs200/afs260/media/135aircraft.xlsx