Supporting Statement Part B

Supporting Statement

Pilot Medical Disclosure Decision Making Model for Safety Risk Assessment Survey

OMB # 2120-XXXX

B. Statistical Methods

1. Describe the potential respondent universe.

The universe of potential respondents includes 148,306 Air Transport pilots, those with the highest level of pilot certification and required to act as captain or pilot-in command of scheduled airline flights, represented within the aerospace medical certification database. Commercial, private, student, and recreational pilots are not included in the population of interest for this survey. Previous administrations of surveys sampling from the aerospace medical certification database (AMCS) have resulted in a 35% response rate.

The minimum sample required to detect an effect from the current collection is N = 1,950. Given prior response rates of 35%, we will sample 51,907.1 of the total number of Air Transport pilots within the database.

2. Describe the procedures for the collection of information.

The sampling frame will be constructed from all cases of Air Transport Pilots having a certificate listed as issued, rather than deferred or denied, within the AMCS database. A simple random sampling strategy will be used. We will draw a random sample of n 1,950 ATP-certified pilots from the AMCS database, using random numbers to select without replacement.

Sample Size Estimation. A power analysis was conducted using the pwrSEM Shiny application to estimate the minimum sample size required to detect direct and interaction effects in a constrained multi-group structural equation model (SEM). The model includes six latent predictor variables, four latent interaction terms, and one latent outcome variable. All structural paths were constrained to be equal across three randomly assigned survey conditions. The three survey conditions corresponding to hypothetical medical conditions critical to aviation safety. Participants will be asked to respond to survey items with one of the three medical conditions in mind.

Effect sizes were specified as small-to-moderate (β = 0.10 to 0.20), and the alpha level was set at .05. Results indicated that with 650 participants per group (total N = 1,950), the study is adequately powered (≥ .80) to detect:

• Main effects as small as β = 0.10 (power ≈ .85)

• Interaction effects of β = 0.15 or greater (power ≈ .86)

These findings confirm that the proposed sample size of N = 1950 (n = 650 per group) is sufficient for detecting theoretically meaningful effects, particularly under the constraints of a multi-group structural equation modeling design.

There are no unusual problems requiring specialized sampling procedures. There is no use of period (less than annual) data collection cycles.

3. Describe methods to maximize response rates.

Methods to maximize response rates include:

• The notification/invitation package will include a cover letter worded to engage the pilots’ interest, the reasons for the survey, and the benefits that they will receive from participating.

• Reimbursement of $25 upon attentive response completion distributed via eGift Card.

• The survey will be made available online.

• Email reminders (n = 2) of the survey will be sent to all pilots who have not completed the survey. Reminder emails will be sent to nonresponders at two-week intervals during the data collection period. Methods to assess generalizability include:

  • Non-response analyses will be conducted using demographic data available in the AMCS system, including gender and geographic region.

  • If survey respondents differ significantly from the population, weighted sample procedures will be used to mitigate the bias.

4. Describe tests of procedures and methods to be undertaken.

Prior to administration, the survey will go through pilot testing with a small group of subject matter experts. Testing will involve having subject matter experts review each survey item for clarity and understanding.

5. Provide the names of consultants and the person who will collect and analyze the information.

The survey development and analysis will be conducted under the direction of Dr. Julia Beckel, a Research Psychologist employed by the FAA at the Civil Aerospace Medical Institute, Oklahoma City (405) 954-6825 (julia.l.beckel@faa.gov).

The survey will be distributed, and the data will be collected by a survey contractor, Cherokee Nation 3S, LLC. The point of contact for data collection will be Janine King (janine.ctr.king@faa.gov).