v1 Supporting Statement Part B 2120-0699 (Final 110624)

Supporting Statement

AVIATOR Customer Satisfaction Survey

B. Statistical Methods

1. Describe the potential respondent universe.

FAA job seekers; this collection is offered to all applicants.

2023 Applicant Survey Metrics: 1 January to 31 December 2023
2,968	Number of cases or job announcements with survey responses from applicants. (64.49% of the total 4602 cases or job announcements available in the time period.)
201,459	Number of applicants that submitted applications and had the opportunity to fill out a survey
17,407	Number of survey respondents or applicants
5.9	Average number of surveys per case or job announcement
1,751	Max number of surveys on a case or announcement
163	Number of cases or announcements with more than 10 responses

2. Describe the procedures for the collection of information.

Online voluntary survey questions are presented after submitting an application or when checking the status of an application in AVIATOR.

3. Describe methods to maximize response rates.

Applicants have the ability to take the survey when submitting an application or when they check the status of a submitted application. The applicant is notified that collection of this information is voluntary. It is estimated that it will take each respondent 3 minutes to complete the survey.

AVIATOR is an employment recruiting and application processing system. As such, the amount of user activity in AVIATOR is directly proportional to the number of people seeking employment with the FAA. AVIATOR was refreshed in 2022 / 2023 to a mobile-friendly layout that was modernized, streamlined, and that better implemented the FAA branding (standard layout for web sites). The AVIATOR database also implemented connection pooling which greatly increases the speed and responsiveness of the AVIATOR system. In addition to the system changes, the FAA Office of Communication (AOC) greatly improved its online and social media presence, facility kiosk advertising, recruitment / employment seminars, and job application workshops. The combination of improved system responsiveness and enhanced recruiting activities have led to a significant increase in survey submissions / responses in 2022 and 2023.

4. Describe tests of procedures and methods to be undertaken.

Surveys are analyzed weekly, quarterly, and annually. Weekly analysis reports of submitted surveys are reviewed in the AVIATOR Change Control meeting that occurs every other week. Comments in the survey are also reviewed. Any items that are actionable for AVIATOR enhancements, USAJobs issues, or Human Resources interest, are discussed as well as items that no immediate action is necessary but should be monitored.  

As an example, a survey respondent identified an issue with the Supplemental Qualifications section of the application which was then provided to the development team and mitigated in the next software release.

Review and analysis include comparing metrics (i.e., number of surveys completed vs number of active job announcements, number of survey respondents for any given survey, average or maximum number of surveys per job announcement, etc.), satisfaction percentages, and categorization of comments (i.e., system navigation issues, unclear job announcement information, etc.). The metrics give us baseline data while the approval percentages let us know our customer’s overall satisfaction with the system and the job application process. The comments review looks for statements from applicants that can be investigated to see if there is an issue or a potential for improvement. The categorization of comments provides us general and specific areas where applicants have indicated displeasure with the system or the process.

5. Provide the names of consultants and the person who will collect and analyze the information.

1. Toni Main-Valentin (toni.main-valentin@faa.gov)

2. John Vandyke (john.vandyke@faa.gov)