Remote Psychoacoustic Test Supporting Statement A

SUPPORTING STATEMENT PART A

TITLE OF INFORMATION COLLECTION: Remote Psychoacoustic Test, Phase 1, for Urban Air Mobility Vehicle Noise Human Response

TYPE OF INFORMATION COLLECTION:

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

This collection is being performed as part of a NASA Technical Direction Notice, TDN C2.01.023. The collection is a laboratory psychoacoustic test that seeks test subject annoyance response to sounds of different vehicles. It is the initial, or feasibility, phase of a larger study for NASA to understand and predict human response to Urban Air Mobility (UAM) vehicle noise.

UAM vehicles are a part of NASA’s vision for Advanced Air Mobility, which seeks to develop new air transportation systems that move people and cargo between places previously not served or underserved by aviation [1]. Representative UAM vehicle concepts involve the use of electrically driven rotors, and the noise from these air vehicles in communities may restrict their operation.

The NASA-led UAM Noise Working Group (UNWG), which consists of researchers from academia, industry, and government to identify and address UAM noise issues, published a white paper containing high-level goals intended to address noise barriers that may hamper UAM vehicle entry into service [2]. The white paper identified the need to perform laboratory studies to understand the perception of UAM vehicle noise and to also collect data on variations in perception between geographically different communities.

Based on these recommendations, members of the UNWG proposed that a psychoacoustic test be conducted using test facilities spanning multiple geographic locations to obtain data on community variation in response. Such a study would require the cooperation of multiple partners, but this cooperation could enable a more diverse collection of noise stimuli and human response. Overarching goals of the cooperative study, which is called the UAM Vehicle Noise Human Response Study, are:

• Assemble a wide range of UAM vehicle sounds through cooperation between multiple agencies and organizations for use in human response studies.

• Conduct psychoacoustic tests using the database of UAM vehicle sounds to provide insights into human response to UAM vehicle noise that would be challenging, in terms of access to stimuli and a wide geographic demographic, for any single agency or organization to acquire.

• Assemble the stimuli and annoyance responses into a database that can be used by members of the UAM community for subsequent analyses.

The UAM Vehicle Noise Human Response Study is divided into a feasibility phase (Phase 1) and an implementation phase (Phase 2). In response to the COVID-19 pandemic and the recognition that annoyance response studies benefit from a diversity of test subjects, it was decided that both phases of the UAM Vehicle Noise Human Response Study will remotely administer a psychoacoustic test. The approval for information collection sought in this document only concerns the Phase 1 remote test.

The Phase 1 remote test is necessary because:

• Technical and administrative challenges to the remote laboratory test method need to be understood, and Phase 1 remote test results need to be compared to previous in-person tests before the remote laboratory test approach can be trusted to gather human response to the Phase 2 test.

• UAM vehicle noise stimuli are largely unavailable, and the Phase 1 remote test needs to demonstrate how sounds will be ranked by their annoyance response to incentivize UAM vehicle manufacturers to provide sounds of their vehicles to serve as stimuli in the subsequent Phase 2 test.

• The Phase 1 test will help researchers understand if providing contextual cues to test subjects in psychoacoustic tests affects responses to sounds. One of NASA’s partners in this UAM vehicle noise study, the FAA, is interested in understanding the link between laboratory psychoacoustic test results and community noise testing, which occurs in real communities outside the laboratory. Previous in-person laboratory tests did not provide contextual cues, but providing contextual cues is standardized in community noise testing [3]. The Phase 1 remote test can potentially help NASA and the FAA understand the applicability of results from these previous in-person laboratory tests to future community noise testing.

• Phase 2 of the UAM Vehicle Noise Human Response Study will test how different test subjects from distinct geographic locations respond to UAM vehicle sounds. The Phase 1 remote test will demonstrate that the remote test method can capture responses from test subjects in at least two distinct geographic locations and understand if there is an overall response difference between the two groups. Phase 1 remote test results will inform how many Phase 2 test subjects should be planned for in each geographic region.

2. Indicate how, by whom, and for what purpose the information is to be used.

The NASA Revolutionary Vertical Lift Technology (RVLT) Project of the NASA Advanced Air Vehicles Program will collect the data from the Phase 1 remote test to determine:

• Technical and administrative challenges to the remote laboratory test method and how to address these challenges for the Phase 2 test.

• How well the Phase 1 remote test results replicate a previous in-person laboratory psychoacoustic test so that the same remote test method can be trusted when applied to Phase 2.

• If providing contextual cues to test subjects significantly affects responses to sounds so that NASA can demonstrate to its partners, like the FAA, the applicability of laboratory psychoacoustic test results to plan future community noise testing.

• A ranking of test stimuli by their annoyance response to demonstrate to UAM vehicle manufacturers how sounds they provide for the Phase 2 test will be ranked. This demonstration will be used as incentive for UAM vehicle manufacturers to provide sounds of their aircraft for the Phase 2 test.

• A demonstration of gathering test subject responses from two distinct geographic regions. Analyzing Phase 1 remote test responses between the two distinct geographic regions can help in planning the number of test subjects in each geographic region for Phase 2 of the UAM Vehicle Noise Human Response Study.

This information collection is new. While Phase 1 remote test stimuli are from a previous in-person psychoacoustic test [4], responses to the stimuli have not been previously acquired by a remotely administered psychoacoustic test. The Phase 1 test responses will be compared against the in-person test responses to determine that the remote test method can be applied to subsequent tests for which there may not be an in-person counterpart.

To address the Phase 1 test objective of understanding if providing contextual cues affects responses to sounds, test subjects will be divided into two groups. One group will be provided a contextual cue and the other group will not. Both test subject groups will be provided the same sound stimuli. For test subjects who are given a contextual cue, the Phase 1 remote test will ask them, for each test sound stimulus, “Imaging hearing this sound several times each day while outdoors and near your home, how annoying would this sound be to you?” Test subjects will then indicate their annoyance response. For test subjects not provided contextual cue, the Phase 1 remote test will ask them, for each test sound stimulus, “How annoying was the sound to you?,” which does not provide context through which they indicate their annoyance response. The question, “How annoying was the sound to you?,” was asked to test subjects in the previous in-person test that used the same stimuli that the Phase 1 remote test will use [4]. Subsequent analyses will compare responses between the two contextual cue groups to determine if having a contextual cue gives a significant response difference.

Test subjects will be divided into the two contextual cue groups after test subject recruitment based on US ZIP code. The two distinct geographic regions will only be specified after the ZIP codes from test subjects have been gathered by email. Test subjects will be divided so that there is the same number of test subjects per contextual cue group and that each of the two distinct geographic regions have approximately the same number of test subjects in each contextual group.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection.

Recruitment of test subjects will be an open call for volunteers by electronic mail (email). The email will be sent to participants of the NASA-led Urban Air Mobility Noise Working Group (UNWG), which consists of researchers from academia, industry, and government, and it will request the UNWG participants to forward the email within their organizations. Being a NASA-led working group, NASA has access to email addresses of UNWG participants.

The Phase 1 remote psychoacoustic test will be conducted electronically using a remote test platform being developed as part of NASA TDN C2.01.023. The test will be hosted from the NASA Amazon Web Services (AWS) cloud service, which is approved by the Federal Risk and Authorization Management Program. When test subjects click on an email link sent to them by test administrators to take the Phase 1 remote test, the Phase 1 test application will open on a web browser on their computer. The document “TestApplicationProcedure.pdf” details the steps that will be taken by test subjects as they navigate the online test application. The test platform provides an electronic means for test subjects to consent to taking the test. See “Verification-consent_text.pdf” for more details. All test responses from test subjects will be returned electronically to the NASA AWS.

Test subjects will be able to take the test at any time during the week the test will be administered. If they exit the test before completing it, they may return and complete the test at any time before the end of the test week. If test subjects leave the online test application idle for more than six hours before completing it, the platform will automatically exit the test, and test subjects will need to retake the test any time before the end of the test week.

The NASA AWS will provide test subjects one sound stimulus at a time. While all test subjects will be provided the same stimuli, each test subject will be provided the stimuli in an unique order. Test subjects will listen to test sounds over their headphones. Calibration, described in “TestApplicationProcedure.pdf” will help keep sound levels comfortable for test subjects.

The Phase 1 remote test stimuli will be 84 of the 103 sound stimuli used for the previous in-person psychoacoustic test conducted at NASA Langley Research Center for which responses will be compared [4]. These 84 sounds consist of 50 recorded sounds of small-unmanned aircraft flyovers, 20 recorded sounds of ground vehicles, 8 auralizations of small-unmanned aircraft flyovers, and 6 auralizations of distributed electric propulsion aircraft flyovers. Auralizations are simulated sounds generated from numerical data, such as those generated from computations of acoustic predictions, as opposed to data generated from microphone recordings of real vehicles. The durations of the 84 sounds vary between a minimum of 12 seconds and a maximum of 52 seconds, with the sample mean duration being approximately 23 seconds. Test subjects will listen and respond to the 84 sounds in 4 sessions, with 21 sounds per session, with a break provided between sessions.

The document, “TestApplicationProcedure.pdf,” explains that test subjects in the group provided contextual cue will be provided a different question when responding to sound stimuli than test subjects in the group who are not provided contextual cue. See items 1 and 2, Part A, for additional details. Information on the NASA AWS will indicate which contextual group a test subject is assigned to, and the appropriate test question will be provided to the test subject.

Test administration is performed entirely electronically since there was no need to conduct these steps non-electronically. Contacting test support, as described in “TestApplicationProcedure.pdf,” is performed over the phone. The test support phone number is (781) 852-3199.

4. Describe efforts to identify duplication.

The Phase 1 remote psychoacoustic test does not duplicate another remote psychoacoustic test. The Phase 1 remote test stimuli are from a previous in-person psychoacoustic test [4]. The previous in-person test used 103 sounds, but the Phase 1 remote test will use 84 of the 103 sounds. Since a major objective of the Phase 1 remote test, as described in item 2, Part A, above, is to assess the performance of the remote test method against the in-person test approach, the remote test needs to use stimuli from the previous in-person test. The in-person test results cannot be used to accomplish the other Phase 1 remote test objectives:

• Without conducting a remote psychoacoustic test, the administrative and technical challenges to administering the remote test cannot be ascertained.

• Since the in-person test did not provide contextual cues to test subjects, the remote psychoacoustic test is needed to understand how contextual cues can affect the annoyance response.

• Without conducting a remote psychoacoustic test, the remote test method of obtaining responses to aircraft sounds cannot be demonstrated to UAM vehicle manufacturers who need to provide sounds of their aircraft for Phase 2 of the UAM Vehicle Noise Human Response Study.

• The in-person test was administered to test subjects from the local geographic area surrounding NASA Langley Research Center in Hampton, VA. The remote Phase 1 test is needed to demonstrate obtaining test subjects from distinct geographic regions and to provide insight as to how many test subjects need to be in different geographic regions that will be used for the Phase 2 test.

5. If the collection of information impacts small businesses or other small entities (Item 5 of the OMB Form 83-I), describe the methods used to minimize burden.

Collection of this information does not have a significant impact on small businesses.

6. Describe the consequence to federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If the Phase 1 remote test is not conducted, then it will likely be more difficult for NASA to pursue Phase 2 of the UAM Vehicle Noise Human Response Study.

UAM vehicle manufacturers will be less likely to provide sounds of their aircraft for Phase 2 if the remote test method that will be used to rate their sounds remains undemonstrated. It is one of the primary reasons why the Phase 2 test cannot be executed in-place of the Phase 1 remote test.

The performance of the remote test platform will be more difficult to ascertain without the Phase 1 remote psychoacoustic test. If the Phase 2 test was conducted without a Phase 1, a subsequent in-person test will need to be conducted with Phase 2 test stimuli in a controlled test environment to assess the performance of the remote test method. Conducting the subsequent in-person test to Phase 2 would increase time and cost to NASA and would cause additional burden to test subjects who will need to take the in-person test, whereas the Phase 1 remote test will be compared to an in-person test that has already been conducted.

If the Phase 1 remote test is not conducted, the NASA RVLT project will be presented with reasons not to pursue the Phase 2 test using an undemonstrated remote testing platform. It will reduce NASA’s ability to predict the human response to UAM vehicle noise without access to the larger stimuli and test subject pool the remote test can provide.

7. Explain any special circumstances that would cause an information collection to be conducted in an exceptional manner:

The information collection will not be conducted in an exceptional manner. Test subjects will need to take the Phase 1 remote test only once, not need to provide a written response in fewer than 30 days, not need to provide documentation, not need to retain records, not provide data that will be under unapproved data classification, not swear a pledge of confidentiality, and not submit proprietary trade secret(s) or other confidential information.

8. Provide the date and page number of publication in the Federal Register for the 60-day and 30-day FNRS, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB.

60-day FRN: Federal Register Volume 87, Number 30, on 2/14/2022. No comments were received.

30-day FRN: Federal Register Volume 87, Number 130, on 7/8/2022.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No incentives of any form are anticipated for participants in the current data collection.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The verification screen for the Phase 1 remote test will state that test subject identity will be kept confidential and their responses will be combined with other responses. See “Verification-consent_text.pdf.” The verification screen will also display the expiration date for OMB approval of the PRA.

Only test subject name, phone number, and email address, gathered by email, will be stored, encrypted, on the NASA cloud service to administer the Phase 1 remote test. This information is only used to authenticate test subject access to the Phase 1 test. It will be deleted once the test is completed.

Test subject ZIP code, also gathered by email, will be used to specify two distinct geographic locations and to evenly divide test subjects into contextual cue groups (see items 2 and 3, Part A), but will only be associated with test subject number, which will be between one and the total number of test subjects. The ZIP code will not be associated with test subject name, phone number, and email address.

A Privacy Impact Assessment on the Phase 1 remote test was submitted for approval to the NASA Langley Research Center Privacy Officer. The Privacy Act System of Record Notice, “NASA 10HERD – Human Experimental and Research Data Records,” applies to the information collected for the Phase 1 remote test.

According to NASA Technical Direction Notice, TDN C2.01.023, which is supporting the Phase 1 remote test, the NASA contractor executing the test should protect test subject personally identifiable information in accordance with the Privacy Act of 1974. The TDN also specifies that the Phase 1 remote test should be approved by an Institutional Review Board. The Institutional Review Board will review the Phase 1 remote test to ensure that it follows the Federal Policy for Protection of Human Subjects, or 45 CFR 46, which provides regulations to ensure that risks to test subjects are reasonable in relation to benefits and are minimized, informed consent is sought, data collection is monitored to ensure test subject safety, and adequate provisions to protect the privacy of subjects and maintain confidentiality are in place.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

Questions of a sensitive nature are not found in this information collection.

12. Provide estimates of the hour burden of the collection of information.

RESPONDENT CATEGORY	NUMBER OF RESPONDENTS	ESTIMATED COMPLETION TIME	ANNUAL BURDEN
Individuals, Main Phase 1 Test	90	2 Hours	180 Hours
TOTAL	90	2 Hours	180 Hours

While 80 test subjects are needed for the Phase 1 remote test, some of these test subjects may not complete the Phase 1 remote test. Increasing the likelihood that 80 test subjects complete the test is why 90 test subjects are being sought for the Phase 1 remote test.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information.

The annual cost burden on respondents and record-keepers, based on a rate of $29.90/hour, is $5382.00.

14. Cost to the Federal Government: Provide estimates of annualized costs to the Federal government.

The annual cost of Federal employees for monitoring the development and execution of the Phase 1 remote test is 0.8 FTE, or $126,000. This estimate includes the time from the Technical Monitor on the project and the time from the Alternate Technical Monitor. It also includes minimal time from the contracting officer and other NASA employees who participate in technical interchange meeting and reviews.

The cost of executing TDN C2.01.023 with contractors who develop the remote test platform and who will help administer the remote test platform is $465,378. This cost is the estimate cost for completion of the TDN C2.01.023 contract, and it excludes the annual FTE cost stated above.

The total cost to the Federal government is $591,378.00.

15. Changes in Burden: Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I, if applicable.

This is a new collection, and hence, a program change.

16. Publication of Results: For collections of information whose results will be published, outline plans for tabulation and publication.

The Phase 1 remote test is currently scheduled for September 10-17, 2022. An informal report will be submitted by the contractors at the end of the TDN C2.01.023 contract in October 16, 2022. The report will inform how test subjects were recruited, how many test subjects took and completed the Phase 1 remote test, and any challenges or issues that occurred during the test administration. The informal report will not include test analyses.

A subsequent publication by NASA on Phase 1 remote test analyses is expected to be submitted to an appropriate technical conference or journal within six months of October 16, 2022.

The previous in-person test from which Phase 1 stimuli will be drawn used augmented linear regression combined with percentile bootstrapping of regression model parameters to determine that sounds from small-unmanned aircraft were significantly more annoying than sounds from ground vehicles [4]. Two regression lines were created with this method, one for responses to small-unmanned aircraft sounds, and the other for responses to ground vehicle sounds. The offset between the regression lines was found as well as a confidence interval on the offset. These same analyses techniques will be used on the Phase 1 remote test results to determine how well the remote test replicates the results of the in-person test. If the Phase 1 remote test annoyance difference between responses to small-unmanned aircraft and ground vehicle sounds has a similar high coefficient of determination and regression line offset confidence intervals as results from the in-person test, the remote test can be claimed to replicate the in-person test [5]. See item 4, Part B, for further details.

Augmented linear regression combined with percentile bootstrapping will also analyze if providing contextual cues affects responses to sounds. Instead of analyzing responses between small-unmanned aircraft and ground vehicle sounds, responses will be analyzed between the test subject groups with and without a contextual cue, which will again produce two regression lines. If the offset between the regression lines is not significant, as determined from confidence intervals, then it will be concluded that providing contextual cues does not affect test subject responses [5]. See item 4, Part B, for further details.

Analysis of Variance (ANOVA) will demonstrate finding differences in the overall mean annoyance response to Phase 1 test stimuli between test subjects from two distinct geographic regions. See item 1, Part B, for further details. As a reminder, the geographic regions will only be specified after collected test subject US ZIP codes.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

NASA will display the expiration date for OMB approval on the verification page (“Verification-consent_text.pdf”) of Phase 1 remote test platform.

18. Explain each exception to the certification statement identified in Item 19, "Certification for Paperwork Reduction Act Submissions," of OMB Form 83-I.

The NASA Revolutionary Vertical Lift Technology Project and NASA Technical Direction Notice (TDN) C2.01.023 take no exception to 5 CFR 1320.9, per Siddhartha Krishnamurthy. No exception to the certification statement is sought.

The NASA office conducting or sponsoring this information collection certifies compliance with all provisions listed above.

Name: Siddhartha Krishnamurthy

Title: Assistant Branch Head, Structural Acoustics Branch, NASA Langley Research Center

Email address or Phone number: siddhartha.krishnamurthy@nasa.gov, 757-864-7656

Date: June 28, 2022

(Certifying individual must be a civil service employee)