Supporting_Statement_A_Final

Department of Transportation

Federal Railroad Administration

INFORMATION COLLECTION SUPPORTING STATEMENT

Experimental Investigation of Automation-induced Human Error in the Locomotive Cab

OMB CONTROL NUMBER 2130-NEW

Summary of Submission

• This information collection request is a new submission, rquesting a three-year approval. The proposed assessment is intended to identify and evaluate the potential for human error within the operation of system and automation in locomotive cabs.

• The estimated total number of burden hours requested for this submission is 168 hours.

• The estimated total number of responses for this submission is 48.

• The required 60 and 30 Day Federal Register Notice were published in the Federal Register on May 2, 2017 (see 82 FRA 20530) and September 13, 2017 (see 82 FRA 43078), respectively. See 82 FR 18341. After better clarification of the purpose of this collection, an additional 30 day FR was published on November 14, 2018. See 83 FR 56913.

• Comments were received from the Association of American Railroads outlining some concerns with the research approach in the human error study. A reply was made clarifying the research approach. All documentation can be found in this submittal package.

INTRODUCTION

Part A. Justification

1. Circumstances that make the collection of information necessary.

The purpose of this work is to identify and evaluate the potential for human error associated with the operation of systems and automation in the locomotive cab. This research addresses the Department of Transportation (DOT) strategic goal of safety. If more is understood about the nature and risk of human error in the operation of automated systems, then error mitigating steps can be taken to provide safer systems and reduce risk of accidents or incidents involving use of these systems. Particular statues, legal or administrative requirements are not applicable.

Distraction and loss of situation awareness have been implicated in rail accidents. One example is an overspeed derailment outside Philadelphia. The engineer was listening to radio reports of trespassers throwing rocks at trains. As he listened to these reports, he became confused on his current location, and did not realize that he was approaching a curve in the track with a considerably lower speed restriction.

The pilot study revealed the potential for distraction to prevent an engineer from noticing a request for information and a change in the Trip Optimizer (TO) mode. In this particular study, where the train was running on level terrain, the consequence was that the train eventually began slowing down. Such an event on a downhill grade would result in increasing speed and potential overspeed.

Similar operator-distraction errors while using automation have been noted in aviation. The following table identifies the results of a review of data from the Commercial Air Safety Team (CAST) to determine how frequently different airline HAI-relevant policy issues were violated. The table lists the frequency with which the policy violation occurred among the 50 incidents. The second column specifies the policy violation, and the third includes our estimate of the relevance to Positive Train Control (PTC) systems. These estimates were based on observations of rail industry issues and accidents. For example, operator distraction and fatigue are significant concerns in rail and are indicated on the below table as being of high relevance to PTC. The table indicates that a failure to resist distraction is the most frequently identified concern.

(Sebok, Wickens, Laux & Jones, 2015)

Sebok, A., Wickens, C., Laux, L. & Jones, M. (2015). Supporting Human-Automation Interaction in the Rail Industry by Applying Lessons from Aviation. Proceedings of the Human Factors and Ergonomics Society 59^th Annual Meeting. Pp 1661-1665.

2. How, by whom, and for what purpose is the information to be used.

Through the use of a locomotive simulator laboratory, contracted researchers will conduct research aimed at answering the following specific questions regarding human-automation interaction and any associated human errors:

1. How does a communication from dispatch, timed to coincide with a change on an automation display, affect the likelihood of noticing the change in automation?

What impact does a “failure to notice” the automation change have on train control performance (e.g., does the automation switch to manual mode and cause unexpected deceleration or overspeed)?

Where is the operator looking, or what are they doing, if they fail to notice the events?

2. How frequently do operators notice an error in automation systems programming?

3. How frequently are operators able to stop in time for the “stop and protect” procedure?

This is a discretionary research project. The integration of automated systems in the locomotive cab space and elsewhere in rail operations is changing rapidly. Results of this research will be used by the agency to aid in recommending training, operational procedures, or automation design requirements that will improve the safe operation of automated systems.

Simulator-Laboratory research.

The work will be conducted through a contracted research organization responsible for experimental design, recruitment of human subjects, conduct and analysis of the research. Human subjects will be recruited as compensated volunteers from railroad participation, the operating unions, and retired operators with the necessary locomotive operating knowledge and experience. Through use of the Cab Technology Integration Lab (CTIL), a full-sized locomotive cab simulator in Cambridge, Mass., a maximum of 24, 2-person locomotive operating crews will be asked to operate the simulator given a particular set of scenarios. The scenarios include voice communication with the crew to simulate the dispatcher.

Objective measures of the crew’s performance, related to workload, will be observed and collected. Workload is defined as task loading, or the number of tasks in a scenario. The high workload scenarios have more tasks than the low workload scenarios. Based on the pilot study, and on research and operational experiences in other industries, high workload is often associated with error. This data will include the number and types of human errors, if any, observed while operating. Experimenters will sit near the simulator to observe and take notes on the crews’ performance during the scenarios.

Automation design concerns and training needs are often revealed through operator error. While the FRA expects that distraction and high workload (tasking) can lead to increased error probability, part of this research effort includes identifying design and training aspects that can be improved to mitigate potential errors.

3. Extent of automated information collection.

Automated event logs from the simulator and videos of the crew (overview of the cab, and from behind), the engineer, views of the displays of the locomotive automated systems, and eye-tracking data. No other form of automated data collection is planned or needed.

4. Describe efforts to identify duplication.

Not applicable. Laboratory research being conducted.

Automation is being developed and implemented in a variety of industries with the goals of improving overall system performance and reducing the probability and impact of human or operator error. However, as automation is introduced, it changes the role of the operator and creates new opportunities for different and unanticipated types of errors. In the rail industry, automated safety protection and optimization systems are being introduced into the locomotive cab. This paper describes a simulator-based investigation of human-automation interaction in the locomotive cab. Three scenarios were conducted in a dynamic and realistic rail simulator with three professional locomotive engineers. This preliminary study identified three distinct errors related to failing to appropriately monitor automation, and suggested potential opportunities for design improvements.

The pilot study was an informal assessment and included three (3) participants and identified Three (3) errors. The currently proposed study is a controlled experiment to investigate with a much larger population of participants, the potential for automation to lead to human error.

4. Efforts to minimize the burden on small businesses.

Not applicable. Laboratory Research

6. Impact of less frequent collection of information.

Not applicable. Laboratory research conducted.

7. Special Circumstances.

No special circumstances. Not applicable. Laboratory research being conducted.

8. Compliance with 5 CFR 1320.8(d).

The required 60 and 30 Day Federal Register Notice were published in the Federal Register on May 2, 2017 (see 82 FRA 20530) and September 13, 2017 (see 82 FRA 43078), respectively. See 82 FR 18341. FRA received no comments in response to this notice

9. Payment or gifts to respondents.

As is consistent with normal experimental practice, FRA will provide monetary payment for participation in the research program in order to compensate participants for their time. A $350 compensation will be provided to respondents associated with this proposed collection of information. This is in compensation for their time and travel costs for participating in the study and as wage compensation for wages lost as a result of taking the time to participate in the study. This compensation is in-line with previous research studies conducted by FRA, such as OMB Control No. 2130-0622, Cab Technology Integration Lab (CTIL) Head-Up Display Study.

10. Assurance of confidentiality.

The research will be presented to an independent/institutional review board for approval as required by 45 CFR 46. The review is required as protection from harm and confidentiality of human subjects of biomedical and behavioral research. Identity of human subjects is not reported. Performance of human subjects in the simulator laboratory is kept anonymous. No laboratory research involving human subjects will be conducted without this approval.

11. Justification for collection of sensitive information.

Not applicable. No sensitive information to be collected.

12. Estimate of burden hours for information requested.

	Respondent Universe	Total Annual Responses	Average Time per Response	Total Annual Burden Hours
Simulator Experience	48 participants	48	3.5 hours	168 hours

24, 2-person crew x 5 scenarios x 35 min each scenario, plus 35 minutes for explanation and debrief per crew = 10080 minutes or 168 hours total

Per the Bureau of Labor and Statistics, Occupational Employment and Wages, May 2016, (53-4099) Rail Transportation Workers, All Others, the median hourly wage is $29.05 per hour. Therefore, the cost would be $29.05 x 96 hours = $2788.80.

13. Estimate of the total annual costs burden.

Not applicable. Laboratory research conducted.

14. Estimates of costs to the Federal Government.

Per question 9 above, there will be a payment to the participants in the estimated amount of $350. Therefore, the total cost for participants is estimated at $16,800 ($350 x 48 participants).

Total cost to the Government for this study, based on contractor proposal, is estimated at $511,600 including the costs for participants ($16,800) above.

15. Explanation of the program change or adjustments.

Not applicable. No changes or adjustments needed.

16. Publication of results of data collection.

Results will be published as FRA technical reports and published in a peer-reveiwed journal.

17. Approval for not displaying the expiration date of OMB approval.

None needed. Not applicable

18. Exceptions to the certification statement.

None. Not applicable.

Page 5