Comments_OMB Part B

National Highway Traffic Safety Administration

Information Collection Request Supporting Statements: Part B

Examining the Effectiveness of Lane Departure Warning and Lane Keep Assist Advanced Driver Assistance Systems

for Improving Driver Response

OMB Control No. 2127-NEW

Abstract:¹

This information collection request (ICR) from the National Highway Traffic Safety Administration (NHTSA) of the U.S. Department of Transportation is to seek approval to conduct 11 voluntary information collections as part of a one-time research study involving up to 80 licensed drivers of various ages. The purpose of the research study is to examine the effectiveness of lateral control advanced driver assistance systems (ADAS) for improving driver response, with specific focus on Lane Departure Warnings (LDW) and Lane Keep Assist (LKA). The respondents will participate in two separate driving data collection sessions. Within this study, NHTSA’s contractor, the Virginia Tech Transportation Institute (VTTI), will instrument two research vehicles with data acquisition systems (DAS). The DAS includes video cameras and sensors that allow for collecting continuous data that encompasses driver behavior and vehicle performance. The completion of the two drives will take place on different days. Recruitment of study respondents is from southwest Virginia, specifically the New River Valley and surrounding areas (Roanoke, Salem, etc.). The target for the study is 50 participants, completing both sessions and with valid data collected for each; therefore, the research team will contact up to an estimated 340 potential research participant, to identify approximately 85 respondents who will be eligible for participation and complete the informed consent process, estimating that up to 80 respondents will participate in the driving sessions. Since the research study involves driving sessions on two different days, recruitment of up to 80 respondents is needed to account for attrition and potential replacements to ensure that the research study includes 50 participants who complete both the Session 1 and Session 2 drives This ICR includes the following information collections: (1) a pre-screening questionnaire; (2) an informed consent form for Session 1; (3) a vision and hearing evaluation for Session 1; (4) Knowledge and Opinions Questionnaire for Session 1; (5) the Session 1 drive; (6) the Session 1 post-condition questionnaire; (7) the Session 1 post-drive questionnaire; (8) the Session 2 drive, LDW subset; (9) the Session 2 LDW subset post-drive questionnaire; (10) the Session 2 drive, LDW/LKA subset; and (11) the Session 2 LDW/LKA post-drive questionnaire. The purpose of the research study and its associated information collections is to help NHTSA better understand the effectiveness of lateral control systems, the human factors that affect the performance of lane departure warning (LDW) and lane-keep assistance (LKA) systems, and about the system characteristics that will favor better acceptance. This will, in turn, help NHTSA to develop, implement, and maintain effective countermeasures that meet the agency’s mandate to reduce the number of deaths, injuries, and economic losses resulting from motor vehicle crashes on the U.S. highways. The planned data collection activities discussed herein have been approved by Virginia Tech’s Institutional Review Board (IRB). NHTSA will use the information collected to produce a technical report containing summary statistics and tables. No identifying information or individual responses will be reported. The technical report will be made available to a variety of audiences interested in improving highway safety through the agency website and the National Transportation Library. This is a new information collection that is estimated at 269 annual burden hours and $9,421 annual cost.

B.1. Describe the potential respondent universe and any sampling or other respondent selection method to be used.

The potential respondent universe is comprised of all U.S. licensed driving adults between the ages of 25 and 65 residing in the New River Valley and Roanoke Valley regions in Virginia. VTTI has an extensive database of individuals who have either participated previously or who have expressed interest and provided contact information. This database is private and only accessible to those tasked with recruiting subjects. This database will be the initial source of recruitment for the study, although additional recruitment methods may be used (social media advertisements, etc.). From this universe, a screening questionnaire will be administered to an estimated 340 potential participants to qualify a total sample of up to 80 volunteer drivers, with gender representative of the population. Participants will be screened over the phone to determine eligibility, with recruitment personnel recording responses on a paper form using an anonymized ID. The target for the study is 50 participants. However, because the research involves driving sessions on two separate days, it is estimated that up to 80 respondents will need to be recruited and enrolled to account for attrition and potential replacements to ensure that the research study includes 50 participants who participate in both study days. This is an experimental study that will examine the effectiveness and preferences of lateral control warnings ADAS for improving driver response. Care will be taken to recruit participants from a wide cross-section of the population in the study area to ensure people of different demographics (e.g., race, ethnicity, sex) have equal opportunity to volunteer to participate.

B.2. Describe the procedures for the collection of information.

Aim and Research Questions for the Collection of Information

The objective of this data collection is to evaluate driver acceptance of and driver performance using lane departure warning (LDW) alone and in combination with lane keep assistance (LKA) systems, which will be accomplished by addressing the research questions (RQs) below:

• RQ5: Why do drivers turn LDW alerts off? What keeps the system turned on?

• RQ6: What alert characteristics (e.g., auditory alert, haptic alert, visual component, or a combination) available for LDW yield the highest rates of driver acceptance?

• RQ7: Where the interface design plays a role keeping it on, how difficult is it to turn the system off?

• RQ8: How does modality influence performance/effectiveness (behavioral outcome) of the alert?

• RQ9: Is there an additional safety benefit associated with LDW alerts when LKA is activated?

• RQ10: Does the timing of the alert affect driver reaction?

• RQ11: If a safety system is active, does alert prior to activation influence the outcome?

Procedures for the Collection of Information

A targeted group of adult study participants with valid U.S. driver’s licenses will be recruited in compliance with Virginia Tech Institutional Review Board protocols. This research project has been designed with a target of 50 participants, although it is anticipated that up to 80 participants will be required to achieve this target, accounting for attrition and potential replacements. The participants will be asked to complete questionnaires and to drive test vehicles developed for the purpose, denoted hereafter as LDW mockup vehicle and LDW/LKA factory vehicle. The LDW mockup vehicle is a reference vehicle developed by the contractor, able to mimic a variety of modality and timing conditions. The LDW/LKA factory vehicle is supported by a vehicle that has factory-installed LDW and LKA. Both research vehicles will be instrumented with a data acquisition system (DAS), which includes video cameras and sensors that allow for collecting continuous data encompassing driver behavior and vehicle performance. The participants will be asked to perform controlled driving tests on the Smart Roads facilities with the LDW mockup vehicle and prescribed driving tests on public roads local to the contractor’s facilities in Blacksburg, Virginia, with either the LDW mockup vehicle or with the LDW/LKA factory vehicle. Each session will include paper-based questionnaires to gather subjective feedback regarding each participant’s experience and feelings towards the systems they are exposed to. Targeted questionnaires include paper forms administered before (first session only), during (first session only), and after the driving session. The Knowledge and Opinions Questionnaire (on paper) will provide insights into participants’ previous knowledge of and experience with the systems under evaluation. During the first session on the test track, an in-vehicle experimenter will ask questions regarding the specific conditions that participants are experiencing at that time. Finally, after each session, participants will answer a post-drive questionnaire capturing feedback regarding the session and what they experienced throughout. Each part of the study will be specifically designed to collect the data necessary to answer RQ5–RQ11. Table 1 summarizes the driving data collection activities designed to answer the research questions. Note that RQ7 will be answered separately via a survey on preferences of switching between on/off the LDW/LKA modes. Details on the procedures that will be followed for the controlled driving on the Smart Roads and the prescribed driving on public roads are provided below.

Table 1. Summary of Driving Data Collection to Answer Research Questions

Road Type	Vehicle Type	Drive Type	LDW Modality	LDW Activation Timing	LDW/LKA Interaction
Smart Roads	LDW mockup vehicle	Controlled driving	RQ5 & RQ6 (subjective)
Public Roads	LDW mockup vehicle	Prescribed driving	RQ8 (objective)	RQ10 (objective)
	LDW/LKA factory vehicle	Prescribed driving			RQ9 & RQ11 (objective)

Controlled Driving on the Smart Roads

To assess preferences regarding LDW modality and timing under dynamic scenarios, the targeted 50 study participants will experience a series of controlled driving tests with the LDW mockup vehicle on the Smart Roads test track. Each participant will drive continuously on closed loops while experiencing modality and timing conditions (independent and in combination, where applicable) incorporated in the LDW mockup vehicle, while data are collected by the DAS. After the driver has performed a few loops to become familiar with the vehicle and the test track without instructions to depart the lane, the drivers will be instructed to gradually deviate towards one of the lines until the warnings are triggered, and to carefully perform a corrective maneuver back to the center of the lane after the warning. Before, during, and following this behind-the-wheel session, drivers will provide feedback via questionnaires administered by the experimenter. By experiencing variations of LDW modality and timing, participants will be better equipped to provide acceptance and preference feedback across the experienced options.

Prescribed Driving on Public Roads

To assess driver response to naturally occurring LDW and LKA actuations, two independent driving data collection efforts will be conducted on public roads in southwest Virginia (the community surrounding the VTTI facility). The targeted 50 drivers who complete the controlled driving sessions will return to the contractor’s facilities for a second session, during which they will be assigned to one of two groups of 30 and 20 drivers and asked to drive a pre-determined (prescribed) route using one of the test vehicles, experiencing different modality, activation timing, and variation of LDW, LKA and LDW/LKA conditions while driving as they normally would.

LDW subset (targeted 30 participants)

Each of the 30 participants in the first group will drive a prescribed route using the LDW mockup vehicle. Each driving session will be part of a sub-study that aims to clarify the effects of the two independent LDW design variables (modality and activation timing) on driver performance safety indicators (e.g., frequency of lateral excursions and unintended departure events, and the magnitudes of these events). The sub-study will be conducted as a 2×3 factorial design with three LDW modalities and two LDW timing activation levels (for a total of six combinations). The LDW activation timing levels will be defined according to a market assessment and vehicle characterization previously conducted. The total driving session duration for each participant will be approximately 4 hours, split into two sub-sessions. At the halfway point, a member of the research team will switch the modality/timing combination. A remote experimenter tool will allow the experimenter to monitor the session and allow interfacing with the DAS. These sub-sessions will generate two data sets from each of the 30 participants, resulting in 60 data sets. This will allow analysis of 20 sets for each modality, 30 sets for each activation timing level, and 10 sets for each of the six combinations of factors in the factorial design (Table 2). Participants will be distributed within the cells such that paired differences are captured. This session will have a total duration of approximately 5 hours and 30 minutes for each driver, including preparation and familiarization with the vehicle, approximately 4 hours of driving with a 15-minute break at the halfway point, and 10 minutes for each repetition of the post-route questionnaire (completed twice, once after each route repetition).

Table 2. Design of Experiments for Prescribed Driving with the LDW Mockup Vehicle on Public Roads (30 Drivers, Two 2-hour Driving Sessions per Driver)

2×3 factorial design (number of data sets)		LDW activation timing
		T1	T2	Total
LDW Modality	M1	10 data sets	10 data sets	20 data sets
	M2	10 data sets	10 data sets	20 data sets
	M3	10 data sets	10 data sets	20 data sets
	Total	30 data sets	30 data sets	60 data sets

LDW/LKA subset (targeted 20 participants)

Each of the 20 participants from the second group will complete the same prescribed drive but using the LDW/LKA factory vehicle rather than the LDW mockup vehicle. This experiment will address objective driver performance and subjective qualitative preferences under four system activation modes (none, LDW only, LKA only, and LDW with LKA). As with the mockup study, the total driving session duration for each driver will be approximately 4 hours, split into two 2-hour sessions in between which the system activation modes will be modified. This will provide a total of 40 data sets, or 10 sets for each of the four activation modes studied (Table 3). Participants will be distributed within the cells such that paired differences are captured. This session will have a total duration of approximately 5 hours and 30 minutes for each driver, including preparation and familiarization with the vehicle, approximately 4 hours of driving with a 15-minute break at the halfway point, and 10 minutes for each repetition of the post-route questionnaire (completed twice, once after each route repetition).

Table 3. Design of Experiments for Prescribed Driving With the LDW/LKA Factory Vehicle on Public Roads (20 Drivers, Two 2-Hour Driving Sessions per Driver)

LDW/LKA combination	None	LDW only	LKA only	LDW & LKA	Total
Number of data sets	10 data sets	10 data sets	10 data sets	10 data sets	40 data sets

Table 4 summarizes the main research questions of interest, the targeted number of participants, the analysis variables and methods, and the data sources that will be applied by the research team to formally answer each question.

Table 4. Main research questions, population, statistical variables and methods and data sources.

Research Questions	Population	Dependent Variable	Independent Variables	Analysis Method	Data Source
RQ5: Why do drivers turn LDW alerts off? What keeps the system turned on?	All (N = 50)	Key words in driver responses and reasons	Driver Experience/ knowledge level Alert characteristics	Chi-Square analysis of key reasons versus systems Pareto chart of reasons	Experience driving/knowledge questionnaire Pre-, during-, and post-driving questionnaires for controlled driving on Smart Roads
RQ6: What alert characteristics (e.g., auditory alert, haptic alert, visual component, or a combination) available for LDW yield the highest rates of driver acceptance?	All (N = 50)	Driver survey responses (Likert scale)	Driver Experience/ knowledge level Alert characteristics	ANOVA Generalized linear model	Experience driving/knowledge questionnaire Pre-, during-, and post-driving questionnaires for controlled driving on Smart Roads
RQ7: Where the interface design plays a role keeping it on, how difficult is it to turn the system off?	All (N = 50)	Ratings of difficulty to turn off (Likert scale)	Driver Experience/ knowledge level	Pareto analysis of interface ANOVA on difficulty to turn off	Experience driving/knowledge questionnaire
RQ8: How does modality influence performance and effectiveness (behavioral outcome) of the alert?	Participants on the prescribed driving on public roads with the LDW mockup vehicle (N = 30)	Minimum lateral position during events Frequency crossing lane line Latency of response	Modality and timing of the alerts	ANOVA Generalized linear model t-test Pareto chart on comments	Vehicle data captured by the DAS Pre- and post-driving questionnaires
RQ9: Is there an additional safety benefit associated with LDW alerts when LKA is activated?	Participants on the prescribed driving on public roads with the LDW/LKA factory vehicle (N = 20)	Minimum lateral position during events Frequency crossing lane line	Which system(s) are active: LDW versus both LWD & LKA	ANOVA t-test Pareto chart on comments	Vehicle data captured by the DAS Pre- and post-driving questionnaires
RQ10: Does the timing of the alert affect driver reaction?	Participants on the prescribed driving on public roads with the LDW mockup vehicle (N = 30)	Minimum lateral position during events Frequency crossing lane line Latency of response	Modality and timing of the alerts	ANOVA Generalized linear model t-test	Vehicle data captured by the DAS Pre- and post-driving questionnaires
RQ11: If a safety system is active, does alert prior to activation influence the outcome?	Participants on the prescribed driving on public roads with the LDW/LKA factory vehicle (N = 20)	Minimum lateral position during events Frequency crossing lane line Latency of response	Which system(s) are active: LKA versus both LWD & LKA	ANOVA t-test Pareto chart on comments	Vehicle data captured by the DAS Pre- and post-driving questionnaires

B.3. Describe methods to maximize response rates.

Participation in this study is voluntary. To maximize participation and minimize recruitment, the contractor’s volunteer recruitment team will contact people via email or phone who indicated a prior interest and agreed to be contacted about future opportunities. Individuals who have participated in past research studies with LDW/LKA systems as part of the study design will be excluded. In addition, the contractor will post the opportunity on social media platforms likely to be seen by eligible participants (e.g., Craigslist, Facebook, Nextdoor, LinkedIn) and provide it to existing contacts in the surrounding communities (e.g., managers at community centers or independent senior living facilities; university professors; local businesses) who have assisted the study team in prior recruiting efforts.

The contractor will also maximize participation and minimize recruitment by compensating volunteers for their time if they qualify for study participation. Study participants who make it past the eligibility screening are paid for their time participating in the study. Participants will receive $360 based on full participation; $140 will be paid after the first session and the remaining $220 after the second session. The entirety of participation in the study is estimated at 9 hours. The $360 total compensation is based on 9 hours of participation at the rate of $40 per hour. This compensation is higher than the mean Virginia hourly rate for labor of $34.91², which equates to a total burden cost of $28,262.50. The compensation helps ensure participants will be timely and cooperative during the experiment. Additionally, the compensation rate is based on rates that the contractor has paid participants in past research studies sponsored by the U.S. Department of Transportation.

The contractor will also provide written assurances of data security and de-identification in analysis files and reporting, such that no individual will be identified in reports of the study’s findings or in data sets received by NHTSA, nor will any drivers’ data be shared with any licensing regulatory authorities.

B.4. Describe any tests of procedures or methods to be undertaken.

Development of the LDW mockup vehicle (for Session 1 and a subset of Session 2 participants) is currently underway. With an installed DAS, the ability to generate a variety of LDW-notifications already exists. However, the research team will continue to work with NHTSA toward finalizing the specific characteristics and conditions to be examined. Ideally, approaches selected will closely mimic common implementations available to the consumer market. The research team continues to monitor trends and implementation specifics, to aid these conversations. Once these test conditions are finalized and developed, the research team will conduct validation testing to ensure that the mock LDW feature is operating as intended, both from a timing and presentation standpoint. Data review and quality control will ensure that key measures of interest are being collected and are valid. Initial piloting will provide an opportunity for continued review of data output, while also verifying procedures and experimenter protocols. Upon completing these steps, formal data collection, as outlined, will begin. The research team will continuously monitor output to ensure that the developed features are working as intended and recorded output continues to collect the necessary measures of interest.

Similarly, a to-be-determined LDW/LKA factory vehicle will be instrumented for use by a subset of participants for Session 2 (prescribed driving on public roads), as outlined in section B.2. Relying on the factory LDW/LKA system, the research team will instrument this vehicle with a similar DAS but without the need for any further feature development. Similar to the steps outlined for the LDW mockup vehicle, the research team will conduct comprehensive data review and quality control steps to ensure that key measures of interest are being collected and are valid. Initial piloting will, again, provide an opportunity for continued review of data output, while also verifying procedures and experimenter protocols. Upon completing these steps, formal data collection, as outlined, will begin. The research team will continuously monitor output to ensure that the developed features are working as intended and recorded output continues to collect the necessary measures of interest.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design.

Luke Neurauter

Director

Division of Vehicle, Driver, and System Safety

Virginia Tech Transportation Institute

lneurauter@vtti.vt.edu

540-231-1522

Ellen Barnes, M.S.

Lead Statistician, Senior Research Associate

Division of Vehicle, Driver, and System Safety

Virginia Tech Transportation Institute

Ebarnes@vtti.vt.edu

1 The Abstract must include the following information: (1) whether responding to the collection is mandatory, voluntary, or required to obtain or retain a benefit; (2) a description of the entities who must respond; (3) whether the collection is reporting (indicate if a survey), recordkeeping, and/or disclosure; (4) the frequency of the collection (e.g., bi-annual, annual, monthly, weekly, as needed); (5) a description of the information that would be reported, maintained in records, or disclosed; (6) a description of who would receive the information; (7) if the information collection involves approval by an institutional review board, include a statement to that effect; (8) the purpose of the collection; and (9) if a revision, a description of the revision and the change in burden.

2 May 2024, Employment and Earnings Summary Table B, Hours and Earnings All Employees, Total Private Average Hourly Earnings, available https://www.bls.gov/web/empsit/ceseesummary.htm (accessed June 24, 2024)