PRA-2126NEW.CMVDriver.SS.PartB.071807.AS.USE

Part B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. SAMPLING METHODOLOGY

The FMCSA has initiated the Commercial Motor Vehicle Driver Risk Factors Study to examine a wide array of driver and situational safety factors and to determine the prevalence of these factors and increased or decreased crash and incident risks associated with them. The major analysis paradigm of the study is “frequency risk.” Project data will measure the frequency, incidence, magnitude, and/or range of each safety factor examined and then compare the baseline incidence of the factor to the frequency and incidence associated with crashes or other measures of risk. We will employ extreme groups (e.g., drivers with at-fault crashes versus drivers with no crashes) to maximize contrast between groups, and thus associations with correlating factors.

The purpose of the Driver Risk Factors Study is twofold:

• The first objective is to identify, verify, quantify, and prioritize commercial driver risk factors. Primarily, these are personal factors such as demographic characteristics, medical conditions, personality traits, and performance capabilities. Risk factors may also include work environmental conditions, such as carrier operations type, and compensation method.

• The second objective is to pilot the methods and procedures used to collect the data to determine whether these methods will be suitable for scaling up to the full study, which is about 10 times as large as the pilot. Therefore, this study includes some variation in data collection methods to gauge the extent to which these methods affect the response rates for our instrument and for individual items.

The study is divided into three phases:

• In Phase 1, we will analyze existing data (e.g., MCMIS, SafeStat; described below) to define and identify two extreme outcome groups of drivers, define and identify extreme groups of carriers, and perform a top-level analysis of available data on antecedent variables.

• In Phase 2, we will survey 600 drivers, selected based on analyses of carriers and drivers from the Phase 1 data set. We will collect additional data from the carriers employing the drivers. The survey content will augment Phase 1 by gathering information on driver characteristics that is not present in any national database (see Appendix C for Phase 2 measures).

• Phase 3 will include in-person interviews, psychological and perceptual testing, and a medical examination of a subset of 72 drivers from Phase 2. The interview and exam will provide information about the physical and psychological risk factors for drivers. (see Appendix D for Phase 3 measures).

The study’s sampling methodology will be based on analyses of existing, publicly-available data from national databases during Phase 1, supplemented by motor carriers’ list of employed drivers to obtain contact information in order to conduct Phases 2 and 3.

Phase 1: Analysis of Existing Public Data

Public data sources for Phase 1 include the FMCSA’s Motor Carrier Management Information System (MCMIS) and SafeStat. FMCSA maintains MCMIS, which is a centralized database of carrier-based information about crashes and roadside inspections of commercial motor vehicles and drivers. MCMIS also contains census information regarding each motor carrier (e.g., address, number of power units, number of drivers, cargo carried).

SafeStat is a national database containing information on commercial vehicle and driver crashes, inspections, and traffic violations. It is used to assess the safety performance of interstate motor carriers.

The MCMIS and SafeStat databases will be linked through the unique combination of driver state and license number to form a larger driver-specific database. Doing so will allow us to examine other potential available factors such as age, sex, and body mass index (BMI), in addition to violations, convictions, and past crashes. We will also relate characteristics of the current employing carrier, such as type of operation, cargo carried, and fleet size, to driver safety outcomes. In addition to individual analyses of each characteristic, we will develop an overall logistic regression model to link multiple factors together in predicting crash risk to determine those most significantly associated with increased crash likelihood.

To identify potential drivers for case and control groups for the telephone interview and paper/online questionnaire components of the study, we will separate the drivers by those with factors that identify them as higher risk in the logistic regression model (cases) who have also been involved in crashes versus those with no or very few of the high-risk factors who have not been involved in crashes (controls).

With the case and control drivers selected, we will determine the carriers with which they were last associated. Using this sample of carriers, we will examine their most recent SafeStat data to further delineate the case and control extreme groups (i.e., carriers in the case group from the initial analysis who also have poor SafeStat data will be prioritized for inclusion in the final case group).

We will analyze MCMIS census data (physical address, operation type, etc.) for the final case and control carrier groups identified from the analysis in order to determine suitability for inclusion in the study. In particular, we will analyze the carriers to determine geographical clusters suitable for study team access. Since study travel funds are limited and key study team staff are located in Boston, MA; Washington, DC; Blacksburg, VA; and Denver, CO, priority will be given to carriers in these areas who also meet other study requirements. Please note that geographic representativeness is not a critical consideration for this study since it is not intended primarily to assess the prevalence of various driver factors. Rather, it compares these factors for good and bad drivers – comparisons that are much less likely to vary geographically than are the simple prevalences of these factors. Moreover, the study design intentionally incorporates case-control matching by geographic area, which will help to reduce confounding due to possible socio-cultural factors related to geographic location.

After selection of the candidate carrier fleets, the study team will examine the characteristics of the drivers in those carriers and compare them to the analysis results for the universe of drivers found during the initial existing data analysis. The results from this will determine the respondent group of carriers (including owner-operators). Although the criteria for classifying drivers by safety outcomes (i.e., cases or controls) will be consistent across the study, different participating carriers may have different percentages of drivers in these categories. An ideal carrier for the study will be one that can contribute both “bad” and “good” drivers to the study, but this may not always be the case. Better carriers will contribute all or mostly good drivers, poor carriers will contribute all ore mostly bad drivers. But, as emphasized, the criteria for classifying individual drivers will be consistent.

Phase 2: Driver Survey

After establishment of the participating carrier sample, we will develop criteria for driver selection for participation in the Phase 2 telephone interview and paper/online questionnaire and Phase 3 in-person interview, psychological and perceptual testing and medical examination. Analysis of public driver records will allow the classification of drivers into various safety criterion groups based on outcomes such as involvement in crashes, convictions for moving violations, and out-of-service (OOS) inspection violations. Additional considerations will be whether data on drivers is available for the full assessment period and any planned matching variables such as driver age.

The case-control methodology involves comparisons of predictor factors (driver characteristics) between or among groups defined by one or more outcome criteria (in this cases at-fault crashes and/or violations). Good research design requires that criterion outcome measures be defined prior to data gathering and that these criteria be explicit and reliable. Although outcome criteria must be explicitly defined prior to data collection, the methodology allows for flexibility in the various post hoc group comparisons among various outcomes. Thus, even though the method is called “case-control,” it is really a group comparison methodology where the characteristics of different outcome groups, however defined, may be explored and compared post hoc to the data collection. Specifically, the current methodology could permit post hoc comparisons of three different driver safety outcome criteria of principal interest:

• Involvement in crashes (and, further, whether the driver is at-fault in the crash)

• Convictions for moving violations

• Involvement in out-of-service (OOS) roadside inspection violations

Given the three separable safety outcome criteria listed above, and a “yes-no” dichotomy for each, one may envision eight possible criterion combinations of crash, moving violation, and inspection violation involvement for drivers in the study. Table 2 below presents the possible combinations of involvement in crashes, moving violations (convictions), and OOS inspection violations for any given assessment period; for discussion, these outcome categories are arbitrarily numbered from 1-8.

Table 2: Classification of Drivers by Three Basic Safety Criteria

Crash Criterion: Violation Criterion:	Crash	No Crash
Moving Violation & Inspection OOS Violation:	1	5
Moving Violation but No Inspection OOS Violation:	2	6
No Moving Violation but Inspection OOS Violation:	3	7
No Moving Violation & No Inspection OOS Violation:	4	8

There can be various group comparisons based on these three safety criteria; most notably, potential “case” groups could include those with crash involvements (Groups 1, 2, 3, 4), convictions for moving violations (1, 2, 5, 6), and/or involvements in OOS inspection violations (1, 3, 5, 7). One could compare only the very “worst” drivers (Group 1) to the very “best” (Group 8), or employ any other desired groupings to compare outcome categories to predictors such as driver medical, personality, performance, or behavioral factors.

The above classification scheme is both straightforward and reliable in that drivers are classified based on public records for a standardized assessment period. However, it does not fully address FMCSA’s interest in determining factors that are predictive of at-fault crash involvement. Therefore, an additional step in the study procedure will be an effort to determine crash culpability (causal, not necessarily legal) and also classify driver subjects by this key safety criterion. The determination of crash culpability for this parallel analysis will be based on both review of carrier records and driver interviews. This second, parallel classification scheme for drivers in the study is shown in Table 3 below.

Table 3: Parallel Classification by Safety Criteria, Including Fault in Crashes

Crash Criterion: Violation Criterion:	At-Fault Crash	No At-Fault Crash
Moving Violation & Inspection OOS Violation:	1	5
Moving Violation but No Inspection OOS Violation:	2	6
No Moving Violation but Inspection OOS Violation:	3	7
No Moving Violation & No Inspection OOS Violation:	4	8

Note, however, that the determination of crash fault for the purposes of the study is problematic. Some carriers will have classified drivers’ crashes as preventable (i.e., at-fault) versus non-preventable (not at-fault). Others will have information on the crash scenario but not a causal judgment. Many carriers are likely to provide driver identification information to allow us to search public records for crash involvements, but the determination of fault may depend on the driver interview. A consistent rule applicable to this analysis will be to classify drivers conservatively. If a reasonable determination of crash culpability cannot be made, the driver’s data will be omitted from the at-fault crash analysis although still retained in the “crash-no crash” analysis.

Note also that, in both analyses, OOS inspection violations may be limited to driver-related (e.g., Hours-of-Service, logbook) or may include vehicle-related violations (e.g., brakes, tires) as well. This decision will be made in consultation with FMCSA following the initial analyses described in steps above.

In this study, we will classify drivers as described above, recruit drivers for participation in the study, and then compare results. Since there will be fewer crash-involved than non-crash-involved drivers for the 30-month assessment period, a higher proportion of them will be recruited for study participation. Beyond the targeting of crash-involved drivers, we will not differentially target violation-involved drivers in this study.

Note that regardless of how drivers are prioritized and sampled there will be multiple types of post hoc comparisons possible. For example, it will be possible to split the “case” drivers into those with crashes and violations (Groups 1-3) versus those with only crashes (Group 4). Results of various group comparisons will have both theoretical and practical implications; for example, one would expect driver medical factors to be associated with crash involvement though not necessarily violation involvement. Personality factors like aggressiveness and impulsivity might be more generally associated with all types of unsafe outcomes.

The initial classification of drivers by safety outcomes will determine which drivers to contact for participation in the Phase 2 driver survey. The driver survey will consist of a 20-minute or less telephone interview and a 30-minute or less written questionnaire. The driver telephone interviews will consist primarily of questions relating to predictive risk factors but will also include questions to validate driver classification per the safety criteria. In particular, the telephone interview will question drivers regarding crash involvement. Information will be gathered to supplement or correct that obtained from public records. Some drivers may be reclassified or omitted from the at-fault analysis based on classification information obtained in the interviews.

Response rates are expected to be around 50% (Majowicz, Edge, Flint, et al., 2004). Procedures taken to maximize response are as follows:

• Distribution of an advance letter tailored to dispel objections. Letters will use attractive graphics and non-bureaucratic language.

• Descriptions of the questionnaire will de-emphasize the government’s role in the questionnaire and emphasize University sponsorship and assurances of confidentiality. The rationale for this is that given the authoritative role of the government in imposing regulations on truck drivers, these drivers may feel more comfortable discussing their medical, psychological, and personal history with third-party researchers who do not have authority over them.

• Advance letters will be personalized and business cards with the name of the interviewer will be enclosed.

• A 1-800 number will be created for respondents to contact interviewers.

• Instruments have been designed to provide the most essential and important information relating to potential risk factors while minimizing items that may be considered offensive or intrusive.

• Appropriate incentive levels will be included for each phase of the study, including the first contact.

• The team will send thank you notes at every stage of the process.

Commercial truck drivers are an independent, hard-to-reach, and somewhat autonomous group. The approach of primarily recruiting drivers through their employing carriers is intended to maximize subject participation, as are other methods to be used in the study. But, given the nature of commercial drivers and their work schedules, response rates are admittedly likely to be somewhat lower than they would be for other groups. In regard to the impact of response bias on the study, the point made above (p. 11) regarding prevalence vs. factor comparisons applies here as well. That is, non-response bias would likely affect prevalence estimates more than it would affect comparisons between good and bad responding drivers. Nevertheless, to assess non-response bias, the study team will make post hoc comparisons of respondents to non-respondents along known parameters, including demographics and factors related to subject selection criteria. These comparisons will be made both for the case and control groups. Based on this assessment and the overall success of the pilot study, methods may be revised for the full study.

Phase 3: Driver Interview, Psychological and Perceptual Testing and Medical Examination

Of these 600 Phase 2 drivers, 12% (n=72; 36 case, 36 control) will be selected to complete the final Phase 3 information collection methods, which are the in-person interviews, psychological and perceptual testing, and medical examinations. These 72 drivers will be selected based on geographical clustering to make the collection logistically feasible based on the project funding structure. They will also be selected from the 600 Phase 2 drivers in a manner to equalize the number of drivers in the various criterion cells per Tables 1 and 2. In addition, because of the demands on the driver’s time that the medical examination and interview will entail, we anticipate some refusals, missed appointments, or other lost subjects. Nevertheless, the Phase 3 sample will be derived from Phase 2 sample and based on the same criteria; it is not expected to involve any new subjects not already included in the study.

2. STATISTICAL METHODS & SOFTWARE

Principal statistical methods for analyzing antecedent and outcome variable relationships are expected to include t-tests (to compare group means) and odds ratios. Odds ratios are used to approximate relative risks of crashes (or other safety outcomes) based on independent variables of interest. For example, if one has crash involvement data for subjects classified as having or not having sleep apnea, one can calculate the odds of crash involvement given sleep apnea versus the odds without sleep apnea. An advantage of odds ratios is that they provide an easily understandable quantitative measure of risk association. The study will also employ correlation/regression analysis, including multiple Ordinary Least Squares and logistic regression techniques for determining optimal risk (outcome) predictions based on multiple antecedent factors. In all of the analyses, statistical significance is a fundamental consideration, although stronger effects are generally required for practical significance.

Should the study design incorporate a “matching” component for some antecedent variables (e.g., driver age, gender), the analysis will require a focus on the matched pair rather than the individual subject. In this type of analysis, only the pairs in which the members differ in the risk factor under study contribute to the test statistic. Hence, we would use McNemar’s test to test for the statistical significance of the difference between the two proportions. The proper method for estimating the odds ratio for matched pairs is to treat each pair as a stratum (Fleiss, Levin & Paik, 2003). The analysis team will also use a conditional logistic regression technique with match variables as strata to produce appropriate measures of relative risk (e.g., PHREG in SAS for a 1:2 case-control ratio).

3. TESTS OF PROCEDURES AND METHODS

The procedures involved in this information collection will be tested on fewer than 10 individuals so as to refine the process and instrument(s) if necessary. Doing so will assure we are minimizing burden and maximizing the utility of the information collection.

4. CONTACT INFORMATION FOR CONTRACTOR CONDUCTING

INFORMATION COLLECTION

Project Leads for this information collection:

Albert Alvarez Linda C. Sharpe

Senior Transportation Specialist Senior Associate

Federal Motor Carrier Safety Administration Booz Allen Hamilton Inc.

400 Virginia Ave. 22 Batterymarch Street

Washington, DC 20024 Boston, MA 02109

(202) 385-2387 (617) 428-4445 v

albert.alvarez@fmcsa.dot.gov (617) 428-4410 f

sharpe_linda@bah.com

REFERENCES

Beirness, D. J. (1993). Do we really drive as we live? The role of personality factors in road crashes. Alcohol, Drugs, and Driving, 9, 129-143.

Craft, R., & Blower, D. (2004). The Large Truck Crash Causation Study. Paper presented and distributed at the November 17, 2004 FMCSA R&T Stakeholder Forum. Arlington, VA.   

Fleiss, J.L., Levin, B. & Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd Ed). New York, NY: John Wiley & Sons.

Hanowski, R. J., Blanco, M., Nakata, A., Hickman, J. S., Schaudt, W. A., Fumero, M. C., Olson, R. L., Jermeland, J., Greening, M., Holbrook, G. T., Knipling, R. R.., & Madison, P. (2005). The drowsy driver warning system field operational test, data collection final report. Contract No. DTNH22-00-C-07007, Task Order 14. Blacksburg, VA: Virginia Tech Transportation Institute.

Knipling, R.R. (2005a). Evidence and Dimensions of Commercial Driver Differential Crash Risk. Proceedings of the Driving Assessment 2005 Conference. Rockport, ME.

Knipling, R.R. (2005b). Individual Differences in Commercial Driver Fatigue Susceptibility: Evidence and Implications. Paper and presentation for the Fatigue Management in Transportation Operations International Conference, Seattle.

Knipling, R. R., Hickman, J. S., Hanowski, R. J. & Blanco, M. (2005). Phase I- preliminary analysis of data collected in the drowsy driver warning system field operational test: Task 4, Phase II research plan. Contract No. DTNH22-00-C-07007 (Task Order No. 21). Blacksburg, VA: Virginia Tech Transportation Institute.

Knipling, R.R., Boyle, L.N., Hickman, J.S., York, J. S., Daecher, C., Olsen, E.C.B., & Prailey, T. D. (2004). Synthesis Report #4: Individual Differences and the High-Risk Commercial Driver. Project Final Report, Transportation Research Board Commercial Truck and Bus Synthesis Program. ISSN 1544-6808, ISSN 0-309-08810-0, available at http://trb.org/news/blurb_browse.asp?id=11.

Lancaster, R., & Ward, R. (2002) The contribution of individual factors to driving behaviour: Implications for managing work-related road safety. Entec UK Limited, Health and Safety Executive, Research Report 020, United Kingdom.

Majowicz, S. E., Edge, V. L, Flint, J., et al. (2004). An introductory letter in advance of a telephone survey may increase response rate. Canada Communicable Disease Report, 30, 121-123.

Murray, D., Lantz, B., & Keppler, S. (2006). Predicting Truck Crash Involvement: Developing a Commercial Driver Behavior Model and Requisite Enforcement Countermeasures. Transportation Research Board Annual Meeting. Paper #06-2850.

Treat, J.R., Tumbas, N. S., McDonald, S. T., Shinar, D., Hume, R. D., Mayer, R. E., Stansifer, R. L., & Catellan, N. J. (1979). Tri-Level Study of the Causes of Traffic Crashes: Final Report Volume I: Causal Factor Tabulations and Assessments. Institute for Research in Public Safety, Indiana University, DOT HS-805 085.

Tsuang, M. T., Boor, M., & Fleming, J. A. (1985). Psychiatric aspects of traffic accidents. American Journal of Psychiatry, 142, 538-546.

9