SS-Part-B-Truck-and-Bus-Maintenance-Requirements-OMB_v17.5 -clean

Version 17.5

_{Department of Transportation}

Office of the Chief Information Officer

Supporting Statement:

Truck and Bus Maintenance Requirements and Their Impact on Safety

Part B. Collections of Information Employing Statistical Methods

1. DESCRIBE the POTENTIAL RESPONDENT UNIVERSE AND ANY SAMPLING SELECTION METHOD TO BE USED

Respondents to this voluntary, online information collection will be maintenance managers, safety managers, or owner-operators of truck or bus carriers. Two sample groups, described as the Recommended Practices (RP) Group and the Intervention Effects (IE) Group, will be targeted using purposive sampling. Carriers eligible for the RP Group will be filtered from the list of all active carriers available in the Motor Carrier Management Information System (MCMIS) and Safety Measurement System (SMS) databases based on the following criteria:

• Carrier “Vehicle Maintenance” Behavior Analysis and Safety Improvement Category (BASIC) percentile (from SMS) must be less than or equal to the 33rd percentile. The BASIC percentile for a carrier is based on its record of seven categories of on-road performance during the preceding 24 months of operation: unsafe driving, fatigue, driver fitness, drugs & alcohol, vehicle maintenance, load securement, and crash history. The carriers that have low BASIC percentiles for vehicle maintenance violations are among the best carriers, which is a criterion for eligibility in the RP group.

• Carrier “Crash Indicator” BASIC percentile (crash data from MCMIS) must be less than or equal to the 33rd percentile. The carriers that have low BASIC percentiles for crashes are among the safest carriers, which is a second criterion for eligibility in the RP group.

• Carrier contact information must be available in SMS (for outreach purposes).

Carriers eligible for the IE Group will be filtered based on the following criteria:

• Must have experienced Federal or State intervention¹ activity in the past 24 months.

• Carrier contact information must be available in SMS (for outreach purposes).

Carriers included in the IE Group will not be included in the RP Group.

Respondents for both groups will be sampled from the list of those carriers meeting the above eligibility criteria. In December of 2016, there were 524,058 interstate motor carriers and intrastate hazardous materials motor carriers with recent activity in MCMIS.⁽²⁾ However, the respondent universe will be much smaller, based on the selection criteria. An estimate of the number of carriers eligible for each of the groups (RP and IE), broken out by size and operation type, is provided in Table 1.

Table 1. Estimated number of carriers eligible for RP Group and IE Group (respondent universe), broken out by size and operation type.

Carrier Size (Power Units)	Truck Carriers		Bus Carriers
	RP Population†	IE Population‡	RP Population†	IE Population‡
1–6	3,126	11,224	78	102
7–20	3,349	4,521	171	62
21–100	4,100	1,740	203	26
101–500	1,855	216	52	5
501+	458	41	12	0
Subtotal	12,888	17,742	516	195
Total	30,630		711

† Source: MCMIS and SMS, 2016-2017.

‡ Source: MCMIS and SMS, 2015-2016.

The survey will be comprised of two parts – Phase I and Phase II (described in more detail in Section 2). Only those who complete Phase I may continue to Phase II after screening (also described in more detail later). For Phase I of the survey, the study contractor —Virginia Tech Transportation Institute (VTTI) plans to initially solicit 578 carriers according to the stratification given in Table 2.

Table 2. Initial sample sizes for each of the survey sampling strata.

Carrier Size (Power Units)	Truck Carriers		Bus Carriers
	RP Group	IE Group	RP Group	IE Group
Very,Very Small/Very Small 1–6*	80	40	30	30
Small 7–20	60	30	24	24
Medium 21–100	40	20	20	20
Large 101–500	40	20	20	20
Very Large 501+	40	20	0	0
Subtotal	260	130	94	94
Total	390		188

* For the purposes of this table, “very, very small” carriers (carriers with 1 PU) and “very small” carriers (carriers with 2–6 PUs) have been combined.

The stratification serves to ensure carriers represent different sizes (ranging from very, very small to very large) and operations (i.e., truck or bus). The carriers are split between operational types (i.e., truck carriers and bus carriers). Each of these carrier types is further split into two subgroups (i.e., RP Group and IE Group). Each row in the table represents a category of carrier size, which is based on the number of power units (PUs) operated by the carrier. VTTI believes that a 50% response rate is reasonable based on past experience. The assumption of a 50% response rate means that the target sample sizes will be one-half of the initial sample sizes represented in Table 2. Table 3 shows the resulting target sample sizes for the survey.

Table 3. Target sample sizes for each of the survey sampling strata.

Carrier Size (Power Units)	Truck Carriers		Bus Carriers
	RP Group	IE Group	RP Group	IE Group
Very,Very Small/Very Small 1–6*	40	20	15	15
Small 7–20	30	15	12	12
Medium 21–100	20	10	10	10
Large 101–500	20	10	10	10
Very Large 501+	20	10	0	0
Subtotal	130	65	47	47
Total	195		94

* For the purposes of this table, “very, very small” carriers (carriers with 1 PU) and “very small” carriers (carriers with 2–6 PUs) have been combined.

All the carriers in the initial sample of 578 that will be solicited to participate in Phase I of the survey will be selected from the 2016-2017 MCMIS database. The list of carriers for the initial sample will be randomly ordered. The process of solicitation will be to send an invitation to the point of contact of a carrier, starting with the first carrier on the above list. If the point of contact for the carrier is not the appropriate person to respond to questions in the survey, the e-mail will ask for the appropriate person to contact. If the point of contact for the carrier fails to respond to the e-mail solicitation, there will be follow-up with a telephone call. If the carrier expresses no interest in participating in the survey, the carrier will be dropped from further consideration for participation. The process is re-iterated until all the target sample sizes in Table 3 are reached. Not all carriers that respond to a solicitation to participate will fit into one of the two groups (i.e., RP Group or IE Group); therefore, VTTI will verify that the carrier meets the criteria for either the RP Group or IE Group by checking the carrier’s status in the MCMIS and SMS databases. This checking will be done to determine whether the carrier will be invited to proceed to Phase II of the survey. In the event that the number of carriers needed for each stratum in Table 3 is not reached based on the method of direct contact, then VTTI may resort to open outreach through advertising. The choice to participate in the survey will remain voluntary and carriers may choose not to responds to the advertisement.The study contractor will not be extrapolating survey findings to produce estimates representative of the entire universe of motor carriers, nor will it be producing estimates having an associated estimated level of accuracy. Therefore, there are no minimum sample size requirements needed per se for achieving a desired level of accuracy from this effort. Rather, VTTI will assess individual survey responses to learn about respondents’ perceptions and experiences with various industry safety procedures and protocols, as well as their perceptions concerning FMCSA’s intervention program.

Although there are no planned target levels of accuracy for the results to be tabulated from this study, VTTI has nevertheless designed the study with the target sample sizes shown, to ensure that a reasonable number of data points are collected from truck and bus carriers of different sizes. This will help the research team glean information from a broad range of entities in the motor carrier industry. In the event there is non-response in certain carrier size groups, the research team should have enough information about the non-responding carrier to track non-response and to target a suitable replacement to meet the sample size requirements.

2. DESCRIBE PROCEDURES FOR COLLECTING INFORMATION, INCLUDING STATISTICAL METHODOLOGY FOR STRATIFICATION AND SAMPLE SELECTION, ESTIMATION PROCEDURES, DEGREE OF ACCURACY NEEDED, AND LESS THAN ANNUAL PERIODIC DATA CYCLES.

   1. Information Collection Procedures

VTTI will use Qualtrics, an online survey platform, to collect the information for this study. VTTI has successfully used Qualtrics to create and administer online surveys in the past. The information collection will be administered in two phases:

• Phase I: Online Recruitment Survey . This voluntary, seven-question survey will screen carriers and verify their eligibility for Phase II participation. To be eligible for Phase II participation, carriers must fall into one of two groups:

• Recommended Practices (RP) Group: Carriers with the lowest Vehicle Maintenance and Crash Indicator BASIC percentiles (i.e., less than or equal to the 33rd percentile).

• Intervention Effects (IE) Group: Carriers that have experienced Federal or State interventions in the last 24 months due to vehicle maintenance violations.

• Phase II: Carrier Maintenance Management Survey . This voluntary, 106-question survey will include questions about demographics; maintenance practices, intervals, personnel, and facilities; and State and Federal inspections, among other things. The Phase II survey will employ branch logic; as such, carriers will be prompted to complete different sections based on their survey group (and for one section, carrier size). No participating carrier will be asked to complete all 106 questions.

1. Phase I: Online Recruitment Survey

The link for the Phase I survey will be sent via email to representatives from a sample of active motor carriers that may be eligible for either the RP Group or the IE Group, based on the pre-established filtering criteria described in Section 1. Responses to the Phase I survey will be used to confirm carrier identification, size (i.e., number of PUs), and eligibility prior to distribution of the Carrier Maintenance Management Survey (Phase II).

The Phase I survey will ask the following seven high-level questions:

1. Name and contact information (email and phone number).

2. Name of carrier and USDOT number.

3. Current position/job title at carrier.

4. Length of time carrier has been in operation.

5. Size of carrier.

6. Types of vehicles operated (e.g., bus, truck, tractor-truck, or tractor-trailer).

7. Involvement in the past 24 months with a Federal or State intervention due to maintenance violations.

The research team will compare carriers’ responses to the Phase I survey against information contained in MCMIS to verify which stratum of power units the carrier respondent fits into (e.g., large, small, and very small) prior to distributing the Carrier Maintenance Management Survey (Phase II). If the carrier indicates that it fits into a stratum different from the one indicated in MCMIS, the one indicated by the carrier will be used. The carrier will be directed to Phase II if the target sample size (given in Table 3) into which the carrier says it fits has not been filled. At this time, VTTI will also verify that a carrier responding to Phase I, which has already been pre-classified in the RP Group, does not have an intervention. Because the SMS database may not be up-to-date, there is a possibility that it may not reflect a recent intervention for a carrier. If a carrier pre-classified in the RP Group is found to have such an intervention, the carrier will be excluded from the RP Group and dropped from further consideration for participation in the survey. The time lag between the end of Phase I and beginning of Phase II of the survey should be one or two business days but no longer than a week because the verification of responses to Phase 1 is a manual process involving allocation of available staff resources.

2. Phase II: Carrier Maintenance Management Survey

The Phase II survey will collect qualitative and quantitative data relating to carrier demographics, maintenance practices, safety practices, and State and Federal inspections. If the respondent is unable to answer questions in the Phase II survey, they will be asked to refer a maintenance manager or other appropriate representative in their organization to respond to the Phase II survey. In some cases, owner-operators who drive and operate under their own authority may be asked to respond to questions based on their experiences.

Before initiating the Phase II survey, the research team will provide potential participants with an online informed consent form (ICF) to review. Tailored versions of the ICF will be provided to carriers in the RP Group and carriers in the IE Group. Both versions of the ICF outline the study objectives and methods, any possible risks, compensation, and participant rights. Participant acceptance of the ICF will be presumed based on participant submission of a completed Online Recruitment Survey. This will be clearly stated on both versions of the ICF.

A “NEXT” button will appear at the bottom of the ICF. Carriers that accept the terms of the ICF and wish to participate will click the “NEXT” button to begin the Carrier Maintenance Management Survey. If respondents do not complete the survey within a reasonable period of time, VTTI will attempt to contact them by telephone and encourage their participation.

The Carrier Maintenance Management Survey employs branch logic and includes eight sections, summarized in Table 4. Table 4 also provides details regarding which respondent groups will complete each section of the survey, along with the purpose of each section and how it is related to achieving the goals of the project.

Table 4. Carrier Maintenance Management Survey sections, respondents, and purposes.

Survey Section	Groups to Complete	Purpose of Section
Demographics (Questions 1–24)	Recommended Practices, Intervention Effects	Gather information on respondents’ experiences with commercial vehicles and the operating characteristics of their carrier.
Systematic Maintenance (Questions 25–50)	Recommended Practices, Intervention Effects	Determine common maintenance intervals, technician training practices, and maintenance facilities that support carriers’ maintenance operations.
Maintenance Personnel and Maintenance Facilities (Questions 51–68)	Recommended Practices	Gather opinions about the minimum capabilities of properly trained maintenance personnel and properly equipped facilities.
Safety Impacts (Questions 69–77)	Recommended Practices	Gather information on common and important vehicle maintenance issues that may impact safety on the roads.
State/Federal Periodic (Annual) Inspections (Questions 78–82)	Recommended Practices, Intervention Effects	Gather detailed information on the periodic inspections that apply to participating carrier fleets.
Miscellaneous (Questions 83–89)	Recommended Practices, Intervention Effects	Gather information about special operations.
Very, Very Small and Very Small Carriers (Questions 90–97)	Recommended Practices, Intervention Effects (Very, Very Small and Very Small Carriers Only)	Gain information and opinions on specific needs of very, very small and very small carriers.
Intervention Effects (Questions 98–106)	Intervention Effects	Gather detailed information on carriers’ experiences with State or Federal interventions.

This information collection is exploratory in nature. The study has three objectives:

• Study Objective 1: Develop an operational definition of “systematic maintenance.” Objective 1 is an overarching objective that can refer to types of specific standards for vehicle maintenance.

• To address this objective, the research team will explore trends in qualitative responses to how “systematic maintenance” should be defined. For example, does it include periodic inspections and maintenance intervals?

• Study Objective 2: Evaluate whether current regulations and the intervention process could be modified to improve compliance with vehicle maintenance requirements. [ The results of the survey will be used only to explore what areas of rulemaking and other areas, such as policy guidance and training, might be needed, but the results will not be used for rulemaking, per se.] Examples of such requirements are as follows: (i) preventative maintenance intervals, (ii) preventative maintenance inspections with adequately trained/equipped mechanics, and (iii) motor carriers’ maintenance facilities. Objective 2 supports, enhances, or enables implementation of Objective 1 by specifying particular standards (e.g., inspection and maintenance intervals for different types of safety-critical equipment).

• To address this objective, the research team will explore trends in qualitative responses to what preventative maintenance intervals, inspections, and maintenance technician training are considered adequate by motor carriers.

• Study Objective 3: Gather information to assist in establishing minimum standards for inspection intervals, mechanic qualifications and training, and certification of maintenance facilities.

• To address this objective, the research team will explore trends in qualitative responses to questions about minimum standards for inspection intervals, maintenance technician qualifications, and maintenance facilities.

As shown in Table 4, all participating carriers will be asked to provide information on demographics, systematic maintenance, State/Federal periodic (annual) inspections, and special operations. Very, very small and very small carriers in both the IE and RP groups will be asked to answer questions that pertain only to carriers of those sizes.

Carriers (of all sizes) in the RP Group will be asked to provide additional information about maintenance personnel and facilities (e.g., mechanic training levels, tools required for adequate inspection, and certification of facilities) and vehicle maintenance issues that may impact safety. Information from the RP Group will seek to address Objective 1, relating to development of an operational definition of “systematic maintenance,” Objective 2, and Objective 3, relating to establishment of minimum standards for inspection intervals, mechanic qualifications and training, and certification of maintenance facilities.

Carriers in the IE Group will be asked to complete the section on intervention effects, which includes questions about the status of active interventions or investigations, results of closed interventions or investigations, interactions with State versus Federal agencies, intervention activities experienced, the accuracy of violations leading to interventions, actions taken in response to interventions, changes in carrier vehicle maintenance practices as a result of an intervention, significant benefits of interventions, and ways the intervention process could be improved. Information provided by the IE Group will address the portion of Objective 2 regarding where interventions could be modified to improve compliance with vehicle maintenance requirements.

The survey will be used by researchers to better understand practices and opinions of RP carriers and IE carriers. The survey will include checkbox, multiple choice, and open-ended questions, which will cover a diverse range of topics within the preventative maintenance field. The survey itself and the study sample size were designed to collect a substantial amount of carrier input on a topic that has not been previously researched.

Survey responses will be summarized and reported using plots, tables, content analysis, and calculated summary statistics. Plots and tables will provide a visual comparison of multiple choice and checkbox survey responses for successful carriers (i.e., carriers in the RP Group) and those receiving interventions in the last 24 months (i.e., carriers in the IE Group). These methods will also allow researchers to summarize responses by carrier operation type (i.e., truck or bus) and size. Bar charts will be used to plot responses to many survey questions. For example, the survey question “What types of operations does your carrier perform?” has five multiple-select/checkbox response options. A bar chart for each carrier-size subgroup will plot the percentage of RP carriers and IE carriers reporting each of the five response options. Another survey question that will be plotted in a bar chart is “Does your carrier maintain pre- and post-trip driver reports or driver-vehicle inspection reports in the following formats?” with three multiple-select/checkbox response options. Responses for both survey groups (RP and IE) can be visually compared by truck or bus operation type and carrier size. This will aid in the identification of patterns in responses for carriers in the RP Group.

Some survey responses may be summarized with tables. Tables will include rows for each of the carrier operation types (truck or bus) and each carrier-size subgroup. Columns will list the question response options. Each cell will display the number and percent of subgroups reporting each response option. Tables will be used to display responses to survey questions such as “Does your carrier provide simple maintenance repair training to drivers?” and “Does your carrier provide laptops, tablets, or handheld devices for technicians to use?”

To explore and summarize responses to open-ended survey questions, researchers will use content analysis methods. Examples of open-ended questions in the survey include “List examples of critical safety-related maintenance activities for trailer vehicle milestones,” and “Please list examples of simple maintenance repair training provided to drivers.” Other questions include “In the last two years, what have been the three most common maintenance violations?” and “Think about the most serious safety-related commercial vehicle component failure you have ever experienced. Briefly describe the vehicle problem.” The survey includes several more free-response questions that will be summarized using the same methodology. The goal will be to identify common answers among carriers in the RP Group.

Summary statistics will include calculations such as means, medians, and ranges. Many of these questions are more demographic in nature, giving us an understanding of the carriers’ features. One example of a survey question that will be summarized using these values asks, “How many total years have you been working in the commercial vehicle industry?” For this question, researchers will report the mean and median values, as well as the minimum and maximum value. These values will be calculated for each of the sample subgroups (truck or bus, then by carrier size) and reported in a table. Two other survey questions that will be summarized using these calculations include, “How many years did your primary role involve driving, maintenance/tech, managing?” (with an answer option for each of these roles) and “How many years and months have you been working at or operating with your current carrier?” Summary statistics will also be calculated for questions on compensation, including:

• What is the average income of your technicians?

• What is the average income of your maintenance supervisors?

• What is the average hourly rate for technician support you use elsewhere?

Possible statistical analysis options include analysis of variance for open-ended questions with continuous values (such as those asking about technician compensation) and Chi-squared tests for multiple-choice and multiple-select/checkbox survey questions. For multiple-select survey questions, responses will be re-calculated as the number of carriers in each subgroup reporting a single answer compared to the number of carriers not reporting the same single answer.

Although there are explicit range restrictions among the RP Group, the nature of the analyses and the objectives of the project will not require solicitation of carriers outside of the study criteria. The research team presumes that survey responses from smaller carriers will be more diverse. To better capture the variability in responses in these carrier size sets, the sampling plan is skewed to more heavily sample these smaller carriers. Carrier size recruitment will occur simply by enrolling different carrier managers until the target number of carrier managers per carrier size category has been reached for the participants in the RP Group and the IE Group.

3. DESCRIBE METHODS TO MAXIMIZE RESPONSES RATE AND TO DEAL WITH THE ISSUES OF NON-RESPONSE.

Participation in the information collection is voluntary. The Phase I and Phase II surveys will be administered online; as such, they can be completed at a time that is convenient for the carrier representative (i.e., the survey does not need to be completed during work hours). Participants who complete the Carrier Maintenance Management Survey (Phase II) will be offered $50 as compensation for their time. Respondents who opt in will be transferred to the Incentive Payment Page, a secure online form independent of their Carrier Maintenance Management Survey responses. Respondents will be asked to enter their name, mailing address (for receipt of the compensation check), and their phone number. VTTI will use this information for payment tracking and mailing purposes.

All potential participants will initially communicate their interest in participating in the study and data collection by completing the Online Recruitment Survey (Phase I), which is described in detail in Section 2.1.1. Since the carriers who will be targeted for the longer Phase II survey will have already communicated their interest in participating in the Phase I recruitment survey, we expect the response rate to be fairly high (i.e., non-responses should be low). In the event there is significant non-response, the research team will have enough information about the non-responding carrier to track non-response and to target a suitable replacement in order to keep the stratified sampling procedure in place.

4. DESCRIBE TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN.

The content of the Carrier Maintenance Management Survey was informed by two focus groups conducted in an earlier task for this project. One focus group comprised representatives from the commercial truck freight transportation industry and the other comprised representatives of the commercial bus passenger transportation industry. VTTI identified and recruited representatives considered to be “best performers” (i.e., carriers with long records of safety programs that go above and beyond the minimum requirements for accountability) to participate in the focus groups. These best performers were identified with the help of organizations including, but not limited to, the American Trucking Associations (ATA), the American Bus Association (ABA), the Commercial Vehicle Safety Alliance (CVSA), and a commercial vehicle maintenance tracking technology vendor. These organizations included industry representatives that regularly experience practical maintenance challenges. Participants in the focus groups also came from carriers of varying sizes, to best represent the industry. Representatives from these stakeholder organizations were aware of opportunities to bring the regulations, best practices, and typical carrier operations into alignment.

Additionally, prior to and during the development process for this ICR, VTTI led presentations and discussions at the following relevant industry meetings:

• Transportation Research Board (January, 2017; Trucking Industry Research Committee, Truck and Bus Data Subcommittee, Motorcoach Safety Subcommittee).

• ABA Marketplace (January, 2017; Bus Maintenance and Repair Council [BusMARC]).

• ATA Technology Maintenance Council (February, 2017).

• Mid-America Truck Show, Seminar (March, 2017).

• ATA Technical Advisory Group Meeting (June, 2017).

VTTI believes that the two focus groups and industry interactions described above increase the chances of succeeding in this information collection effort.

5. PROVIDE NAME AND TELEPHONE NUMBEER OF INDIVIDUALS WHO WERE CONSULTED ON STATICTICAL ASPECTS OF THE INFORMATION COLLECTION AND WHO WILL ACTUALLY COLLECT AND/OR ANALYZE THE INFORMATION

Project leads for this information collection request:

Quon Y. Kwan Program Manager, Technology Division Federal Motor Carrier Safety Administration 1400 New Jersey Ave., SE Washington, DC 20590 (202) 385-2389 quon.kwan@dot.gov	Andrew Krum Principal Investigator Virginia Tech Transportation Institute 3500 Transportation Research Plaza Blacksburg, VA 24061 (540) 231-0353 akrum@vtti.vt.edu
Rich Hanowski, Ph.D Principal Investigator Virginia Tech Transportation Institute 3500 Transportation Research Plaza Blacksburg, VA 24061 (540) 231-1513 rhanowski@vtti.vt.edu	Feng Guo, Ph.D. Statistician Virginia Tech Transportation Institute 3500 Transportation Research Plaza Blacksburg, VA 24061 (540) 231-1038 fguo@vtti.vt.edu

1 An intervention is an action against a carrier taken by FMCSA or a state commercial vehicle enforcement agency in the form of a warning letter, on-site or off-site investigation, nonrated review, or other follow-on enforcement action. For the purpose of this study, targeted roadside inspections, cargo tank facility reviews, shipper reviews, terminal investigations, and security contact reviews are not considered interventions.

2 FMCSA, MCMIS, data snapshot as of December 30, 2016.

6