drfT&EStdy(2)-Part B-Apr. 17

INFORMATION COLLECTION

SUPPORTING JUSTIFICATION - WORK SCHEDULES AND SLEEP PATTERNS OF RAILROAD TRAIN AND ENGINE SERVICE EMPLOYEES

FRA Form Numbers F 6180.127; F 6180.128

Part B: Collections of Information Employing Statistical Methods

1. DESCRIBE (INCLUDING A NUMERICAL ESTIMATE) THE POTENTIAL RESPONDENT UNIVERSE AND ANY SAMPLING OR OTHER RESPONDENT SELECTION METHODS TO BE USED. DATA ON THE NUMBER OF ENTITIES (E.G., ESTABLISHMENTS, STATE AND LOCAL GOVERNMENT UNITS, HOUSEHOLDS, OR PERSON) IN THE CORRESPONDING SAMPLE ARE TO BE PROVIDED IN TABULAR FORM FOR THE UNIVERSE AS A WHOLE AND FOR EACH OF THE STRATA IN THE PROPOSED SAMPLE. INDICATE EXPECTED RESPONSE RATES FOR THE COLLECTION AS A WHOLE. IF THE COLLECTION HAD BEEN CONDUCTED PREVIOUSLY, INCLUDE THE ACTUAL RESPONSE RATE ACHIEVED DURING THE LAST COLLECTION.

The potential respondent universe is the 80,000 railroad train and engine (T&E) service employees in the United States. The majority of U.S. T&E employees -- 47,000 -- are members of the Brotherhood of Locomotive Engineers and Trainmen (BLET). The remainder-- 33,000 -- are members of the United Transportation Union (UTU). These two groups provide a comprehensive universe of U.S. T&E employees. There is a very small number of T&E employees who work for shortline railroads and are not represented by a labor organization. As such, they could not be part of this study.

There are five primary types of T&E jobs: yard operations, road freight, local freight, passenger or commuter operations, and hostler. Because T&E employees frequently change from one type of job to another, the type of job is not in the union databases. Road freight jobs have irregular schedules that vary from week to week. The other four types of jobs have regular starting times and assigned work days.

Both labor organizations maintain a database with the names, mailing addresses, date of birth, and date joined the union for all of its members. Both the BLET and the UTU have agreed to generate a random sample of their members in accordance with the method described in item number 2 below. The database will be current as of the date that the random sample is drawn. Only actively working (i.e., non-retirees) T&E employees living in the United States will be included in the sampling frame. Full-time union officials and railroad officials who are union members will be excluded from the sampling frame. Respondents will be selected randomly from this sampling frame. The only type of stratified random sampling from the union databases that is possible would be a geographic stratification. However, this study does not seek to look at the characteristics of subpopulations defined by geography. If type of job were available in the union database, then a stratified sample on this basis would have been used.

This collection has not been conducted previously, so the response rate must be estimated from other similar efforts. As described in more detail in item number 3 below, the expected response rate is at least 42 percent. (Note: In the previously approved Foster-Miller fatigue studies, the response rate for Railroad Signalmen (OMB No. 2130-0558) was 50 percent; the response rate for the Maintenance of Way Employees (OMB No. 2130-0561) was 30 percent; the response rate for the Dispatchers (OMB No. 2130-0570) was 46 percent.)

2. DESCRIBE THE PROCEDURES FOR THE COLLECTION OF INFORMATION INCLUDING:

• STATISTICAL METHODOLOGY FOR STRATIFICATION AND SAMPLE SELECTION

• ESTIMATION PROCEDURE

• DEGREE OF ACCURACY NEEDED FOR THE PURPOSE DESCRIBED IN THE JUSTIFICATION

• UNUSUAL PROBLEMS REQUIRING SPECIALIZED SAMPLING PROCEDURES, AND

• ANY USE OF PERIODIC (LESS FREQUENT THAN ANNUAL) DATA COLLECTION CYCLES TO REDUCE BURDEN

One of the most important issues in conducting this study is determining how large a sample is necessary for the estimates obtained in the sample survey to be reliable enough to meet the objectives of the study. In general, the larger the sample, the greater the reliability of the resulting estimates, but this must be traded off against the expense of a larger sample. The first step in this process is to specify the level of reliability needed for the resulting estimates.

The purpose of this study is to obtain descriptive information about work hours, sleep, and level of alertness. Foster-Miller assumes that 95 percent confidence is adequate for this purpose. In addition, we assume our estimate should be within 15 percent of the “true” value.

The appropriate sample size, n, for estimating the mean daily sleep time can be computed from the following (Levy & Lemeshow):

where z = reliability coefficient (1.96 for 95 percent confidence level)

N = population size

V_x = unknown population variance

ε = error tolerance

This estimation for sample size applies as well to other mean values, such as work and commute time, that the study seeks to estimate.

Webb (1992) estimates the standard deviation for daily sleep for the general population is 1 hour (Webb, p. 72). Applying this estimate of standard deviation (and hence V_x, variance) to the T&E employee population (N=80,000) and using an ε = .15, there must be 340 (n) in the sample.

A simple random sample will be selected from the sampling frame described in item 1 above. The number drawn from each of the two union databases will be proportional to that organization’s share of the total number of T&E employees in the sampling frame. Based on our preliminary estimate of the number in each union, 59 percent of the random sample will come from the UTU and the remainder from the BLET. Foster-Miller will use sampling without replacement. Each candidate T&E employee will be assigned a number sequentially from 1 to the total number of candidates, C. Using an integer random number generator, numbers in the range 1 to C will be drawn until the desired number is reached. (See item number 3 for discussion of target number of names to be drawn.) In the event of a duplicate number, another will be drawn.

Foster-Miller/FRA plans to limit the analysis of this data to characterizing all T&E employees as well as subgroups of those holding jobs with regular starting times and those holding jobs that do not have regular starting times. No attempt will be made to establish subgroups based on age, years of work experience, or work or sleep time.

The budget available for this study will not allow for a larger sample size. The error level chosen is consistent with the known variance of daily sleep in the general population (Webb, 1992).

It is not possible to use individual-level administrative records to augment the study data. Individual railroads maintain records of the hours worked by their employees for payroll purposes. However, using this information is not feasible for three reasons. First, participants will not be asked for the name of their employer as the study does not intend to report results by railroad. Second, obtaining actual work hours in this manner would be extremely time consuming and far more expensive to the government than the proposed approach. Third, and perhaps most important, the purposes of this study require information on both work hours and sleep patterns for the same time period. Attempting to coordinate individually recorded data with railroad records would be prone to error.

References

Levy, P. and Lemeshow, S. (1999). Sampling of Populations: Methods and Applications. New York: John Wiley & Sons, Inc.

Webb, W. B. (1992). Sleep, the Gentle Tyrant. Bolton: Anker Publishing.

3. DESCRIBE METHODS TO MAXIMIZE RESPONSE RATES AND TO DEAL WITH ISSUES OF NON-RESPONSE. THE ACCURACY AND RELIABILITY OF INFORMATION COLLECTED MUST BE SHOWN TO BE ADEQUATE FOR INTENDED USES. FOR COLLECTIONS BASES ON SAMPLING, A SPECIAL JUSTIFICATION MUST BE PROVIDED FOR ANY COLLECTION THAT WILL NOT YIELD “RELIABLE” DATA THAT CAN BE GENERALIZED TO THE UNIVERSE STUDIED.

Since not every BLET and UTU member who is selected to participate in this study will choose to do so, over sampling is necessary. The extent of over sampling is a function of the anticipated response rate. Conservatively, the planned study can likely achieve at least a 42 percent response rate. (As mentioned previously, this estimate for the likely response rate is an average of the response rates of the already completed signalmen, maintenance of way worker, and dispatcher surveys, which had response rates of 50 percent, 30 percent, and 46 percent, respectively.). If 42 percent of the selected individuals in the random sample express interest in participating in the study, then the random sample must be 810 (810 x .42) to yield 340 participants.

Due to the nature of the railroad industry, FRA doubts that the response rate for this survey will achieve what is possible with other non-railroad populations. Based on experience with other FRA research efforts that sought participation from railroad workers, FRA researchers have found that many of these workers are suspicious of any efforts to collect data, even if the effort has the endorsement of their labor union and the researchers assure the confidentiality of the information. Moreover, the proposed survey differs significantly from most mail surveys in that this survey requires responses every day for a 14-day period. A similar survey by Pollard (1996) only achieved a 25 percent response rate. Although Pollard’s low response rate is, in part, attributable to a lack of follow-up and other methodological problems, the commitment required for participation in an activity log is also certain to lower participation.

The FRA study plan includes several specific actions designed to encourage participation in the study. These actions are based on the Total Design Method, a standard set of mail procedures designed to maximize response rate (Dillman, 1983), and include the following:

• The survey materials will be sent via first class mail with a personally addressed and dated letter printed on high quality paper. The package will contain a cover letter, daily log, background survey, instruction sheet, and a first class postage paid envelope for return of the study materials. (Copies of the survey materials accompany this justification.)

• Ten days after distribution of the materials, a follow-up reminder postcard will be sent to all recipients to encourage them to participate in the study. (text for this postcard provided as postcard1.doc)

• Two weeks after distribution of the materials, a follow-up reminder/thank you postcard will be sent to all recipients to thank those who have participated and to encourage those who have not to participate in the study. (text for this postcard provided as postcard2.doc)

• Both the background questionnaire and the daily log are designed as booklets. The questionnaire is 8.5 x 5.5 in., printed on white paper with no questions on the cover page. The cover page contains only the title “Train and Engine Service Employee Background Survey” and the participant’s i.d. number. The required OMB statement, along with the assigned OMB Control number and confidentiality assurance statement, will be placed on the inside cover page of the background questionnaire booklet and on the inside of the back cover of the daily log book. Both survey instruments are designed to be visually pleasing and easy to read.

• The survey cover letter is from either the BLET or UTU International President, depending upon the participant’s affiliation. In addition to explaining the survey procedure, this letter describes the benefits of the study to the union and its members, and encourages members to participate in the study. This letter, as well as an article that will appear in the labor organizations’ member newsletters prior to the survey, are intended to familiarize T&E employees with the effort and to legitimize the study.

• Participants will be compensated for their data collection efforts with a $75 gift certificate to a national retail establishment upon return of the survey materials. In addition, each survey packet will include a $5 bill to encourage participation.

In the event that the response rate is below 75 percent, FRA plans to conduct a non-response bias study. From the union membership databases, it is possible to determine the age of each T&E employee. The non-response bias study will involve comparing the age distribution of the survey non-respondents with the age distribution of the respondents. The mean age for each group will be compared. For a number of reasons, age is an important characteristic for assessing potential bias. First, human sleep patterns change with age. In addition, age is highly correlated with years of work experience and seniority. If we can establish that the respondent and non-respondent populations are the same in terms of age distribution, then the results from the study will be valid.

References

Dillman, D. (1983). Mail and Other Self-Administered Questionnaires. In Rossi, P, Wright, J, and Anderson, A.(eds.) Handbook of Survey Research. Orlando:Academic Press, Inc.

Pollard, J. (1996). Locomotive Engineer’s Activity Diary. (Report No. DOT/FRA/RRP-95/02). Washington, DC: Federal Railroad Administration.

4. DESCRIBE ANY TESTS FOR PROCEDURES OR METHODS TO BE UNDERTAKEN. TESTING IS ENCOURAGED AS AN EFFECTIVE MEANS OF REFINING COLLECTIONS OF INFORMATION TO MINIMIZE BURDEN AND IMPROVE UTILITY. TESTS MUST BE APPROVED IF THEY CALL FOR ANSWERS TO IDENTICAL QUESTIONS FROM 10 OR MORE RESPONDENTS. A PROPOSED TEST OR SET OF TESTS MAY BE SUBMITTED FOR APPROVAL SEPARATELY OR IN COMBINATION WITH THE MAIN COLLECTION OF INFORMATION.

A pilot survey with nine participants will be conducted to refine the data collection procedures and instruments. Because this pilot is designed solely to test the study methods and not for analysis of the data, the BLET and UTU will select the pilot survey participants. Participants will include T&E employees working jobs with regular starting times as well as those working jobs that do not have regular schedules. This pilot survey will collect one week of data. In addition to completing the Background Survey and the Daily Log, pilot participants will also complete a brief Post-Survey Form to provide feedback on the survey instruments and procedures. Similar to the plans for the full survey, participants will be compensated with a $75 gift certificate to a national retail establishment. The Train and Engine Service Employee Background Survey and Train and Engine Service Employee’s Daily Log that accompany this submission will be revised, as necessary, based on the results of the Pilot Survey. The Daily Log contains pages for 14 days of data, but the pilot survey participants will complete data for only 7 days.

5. PROVIDE THE NAME AND TELEPHONE NUMBER OF INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS OF THE DESIGN AND THE NAME OF THE AGENCY UNIT, CONTRACTOR(S), GRANTEES, OR OTHER PERSONS(S) WHO WILL ACTUALLY COLLECT AND/OR ANALYZE THE INFORMATION FOR THE AGENCY.

FRA has engaged the services of Foster-Miller, Inc., 350 Second Ave., Waltham, MA 02451, for the conduct of this study. Foster-Miller will be responsible for data collection, information coding, and analysis.

The Foster-Miller primary point of contact for this work is:

Judith Gertler

(781) 684-4270

jgertler@foster-miller.com