Mariner Survey Pre-Test
Supporting Statement Part B
Maritime Administration (MARAD)
U.S. Department of Transportation
Washington, D.C.
B.3 Methods to Maximize Response Rates and Deal with Issues of Nonresponse 5
B.4 Tests of Procedures or Methods 6
B.5 Individuals Consulted on Study Design 6-8
The respondent universe consists of two broad categories of U.S. mariners: first, all officer (licensed) and rating (unlicensed) mariners with unlimited credentials and qualifications to sail on oceangoing sealift vessels who have a current Merchant Mariner Credential (MMC), and second, all officer (licensed) mariners with unlimited credentials and qualifications to sail on oceangoing sealift vessels who have an MMC in a continuity status.1 The sampling frame containing the respondent universe is the United States Coast Guard’s Merchant Mariner Licensing and Documentation (MMLD) system, a complete listing of every mariner that has been issued an MMC. The sampling frame includes a total of 105,718 mariners; 102,881 mariners with a current MMC and 2,837 mariners with an MMC in a continuity status.
Within the group of officer (licensed) mariners, there are eight strata: four subgroups of Deck Officers, and four subgroups of Engineer Officers. Within the group of rating (unlicensed) mariners there are four strata.
For the cognitive interviews, the goal is not to select a sample that is representative of the overall population of mariners with regard to specific subpopulations, nor is the goal to obtain a specific response rate. Rather, the intent is to ensure enough variability in the respondents to detect any potential issues across the entire range of the questionnaire.
In instances where there are known or suspected differences in how subgroups may interpret survey questions and response options, the sampling approach must assure sufficient participation from each subgroup. It is likely that there will be differences in how mariners with a current Merchant Mariner Credential (MMC) and those with an MMC in continuity status experience or respond to some survey constructs. Within the group of mariners with a current MMC, responses are also expected to differ between officer (licensed) and rating (unlicensed) mariners. It should also be noted that for international endorsements, terminology is used that separates Deck Officers (Officer in Charge of a Navigational Watch) and Engineer Officers (Officer in Charge of an Engineering Watch) from more senior Officers (Management). Similarly, terminology for international endorsements for Ratings are separate for the deck and engineer unlicensed crew. These subdivisions result in six key subgroups:
Deck Officers (licensed) with a current MMC
Engineer Officers (licensed) with a current MMC
Able Seamen/Qualified Members of the Engine Department with a current MMC – Ratings (Unlicensed)
Ordinary Seamen/Wipers/Stewards with a current MMC – Ratings (Unlicensed)
Deck Officers (licensed) with an MMC in continuity status
Engineer Officers (licensed) with an MMC in continuity status
The cognitive interviewing component of the pre-test will seek to conduct six to seven interviews per subgroup, for a total of 40 interviews. Initially, twenty-five mariners will be randomly selected from each of the six subgroups of mariners. If needed, additional mariners will be sampled until six to seven interviews have been conducted with mariners in each of the six subgroups.
For the pilot test, a total of 650 mariners will be randomly selected to participate: 500 mariners who have a current MMC (350 officer (licensed) and 150 rating (unlicensed)) and 150 officer (licensed) mariners who have an MMC in a continuity status. A proportionate random sample will be drawn from each of these three major groupings. A minimum response rate of 35% is expected for mariners with a current MMC and a minimum response rate of 20% is expected for mariners with an MMC in continuity status. These estimates are derived from the 2001 Mariner Survey response rate (43%), taking into consideration the general decline in survey response rates over recent decades and aspects of the methodological design for the current effort that should boost response (e.g., offering multiple methods of contact, offering multiple modes of survey administration, conducting telephone non-response prompting). Additionally, mariners without a current MMC are likely to have lower survey buy-in than mariners who have a current MMC. However, one objective of the pilot test is to ascertain the accuracy of these estimated response rates, and the volume of pilot test cases is large enough to provide a good estimation of the response to expect from the main survey effort using the same data collection methodology. If response in the pilot is lower in general, or lower for some subgroups, revisions will be made to the data collection methodology for the main survey effort to improve the response so that the survey response goals are met for the main survey effort.
Tables B.1.1 provides the number and percentage of USCG credentialed mariners by subgroup for mariners with a current MMC and mariners with an MMC in continuity status. The table also provides the corresponding estimated sample sizes for the Mariner Survey pilot test.
Table B.1.1. Mariner Population by Subgroups and Pilot Test Sample Sizes
Mariner Subgroups |
Mariners with Current MMC |
Mariners with MMC in Continuity Status |
||||
Population Size |
% of Group |
Estimated Sample Size |
Population Size |
% of Group |
Estimated Sample Size |
|
Officer (Licensed) Mariners |
|
|
|
|
|
|
Deck Officers |
|
|
|
|
|
|
Master |
3,236 |
20% |
71 |
495 |
18% |
26 |
Chief Mate |
766 |
5% |
17 |
121 |
4% |
7 |
Second Mate |
2,495 |
15% |
53 |
278 |
10% |
15 |
Third Mate |
2,674 |
17% |
58 |
690 |
24% |
36 |
Engineer Officers |
|
|
|
|
|
|
Chief Engineer |
2,225 |
14% |
49 |
317 |
11% |
17 |
First Assistant Engineer |
910 |
6% |
20 |
117 |
4% |
6 |
Second Assistant Engineer |
1,951 |
12% |
43 |
195 |
7% |
10 |
Third Assistant Engineer |
1,792 |
11% |
39 |
624 |
22% |
33 |
Subtotal |
16,049 |
100% |
350 |
2,837 |
100% |
150 |
Rating (Unlicensed) Mariners |
|
|
|
|
|
|
Able Seaman Unlimited |
11,928 |
14% |
21 |
n/a |
n/a |
n/a |
Able Seaman Special |
5,651 |
6% |
10 |
n/a |
n/a |
n/a |
QMED |
4,262 |
5% |
7 |
n/a |
n/a |
n/a |
Ordinary Seaman/Wiper/ Steward |
64,991 |
75% |
112 |
n/a |
n/a |
n/a |
Subtotal |
86,832 |
100% |
150 |
n/a |
n/a |
n/a |
Grand Total |
102,881 |
|
500 |
2,837 |
|
150 |
This collection has not been conducted previously. A survey of mariners was last conducted in 2002, but a pre-test was not conducted.
For the cognitive interviews, a simple random sample will be drawn from each of the six mariner groupings. For the pilot test, a proportionate stratified random sample will be drawn from each of the three major groupings of mariners. Both samples will be drawn using randomizing functions in Microsoft Excel.
Given that the cognitive interviews and pilot test are testing the survey items and data collection methods, no attempt will be made to produce estimations from the responses given in the survey. However, some analysis will be performed on the pilot test data with the goal of identifying improvements to the survey instrument for the full survey effort. In particular, descriptive analysis will be conducted for all survey items. Additionally, analysis will be conducted to examine the relationships between the mariner demographics captured in Section 1 and the willingness to serve items in Section 2. This analysis will inform decisions on whether to revise demographic items or drop them from the full survey effort if they are not found to be predictive.
The Mariner Survey Pre-Test will be conducted one time. The results will be used to test the survey instrument and other study components and identify improvements. Following the pre-test, MARAD will submit a separate request for the full survey effort.
The goal of the cognitive interviews is not to achieve a certain response rate, rather it is to conduct six to seven interviews with mariners in each of the six groupings. Sampled mariners will be contacted via regular mail and email; the invitation letter will include information about the incentive for participating ($50). If there is not sufficient response to the initial letter, non-responding mariners will be contacted by phone. If needed, additional mariners will be sampled and invited to participate using the same methods until six to seven interviews are conducted with mariners in each of the six subgroups.
The pilot test is a pre-test of the main survey effort and will consist of the same data collection methods planned for the main survey. The main difference between the pilot test and main survey methodology is that the pre-test data collection period will be shorter. Several strategies will be used to maximize response rates for the pilot test component of the Mariner Survey Pre-Test. These strategies include keeping the survey as brief as possible, offering a web and mail version of the survey, and using multiple methods to contact mariners. The effectiveness of these strategies will be used to identify improvements to the full survey methodology. Details about these strategies are provided below.
The current version of the survey consists of 25 questions in two sections; depending on their particular situation, some mariners will be skipped out of some questions. The survey underwent multiple iterations, with the goal of minimizing respondent burden to the extent possible while still capturing all information required to answer the study’s research question. Section 1 obtains key information about mariners’ experience and potential availability in the event of a National Need. The information asked in Section 1 is not available from the MMLD or any other sources. Section 2 assesses mariners’ willingness to volunteer to serve in the event of a National Need, and the conditions under which they might be willing to volunteer to serve.
At different stages of data collection, mariners will be contacted via different methods, an approach that has been shown to increase response rates.2 Since a regular mail address is available for nearly all mariners in the frame, all mariners will first be mailed a survey invitation letter. The letter will introduce the survey and its purpose, emphasize the importance of participating, provide contact information in case they have any questions, and provide the survey website and the mariner’s passcode for accessing the survey. About a week-and-a-half later, a survey invitation will be sent via email to all mariners for whom an email address is available (approximately 92% of mariners).
In week four of data collection, a survey packet will be mailed to all non-responding mariners who do not have a valid email address (as determined from the email survey invitation). The packet will include a personalized cover letter (which, again, will provide the survey website and their passcode in case they would prefer to take the survey online), the survey questionnaire, and a business reply envelope. These mariners will be mailed a second survey packet approximately three weeks later if they have not yet returned the mail survey, completed the survey online, or called Strategic Research Group to opt out.
All non-responding mariners who have a valid email address will be sent a reminder email on a weekly basis until either they complete the survey, or the survey window closes ten weeks after the initial invitations are sent. There will be multiple versions of the follow-up reminder emails, so there is some variation in phrasing as the data collection phase progresses. In addition to sending email reminders to non-responders, email reminders will be sent to mariners who have started but not yet completed the survey. These reminders would convey appreciation to the mariner for having started the survey and encourage them to finish.
Approximately eight weeks after sending the initial survey invitations, nonresponse telephone prompting will begin. Response will be closely monitored for the subpopulations and nonresponse efforts will be focused on subgroups with low response, in order to determine the level of effort necessary for the full survey effort to achieve the desired response rate. The nonresponse prompting effort will also allow us to contact cases for which we have a telephone number for a mariner, but no valid email or mail address.
In summary, response rates will be maximized by using multiple methods of contact, offering multiple modes of the survey, sending multiple reminders, and using telephone non-response prompting.
Copies of the mail versions of the Cognitive Interview Invitation Letter and Pilot Test Survey Invitation Letter are provided in Appendix 1.f.
This current request is for a pre-test of the Mariner Survey and will be used to inform the full study request for clearance to be submitted at a later date.
Strategic Research Group (SRG) is the contractor providing survey design, methodology, survey administration, survey analysis, and reporting services for the Mariner Survey Pre-Test to the Maritime Administration (MARAD). The SRG team is led by:
Dr. Kristen Heimdal
Principal Investigator
Strategic Research Group
614-220-8860
kheimdal@strategicreseachgroup.com
SRG and MARAD consulted with the following staff from the Bureau of Transportation Statistics:
Patricia Hu
Director
Bureau of Transportation Statistics
U.S. Department of Transportation
(202) 366-5277
patricia.hu@dot.gov
Dr. Cha-Chi Fan
Director, Office of Data Development & Standards
Bureau of Transportation Statistics
U.S. Department of Transportation
(202) 366-0483
chachi.fan@dot.gov
The MARAD team which was consulted on the design of the information collection for the Mariner Survey Pre-Test includes:
Dr. Shashi N. Kumar
Deputy Associate Administrator and National Coordinator for Maritime Education & Training
Maritime Administration
U.S. Department of Transportation
202 366-2105
Christopher Wahler
Director, Maritime Labor and Training
Maritime Administration
U.S. Department of Transportation
(202) 366-5469
Douglas McDonald
Director, Office of Policy and Plans
Maritime Administration
U.S. Department of Transportation
(202) 366-5469
202-366-2145
Matthew Mueller
Maritime Workforce Development Officer
Maritime Administration
U.S. Department of Transportation
202-366-7173
The MARAD team input to SRG was provided through Nuns Jain, the designated MARAD Contracting Officer’s Representative for the MARAD-SRG Mariner Survey Contract. Mr. Jain retired from federal service effective December 31, 2021, and Mr. Matthew Mueller assumed coordinating responsibilities.
Documents Included in Appendix 1
1.a. Copies of Statutes and Executive Orders Requiring the Data Collection
1.b. Mariner Survey Questionnaire
1.c. Summary of Merchant Mariner Risk and Compensation Factors
1.d. Cognitive Interview Guide
1.e. Summary of Available Historical Revealed Data from Past Major Mariner Mobilizations
1.f. Cognitive Interview Invitation and Pilot Survey Invitation
1 In the survey effort, MARAD is including mariners with an MMC in continuity status. Continuity status means that the license is not active, but it can be renewed if the mariner completes refresher training required by the U.S. Coast Guard. These mariners may be open to renewing their expired credentials by completing the necessary training once they are able to meet the current medical standards. Only licensed mariners are included in this group; because the pool of unlicensed mariners with a current MMC is so large, it is unlikely that unlicensed mariners with an MMC in continuity status would ever be needed to crew sealift vessels during periods of National Need.
2 See, for example, Dillman, D. A., Smith, J. D., and Christian, L. M. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, New Jersey: John Wiley & Sons, Inc.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kaitlyn Bregman |
| File Modified | 0000-00-00 |
| File Created | 2022-09-27 |