0591 Supporting Statement 100815 Part B

SUPPORTING STATEMENT

ANNUAL ECONOMIC SURVEY OF FEDERAL GULF AND SOUTH ATLANTIC SHRIMP PERMIT HOLDERS

OMB CONTROL NO. 0648-0591

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The population of interest is all vessels fishing for penaeid and rock shrimp in the federal waters of the Gulf of Mexico and South Atlantic, i.e. off the States of North Carolina, South Carolina, Georgia, Florida, Alabama, Mississippi, Louisiana, and Texas, during one calendar year. An excellent sampling frame is available for this and future survey efforts, because vessels shrimping in the federal waters of the Gulf and South Atlantic are required to have a federal permit. Their contact information should be up-to-date due to the annual permit renewal process. The sampling frame consist of all fishermen holding at least one of four federal shrimp permits at any time during the previous calendar year (including individuals whose permits might have expired but are still legally renewable, i.e. “latent permit holder”). Note that the survey effort conducted each year (e.g., 2016), will be collecting the previous year’s annual economic data ( e.g., 2015).

Roughly, we aim to randomly sample without replacement about a third of the whole population each year, covering the population once every three years. As of January 2015 (see also Table 1), the total population was 1,784 unique vessels with one or more federal shrimp permits. For the 2014 survey, we sampled 643 vessels. Due to the management and political importance attributed to delineation by state, we stratify the total population by state. Within each stratum we randomly sample vessels in proportion to each stratum’s weight in the total population. By sticking to a simple, straightforward design, we hope to avoid many potential problems.

Currently, the best estimate of the size of the sampling frames for 2015 through 2017 would be 1,784 vessels. Table 1 below breaks down the 2014 sampling frame into the strata, lists the permits held, offers some descriptive data for the vessels in each, and provides the number of respondents sampled and surveys returned (preliminary) in the most recent survey effort. Of the total sampling frame, the majority of vessels hold only a Gulf shrimp permit and thus represent the dominant group. There is significant variation within the industry across several variables, but none seems to further divide the population into discrete groups (offering no advantage of further stratification). These numbers are unlikely to change dramatically in the coming years. The actual number of permit holders in the fishery might change a little due to new entrants (the South Atlantic penaeid permit and rock shrimp permit (Carolinas zone) are open access permits), owners and vessels leaving the fishery (permits non-renewed or terminated), or changes in vessel ownership or state of registration. The final sampling frame will use all the information available just prior to the survey implementation.

The raw response rates for the Annual Economic Survey of Federal Gulf Shrimp Permit Holders have ranged from 78% to 92% between 2007 and 2012. After adjusting for vessels that were deemed ineligible because their permits were sold or terminated, the adjusted response rate has ranged from 86% to 95%. For South Atlantic permitted-vessels, 2009-2011, the annual raw and adjusted response rates ranged between 79%-80% and 85%-90%, respectively.

The new crew survey component: For each permitted vessel, NMFS already collects the average number of crew members, as well as if the vessel is owner operated. Based on these numbers, we estimate that the total number of crew working on the active federally permitted shrimp fleet in the Southeast (at any one point in time) is somewhere around 5000 individuals. By contacting crew respondents through the vessel owner, we maintain the statistical method/properties of the primary survey. The crew survey can be conceived as a cluster sampling approach (vessel as PSU), with a ‘take-all’ second stage (as we will ask owners to hand out one survey to each crew on their vessel). We do not intend to send the crew survey to all selected vessel owner respondents. Instead, we will randomly subsample among cost-survey-selected vessel owners to have their crew participate in the crew survey. On (annual) average, we will sample the number of vessels needed to contact (in principle) approximately 1200 crew---hoping for 500-1000 annual crew responses. In fact, we will slowly ramp up the effort from year 1 to year 3, with the first year effectively functioning as a “pretest” of the survey protocols.¹ We are attempting to make the process and survey as simple, straightforward, and uncontroversial as possible to encourage response. The actual response rate is not known at this time; but faces two obstacles: a) owners will need to hand out the survey to crew members, and b) crew members must respond.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For sampling, we will stratify the population by state as this is a policy relevant variable. We will then randomly sample in each strata proportional to each strata’s weight in the population. Each year, we will sample approximately a third of the population (see also Table 1). The very tractable proportional random sampling approach should require only simple adjustments to the inclusion probabilities used for the estimation of population means and other aggregate statistics if non-response is significant and skewed across the strata.

The owner of each vessel selected will be contacted by mail in late February/early March of each year, first by a selection letter, followed by the survey package. The package will contain a cover letter, information material, instructions, the two-page survey instrument and a return envelope. They will be asked to return the completed survey instrument to us in the enclosed, pre-paid envelope. If no response is received by April 30, up to two further letters will be sent (including additional survey instruments). We will also attempt to contact the non-responders by phone and urge them to return the survey. Information will not be collected during the phone call, and a further survey instrument will be sent – by mail, fax, or email – if requested.

After data entry, verification and cleaning, descriptive statistical analysis will be conducted on the relevant variables collected (costs and profits). Results will be reported for different definitions of the fleet (all permitted, Gulf shrimp vessels, active, inactive, etc.) and by state. The accuracy for the population level totals and means of the important variables should exceed the standard +/- 10% confidence interval at a 95% significance level for the larger groups. Given the overall uncertainty inherent to policy assessments of economic conditions in fisheries and given the quality and accuracy of other data used, the standard accuracy should suffice.

The use of periodic instead of annual collection will be considered in the future. The burden on the public will depend on how frequently significant changes occur in this industry. Currently, the fishery is still undergoing substantial changes making the annual collection of data necessary.

With the last OMB PRA renewal, we were asked to evaluate if our results---estimates of population means---should be weighted/post-stratified to improve accuracy. We enlisted the help of some local statisticians (and did a bunch of reading) to explore and understand this issue. We have concluded that weighting/post-stratification should be conducted going forward. The weights should correct for variation in state-strata non-response and for Gulf shrimp activity status (known only after survey is conducted).

To look at the potential bias introduced in our already published results (by not weighing), we constructed weights for the 2011 data and calculated weighted averages in SAS for the two most important (sub) domains in our data collection: All Gulf shrimp (SPGM) federally-permitted vessels (Table 3) and only active Gulf shrimp vessels (with SPGM permit) (Table 4). Not entirely unexpected, given high response rates and little bias, the population estimates calculated by arithmetic mean vs. weighted mean do not differ materially in absolute or relative terms for almost all variables. The only (calculated) variable for which we find a material difference between weighted vs. non-weighted estimates is the net revenue from operations (for the active Gulf fleet only). It should be noted that this estimate (and the economic return calculation based on it) are the only two variables that are not significantly different from zero (t=0.41 and 0.28). In light of these results, we do not intend to retrospectively introduce weighting into our published reports.

We agree that---all else being equal---reporting weighted means is preferable and intend to take this approach going forward. We are in the process of rebuilding our data processing, cleaning, analysis, and report-generation approach. Originally, we used a variety of software, including Oracle, SAS, Excel, and Word, with Excel featuring prominently for pivot tables (e.g., arithmetic means), charts, and formatting. Introducing weighting into our current approach is notoriously labor intense. We are currently coding a new approach centered on the statistical program R and knitr to (almost) auto-generate our reports as PDFs. Dealing with weighting in R is trivial. We expect full implementation of proper weighting with our next set of reports. The new approach will also consolidate data/code across time, allowing for better time-series analysis (a significant weakness of the current reports).

3. Describe the methods used to maximize response rates and to deal with non-response. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The central approach to maximizing the response rate is to make answering a very concise and simple survey a requirement for future permit renewal. The first cover letter will politely emphasize this point. The second and third reminder letters will be more explicit. The telephone call will also explain the consequences of not complying. The call has the further advantage of being a different mode of contact and should discover non-response due to an incorrect address. Given the potential loss of permit, we expect compliance from all fishermen wanting to continue to fish for shrimp in federal waters. The behavior by those who have left the fishery by the time of the survey, or are planning to leave it before their current permit expires, will not be influenced by the implicit threat. Since the data will be used primarily for assessments and predictions about future developments, under-reporting by individuals leaving the fishery is less problematic.

A good sampling frame, with annually updated contact information (through the ongoing permit renewal), will help to reduce the non-contact component of non-response. If necessary due to low response, at the conclusion of the survey, we will contact port agents (local federal employees who collect data and report from a limited area) and ask them for any information on non-responding vessels/individuals. Should non-response be a significant factor, we might even ask port agents to inquire themselves, and/or we will debrief a few (<10) individuals about reasons for not responding in order to establish potential non-response biases.

Beyond the above, we will take every action available to us to facilitate completing and returning the survey by the fishermen. General survey design techniques (Dillman method) and experience from the previous surveys will guide us. Noteworthy actions include:

• Timing of the survey during the slow shrimp fishing season (winter and spring) and coinciding with tax time, when business records are being consulted and financial concerns are “top of mind.”

• Disseminating together with the survey effort-specific outreach material.

• Using plain language and translating the survey into “language” spoken by southeast shrimp fishermen (including a Vietnamese version).

The statistical design and size of this sample survey will allow for valid generalizations of the results to the population and larger subpopulation levels. The anticipated accuracy of the results is discussed in more detail in the previous question (Part B, Question 2).

For the crew survey, as explained below, there is currently no sampling frame. This first attempt will in part be to see how feasible it would be to continue such a survey.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The crew survey component described above is a novel element to this data collection. Given the lack of a crew sampling frame, contacting crew through vessel owners---if successful (e.g., reasonably high response rate---would be a very cost effective way of collecting some data on these important stakeholders in the federal SE shrimp fisheries). In the absence of a frame, the only other approach would be dockside intercept sampling which is cost prohibitive and has its own statistical problems (shrimp trips are often 3+ weeks long). A draft of the crew survey questions was distributed to all SE NMFS/Council social scientists for comment. We received and integrated comments from 10 individuals.

Sometime during this 3 year cycle, we would like to once again ask a question about if respondents would prefer to fill-out the survey online. Roughly 10 years have passed since we last asked a question of this nature, and attitudes might have changed. If we received a large affirmative answer, we might go to a multi-method approach (with the next OMB/PRA renewal process). While an online system would require resources to construct, it would also save resources (less data entry; possibly less data entry errors and verification; possible email instead of phone contact and follow-up).

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Individual consulted on the statistical aspects of the design:

Elizabeth Overstreet, MSc in Applied Statistics

National Marine Fisheries Service

Southeast Fisheries Science Center

Social Science Research Group

(305) 361-4496

Kaming Lo, M.P.H.

Biostatistics Collaboration and Consulting Core (BCCC)

Department of Public Health Sciences

University of Miami, Miller School of Medicine

(305) 243-2506

Persons who will actually collect and analyze the information:

Christopher Liese, Ph.D.

National Marine Fisheries Service

Southeast Fisheries Science Center

Social Science Research Group

(305) 365-4109

Table 1: 2014 Sampling Frame, Sampling Strata, Population Characteristics, Sample Size, Response, and Response Rate
	Population	Permits (% of Population by Permit Type)					Vessel Characteristics						Sample	Response5	Response Rate5
		SPA1	RSCZ2	RSLA3	SPGM4		Length	HP	Year	Steel Hull	Freezer
NC	129	94%	29%	16%	29%		63	495	1985	43%	20%		49	36	73%
SC	51	98%	16%	2%	8%		54	373	1977	8%	6%		26	18	69%
GA	88	99%	6%	6%	14%		58	450	1977	15%	18%		35	21	60%
FL	291	56%	5%	10%	74%		54	418	1982	24%	42%		114	88	77%
AL	103	36%	2%	30%	99%		66	559	1992	73%	55%		36	34	94%
MS	104	18%	5%	5%	99%		72	655	1989	83%	56%		39	35	90%
LA	411	1%	1%	0%	100%		64	506	1990	78%	31%		143	116	81%
TX	546	6%	2%	1%	100%		72	597	1987	88%	83%		173	153	88%
Other	61	48%	43%	10%	69%		71	620	1990	80%	39%		28	19	68%
Total	1784	31%	6%	6%	82%		65	528	1986	65%	50%		643	520	81%
Permit Type Count:		546	115	102	1469
Sample by Permit:		217	34	58	509

¹ SPA: South Atlantic penaeid shrimp permit (open access).

² RSCZ: South Atlantic rock shrimp permit – Carolina Zone (open access).

³ RSLA: South Atlantic rock shrimp permit – Limited Access (limited access).

⁴ SPGM: Gulf of Mexico shrimp permit (limited access).

⁵ 2014 response and response rate are preliminary as we are just concluding the data collection for the year (Sept 2015).

Table 2: Example of the Presentation of Results (2011)

Table 3: Comparison of 2011 results with and without post-stratification (SPGM fleet)

Table 4: Comparison of 2011 results with and without post-stratification (active Gulf)

1 We realize that if major changes to the protocols---or any different questions---are needed, we will need to submit a change request to OMB/PRA.

5