0648-0652 Supporting Statement Part B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Marine Recreational Information Program Fishing Effort Survey

OMB Control No. 0648-0652

2. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The MRIP Fishing Effort Survey (FES) is a bi-monthly (wave), cross-sectional mail survey designed to estimate the total number of private boat and shore-based recreational, saltwater fishing trips taken by residents of coastal states. For each administration, the FES utilizes address-based samples (ABS) covering Hawaii and 16 coastal states along the Atlantic coast and Gulf of Mexico (Maine through Alabama). The sample frame is derived from the USPS Computerized Delivery Sequence File (CDS) and includes all full-time (non-seasonal), residential addresses, with the exception of PO boxes that are not flagged as the only way to get mail. Sampling is stratified both geographically and by angler license status. Within each state, sampling is stratified into coastal and non-coastal sub-state regions defined by geographic proximity to the coast. For the purpose of the FES, counties with borders that are within 25 miles of the coast are in the “coastal” stratum and all other counties are in the “non-coastal” stratum. Rhode Island, Connecticut, Delaware, and Florida are not geographically stratified due to relatively consistent rates of fishing among counties.

Within the geographic strata, addresses are matched to the National Saltwater Angler Registry (NSAR), which consists of state lists of licensed saltwater anglers. This creates two additional strata; license matched (households with one or more licensed anglers) and license unmatched (households that cannot be matched to NSAR). This stratification provides additional information to optimize sampling. Within each stratum, addresses are selected in a single stage using simple random sampling.

Table 1 provides the sample universe, annual target sample sizes, and estimated number of completed household interviews for each state. The sample sizes for each state and wave are expected to result in estimates of total fishing effort with coefficients of variation of 0.20 or less. Within each state and wave, sample are allocated using a Neyman approach, where the sample is distributed among strata in proportion to the product of the population size and the standard deviation. Standard deviations are based upon historical FES data and estimates.

Table 1. Estimated size of the sample universe, annual target sample sizes, expected response rates and estimated number of completed household surveys.

State	Estimated Number of Households1	Target FES Sample Size	Expected Response Rate (%)	Estimated Completed Interviews2
AL	1,861,977	23,815	30.7	7,321
CT	1,356,762	20,161	32.7	6,588
DE	357,937	16,128	34.1	5,492
FL	7,694,069	10,116	30.7	3,108
GA	3,740,163	33,800	26.6	9,007
HI	458,075	15,458	39.6	6,128
ME	549,870	9,055	36.9	3,338
MD	2,206,854	19,151	32.2	6,168
MA	2,605,314	30,280	32.8	9,943
MS	1,106,063	21,598	30.9	6,677
NH	529,216	12,751	34.6	4,407
NJ	3,218,798	22,128	28.5	6,307
NY	7,311,723	33,416	23.1	7,716
NC	3,955,354	20,368	33.6	6,838
RI	408,675	20,624	34.3	7,080
SC	1,910,576	19,518	34.5	6,725
VA	3,133,590	21,700	33.0	7,156
Overall	42,405,016	350,065	31.4	110,000

*The denominator for the calculation of response rates includes all addresses, including those returned by the postal service as non-deliverable. Based upon historical FES administrations, approximately 6% of addresses will be returned as non-deliverable. In 2020, the overall response rate, excluding non-deliverable addresses, was approximately 33.3%.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

2.1. Statistical Methodology and Data Collection Procedures

The FES is a self-administered mail survey. As described in detail in Question 1, sample selection is based on stratification of the target population by geography and angler license status. Data collection procedures have been extensively tested through several pilot studies (Andrews et al. 2010, 2014; Brick et al. 2012a). Each year, the survey is administered for six, two-month reference waves. The data collection period for each wave begins one week prior to the end of the wave with an initial survey mailing. The timing of the initial mailing is such that materials are received prior to the end of the reference wave. The initial mailing is delivered by regular first class mail and includes a cover letter stating the purpose of the survey, a survey questionnaire, a post-paid return envelope, and a prepaid cash incentive (as described in section A.9).

One week following the initial mailing, a thank you/reminder postcard is sent via first class mail to all sample units. Three weeks after the initial survey mailing, a follow-up mailing is delivered to all sample units that have not responded to the survey. The follow-up mailing is delivered via first class mail and includes a nonresponse conversion letter, a second questionnaire and a post-paid return envelope.

2.2. Estimation Procedures

The FES estimates fishing effort (angler trips) by residents of sampled states. An adjustment to account for non-resident fishing activity is derived from the MRIP Access-Point Angler Intercept Survey (APAIS, OMB Control No. 0648-0052).

Final FES weights are calculated in stages. In the first stage, base weights ) for each sampled address within a given stratum are calculated as the inverse of the inclusion probabilities

where π_i is the probability that unit i is included in the sample.

In the second stage, base weights are adjusted to compensate for unit nonresponse (e.g. when households fail to mail back the completed survey). The sample is partitioned into nonresponse adjustment cells, or weighting classes, by state, sub-state region (coastal or non-coastal) and license match (matched or unmatched). In addition to these stratification variables, samples are matched by address to state lists of registered boats, which is subsequently included in the formation of nonresponse adjustment cells. The base weights of the respondents in each adjustment cell ( ) are then divided by the response rate for that cell ( ) to calculate the adjusted weight ( )

where ,

is the sum of the base weights of each respondent within adjustment cell c, and

is the sum of the base weights of each nonrespondent within adjustment cell c.

In the third stage, nonresponse weights are further adjusted through a process known as raking, which adjusts weights so that the separate or marginal distributions for select variables in the sample data conform to corresponding distributions from independent data sources (Brick and Kalton 1996). For the FES, auxiliary variables are derived from the American Community Survey, Current Population Survey and National Health Interview Survey, and include households with seniors, households with children, household tenure (own/rent), households with three or more household members, and wireless-only households. Raking is an iterative procedure that sequentially adjusts weights to force sample distributions to match marginal distributions for each auxiliary variable. The weights are repeatedly adjusted until the sample marginal distributions match the auxiliary marginal distributions for all selected variables. Raked weights are calculated as

where R_s is a generalized raking adjustment in state s.

During the fourth stage, raked weights are post-stratified to account for incomplete coverage of the target population. Post-stratification is commonly used to make respondent data conform to target population totals from other sources independent from the survey (Brick and Kalton 1996). The most recent estimates of the number of residential households available from the American Community Survey are used as population control totals. Raked weights are post-stratified to household-level control totals within coastal and non-coastal strata. The resulting post-stratified weight ( ) of address in stratum is calculated as

where the adjustment factor is equal to the ratio of the control total ( , from the American Community Survey) to the estimated total based upon the sum of nonresponse adjusted weights ( ).

Estimates of fishing effort by residents of coastal states, as well as associated estimates of variance, are calculated in SAS Version 9.4 using the survey means procedure. For each state and wave, total resident effort is calculated as a weighted sum over the sample

where and are the final weight and reported number of recreational fishing trips, respectfully, for address i in stratum h.

Variance is estimated using the Taylor series linearization

Adjustments to account for fishing activity by non-resident anglers are estimated from the APAIS. For each coastal state and wave, resident effort is adjusted by the inverse of the estimated proportion of fishing trips taken by resident anglers ( ) to estimate total effort (

and

where the proportion is estimated from APAIS data as the weighted mean of an indicator variable.

and

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Previous administrations of the FES resulted in response rates ranging from 25-40%. We expect a similar response for future administrations of the survey.

The expected response rates will be achieved by using standard mail survey protocols (Dillman et al, 2008). An initial mailing will include an introductory letter stating the purpose of the survey, the survey questionnaire, a business reply envelope, and a prepaid, $2.00 cash incentive. During testing of the FES design, a $2.00 incentive was found to be optimal in terms of maximizing response and minimizing data collection costs. A thank-you/reminder postcard will be administered to all sample units one week following the initial mailing. A final mailing, including a second questionnaire, a nonresponse conversion letter, and a business reply envelope will be sent to all nonrespondents three weeks after the initial mailing.

We will minimize nonresponse bias by using a questionnaire that maximizes responses by the entire sample population, including both fishing and non-fishing households. Testing of the FES design included two versions of the survey instrument, a fishing-specific version and a more general version that included non-fishing questions. The FES will utilize the more general “Weather and Outdoor Activity Survey” instrument, which provided the most representative sample of the general population during testing.

FES testing also included a nonresponse follow-up study to assess nonresponse bias in the data collection design. Each wave, 400 nonrespondents were sampled for the follow-up study. Data collection for the nonresponse study was initiated six weeks after the final contact for the FES with the delivery of an advanced letter via regular first-class mail. Five days later, a survey packet, including a cover letter, questionnaire (the same questionnaire used in the FES), post-paid return envelope, and a $5.00 cash incentive was delivered via FedEx (USPS Priority Mail was used where FedEx is unavailable). A thank you/reminder postcard was delivered eight days after the FedEx.

The nonresponse follow-up study achieved a 40% response rate, and respondents to the nonresponse follow-up study were not significantly different from FES respondents in terms of recreational fishing activity. These findings suggest that nonresponse bias in the FES is minimal. A second nonresponse follow-up study was implemented in 2020 and found similar results – follow-up study respondents were not significantly different from FES respondents with respect to fishing activity (Andrews 2021).

Finally, the FES will use information from the sample frame to define weighting classes for nonresponse weighting adjustments (as described above). Weighting classes are defined such that response rates and fishing activity are similar within classes. Andrews et al. (2010) and Andrews (2021) describe the effect of weighting adjustments on reducing nonresponse bias.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Reporting Sensitivity Experiment: In 2009 NOAA Fisheries implemented the National Saltwater Angler Registry (NSAR) and State Exemption Program (OMB Control No. 0648-0578), which mandates the registration of anglers fishing in the Federal Exclusive Economic Zone (EEZ) or for anadromous species. The intent of NSAR (National Research Council, 2006) was to provide a comprehensive sample frame to improve the quality and efficiency of recreational fishing surveys. More recently, the National Academies of Sciences, Engineering and Medicine (2021) recommended that NOAA Fisheries evaluate the effectiveness of license or permit frames for surveying recreational anglers who target federally permitted species.

MRIP has tested the feasibility of NSAR as a survey sample frame and concluded that license exemptions (e.g. minors < 16 years of age, anglers fishing on state-licensed piers) and illegal, unlicensed fishing activity result in significant coverage gaps that limit NSAR’s utility as a stand-alone sample frame (Andrews et al. 2010, Andrews et al. 2013, Andrews et al. 2014). The FES compensates for NSAR coverage gaps by utilizing the database as auxiliary data to stratify address-based sample frames rather than as a stand-alone sample frame. This design enhances the efficiency of household sampling while maximizing coverage of the household population.

An alternative to the FES design is to estimate license coverage rates through an independent survey and apply these rates to fishing effort estimates derived from license samples. A similar design is used to adjust FES estimates to account for nonresident fishing activity, which is not covered by the FES sample frame (FES sampling is limited to coastal states). The source of the FES adjustment, the Access-Point Angler Intercept Survey (OMB Control No. 0648-0659), could also be used to estimate license coverage rates. The challenge to this approach is accurately determining the license status of intercepted anglers. We hypothesize that unlicensed anglers who have just completed a fishing trip would be reluctant to report participating in illegal fishing activity and that estimates of license coverage rates derived from intercept samples would be severely biased. In this sense, asking an angler to report his or her license status at the completion of a fishing trip would be a sensitive question, similar to the reporting of drug use, or other activities perceived to be socially undesirable.

The RSE will test this hypothesis using the FES sampling, data collection and estimation designs and adding a license question to the FES instrument. The experiment will utilize three experimental survey instruments (Appendix 2). Two instruments (License V1 and License V2) add a single license question to the existing FES instrument, but vary the location of the question relative to standard FES questions about fishing activity. In V1, the license question precedes fishing questions, and in V2 the license question follows fishing questions. A third, control instrument (License V3) will be used to estimate the prevalence of saltwater fishing licenses in the absence of fishing activity questions. The license question in this instrument is embedded within more general questions about outdoor recreation activities that will appeal to a broader audience.

The study will include the following comparisons:

1. V1,V2 vs. V3: measure the impact of the fishing questions on license reporting. We expect that reported license prevalence will be greater when fishing questions are included.

2. V1,V2 vs. FES: measure the impact of the license questions on reporting of fishing activity. We expect estimated fishing prevalence to be lower when a license question is included.

3. V1 vs. V2: measure the impact of question order on reporting of fishing activity. We expect estimated fishing prevalence to be lower when trip questions follow the license question. We also expect that license prevalence will be greater when the license question follows trip questions. We may also expect differential item nonresponse between the treatments for the license question.

The RSE will be conducted in FL during 2023. The total sample size will be 30,000 addresses, allocated equally among the three experimental instruments. Based upon historical FES response rates, we anticipate this will result in approximately 3,000 completed surveys for each treatment. This sample size will allow a minimum detectable difference (80 percent power and a significance level of 0.05) of approximately 3 percentage points for a proportion of 0.25 or less (e.g. 25 percent of households reporting saltwater fishing).

The goal of this research is to quantify the magnitude of unlicensed fishing activity and measure the sensitivity of respondents to license reporting and associated effects on trip reporting under different scenarios. By evaluating two important sources of nonsampling errors, coverage error and reporting error, the project will assess the benefits and limitations of angler license sampling

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical support was provided by the following:

Dr. J. Michael Brick, Westat, 301-294-2004

Rob Andrews, Fisheries Biologist, NOAA Fisheries Service, Office of Science and Technology, 301-427-8105 is the point-of-contact for the Agency.

John Foster, Chief, Recreational Fisheries Statistics Branch, NOAA Fisheries Service, Office of Science and Technology, 301-427-8130.

References

Andrews WR (2021). Evaluating nonresponse bias in the mrip fishing effort survey. Retrieved from https://apps-st.fisheries.noaa.gov/pims/main/public?method=DO WNLOAD_FR_DATA&record_id=2018.

Andrews, W.R., J.M. Brick, and N.A. Mathiowetz (2013). Continued development and testing of dual-frame surveys of fishing effort: testing a dual-frame, mixed-mode survey design. Available: https://www.st.nmfs.noaa.gov/mdms/public/finalReport.jsp?ReportID=831.

Andrews, W.R., J.M. Brick, and N.M. Mathiowetz (2014). Development and Testing of Recreational Fishing Effort Surveys: Testing a Mail Survey Design. Retrieved from http://www.st.nmfs.noaa.gov/Assets/recreational/pdf/2012-FES_w_review_and_comments_FINAL.pdf.

Andrews, W.R., J.M. Brick, N.M. Mathiowetz, and L. Stokes (2010). Pilot Test of a Dual Frame Two-Phase Mail Survey of Anglers in North Carolina. Retrieved from http://www.countmyfish.noaa.gov/projects/downloads/Final_Report%20NC%202009%20Dual%20Frame%20Two%20Phase%20Experiment.pdf.

Brick. J.M., W.R. Andrews, and N.M. Mathiowetz (2012a). A Comparison of Recreational Fishing Effort Survey Designs. Retrieved from https://www.st.nmfs.noaa.gov/mdms/doc/08A_Comparison_of_Fishing_Effort_Surveys_Report_FINAL.pdf.

Brick, J.M., W.R. Andrews, P.D. Brick, H. King, and N.M. Mathiowetz (2012b). Methods for Improving Response Rates in Two-Phase Mail Surveys. Survey Practice 5(4).

Dillman, D.A., J.D. Smyth, and L.M. Christian (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley and Sons.

Lohr, S. (2009). Multiple Frame Surveys. Chapter 4 in Pfeffermann, D. (Ed.) Handbook of Statistics: Sample Surveys Design, Methods and Applications (vol. 29A). Elsevier, Amsterdam.

National Academies of Sciences, Engineering, and Medicine (2021). Data and management strategies for recreational fisheries with annual catch limits. Retrieved from https://www.nationalacademies.org/our-work/dataand-management-strategies-for-recreational-fisheries-with-annual-catchlimits.

National Research Council (2006). Review of recreational fisheries survey methods. Retrieved from: https://www.nap.edu/catalog/11616/review-ofrecreational-fisheries-survey-methods.

1 Source: American Community Survey, 2019

2 Results may vary due to rounding errors associated with the Expected Response Rate.

10