MLSR_SS_Part_B_081513

SUPPORTING STATEMENT

MARINE RECREATIONAL INFORMATION PROGRAM LONGITUDINAL SURVEY OF RECREATIONAL FISHING PARTICIPATION

OMB CONTROL NO. 0648-XXXX

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

1.1. MRIP Longitudinal Survey of Recreational Fishing Participation

The MRIP Longitudinal Survey of Recreational Fishing Participation (MLSR) is a fixed panel mail survey designed to produce a reliable estimate of the number of adults who participated in marine recreational fishing during the last year in the study state (annual participation). The sampling design is very similar to the MRIP Fishing Effort Survey (MFES, OMB Control #0648-0652) and the results will be used to assess recall error in annual participation estimates derived from MFES data. The comparison of the annual participation estimates from MLSR and MFES is just being done in North Carolina which is the only state where both surveys are being conducted. The sample selection methodology is identical in the two surveys and the mode of contact is the same; the difference between the MLSR and MFES is due to the differences between questions in the surveys (and the recall associated with them). The questionnaires are not identical (for example, the MLSR asks for the first name of the adults and the MFES does not). These differences are a potential source of error that is not recall error. While it is not recall error, it is part of the measurement error. The MLSR estimates an annual participation for May, 2013 to April, 2014. The MFES is conducted every two months so the 2013 Wave 4, which is scheduled to be mailed at the same time as the MLSR, and subsequent waves all can be used for comparison. We expect there will be some differences depending on the wave, with one hypothesis being that waves closer to the peak fishing season will have less measurement error since less recall is required for most participants.  We plan to examine estimates for all the waves for the same year as the MLSR. Combining of MFES data from multiple months is possible, but not currently anticipated. 

The MLSR consists of two independent components, resident angler sampling and non-resident angler sampling. Resident anglers are sampled from the CDS and non-resident anglers are sampled from a database licensed saltwater anglers maintained by the North Carolina Division of Marine Fisheries.

1.2. Resident Angler Sampling

Addresses within North Carolina are sampled form the CDS. Sampling is stratified by geographic proximity to the coast. Specifically, counties with any border that is within 25 miles of the coast are in the coastal stratum, and all other counties are in the non-coastal stratum. This stratification serves two purposes. First, residents of coastal counties are more likely to participate in recreational saltwater fishing than residents of non-coastal counties – historical estimates from the Marine Recreational Fisheries Statistics Survey (MRFSS) demonstrate that 65-90% of recreational saltwater fishing trips in the study state are taken by residents of coastal counties within those state. Stratification also provides an opportunity to sample at different rates among strata and subsequently increase the efficiency of data collection.

A simple random sample of addresses is selected within each stratum in a single stage. The sample size is sufficient to permit sub-sampling, as described below. Addresses are selected from a comprehensive list of residential addresses maintained by a vendor licensed to distribute the CDS. Following selection, sampled addresses in each stratum are matched, by address and telephone number, to databases of anglers licensed to participate in saltwater fishing in the respective state. Databases of licensed anglers are provided to NMFS by state natural resource agencies approximately one month prior to the beginning of data collection. Prior to matching, addresses within the license databases are formatted to conform to USPS postal addressing standards, and duplicate angler records.

Matching addresses to license databases screens the ABS sample to identify households with (matched) and without (unmatched) licensed anglers, effectively stratifying the sample into matched and unmatched strata (Lohr, 2009). Stratification provides an opportunity to optimize sampling among strata - previous studies (Andrews et al., 2010, Brick et al., 2012) have demonstrated that residents of households that match to license databases respond to fishing surveys at a higher rate and are more likely to have fished than residents of unmatched households. The survey instrument collects information about the recent saltwater fishing activity for all residents of each sampled address (i.e. each address is a cluster of individuals who reside at the address).

Table 1 provides the sample universe, initial ABS sample sizes, final target sample sizes and estimated number of completed household interviews for each stratum. The final target allocation is achieved by retaining all matched addresses in the sample and sub-sampling unmatched addresses. Target sample sizes are expected to result in a completed number of household surveys that achieve a half-width of the 95% confidence interval of 2 percentage points on estimates of total participation. Sampling requirements are based upon results from previous MRIP pilot studies.

1.3. Nonresident Angler Sampling

Non-resident anglers are sampled from lists of individuals who are licensed to participate in saltwater fishing in the study state. The sample frame for the state consists of anglers who were licensed to fish in North Carolina during the reference wave but reside in another state. Databases of licensed anglers are provided to NMFS by state natural resource agencies approximately one month prior to the beginning of data collection. Prior to sampling, addresses within the license databases are formatted to conform to USPS postal addressing standards, and duplicate angler records, as well as records for individuals less than18 years of age are identified

and removed.

Table 1. Estimated size of the sample universe, initial and final sample sizes, expected response rates and estimated number of completed household interviews for the resident angler sampling.

Wave A (May-August, 2013)
State	Geographic Stratum	License Stratum	Estimated Number of Households1	Initial ABS Sample Size2	Estimated Final ABS Sample Size34	Expected Response Rates5	Household Fishing Status6	Wave A Estimated Completed interviews7
NC	Coastal	Matched	199,839	3,000	3,000	57%	current/likely	1,000
							unlikely	539
	Coastal	Unmatched	557,660	8,372	5,280	43%	current/likely	1,329
							unlikely	715
	Noncoastal	Matched	222,650	480	480	53%	current/likely	149
							unlikely	80
	Noncoastal	Unmatched	2,812,924	6,064	2,520	41%	current/likely	605
							unlikely	326
Total			3,793,073	17,916	11,280	48%		4,742

Table 1. (Continued.)

Wave B (September-December, 2013) and Wave C (January-April, 2014)

State	Geographic Stratum	License Stratum	Household Fishing Status	Expected Response Rates	Wave A Estimated Completed Interviews	Wave B Estimated ABS Sample Size8	Wave B Estimated Completed Interviews	Wave C Estimated ABS Sample Size	Wave C Estimated Completed Interviews
NC	Coastal	Matched	current/likely	57%	1,000	1,000	570	1,000	570
			unlikely	57%	539	180	102	539	307
		Unmatched	current/likely	43%	1,329	1,329	571	1,329	571
			unlikely	43%	715	238	103	715	308
	Noncoastal	Matched	current/likely	53%	149	149	79	149	79
			unlikely	53%	80	27	14	80	42
		Unmatched	current/likely	41%	605	605	248	605	248
			unlikely	41%	326	109	44	326	133
Total				48%	4,742	3,636	1,732	4,742	2,259

A simple random sample of licensed anglers is selected from the state’s license frame. The survey instrument collects information about recent saltwater fishing activity for the sampled angler, as well as any other individuals who reside at the same address as the sampled angler; each sampled angler represents a cluster of anglers who reside at the same address. Table 2 provides the sample universe, sample size, expected response rates and estimated number of completed surveys for the state.

Table 2. Estimated size of the sample universe, initial and final sample sizes, expected response rates and estimated number of completed interviews for the nonresident angler sampling.

Wave A (May-August, 2013), Wave B (September-December, 2012) and Wave C (January-April, 2014)
State	Estimated Number of Nonresident Anglers9	Wave A Sample Size10	Expected Response Rate11	Household Fishing Status12	Wave A Estimated Completed Interviews13	Wave B Estimated Sample Size14	Wave B Estimated Completed Interviews	Wave C Estimated Sample Size	Wave C Estimated Completed Interviews
NC	159,743	720	60%	current/likely	311	311	187	311	187
				unlikely	78	26	16	78	47
Total					389	337	202	389	234

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

2.1. Data Collection Procedures

The MLSR is a self-administered mail survey. The data collection procedures have been extensively tested through previous MRIP pilot studies (Andrews et al. 2010, Brick et al. 2012). The surveys are administered for one fixed panel survey, which consists of three waves over a year at 4-month intervals.

The data collection period for each wave begins one week prior to the end of the wave (e.g. target starting date for wave A is 8/25/2013) with an initial survey mailing. The timing of the initial mailing is such that materials are received prior to the end of the reference wave.

The initial mailing is delivered by regular first class mail and includes a cover letter stating the purpose of the survey, a survey questionnaire and a post-paid return envelope and a prepaid cash incentive (as described in Part A, Question 9).

One week following the initial mailing, a thank you/reminder postcard is sent via regular first class mail to remind sample units to complete and return the questionnaire.

Three weeks after the initial survey mailing, a follow-up mailing is delivered to all sample units that have not responded to the survey. The follow-up mailing is delivered via first class mail and includes a nonresponse conversion letter, a second questionnaire and a post-paid return envelope.

The Wave B expected sample size is smaller due to the subsampling of those households classified as having unlikely saltwater participants based on the Wave A response (see footnote 8 and 14). The subsampling is a cost saving and burden reduction device. The alternative that was considered, and is used in some other surveys, is to use the first wave response as definitive and exclude these households from both subsequent waves. In this investigation we felt it safer to subsample and recontact all of the unlikely households in Wave C. 

2.2. Estimation Procedures

Final sample weights for the resident angler sample and the non-resident angler sample are calculated in stages. In the first stage, base sample weights within each stratum are calculated as the inverse of the selection probability ( , where π_i is the probability of selecting unit i for the sample). For the resident sample, base weights for addresses that cannot be matched to an angler license database (sample units in the unmatched strata), are adjusted to account for subsampling by multiplying the base weight by the inverse of the subsampling rate.

In the second stage, base weights (or adjusted base weights in the unmatched strata) are adjusted to account for nonresponse. Specifically, the weights of nonresponding units are increased by the inverse of the weighted response rate within nonresponse adjustment cells

where

and and are the sums of base weights in cell c for respondents and nonrespondents, respectively. In this case, a respondent is a sampled unit that has responded in the first wave and the third wave (so estimates of the full year can be made retrospectively from their responses). Additional definitions of response might be used for other analysis, but the basic procedures outlined above would also be used in these cases.

In the resident sample, nonresponse adjustment cells will be defined by coastal/non-coastal stratum, matched/unmatched designation, and whether or not the address was successfully matched to a landline telephone number. In the non-resident sample, adjustment cells will be at the stratum level (license state).

Estimates of total participation, as well as associated estimates of variance, are calculated in SAS Version 9.3 using the surveymeans procedure. For a given coastal state, total participation is the sum of resident angler participation and nonresident angler participation, both of which are calculated as weighted sums.

where is the final weight and is the number of participants for unit j at address i of stratum h.

Variance of the total effort estimate is estimated using the Taylor series method

where

For estimating participation, we expect stratification to be more effective than simple random sampling due to the oversampling of coastal and licensed households. Gains in efficiency will be offset somewhat by weighting effects, which will increase the variance of participation estimates. Given these two factors, we expect a design effect of approximately 1.2 (i.e., 20% increase in the variance).

Any respondent who completes both Wave A and Wave C will be considered a complete because we have information on their annual fishing. Note Wave C has a special instrument for those that do not complete Wave B. In addition, imputation of participation is planned for the MLSR for those who complete Wave A and do not complete subsequent waves. This is predicated on being able to produce imputations of participation that have reasonable error properties. The Wave A responses from those who participate in Wave C but not in Wave B are expected to provide reasonable donors for a hot deck imputation for those who did not report participation in Wave A (those are participants even without any later response).  As a result, we expect the sample size used in the longitudinal file will equal the number who completed in Wave A.

The MLSR and the MFES samples are selected independently (although no households are in both) so the estimation procedures for each sample will be applied and then the estimated number of annual participants will be computed for each sample from its standard weight. The prime analytic interest is in the difference in the estimated number of annual participants. Since the two samples are independent, the variance of the estimated difference in estimates of annual participation is simple to compute. The MFES also contains other reports of participation (in the last 2 months, in the last 4 months, and in the last 8 months) and these responses will be examined to determine if there is some way to reduce measurement error if any substantial error is found in the 12 month recall. The overall objective is to determine if there are ways to estimate annual participation in recreational saltwater fishing that do not require a separate, ongoing MLSR.

The power for the comparison is largely a function of the size of the MLSR sample. We anticipate the number of respondents will be 389 (this implies the data from the Wave A respondents can be imputed successfully if other waves are not obtained).  The number of respondents in North Carolina from the MFES survey for a wave is about 1,800. If the true percentage of adults who annually participate is 30%, then a difference of about 5 percentage points can be detected with a power of 80% and an alpha at .05.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The expected response rates for the resident and non-resident samples are 48% and 60%, respectively. Previous MRIP pilot studies utilized similar data collection procedures and achieved similar response rates.

The expected response rates will be achieved by using standard mail survey protocols (Dillman et al, 2008). An initial mailing will include an introductory letter stating the purpose of the survey, the survey questionnaire and a business reply envelope, and a prepaid cash incentive. A thank-you/reminder postcard will be administered to all sample units one week following the initial mailing. A final mailing, including a second questionnaire, a nonresponse conversion letter, and a business reply envelope will be sent to all nonrespondents three weeks after the initial mailing.

We will minimize nonresponse bias by using a questionnaire that maximizes responses by the entire sample population, including both anglers and non-anglers. The MLSR questionnaires are derived from the MFES questionnaires, which have been developed through field testing and cognitive interviews.

The MLSR will interview members in the sampled household three times over the year at 4-month intervals and ask about their fishing activities during the past four months. We expect that errors in placing events in time will be reduced with the shorter recall period.

Nonresponse bias will be assessed by:

1. Comparing early and late responders with respect to reported fishing activity. This analysis will identify differences in respondents based upon the level of effort required to solicit a response. Previous studies (Brick et al., 2012) demonstrated that early and late responders are similar in terms of reported recreational fishing activity,

2. Utilizing information from sample frame to define weighting classes for postsurvey weighting adjustments. Weighting classes will be defined such that response rates and fishing activity are similar within classes. Nonresponse bias will be measured by comparing unadjusted estimates to estimates that have been adjusted to account for differential nonresponse among weighting classes. Previous studies identified differential nonresponse and reported fishing activity between households with and without licensed anglers and demonstrated that nonresponse weighting adjustment decreased estimates of fishing effort by 25% over unadjusted estimates (Andrews et al., 2010), and

3. Comparing the response rates and survey measures among different demographic groups, and post-stratifying the survey weights such that weighted estimates of demographic characteristics (e.g. gender, race, ethnicity) conform to population totals for the state (Kalton and Flores-Cervantes, 2003).

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

No additional testing is planned.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical support was provided by the following:

Dr. Jean Opsomer, Colorado State University, 970-491-3841

Dr. J. Michael Brick, Westat, 301-294-2004

Dr. Richard Aiken, U.S. Fish and Wildlife Service, 703-358-1839

Dr. Han-Lin Lai, NOAA Fisheries Service, 425-415-0460

Dr. Jun Rossetti, ICFI, 301-427-8170

Dr. David A. Van Voorhees, NOAA Fisheries Service, Office of Science and Technology, 301-427-8189 is the point-of-contact for the Agency.

References

Andrews, W.R., J.M. Brick, N.M. Mathiowetz, and L. Stokes (2010). Pilot Test of a Dual Frame Two-Phase Mail Survey of Anglers in North Carolina. Retrieved from http://www.countmyfish.noaa.gov/projects/downloads/Final_Report%20NC%202009%20Dual%20Frame%20Two%20Phase%20Experiment.pdf.

Brick. J.M., W.R. Andrews, and N.M. Mathiowetz (2012). A Comparison of Recreational Fishing Effort Survey Designs. Retrieved from https://www.st.nmfs.noaa.gov/mdms/public/finalReport.jsp?ReportID=362.

Brick, J.M., D. Williams, and J.M. Montaquila (2011). Address-Based Sampling for Subpopulation Surveys. Public Opinion Quarterly 75: 409-428.

Chu, A., D. Eisenhower, M. Hay, D. Morganstein, J. Neter, and J. Waksberg (1992). Measuring the recall error in self-reported fishing and hunting activities, Journal of Official Statistics, 8, 19-39.

Chu, A., D. Eisenhower, D. Morgenstein, and J. Waksberg (1989).  Investigation of Possible Recall/Reference Period Bias in National Surveys of Fishing, Hunting and Wildlife-Associated Recreation.  269 pp.

Church, A.H. (1993). Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis. Public Opinion Quarterly 57:62–79.

Dillman, D.A., J.D. Smyth, and L.M. Christian (2008). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley and Sons.

Groves, R., M. Couper, S. Presser, E. Singer, R. Tourangeau, G. Acosta, and Nelson, L. (2006). Experiments in Producing Nonresponse Bias. Public Opinion Quarterly 70: 720–736.

Kalton, G., and I. Flores-Cervantes (2003). Weighting Methods. Journal of Official Statistics 19(2):81-07.

Lohr, S. (2009). Multiple Frame Surveys. Chapter 4 in Pfeffermann, D. (Ed.) Handbook of Statistics: Sample Surveys Design, Methods and Applications (vol. 29A). Elsevier, Amsterdam.

National Research Council (2006). Review of Recreational Fisheries Survey Methods. Washington, D.C.: National Academies Press.

Trussell, N. and P.J. Lavrakas (2004). The influence of incremental increases in token cash incentives on mail survey response: Is there an optimal amount? Public Opinion Quarterly 68: 349-367.

1 Estimated number of households in the matched stratum is based upon the number of unique addresses in state databases of licensed saltwater anglers as of 8/29/2012. Estimated number of households in the unmatched stratum is the difference between the estimated number of total occupied housing units (Census 2010) and the number of unique addresses in the state license databases.

2 Estimated amount of ABS sample required to achieve final sampling targets.

3 Final ABS sample sizes after subsampling from the unmatched strata. All matched addresses are retained in final sample.

4 Approximately 10% of addresses will be returned as invalid reducing the final sample size to 10,152.

5 Response rates estimated from previous MRIP pilot studies.

6 Respondents in each geographic and license stratum are classified into current/likely or unlikely based on the wave A responses.

7 Estimated numbers after classifying wave A respondents into current/likely or unlikely status (proportions are assumed to be 0.65 and 0.35, respectively).

8 Samples classified as unlikely are sub-sampled at a rate of 0.33 for wave B, resulting smaller numbers of estimated sample size and completed interviews.

9 Based upon participation estimates from the Marine Recreational Fisheries Statistics Survey.

10 Approximately 10% of addresses will be returned as invalid reducing the final sample size to 648.

11 Estimated from previous MRIP pilot studies.

12 Respondents in each geographic and license stratum are classified into current/likely or unlikely based on the wave A responses.

13 Estimated numbers after classifying wave A respondents into current/likely or unlikely status (proportions are assumed to be 0.80 and 0.20, respectively).

14 Samples classified as unlikely are sub-sampled at a rate of 0.33 for wave B, resulting smaller numbers of estimated sample size and completed interviews.

2