0596-NEW_Flathead__McKenzie_River_Visitor_survey_supporting_statement_B_4-6-12

The Supporting Statement for OMB 0596-NEW

McKenzie River and Trail Visitor Surveys

Flathead Wild and Scenic River Visitor Survey

USDA Forest Service, Flathead and Willamette National Forests,

National Park Service, Glacier National Park

February 2012

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Collection will be limited to 1188 persons for the Willamette National Forest (NF), 1100 for the Flathead National Forest, and 385 for Glacier National Park (NP) over the 3-year life of this approval. We set a target of approximately 1200 completed surveys (600 each of a river-user survey and a trail-user survey) for the McKenzie River, and 1,500 for the Flathead River (combined total of Flathead NF and Glacier NP), which will provide for comparisons among managerially relevant subgroups of visitors, such as weekday vs. weekend visitors, hikers vs. mountain bikers, or whitewater boaters vs. anglers.

This size sample also fulfills the requirements of exploratory factor analysis, which requires at least 20 respondents per item in sets of items that will be collapsed into latent factors (Hair et al. 2010, p. 102). It is generally advisable to have a larger number of respondents per item (e.g., 25) when the anticipated number of factors is more than three (Hogarty et al. 2005). Our largest set of items is from the McKenzie River survey, which includes 25 “experience” items, and given the diversity of items, these may give rise to multiple latent factors. With a target of 25 responses per item, this would require approximately 625 respondents. The data will be subjected to exploratory factor analysis with oblique rotation; items loading > 0.30 on one and only one factor will be retained (Worthington & Whittaker 2006), and Cronbach’s alpha will be used to confirm factor reliability (target 0.80).

Sampling for this collection will be focused on, and limited to, adult (age 16 years and older) recreational visitors to the Flathead River in Montana and Upper McKenzie River and McKenzie River Trail, Oregon, during the primary recreational use seasons. The sampling schedule (stratified random sample) will ensure that the collection obtains a random sample of adult visitors. As is nearly always the case in this type of information collection, precise information about the number of people in the potential respondent universe is not available. There are no data about the number of private boaters, anglers, and trail users. However, records of outfitter-guide use and campground use are available. These numbers, and the results of the 1996 McKenzie River study, suggest that the proposed sampling scheme will generate the target sample size.

Sampling intensity was based on an estimate of the likely number of surveys obtainable. In the 1996 McKenzie River study, on average, 7 surveys per site per day were obtained (with an 81% response rate). We have no reason to believe that the response rate will have changed, so for our calculations we used an estimate of 81% compliance. Because use of the river and trail has increased since 1996, we estimate that we will obtain 9 completed surveys per day.

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection

The visitor surveys will take place in 2012, during the primary and secondary use seasons. The specific dates (April 15 to September 30) were chosen because those were the dates of the 1996 McKenzie river study, and replicating the methods of the 1996 study will permit confident conclusions about changes in visitor experiences and opinions. The Flathead River study will be conducted from May through September, 2012.

No sample frame is available before commencement of information collection. However, the sampling approach will ensure that the collection obtains a random, representative sample of visitors.

For the McKenzie survey, three strata were generated by dividing the study season into thirds (April 15 to June 9; June 10 to August 5, and August 6 to September 30). (This was done because different types of visitors use the McKenzie River during different times of year, and the conditions they encounter – such as weather and crowding – also vary across time.) Within each stratum, for each of the 11 sites, a random sample of four dates was selected using a random number generator (See Table 5). Some of these sites provide access to both the trail and river, while others provide access just to the river or the trail. Surveying will occur in the three developed campgrounds located along Highway 126 (all of which provide both trail and river access). Surveyors will be present during six hours on each sample day. All adult visitors present at the study site during the sample periods will be invited to participate in the study by completing a questionnaire on site. River users (anglers and boaters) will complete the “river user” questionnaire and hikers and mountain bikers will complete the “trail user” questionnaire. The two questionnaires contain many of the same questions, but activity- and experience-specific items are different, to appropriately assess the different experiences of river and trail users. Our estimate of 9 completed surveys per site per day will generate a final sample of 1188 surveys for the Willamette NF.

For the Flathead survey, Forest Service and National Park Service staff, volunteers, and contractors will contact visitors at 33 recreation access points on the Flathead River. A random sample of five dates will be chosen for each study location, and staff will be on site between six and eight hours on sampled days. All visitor groups completing their recreational activities will be approached, and adults (age 16 years and older) will be asked to participate in the survey by completing a questionnaire on site. The estimate of 9 completed surveys per site per day will generate a final sample of 1485 surveys for the Flathead River, which is adequately close to our combined target of 1,500 for both the Flathead NF and Glacier NP.

In both studies, the invitation to participate will follow approved protocols of Institutional Review Boards, assuring visitors that participation is voluntary. No identifying information will be collected at any time, so participation will be fully anonymous. See Survey Methodology for specific sites and dates.

• Estimation procedure

No estimation procedures are needed beyond the stratified random sampling of selected date-location combinations, as described above.

• Degree of accuracy needed for the purpose described in the justification

Several statistical tests and comparisons will be made using the data. In data from 1996, the means and standard deviations for variables were quite variable, so conservative values (those with larger standard deviations) were used in computing power. For example, on one variable, boaters had a mean of 5.0 (7-point scale) with a standard deviation of 1.9, while anglers had a mean of 4.5 (SD= 1.8). (Differences of 0.5 points tend to be fairly typical in this type of research.) These values generate a Cohen’s d value of 0.27. With an alpha of 0.05 and a sample of 600, the power would be 0.91 for this comparison. We do not know what the actual size of differences between groups will be, or the strength of relationships among variables, both of which affect observed power (Cohen 1992), but we believe that, using conservative estimates from previous research, we will have power  0.80 in all but very unusual circumstances.

• Unusual problems requiring specialized sampling procedures

None

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This survey is designed to be administered once for each respondent. If respondents make repeat visits, they will not be asked to complete the survey a second time. This will be ensured by asking a verbal screening question, “have you completed a questionnaire along the McKenzie [Flathead] River previously this year?”

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Accepted survey procedures for wording questions and approaching visitors, as outlined by Dillman (2009), Vaske (2008), and others, will be used. Using an on-site survey, rather than a mail return questionnaire, will greatly increase the response rate. Contacting people at the end of their visits will minimize intrusions upon their experiences. Clear, concise language, and a clear explanation of the purpose of the information collection, will help ensure a high degree of accuracy in obtaining valid responses.

For non-response analysis, non-respondents are considered those people approached and asked to complete the survey who verbally decline, for any reason. We are planning to collect observable information about non-respondents because such information allows systematic comparison of response to nonresponse to identify possible bias. We have not had trouble collecting group size and composition data for non-respondents at similar sites in the past.

Because non-response bias is as important as sample size or high response rate, non-response bias checks will also be conducted. We will record information about all parties approached (see Survey Log), so that we can address potential non-response bias. At each site, on each sample day, surveyors will record the number of people who agree and decline to complete surveys. They will also document information about potential participants’ group size and composition, activities, type (guided vs. private), and activity. We do not anticipate problems of missing groups on busy days for two reasons: 1) these sites rarely have more than 10 people present at one time, and our sampling intervals have been established to permit some time between respondents; 2) if certain days are busier than anticipated, we will have multiple enumerators present. If a group is missed for some reason, it will be considered unknowns.

This information will be statistically contrasted between respondents and non-respondents, which will reveal the presence of any systematic non-response patterns (such as groups with children being less likely to complete the survey). Additionally, any verbal reasons for not completing the survey (e.g., weather, lack of time) will be recorded.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Because this information collection uses the same procedures as a previous study on the McKenzie River, testing of methods is not needed. However, new survey questions will be pilot tested on university or agency staff with fewer than 10 knowledgeable individuals. The time required to complete the instruments will be verified using 5-10 university students.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

James Absher, Ph.D, Research Social Scientist, USDA Forest Service, Troy E. Hall, Ph.D., Professor at the University of Idaho, and Dr. Kreg Lindberg, Ph.D., Associate Professor at Oregon State University headed the preparation and review of this submission package and attached instrument. All three are extensively trained in statistical and methodological procedures, have substantial experience in the fields of leisure and recreation research, and have considerable experience in survey design and implementation. Dr. Absher holds a degree in statistics (Stanford, 1970) and Dr. Hall has taught undergraduate and graduate courses on research methods. Dr. Lindberg is an internationally known expert in environmental economics and teaches undergraduate and graduate courses on statistics and research methods. Both Drs. Absher and Hall have individually conducted research nearly identical to the present information collection for the US Forest Service, National Park Service, and Bureau of Land Management.

Data collection and analysis will be overseen by Dr. Hall and Dr. Lindberg, with the assistance of a graduate research assistant with statistical expertise. The analysis will be very straightforward, consisting of descriptive statistics (means, standard deviations), data reduction (exploratory factor analysis), and comparisons among managerially relevant subgroups (t-tests and ANOVA). Ordinary Least Squares Regressions may be used to investigate how different conditions experienced affect visitor experience quality.

James Absher, Ph.D.

Research Social Scientist

Pacific Southwest Research Station

USDA Forest Service

4955 Canyon Crest Drive

Riverside, CA 92507

(951) 680-1559; jabsher@fs.fed.us

Dr. Troy Hall, Professor

Department of Conservation Social Sciences

University of Idaho

Moscow, ID, 83844

208 885-7911

troyh@uidaho.edu

Dr. Kreg Lindberg

230 Cascades Hall

Oregon State University—Cascades Campus

2600 NW College Way

Bend, OR 97701

541 322-3126

Appendix 1: Literature cited in the responses above

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: Wiley & Sons.

Hair, J.F. Jr., W.C. Black, B.J. Babin, R.E. Anderson. 2010. Multivariate Data Analysis, 7^th ed. Prentice Hall: Upper Saddle River, NJ

Hall, T. E. & Bentley, A. G. (2012). Visitor surveys at Lake Roosevelt National Recreation Area. Report prepared for the National Park Service. Moscow, ID: University of Idaho, Department of Conservation Social Sciences.

Hall, T. E., & Shelby, B. (1996). McKenzie River boater study. Report prepared for McKenzie Ranger District, Willamette National Forest and Eugene District BLM. Corvallis, OR: Oregon State University.

Hogarty, K. Y., Hines, C. V., Kromrey, J. D., Ferron, J. M., & Mumford, K. R. (2005). The quality of factor solutions in exploratory factor analysis: The influence of sample size, communality, and overdetermination. Educational & Psychological Measurement, 65(2), 202-226.

Manning, R. (2007). Parks and carrying capacity: Commons without tragedy. Washington, DC: Island Press.

Shelby, B., & Heberlein, T. A. (1986). Carrying capacity in recreation settings. Corvallis, OR: Oregon State University Press.

Vaske, J. J. (2008). Survey research and analysis: Applications in parks, recreation and human dimensions. State College, PA: Venture Publishing.

Worthington, R. L., & Whittaker, T. A. (2006). Scale Development Research: A Content Analysis and Recommendations for Best Practices. The Counseling Psychologist, 34(6), 806-838.

Page 1 of 4