Colorado River Supporting Statement B 8-12-2015 CN

Supporting Statement B

Colorado River Valuation Survey

OMB Control Number 1024-NEW

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed.

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The universe of potential respondents for this collection is based upon the 2013 estimated visitor statistics (https://irma.nps.gov/Stats/) and a list of 2014-2015 whitewater river floaters provided by the NPS, Concessionaires and anglers on-site the at the sampling sites along the Colorado River corridor from Glen Canyon Dam to the head of Lake Mead.

The choice of a sampling frame including whitewater and angler visitors to the area in the previous 12 months was based in a number of factors.

1) A primary objective of the current study is to perform a substantive replication of the previous work by Bishop et al. (1987)¹ Bishop also utilized this “previous year” sampling frame methodology. Thus our proposed method is consistent with this earlier work.

2) The proposed surveys (like the earlier versions by Bishop et al.) ask the respondent s to recall details from their trips and to respond accordingly. Any lag between taking the trip and responding to the survey introduces the potential for recall bias on the respondents’ part. Extending the pool of potential respondents to include whitewater visitors from earlier years would exacerbate issues of potential recall bias inherent in all non-contemporaneous experiential surveys.

3) While annual precipitation and reservoir levels vary substantially year to year, for both whitewater floaters and Glen Canyon anglers, the most recent year of river flow levels is representative of average flow levels since 2012.

4) Sampling from a limited time period is a standard practice for many Federally-funded studies. The NPS Visitor Services Project conducted nearly 300 surveys of NPS visitors at specific national park units over 20 years using only one to two week “grab samples” of visitors.² Within the context of setting NPS policy, one-year or shorter visitor sample have been successfully used in the context of some of the highest profile NPS policy issues to date, including reintroduction of wolves to the Greater Yellowstone Area, and development of a Winter Use Policy for Yellowstone and Grand Teton NP.³

Overall, we conclude that for the reasons stated above the use of a one-year sample frame for Colorado River angler and whitewater visitors is consistent with prior research, current and past standard practice, and the best potential outcome for the current study.

5) One additional factor reduces the likelihood of significantly different pools of whitewater visitors to the Colorado River from year to year. Private boaters generally have been on a waiting list for a trip permit for several (or many) years. The decisions to float the river are made by private floaters years in advance of any knowledge of water, weather, or economic conditions. Similarly, commercial rafting clients in the Grand Canyon often book their trips up to a year in advance, again significantly diminishing the potential for dramatic shifts in the composition of the pool of potential whitewater respondents due to year-to-year conditions.

Respondent types will include all visitors (18 years and older) who are:

• Whitewater Floaters: people floating the Colorado River through the Grand Canyon

• Glen Canyon Anglers: people fishing in the stretch of the Colorado River between the base of Glen Canyon Dam to just below Lee’s Ferry

The table below shows the approximate respondent universe, expected sample size, response rate, and the estimated final number of expected responses for each of the survey components.

Sample Frame	Respondent Universe	Sample Size	Response Rate %	Estimated Final Responses
Survey 1: Whitewater Floaters	25,000	2,850	65%	1,852
Survey 2: Glen Canyon Anglers	15,000	750	65%	488

Based on our estimations, we anticipate a response rate of 65% for both surveys. This effort will produce sample sizes that are considered robust for the aggregate and accepted margins of error of between ±5% and ± 2% at the 95% confidence level for the population sampled.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Survey 1: Whitewater Floaters

The whitewater floater sample will be randomly selected and stratified into commercial and private samples⁴. The commercial sample will further be stratified into commercial motor trip participants and commercial oar trip participants. Two populations of users will be stratified in a manner to result in equal samples for this collection:

• Private (n=1,425) – these names and address will be drawn from NPS whitewater floater visitor logs from the previous 12 months.

• Commercial (n=1,425) – these names and address will be provided by concessionaires affiliated with commercial float trips.

Based on the results of similar surveys using similar methods of administration in Yellowstone National Park (Duffield, Neher, & Patterson, 2006) ⁵ we expect a response rate of 65% resulting in 1,852 completed surveys. The expected standard errors associated with the simple survey results (proportions) would be + or – 0.002 at a 95% confidence interval (based on an estimated proportion of 0.5).

Survey 2: Glen Canyon Anglers

As part of an ongoing survey, the Arizona Fish and Game Department (AZGFD) collects creel data from anglers intercepted while fishing along the Colorado River. AZGFD has agreed to share contact information for intercepted anglers’ participating in their creel survey. The contact information will only be used for the purpose of developing a mailing list for this study. A total of 750 names will be randomly selected from the list provided by the AZGFD. Every person selected will be sent a mail-back survey. AZGFD reports extremely high cooperation with the on-site creel survey with less than 5% non-response/refusal rate, and these refusals due more to congestion and practicality than to an unwillingness to cooperate. It is not expected that intercept non-response will significantly impact overall response rates.

We expect a response rate of 65% of the 750 surveys sent will result in 488 completed surveys. The expected standard error associated with the simple survey results would be ± 0.04 at a 95% confidence level (based on an estimated proportion of 0.5).

Estimation Procedure: At the end of the survey sampling period, all data will be cleaned, coded, and edited. Sampling weights will also be calculated if needed. The point estimates and their estimated variances for the stated-preference survey data will be calculated using SAS statistical software. We will also perform statistical tests on responses to key survey questions among the subpopulations, for example, commercial vs. private whitewater floaters. We will generate statistics to summarize and compare responses, response rates, and individual characteristics across groups. A post-stratification adjustment will also be generated to correct any detected nonresponse bias.

Estimating Household’s Total Willingness-to-Pay (WTP)

We will use the Stated Preference (SP) data to estimate recreational visitors’ total value for potential outcomes associated with alternative trip attributes. To analyze data from the dichotomous choice Contingent Valuation (CV) version of the surveys, we will apply standard bivariate and multivariate logistic regression techniques. To analyze the data from the conjoint/discrete choice experiment questions, we will apply a random utility modeling (RUM) framework, which is commonly used to model discrete choice decisions in SP studies. The RUM framework assumes that survey respondents implicitly assign utility to each choice option presented to them. This utility can be expressed as

,

• U_ij is individual i’s utility for a choice option (i.e., restoration option) j

• V() is the nonstochastic part of utility, a function of X_ij,

• X_ij represents a vector of attribute levels for the option j (including its cost) presented to the respondent

• Z_i, a vector of personal characteristics

• ⁱ, a vector of attribute-specific preference parameters

• e_ij is a stochastic term, which captures elements of the choice option that affect individuals’ utility but are not observable to the analyst. On each choice occasion, respondents are assumed to select the option that provides the highest level of utility. By presenting respondents with a series of choice tasks and options with different values of X_ij, the resulting choices reveal information about the preference parameter vector.

There is a growing body of work that suggests that CV and choice experiments can yield valid measures of Willingness to Pay TP (see for example, Vossler and Evans 2009,⁶ and Vossler et al. 2012⁷).

Conditional Logit Estimation

To estimate the parameters of the conjoint model, we will use a standard conditional logit (CL) model (McFadden 1986⁸), which assumes the disturbance term follows a Type I extreme-value error structure and uses maximum-likelihood methods to estimate ₁ and _2. The conditional logit is a computationally straightforward estimation approach that can provide useful insights into the general pattern of respondents’ preference, trade-offs, and values.

The parameter estimates from the CL model will then be used to estimate the average marginal value of each non-cost attribute:

,

where k refers to the k^th element of the X and ₁ vectors. They will also be used to estimate the average WTP for acquiring the combination of attributes associated with one management scenario (X₁) compared to the attributes of another scenario (e.g., the no action alternative) (X₀):

.

The standard errors and confidence intervals for these value estimates will be estimated using the delta method (Greene 2003)⁹ or the Krinsky and Robb (1986)¹⁰ simulation method.

To examine, test for, and estimate differences in preferences across our sample, we will estimate both separate and pooled models for subsamples of the data and test the restrictions of the pooled models using log-likelihood ratio tests. We will also estimate varying parameter models by interacting the attribute vector (X_ij) with elements of the respondent characteristics vector (Z_i). The parameter estimates from the interaction terms will allow us to examine whether and how the marginal values associated with program attributes vary systematically with respect to respondent characteristics.

Degree of accuracy for the purpose described in the justification

Accuracy of the WTP values to be estimated from the CV and Conjoint models cannot be specified a priori through the use of an analytical formula. Work by Orme (2006)¹¹ indicates that based on standard rules of thumb, the sample sizes planned for the conjoint model estimations are more than adequate to estimate direct effect values for all included attributes. The accuracy of simple proportions are easily calculated from a standard formulae based on sample size. The accuracy of a simple mean value of a continuous variable depends on sample size and the dispersion of survey responses around the mean. The accuracy of WTP values estimated from either CV or Conjoint-based data depends on number of factors including sample size, model specification, and fit, and significance of estimated parameters. Similar CV or conjoint estimates of mean household WTP from national surveys have reported 95% confidence intervals in the range of + or – 25% of the mean value. However, reported model results vary widely based on model fit and sample size. Our a priori expectation is for full sample results to return estimates of WTP with 95% confidence intervals in the +/- 25% range.

The inherent uncertainty regarding the ultimate precision of estimated models of WTP from the CV and conjoint surveys significantly complicates a priori power analysis for determining optimum sample sizes. Of primary interest to the PIs in their inclusion of a conjoint survey variant is gaining a better understanding of the part-worth, or marginal values associated with individual trip characteristics described in the conjoint questions. We do however anticipate that the resulting sample sizes and the data they provide will be more than adequate for comparisons of estimated WTP values between models and treatments. The smallest anticipated sub-sample sizes (244 for angler CV and conjoint sub-samples) provide sufficient samples for precise estimation of both CV and conjoint models. This expectation is based on two primary factors:

1) Given the inherent uncertainties associated with the precision of willingness to pay estimates derived from complex statistical models, the best a priori “power analysis” is (if available) evidence from prior similar studies. The original study of Colorado River anglers by Bishop et al. (1987) utilized a smaller final sample size for their CV models of WTP (n=235) than the anticipated sample for each treatment in the current proposed study (n=244). Additionally, Bishop was successful in estimating statistically precise WTP estimates not only from this sample of 235, but also from splitting this sample (into samples of 114 and 116) depending on the flows experienced. Further, WTP estimates from these samples of 114 and 116 were shown to be significantly different at the 95% level of confidence (Bishop et al, at p. 120). This prior experience for the same population and significantly smaller samples than are anticipated in the current study provides a solid basis for our target sample size.

2) The conjoint version of the angler survey relies on two characteristics in order to provide sufficient data for statistically significant model estimation: 1) number of respondents, and 2) number of choice occasions (conjoint questions) asked per respondent. While the CV variant of the angler survey asks only one of each of the specific valuation questions per respondent, the conjoint version asks each respondent to answer six valuation questions. Responses to these six conjoint questions are aggregated, and thus provide dramatically more information per respondent on a specific research question than in the case of the CV version of the survey. Prior conjoint surveys of sportsman visitors by the PIs support the expectation that the anticipated angler conjoint sample of 288 responses with 6 data point per response will provide ample data to precisely estimate visitor WTP.¹²

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

A mail-back method will be used to administer the surveys in this collection. We will use a modified Dillman method to maximize the response rate (Dillman 2007)¹³. We will start the process by sending an initial postcard as an address check and serve as a notice to expect a copy of the survey in the mail within the next two weeks. After that, all potential respondents will be mailed a survey packet that will include a cover letter, questionnaire, and a self-addressed stamped envelope. Ten days following the mailing of the survey packet, we will send a reminder postcard, and two weeks afterwards, a packet including a replacement survey, cover letter and a postage paid return envelope will be mailed to all non-respondents.

Non-response testing

For both the whitewater and the angler survey respondents, the PIs will have available supplementary data on a number of characteristics for all potential respondents as well as actual respondents. It is anticipated that (if appropriate) the approach developed by Kanninan, Chapman and Hanemann (1992)¹⁴ could be used to develop final weighted samples with additional characteristics represented in proportions identical to those for the entire potential respondent pool. Comparison characteristics differ between the two surveys (below) but this potential weighting process to adjust for observed non-response bias is the same for both samples.

In a staged mail survey additional information on potential non-response bias can be revealed by comparisons of different groups of respondents. The respondents from the later mailings can be viewed as less cooperative than the respondents from the first mailing. The respondents to later mailings tend to have low response propensity, and they are likely to have characteristics similar to non-respondents. Comparing the estimates for the respondents from the second mailing with the respondents from the first mailing potentially allows us to measure the correlation between the response propensity and responses to survey outcome variables indirectly.

As noted, the proposed study is designed as 1) an update of a dated survey of the same user groups, and to provide additional insight into relationships between CV and conjoint benefit estimates. The study is not intended to specifically inform any single (or group of) policy decisions. Rather, it is hoped the results will provide better understanding on the stability of WTP values over time, and relationship between estimates derived from alternative stated value question formats. Past research on both floater and angler populations indicate these to be highly engaged groups with excellent response rates to surveys on their personal experiences. All of these factors lead the PIs to reasonably anticipate that the non-response adjustments and analysis proposed will be appropriate for the setting and purpose of the study.

Survey 1: Whitewater Floaters

For the whitewater floaters we will use information from the survey that includes: state/zip code, age, type of float trip taken, time of year trip taken, and length of trip. We will compare these characteristics between survey respondents and the entire whitewater floater pool received from NPS. If significant differences between the respondent and population groups are found, data will be weighted to adjust for any observed biases in respondent characteristics.

Survey 2: Glen Canyon Anglers

During their annual creel survey, AZGFD requests that Anglers provide the following information: name, address, age, avidity, and type of fishing to complete their survey. We will use this information to compare the respondents returning a completed survey to the non-respondents. The comparison will be used to allow weighting adjustment for non-response bias in the data.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The survey components of this collection intentionally parallel Bishop’s (1987) surveys from more than 25 years ago. However, results from the Bishop study were examined to ensure questions which provided unclear results, not statistically significant results, or results not informative to the overall goals of the current study were dropped from the newly drafted survey instruments.

Earlier drafts of the survey instruments were reviewed by a number of peer reviewers, both inside and outside of USGS and NPS. Suggestions on the content and format have been integrated into the final version of the survey instruments. The reviewers examined the surveys from their perspectives as a policy maker and as a visitor familiar with recreating within the river corridor. We asked for both levels of insight in order to identify any additional unclear or redundant questions, or possible changes that could streamline the survey instruments.

In addition to the peer reviewers, the survey instruments were administered to a small sample (n=5) of individuals who had previously either floated the whitewater reach of the Colorado River through Grand Canyon NP or had fished the stretch between Glen Canyon Dam and Lee’s Ferry. These individuals provided feedback on question clarity, and length of the survey. Numerous small edits were made to the surveys based on this input.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Person consulted on the statistical aspects of survey and sampling design	Dr. David Patterson Department of Mathematical Sciences University of Montana Missoula, MT 59812 (406) 243-6748
Principal Investigator for data collection and analysis	Dr. John Duffield Department of Mathematical Sciences University of Montana Missoula, MT 59812 (406) 243-5569

1 Bishop, R., Boyle, K., Welsh, M., Baumgartner, R., & Rathburn, P. (1987). Glen Canyon Dam Releases and Downstream Recreation: An Analysis of User Preferences and Economic Values. Salt Lake City: Bureau of Reclamation, Upper Colorado Region.

2 A compendium of all Visitor Services Studies can be found at, http://psu.uidaho.edu/c5/vsp/vsp-reports/

3 Seasonal visitor surveys underlie several policy decisions outlined on the YNP Management webpage at, http://www.nps.gov/yell/learn/management/index.htm

4 Management of float permits though the Grand Canyon are based on a target 50/50 split between commercial and non-commercial float days. In 2013 use was split 47% commercial user days and 53% non-commercial. http://www.nps.gov/grca/learn/management/upload/2014-grca-park-profile.pdf

5 Duffield, J., Neher, C., & Patterson, D. (2006). “Wolves and People in Yellowstone: Impacts on the Regional Economy.” Yellowstone Park Foundation.

6 Vossler, C. A. & Evans, M.F. (2009). Bridging the gap between the field and the lab: Environmental goods, policy maker input, and consequentiality. Journal of Environmental Economics and Management, Elsevier, vol. 58(3):338-345.

7 Vossler, C.A., Doyon, M., and Rondeau, D. (2012). Truth in Consequentiality: Theory and Field Evidence on Discrete Choice Experiments. American Economic Journal: Microeconomics, 4 (4):145–71.

8 McFadden, D. (1986). The Choice Theory Approach to Market Research. Marketing Science 5(4):275 - 97.

9 Greene, William H. (2003). Econometric Analysis, Fifth Edition. Upper Saddle River, NJ: Prentice Hall.

10 Krinsky, I., and Robb, A.L. (1986). On approximating the statistical properties of elasticities. The Review of Economics and Statistics 68(4):715-719.

11 Orme, B. K. (2006). Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research. Madison, WI: Research Publishers LLC.

12 See for example, “Modeling the Behavior of Marlin Anglers in the Western Pacific” by John Duffield, Chris Neher, Stewart Allen, David Patterson, and Brad Gentner (Marine Resource Economics Vol. 27:4 pp. 343-357.

13 Dillman, D.A. (2007). Mail and Internet Surveys: The Tailored Design, Second Edition. New York: Wiley and Sons, Inc.

14 Kanninan, B. J., D. J. Chapman, and M. Hanemann. 1992. Survey Data Collection: Detecting and Correcting for Biases in Responses to Mail and Telephone Contingent Valuation Surveys. Paper presented at the 1992 Annual Research Conference, U.S. Bureau of the Census, Washington, D.C.

10