Supporting Statement

SUPPORTING STATEMENT
Request to Conduct Focus Groups
§316(b) Benefits Survey
June 17, 2010
(1) Title of the Information Collection
Estimating the Social Benefits of §316(b) Requirements for Cooling Water Intake
Structures under the National Pollutant Discharge Elimination System Permit Program
(2) Short Characterization/Abstract
On February 16, 2004, the U.S. Environmental Protection Agency (EPA) took final action
on the Phase II rule governing cooling water intake structures at existing facilities that are
point sources; that, as their primary activity, both generate and transmit electric power or
generate electric power for sale to another entity for transmission; that use or propose to
use cooling water intake structures with a total design intake flow of 50 MGD or more to
withdraw cooling water from waters of the United States; and that use at least 25 percent
of the withdrawn water exclusively for cooling purposes. See 69 FR 41576 (July 9,
2004). Industry and environmental stakeholders challenged the Phase II regulations. On
judicial review, the Second Circuit (Riverkeeper, Inc. v. EPA, 475 F.3d 83, (2d Cir.,
2007)) remanded several provisions of the Phase II rule. Some key provisions remanded
are as follows: EPA improperly used a cost-benefit analysis as a criterion for determining
Best Technology Available (BTA), and EPA inappropriately used ranges in setting
performance expectations. In response, EPA suspended the Phase II regulation in July
2007 pending further rulemaking. The U.S. Supreme Court granted Entergy
Corporation’s petition for writ of certiorari, solely on the question of whether EPA had
the authority under §316(b) of the Clean Water Act to consider costs and benefits in
decision-making. On April 1, 2009, the Court, in Entergy Corp. v. Riverkeeper Inc.,
decided that “EPA permissibly relied on cost-benefit analysis in setting the national
performance standards … as part of the Phase II regulations.” EPA is now taking a
voluntary remand of the rule, thus ending Second Circuit review.
In June of 2006, EPA promulgated the 316(b) Phase III Rule for existing manufacturers,
small flow power plants (facilities that use cooling water intake structures with a total
design intake flow of less than 50 MGD to withdraw cooling water from waters of the
United States; and that use at least 25 percent of the withdrawn water exclusively for
cooling purposes), and new offshore oil and gas facilities. Offshore oil and gas firms and
environmental groups petitioned for judicial review, which was to occur in the Fifth
Circuit, but was stayed pending the Supreme Court decision on the Phase II case. EPA is
also voluntarily remanding the existing facilities portion of the Phase III rulemaking.
EPA is now combining the two phases (Phase II and III) into one rulemaking covering all
existing facilities. EPA will develop regulations to provide national performance

1

standards for controlling impacts from Phase II and Phase III cooling water intake
structures under Section 316(b) of the Clean Water Act (CWA).
Under Executive Order 12866, EPA is required to estimate the potential benefits and
costs to society. The purpose of this set of focus groups is to finish the development of a
survey instrument that will be used to elicit values for changes in fish mortality and the
associated ecological gains from a section 316(b) regulation covering Phase II and Phase
III facilities.
The survey will be used to estimate total values (willingness to pay, or WTP) derived by
average households for changes related to the reduction of fish losses at CWIS. As
indicated in the prior literature (Cummings and Harrison 1995; Johnston et al. 2003,
2005), it is virtually impossible to justify, theoretically, the decomposition of empirical
estimates of use and nonuse values. The survey will provide the flexibility to estimate
nonuser values, using various nonuser definitions drawn from responses to survey
questions 11 to 13. The structure of choice attribute questions will also allow the analysis
to separate value components related to the most common sources of use values—effect
on harvested recreational and commercial fish. The remaining welfare estimates are
expected to be composed primarily of nonuse components, although some use value
components might also be included for some respondents. In summary, the survey will
provide estimates of total values, will allow estimates of value associated with specific
choice attributes (following standard methods for choice experiments), and will also
allow the flexibility to provide some insight into the relative importance of use versus
nonuse values in the 316(b) context.
Within rulemaking, among the most crucial concerns is the avoidance of benefit (or cost)
double counting. Here, for example, the WTP estimates will include use and nonuse
values among a representative population sample. These will overlap—to a potentially
substantial extent—with use value estimates that might be provided through some other
methods, including revealed preference methods that might be used to estimate use
values of recreational anglers for fish kill reductions (i.e., through related improvements
in fishing quality). While using the proposed stated preference value estimates for
benefit estimation, particular care will be given to avoid any possible double counting of
values that might be derived from alternative valuation methods. In doing so, the Office
of Water will rely upon standard theoretical tools for non-market welfare analysis, as
presented by authors including Freeman (2003) and Just et al. (2004). From a purely
mechanistic perspective, survey results will be used to derive use and nonuse values
following standard practice for choice experiments (Adamowicz et al. 1998).
To obtain reliable estimates of WTP, the Agnecy will test and design all survey elements
to promote incentive compatible preference elicitation mechanisms. Incentive compatible
stated preference surveys provide no incentive for non-truthful preference revelation
(Carson and Groves 2007). The literature is clear regarding the importance of incentive
compatibility in stated preference value elicitation and the role of both question format
and scenario consequentiality in ensuring this property (Carson et al. 2000; Carson and
Groves 2007; Collins and Vossler 2009; Herriges et al. 2010; Johnston 2006; Vossler and

2

Evans 2009). It has been established that referendum-type stated preference choices are
incentive compatible given that certain conditions are met, including the condition that
responses are believed by respondents to be consequential, or potentially influencing
public policy decisions (Carson and Groves 2007; Herriges et al. 2010). Additional
requirements for consequentiality in choice experiment surveys are detailed by Collins
and Vossler (2009), who find that common choice experiment (or conjoint) formats are
similarly incentive compatible under a broad range of assumptions, including those
underlying the proposed survey instrument.
This supporting statement addresses the testing of a draft survey instrument.
(3) Need for the Collection
The project is being undertaken pursuant to section 104 of the Clean Water Act dealing
with research. Section 104 of the Clean Water Act authorizes and directs the EPA
administrator to conduct research into a number of subject areas related to water quality,
water pollution, and water pollution prevention and abatement. This section also
authorizes the EPA administrator to conduct research into methods of analyzing the costs
and benefits of programs carried out under the Clean Water Act.
This project is exploring how public values for fishery resources (covering both use and
non-use values) are affected by fish losses from impingement and entrainment at cooling
water intake structures (CWIS). Understanding total public values, including the more
difficult to estimate non-use values, for fishery resources is necessary to determine the
full range of benefits associated with reductions in impingement and entrainment losses,
and whether the benefits of government action to reduce impingement and entrainment
losses at Phase II and Phase III facilities are commensurate with the costs associated with
such actions. Because non-use values may be substantial, failure to recognize such values
may lead to improper inferences regarding policy benefits and costs. The findings from
this study will primarily be used by EPA to improve estimates of the economic benefits
of the section 316(b) regulation for Phases II and III facilities as required under Executive
Order 12866.
(4) Non-duplication
EPA has not identified any previous studies that would allow direct estimation of the
social benefits of quantified nationwide changes in populations of fish species (including
forage, recreational, and commercial species) affected by the proposed 316(b) regulation.
Therefore, to the best of our knowledge this survey is unique and does not duplicate other
efforts.
There are many studies in the environmental economics literature that quantify benefits
or willingness to pay (WTP) associated with various types of water quality and aquatic
habitat changes. However, none of these studies allows the isolation of non-market WTP
associated with quantified reductions in fish losses (or increases in fish populations) for
forage fish. Most available studies estimate WTP for broader, and sometimes
ambiguously defined, policies that simultaneously influence many different aspects of
3

aquatic environmental quality and ecosystem services, but for which WTP associated
with fish or aquatic life alone cannot be identified. Other studies provide benefit
estimates associated with improvements in fish (or aquatic) habitat, but do not link this to
well-defined and quantified changes in affected or supported organisms. Still other
studies address willingness to pay for changes in charismatic or recreational species that
have little relationship to the types of forage fish that are the main species affected by
cooling water intake structures.
For example, choice experiment studies such as Hanley et al. (2006a, b) and Morrison
and Bennett (2004) estimate WTP for aquatic ecosystem changes that affect fish, but
effects on fish are quantified and valued solely in terms of the presence/absence of
different types of fish species. This approach renders associated results unsuitable for
316(b) benefit estimation. Also, many of these studies were conducted outside the U.S.
(e.g., the EU or Australia), making their use for benefit transfer to a U.S. policy context
much more challenging.
Other studies have estimated the value of changes in catch rates or populations of select
recreational and commercial species, charismatic species such as salmon, or changes in
water quality that affect fish, but none have specifically valued changes in forage fish
populations. For example, Olsen et al. (1991) conducted a survey of Pacific Northwest
residents, including both anglers and non-anglers, to determine their WTP for doubling
the size of the Columbia River Basin salmon and steelhead runs. EPA’s proposed survey
approach differs from this study and others like it (such as Cameron and Huppert, 1989)
in that it would include respondents from various geographic regions in the United States
and would provide values for the full range of forage, recreational, and commercial
species affected by 316(b) regulations, instead of valuing a few recreational species in
one specific geographical area.
Among available studies, the most closely related is Johnston et al. (2009), which
estimates total willingness to pay (WTP) for multi-attribute aquatic ecosystem changes
related to improvements in small migratory (diadromous) fish. Unlike other studies, the
choice experiment data of Johnston et al. (2009) allow estimation of WTP associated with
quantified changes in forage fish (e.g., WTP per fish or percentage change in fish),
holding other ecological effects constant. That is, unlike results provided by other studies
in the literature, WTP estimates of Johnston et al. (2009) are not confounded with values
for other changes including water quality, habitat, overall ecological condition, etc. In
addition, the choice experiment of Johnston et al. (2009) addresses species such as
alewife and blueback herring that are neither subject to recreational or commercial
harvest in Rhode Island, nor are charismatic species. Hence, the species affected are a
close analog to the forage fish that are at primary issue in the 316(b) policy context.
Although the methods and data of Johnston et al. (2009) allow quantification of
combined use and nonuse values associated with specific improvements in forage and/or
recreational fish, and also provide limited ability to isolate nonuse values from use
values, the policy context and scale prevent direct use for analysis of national benefits of
the 316(b) regulation. Specifically, Johnston et al. (2009) estimate Rhode Island

4

residents’ preferences for the restoration of migratory fish passage over dams in the
Pawtuxet and Wood-Pawcatuck watersheds. Hence, the case study is for a watershedlevel policy with statewide welfare implications. In contrast, 316(b) policies would have
nationwide implications, both on ecosystems and on affected facilities.
(5) Consultations
The Principal Investigator for the stated-preference portion of this effort is Dr. Robert
Johnston. Dr. Johnston is assisted by Dr. Elena Besedin, a Senior Economist at Abt
Associates Inc. Dr. Erik Helm at the U.S. Environmental Protection Agency serves as the
project manager and a contributor to this research.
Robert J. Johnston is Director of the George Perkins Marsh Institute and Professor of
Economics at Clark University. He is President-elect of the Northeastern Agricultural
and Resource Economics Association (NAREA), on the Program Committee for the
Charles Darwin Foundation, the Science Advisory Board for the Communication
Partnership for Science and the Sea (COMPASS), and is the Vice President of the Marine
Resource Economics Foundation. Professor Johnston has published extensively on the
valuation of non-market commodities (goods, services, and resources), benefit cost
analysis, and resource management. His recent research emphasizes coordination of
ecological and economic models to estimate ecosystem service values, with particular
emphasis on the role of aquatic ecological indicators. He has also worked extensively in
methodologies for benefit transfer, including the use of meta-analysis. Professor
Johnston’s empirical work on non-market valuation and benefit transfer has contributed
to numerous benefit cost analyses conducted by federal, state and local government
agencies in the US, Canada and elsewhere.
Elena Y. Besedin, a senior economist at Abt Asspciates Inc., specializes in the economic
analysis of environmental policy and regulatory programs, Her work for EPA has
concentrated on analyzing economic benefits from reducing risks to the environment and
human health and assessing environmental impacts of regulatory programs for many EPA
program offices. She worked extensively on valuation of non-market benefits associated
with environmental improvements of aquatic resources. Dr. Besedin’s empirical work on
non-market valuation included design and implementation of stated and revealed
preference studies and benefit transfer methodologies. Her recent work focused on
developing integrated frameworks to value changes in ecosystem services stemming from
environmental regulations.
EPA has conducted an external peer review of the previous version of the survey
instrument. The previous version of the survey instrument was reviewed by Dr. Richard
C. Bishop, Dr. Kevin J. Boyle, Dr. Timothy C. Haab, Dr. James R. Kahn, and Dr. Alan J.
Krupnick. The review focused on (1) the appropriateness of the survey design for
measuring fish resources affected by cooling water intake structures (CWIS), (2)
sufficiency and appropriateness of information provided in the survey on the baseline
ecological conditions, the expected changes, and the expected costs, and (3)whether the
answers to this survey lend themselves to straightforward economic interpretation of

5

WTP. The reviewers acknowledged the extensive effort that was placed into survey
design and noted that the 316(b) survey was a well-designed study. They also offered
concrete suggestions for further improvements of the survey instrument. EPA has
incorporated many of the reviewers comments in subsequent survey revisions (Versar
2006).
(6) Peer Review Plans
Internal Agency reviewers provided and will continue to provide input on the content and
format of the focus groups, questionnaire design issues, and issues related to survey
sampling design and methodology. Survey materials will be subject to routine internal
review by Dr. Helm and EPA’s National Center for Environmental Economics (NCEE)
staff.
We note that the current survey instrument is built upon an earlier version that was peer
reviewed in January 2006. It incorporates recommendations received from the first peer
review panel. Because the final product of this study meets the major technical work
criteria specified in the Peer Review Handbook (U.S. EPA 2000) the Agency also plans
to convene a peer-review panel to review the entire survey process. First, the Agency
plans to review the survey instrument prior to fielding it. After the survey is completed,
EPA will peer-review study results, and EPA’s final estimated results for the 316(b)
Existing Facilities rulemaking,
(7) Confidentiality
The survey instrument will fully conform to federal regulations – specifically the Privacy
Act of 1974 (5 U.S.C. 552a), the Hawkins-Stafford Amendments of 1988 (P.L 100-297),
and the Computer Security Act of 1987. Each prospective respondent will be informed
that their participation in the survey is voluntary, and that their identities will be kept
confidential by the investigators and not associated with their responses. Neither EPA
nor any other federal or state agency will have access to the names of respondents.
The focus groups will be recorded using audiotapes, which will later be transcribed.
However, individuals will not be identified on the transcripts. To insure that the focus
groups include a representative and diverse sample of individuals, EPA will tabulate
basic demographic information for participants, such as age, ethnicity, occupation, and
income. However, no personal identifying information about participants, such as names,
phone numbers, or addresses, will be included in the focus group transcripts. Prior to
commencing any audio recordings, respondents would be informed of EPA's intentions to
audiotape the focus group proceedings, and informed that all taped responses will be
strictly anonymous and confidential.
(8) Sensitive Questions

6

There are no questions on sexual behavior and attitudes, religious beliefs, and other
matters that are commonly considered private or sensitive in these focus group materials.
Answers to standard demographic questions (i.e., age, income) will be included.
(9) Respondents
EPA intends to conduct eight focus groups as part of the testing of the survey instrument.
There are a total of seven study regions for purposes of evaluating the 316(b benefits. For
the purpose of the stated preference survey implementation, EPA uses five geographic
regions: California, Great Lakes, Inland, Northeast, and Southeast. The Northeast region
includes the North Atlantic and Mid Atlantic regions and the Southeast region includes
the South Atlantic and Gulf of Mexico regions. Because focus groups in different study
regions might provide distinct information relevant to survey design the Agency will
conduct focus groups in multiple regions. The Agency will modify the survey instrument
for each region relative to the current survey draft for the Northeast region as follows:
• Cover – The text on the cover currently reads “A survey of Northeast Residents
(CT, DE, MA, MD, ME, NH, NJ, NY, RI, VT)”. “Northeast Residents” and the
list of included states will be changed to match the respondent’s region.
• Cover – The photo on the cover is of a forage species (silversides) in found in the
Northeast region. If that species isn't found in other regions, we will replace the
cover photo with a similar substitute photo for a forage species relevant to that
region.
• Page 1 – The survey states that it “asks for your opinions regarding policies that
would affect fish and habitat in the Northeast US.” “Northeast US” will be
replaced with the respondent’s region.
• Page 1 – Includes the statement that “Northeast fresh and salt waters support
billions of fish.” “Northeast” will be replaced with the respondent’s region and
“salt water” will be removed from the Inland and Great Lakes regions.
• Page 2 - The survey states that “Cooling water use affects fresh and salt waters
throughout the Northeast US, but almost all fish losses are in salt waters such as
coastal bays.” For the California and Southeast regions, which include both salt
water and freshwater facilities, reference to the Northeast region would be
replaced with the name of the respondent’s region. For the Great Lakes and
Inland survey regions, which only include freshwater facilities, reference to salt
water will be removed.
• Page 2 – A map of the Northeast region and facility locations is presented. A
comparable map will be produced for each region, and map included in the survey
will correspond to the respondent’s region.
• Page 4 - This page provides an example for two Northeast coastal species and
includes a figure comparing estimates of young fish lost to the total number of
adult fish. The estimate of total losses and example species will be modified to
describe species and losses occurring within the respondent’s region.
• Page 5 - The total losses (1.1 billion) currently included within the survey
corresponds to losses from facilities in Northeast coastal waters. This total would
be replaced with the total losses for the respondent’s region. The pie charts will

7

•
•
•

•

•

•
•

be updated based on the regional total while maintaining the 25% and 95%
reductions used in the example.
Page 7 - The table text describing each score will be changed to include the
current scores for the respondent’s region.
Page 8 - Text describing the commercial fish sustainability score and fish
population score will be changed to describe the current score for the respondent’s
region relative to its maximum.
Page 9 – The text states that the aquatic ecosystem score measures “how close
affected Northeast waters are to the most natural, undisturbed condition that is
possible.” “Northeast waters” will be replaced with a reference to waters within
the respondent’s region.
Page 12 – The included figure illustrates the location of “Coastal Facilities
Affected by the Proposed Policies”. For other regions which include both salt
water and freshwater facilities, the figure will be replaced with a similar figure
circling salt water facilities within the respondent’s region. For the Great Lakes
and Inland regions (which do not include salt water facilities), only facilities
within the region will be shown in the figure, making the circle unnecessary.
Pages 13-16 – Regional references within the table headings in Questions 4, 5,
and 6 (e.g., “Policy Effect NE Coastal”, “Option A NE Coastal”) will be modified
to refer to the policy effects and options for the respondent’s region. The “Fish
Saved per Year” score includes a note reminding the respondent of total fish lost
within the region due to I&E (e.g., 1.5 billion); this number will be changed to
reflect total losses within the respondent’s region. The values describing the
current situation and Options A and B within experiment questions 4, 5, and 6
will also vary across regions.
Page 17 – Text describing “National versus Northeast policies” will be modified
to refer to the respondent’s region.
Page 21 – Question 12 asks the respondent how times he participated each of six
activity categories over the last year within the “Northeastern US only”. Text will
be changed to reflect the respondent’s region including the list of states which the
region includes.

EPA will edit the Northeast survey instrument so that respondents in each of the 5 study
regions receives a survey that is representative of their region. The specific details of the
presentation of this region specific material is to be informed and tested by the focus
group process. In general the instrument will be edited so that all references to
geographic location, fish species and environmental baseline information, and regionally
specific policy outcomes will match the study region.
Each focus group will last approximately 90 minutes. We anticipate that eight to ten
people will attend each focus group. For each focus group, individuals will be recruited
from one of the study regions to reflect the range of ultimate potential respondents to a
survey. The groups will be led by an experienced moderator, and held in facilities with
audio/video recording capabilities.

8

Focus group participants will be randomly selected by marketing research firms from
panels of focus group participants maintained by each firm. Participants are eligible if
they are 18 years of age or older; if they are not full-time students; if they do not have
occupations related to the environment, including water treatment and sewage, electric or
nuclear power companies; and if they have not participated in a survey or focus group in
the last six months. Each of the focus groups will include a mix of individuals with
diverse socioeconomic backgrounds, based on characteristics such as income, marital
status, age, race, education, occupation, and gender. Participants will be asked to attend a
focus group session and participate in a discussion of specific topics led by a moderator.
Participation in the focus group sessions is voluntary. Participants will have to expend
time, effort, and travel to participate in the focus group sessions. Following standard
practice in marketing research, participants will be compensated for their time and effort.
As described by Dillman (2000) in the context of population surveys, compensation can
increase response rates and provide an incentive to participate among respondents who
may not have a strong ex ante interest in the subject matter. Therefore, compensation can
help to avoid self selection biases associated with ex ante interests in environmental
policies, and can thereby encourage a more representative sample of focus group
participants. The proposed incentive fee is $75.00 per participant.
(10) Collection Schedule
The proposed timeline for the data collection is as follows.
Task:
Date:
Submit ICR package for focus groups to OMB
March 2010
Conduct focus groups
June - July 2010
Finalize draft survey instrument
June 2010
Submit ICR package to OMB
July 2010
Receive approval to conduct survey
December 2010
Field survey to sample
March2011
Allow 8-10 weeks for data collection
Analyze data
May - June 2011
Prepare final report and papers
July 2011
Internal EPA peer review of survey methodology
TBD
and results
(11) Respondent Burden
For each of the eight groups we will recruit ten participants. Participants will be given
directions to a marketing research firm where the focus group will occur. They will be
provided a light snack and beverages as well as $75 compensation for their time. We
anticipate that there will be some attrition between recruiting and the actual time of the
focus group, so we anticipate eight to nine participants in each session.
For the purposes of calculating burden, the maximum total number of respondents is 80.
Each individual will participate in a 90 minute session. We also anticipate that

9

individuals will spend up to 30 minutes of travel time. Therefore, the total number of
respondent hours requested is 160 hours.
(12) Information Requested
Individuals who attend the focus groups will at some point during the session be asked to
review the draft survey, answer draft survey questions, and provide feedback on these
presentation materials and survey questions. After reviewing introductory materials and
administering draft survey questions, the moderator of the focus group will read and ask
participants debriefing questions. A script containing relevant questions that will be used
to guide the discussion is attached to this request. EPA notes that the exact wording of
debriefing questions in focus groups may be adjusted somewhat by focus group
moderators to conform to the flow of ongoing discussion. Survey materials will be
revised between groups based on respondent reactions and comments to ensure that the
draft instrument used in each focus group is as clear, concise, and comprehensive as
possible.
EPA notes that not all discussion topics or questions presented in the attached script can
be covered in any single focus group session, due to time constraints. Instead, these
questions will be covered over a number of different focus groups. In some focus groups,
the moderator will first offer some draft valuation questions to focus group participants,
as noted above, followed by debriefing questions. In other cases, the focus group may
begin with general discussion questions, prior to the consideration of draft survey
questions. The order of questions either within or between the categories above may also
be altered, as appropriate to the flow of discussion within a particular focus group.
Freedom to alter question order is important, as it is often impossible to predict the
direction in which respondents will direct their discussions and/or answers.
Some of the focus groups will incorporate individual one-on-one protocol interviews of
the survey instrument. One-on-one protocol interviews will allow in-depth exploration of
the cognitive processes used by respondents to answer survey questions, without the
potential for interpersonal dynamics to sway respondents’ comments (Kaplowicz et al.
2004).
REFERENCES
Adamowicz, W., P. Boxall, M. Williams, and J. Louviere. 1998. Stated Preference Approaches
for Measuring Passive Use Values: Choice Experiments and Contingent Valuation. American
Journal of Agricultural Economics 80(1): 64-75.
Cameron, T.A. and Huppert, D.D. (1989) OLS versus ML Estimation of Non-market Resource
Values with Payment Card Interval Data. Journal of Environmental Economics and
Management. 17, 230-246.
Carson, R. T., T. Groves, and M. J. Machina. 2000. “Incentive and Informational Properties of
Preference Questions.” Working Paper, Department of Economics, University of California,
San Diego.

10

Carson, R.T. and T. Groves. 2007. Incentive and Informational Properties of Preference
Questions. Environmental and Resource Economics 37(1):181–210.
Collins, J.P. and C.A. Vossler. 2009. Incentive Compatibility Tests of Choice Experiment Value
Elicitation Questions. Journal of Environmental Economics and Management 58(2): 226-235.
Cummings, Ronald G., and Harrison, Glenn W., "The Measurement and Decomposition of
Nonuse Values: A Critical Review", Environmental and Resource Economics, 5, 1995, 225247.
Freeman III, A.M. 2003. The Measurement of Environmental and Resource Values: Theory and
Methods. Resources for the Future, Washington, DC.
Hanley N., Colombo, S., Tinch, D., Black, A. and Aftab, A. (2006) Estimating the benefits of
water quality improvements under the Water Framework Directive: are benefits
transferable? European Review of Agricultural Economics 33: 391-413.
Hanley, N., Wright, R.E. and Alvarez-Farizo, B. (2006) Estimating the economic value of
improvements in river ecology using choice experiments: an application to the water
framework directive. Journal of Environmental Management 78: 183-193.
Herriges, J., C. Kling, C. Liu and J. Tobias. 2010. What are the Consequences of
Consequentiality? Journal of Environmental Economics and Management 59(1): 67-81.
Johnston, R.J. 2006. Is Hypothetical Bias Universal? Validating Contingent Valuation Responses
Using a Binding Public Referendum. Journal of Environmental Economics and Management
52(1): 469-481.
Johnston, R.J., E.Y. Besedin and R.F. Wardwell. 2003. Modeling Relationships Between Use and
Nonuse
Values for Surface Water Quality: A Meta-Analysis. Water Resources
Research 39(12): 1363-1372.
Johnston, R.J., E.Y. Besedin, R. Iovanna, C.J. Miller, R.F. Wardwell and M.H. Ranson. 2005.
“Systematic Variation in Willingness to Pay for Aquatic Resource Improvements and
Implications for Benefit Transfer: A Meta-Analysis.” Canadian Journal of Agricultural
Economics 53(2-3):221-248.
Johnston, R.J., E.T. Schultz, K. Segerson and E.Y. Besedin. (2009) Bioindicator-Based Stated
Preference Valuation for Aquatic Habitat and Ecosystem Service Restoration, in Bennett,
J. ed. International Handbook on Non-Marketed Environmental Valuation. Cheltenham,
UK: Edward Elgar, forthcoming.
Just, R.E., D.L. Hueth and A. Schmitz. 2004. The Welfare Economics of Public Policy: A
Practical Approach to Public Policy. Cheltenham, UK: Edward Elgar.
Kaplowitz, M.D., F. Lupi and J.P. Hoehn. 2004. “Multiple Methods for Developing and
Evaluating a Stated-Choice Questionnaire to Value Wetlands.” Chapter 24 in Methods for
Testing and Evaluating Survey Questionnaires, eds. S. Presser, J.M. Rothget, M.P. Coupter,
J.T. Lesser, E. Martin, J. Martin, and E. Singer. New York: John Wiley and Sons.
Morrison, M. and Bennett, J. (2004) Valuing New South Wales rivers for use in benefit transfer.
The Australian Journal of Agricultural and Resource Economics 48: 591-611.
Olsen, D., J. Richards, and R.D. Scott. 1991. “Existence and Sport Values for Doubling the Size
of Columbia River Basin Salmon and Steelhead Runs.” Rivers. 2(1):44-56.
U.S. Environmental Protection Agency (U.S. EPA). 2000. Science Policy Council Handbook:
Peer Review. U.S. EPA,Office of Research and Development, Washington D.C., EPA
100-B-00-001.
Versar. 2006. Comments Summary Report: Peer Review Package for "Willingness to Pay Survey
Instrument for §316(b) Phase III Cooling Water Intake Structures.” Prepared by Versar
Inc., Springfield, VA.

11

Vossler, C.A. and M.F. Evans. 2009. Bridging the Gap Between the Field and the Lab:
Environmental Goods, Policy Maker Input and Consequentiality. Journal of Environmental
Economics and Management 58(3): 338-345.

12