Comments on FRN and responses

October 16, 2006
Ms. Diana Hynek
Departmental Paperwork Clearance Officer
Department of Commerce
Room 6625
14th and Constitution Ave. NW
Washington, D.C. 20230
Dear Ms. Hynek:
I write to share my professional opinion of the draft survey entitled “The Future of Steller
Sea Lions: What is Your Opinion?” This is the survey announced in 71 FR 47177, an
announcement that was corrected in 71 FR 54472.
I am a researcher who specializes in knowledge and attitude surveys about marine
mammal-related issues, including recovery programs for marine mammals protected
under the Endangered Species Act. I have written and conducted targeted and general
public surveys concerning marine mammals, and analyzed and published their results in
books and journals such as Conservation Biology and Endangered Species Update. I
write to you now to share my comments on the National Marine Fisheries Service’s
(NMFS) proposed survey and the agency’s justification for the use of the survey.
First, the survey. I find the survey to be an ill-considered and poorly constructed research
instrument. As constructed, its (I assume unintended) purpose will be to bias respondents
toward a view of Steller sea lion recovery that gives greater credence to economic
considerations in Endangered Species Act (ESA) implementation than they are statutorily
and judicially allowed. The survey is written in a fashion that insufficiently reflects the
complexity of either the endangered species or the Steller sea lion policy arenas, will
likely mislead the respondents about the nature of endangered species protection (or, at
least, NMFS’s approach toward its ESA responsibilities), and is guaranteed to bias
respondents’ answers. I offer the following specific comments in support of these general
criticisms:
•

On page 2, in the phrase “Some people are concerned about the costs of
protecting threatened and endangered species because the protection activities
may:”, the use of the word “costs” is misleading. Most respondents will read the
use of that word in this phrase as implying economic costs, but the first bullet that
follows lists costs that are not necessarily economic for those who are deprived of
them (such as “limiting recreation” and limiting “fishing activities”). For most
recreational fishermen or nature-based tourists, protection activities’ limitations
on these activities are not economic limitations, but actual limitations on the
freedom to undertake the activities themselves. However, the second bullet then
lists economic costs, such as “the cost of producing and providing goods such as
food, drinking water, and lumber.” This mixing of terms – the use of ‘costs’ to
imply different types of meanings without explaining the differences between the

1

meanings – will cause dissonance within and confusion by your respondents, and
will likely bias the results.
•

On page 2, in Q3, the second statement is of questionable utility because it is
provided without sufficient context. The statement reads “Protecting jobs is more
important than protecting threatened and endangered species” and respondents are
asked to state whether they agree or disagree with this statement. However, it is
not clear what the point is in asking this question without also asking respondents’
opinions on many other comparisons. This question is likely to set up in the minds
of respondents a tension between jobs and endangered species (and thus Steller
sea lion) protection. Creating this tension at the beginning of the survey is likely
to bias respondents’ answers for the remainder of the survey. As well, in my
opinion, it establishes a false dichotomy between Steller sea lion protection (and
endangered species protection generally) and “protecting jobs.” This comparison
is so often a symptom of the use and misuse of propaganda in public conflicts
regarding endangered species and habitat protection that you risk promoting the
false sense among the public that this dichotomy – jobs versus wildlife protection
– is well established and a concern of NMFS. In fact it is not well established and
under the ESA must not be a concern of NMFS when the agency makes decisions
about actions under the ESA. Nevertheless, the inclusion of this language is likely
to bias the responses after this point in the survey by influencing them to think
that they should worry about jobs being threatened by Steller sea lion protection.

•

On page 3, the selected information provided for each species – specifically, the
use of population size estimates – is misleading, out of context, and will likely
bias respondents’ perceptions of the species’ actual status. It will do so because
the absolute numbers of animals in each species or population is not the pertinent
figure used to determine the need for protection under the ESA. Rather, it is the
trends in population size. By providing only absolute numbers, you risk giving
respondents a false sense of the actual status of the populations. That is, you have
no way of knowing whether a respondent will consider 90,000 Steller sea lions as
“enough” and thus not warranting protection. This sort of acontextual perception
may be fed by the fact that directly below the Steller figure you describe northern
elephant seals as “stable” at 100,000 animals. Given the marginal difference
between the sizes of those two populations, it is likely to be confusing to a
member of the general public to try to decipher why the Steller sea lion deserves
protection, but the elephant seal does not. In addition, the largest absolute
population figure you provide on this page is 750,000 for the northern fur seal, a
number that will seem large to any general audience, but which masks the
pressures and problems that population has experienced over the last half-century,
which is more important information to the issues raised by the survey than the
actual population numbers. While I recognize that you provide trend information
for the two populations of the Steller sea lion, you do not do so until page 6 of the
survey, three pages after the introduction of these influential and misleading
numbers.

2

•

On page 4, the second bullet after Q4 again promotes the false dichotomy of jobs
versus wildlife. The statement “…fish like pollock, mackerel, herring, cod, and
salmon that commercial fishermen catch for people to eat” is written using
language that risks biasing respondents in the same fashion as the earlier
statements about economic considerations. I hope that NMFS would not want bias
respondents to favor economic concerns over species protection, but this is likely
to be the effect of your use of this language here and elsewhere.

•

On page 6, in Q7, what is the point of the question? What will you do with the
information, and especially of what relevance to NMFS decision making will the
data be if they come back showing an overwhelming majority of respondents are
“not at all concerned” about the two stocks of Steller sea lions? As a general rule,
a survey ought to be transparent – that is, the utility of the questions ought to be
apparent to the respondent. In this and other questions, it is not clear how you
will use the data or of what utility it will be to NMFS, especially given the
statutory mandates of the ESA.

•

On page 7, Q8, I have the same concerns as those I state immediately above about
Q7. That is, what is the purpose of the question to NMFS’s decision making for
Steller sea lions and what will you make of the data if the responses come back
overwhelmingly concerned with the loss of jobs due to Steller sea lion protection?

•

On page 8, I have serious concerns about the statement “Doing more to protect
the Western stock of Steller sea lions will cost every U.S. household more
money” and the two bullets that follow it. My concerns are related to those I state
above – that the survey is unmistakably preoccupied with economic concerns and
with the economic and (ostensibly) other costs of protecting Steller sea lions, so
much so that its bias is unmistakable. By extension, the agency will be perceived
as biased in the same fashion, thereby fostering a sense among its constituents that
it is beholden to political influences that have pressured the agency to implement
an economic survey even though the data that will result from such a survey are
not viable data in the decision making process authorized under the ESA and
would likely spur litigation against the agency were it to attempt to use the survey
results as a basis for taking anything less than the strongest protective actions to
support Steller sea lion recovery. This bias is promoted so strongly in this survey,
and especially brought home in this passage on page 8 because the survey, while
it is repeatedly preoccupied with the economic costs of protecting Steller sea
lions, fails to address in even the mildest possible fashion the benefits of
protecting Steller sea lions. I expand on this concern in my comments on the
rationale for the survey, below.

•

On page 8, Q9, I have the same concerns I have stated regarding Q7 and Q8,
above. That is, what is the purpose of the question to NMFS’s decision making
for Steller sea lions and what will you make of the data if the responses come
back overwhelmingly against spending more money to protect the western stock
of Steller sea lions?

3

•

On page 9, Q10, I have the same concerns I have stated regarding Q7, Q8, and
Q9, above. That is, what is the purpose of the question to NMFS’s decision
making for Steller sea lions and what will you make of the data if the responses
come back overwhelmingly in favor of the option that costs less?

•

On pages 10, 11, and 12, the questions Q11, Q13, and Q14 are most likely too
complex for the general public to digest and make an educated response to – there
are simply too many variables at play in the matrix to have a high degree of
confidence in the responses.

•

On page 13, in Q15, the use of the phrasing “some people” is both awkward and
unprofessional in this context. There is no particular reason why the public – i.e.,
respondents – will relate to NMFS’s definition of “some people” or the opinion of
an unidentified group of people. Who are “some people” according to NMFS?
Are they a sample of the general public? If so, you should say so. Are they midlevel government bureaucrats? Are they academics? Are they paid survey-takers
with whom you pre-tested the survey? The term is simply not useful in instilling
any kind of confidence by the respondents that what “some people” say is
meaningful.

•

On page 13, again in Q15, the first three of the statement raise serious concerns
about your survey methodology. Taking each in order, my concerns are as
follows:
1. “I did not feel it was my responsibility to pay for the protection of Steller
sea lions….” This statement is misleading. Respondents’ conceptions of
responsibility will differ and may not accurately reflect what their actual
responsibility is to the protection of marine mammals. When citizens elect
their representatives to Congress, they both entrust and ask those
representatives to fashion and enact the laws of the United States. By
electing their representatives to Congress, the public (and thus we assume
your respondents) are accepting responsibility for the laws crafted by
those representatives. It is therefore all United States citizens’
responsibility to “pay” for Steller sea lion protection just as it is our
responsibility to pay for clean water or clean air, or to make government
buildings accessible to people with disabilities, or to maintain a strong
national defense. Because this is the actual responsibility that citizens have
to paying for Steller sea lion protection, but because you do not clarify
what is meant by the word “responsibility” in the question, your results
will be necessarily without context and thus meaningless as respondent
data.
2. “There was not enough information for me to make an informed choice
between the alternatives….” For good reason your respondents might feel
this way; you’ve given them a very narrow and stilted view of endangered
species protection with this survey. If the survey actually acts as an

4

introduction to endangered species protection for any respondents – which
it is bound to do – then you will have done them, the ESA, and the Steller
sea lion a great disservice through the survey’s narrow scope, biased
wording and phrasing, and poor execution. I expand on this point in my
comments about the rationale, below.
3. “The added costs I was willing to pay were just to protect Steller sea lions,
and not to protect other species….” The results of this question will not be
meaningful because you have not given respondents nearly enough of an
introduction to other species and their respective protection needs and you
have not asked respondents to describe the other protected species about
which they might already know and care. So, you have no common basis
for comparison of the results of this question. The data that results from
this question will therefore be without merit.
Second, the rationale. I will focus my comments on the rationale NMFS put forth in 71
FR 54472, which was a modification of the rationale offered in the original Federal
Register notice. In the modified rationale, NMFS states that
“The public benefits associated with the results of protection actions on the
endangered Western and threatened Eastern stocks of Steller sea lions (Eumetopias
jubatus), such as population increases, are primarily the result of the nonconsumptive value people attribute to such protection (e.g., active use values
associated with being able to view Steller sea lions and passive use values unrelated
to direct human use). Little is known about these values, yet such information is
needed for decision makers to more fully understand the trade-offs involved in
choosing among protection alternatives and to complement other information
available about the costs, benefits, and impacts of the protection alternatives.” (71 FR
54472)
This rationale offers a strong statement in support of collecting data about nonconsumptive use values. However, the draft survey itself fails to embrace the justification
given it, and instead reads as a subjective document designed to bias respondents into
favoring the perceived economic costs of sea lion recovery over actual recovery options.
As I indicate in my comments above, I find the survey instrument to be rife with
language and phrasing and choices of contextuality that appear to indicate NMFS’s
sympathy for (at the very least) exploring economic concerns over all other concerns, and
(at the very worst) seeking a justification for reducing Steller sea lion protection in the
face of mounting economic costs, or even the perception of such costs, associated with
protection. The survey and its justification fail in many ways to provide information
necessary to making the instrument a balanced, objective, and useful research tool for the
purposes of adding valuable knowledge to the species protection process. For example:
•

The survey is likely to play an important educational role among respondents who
will likely never have been asked about or even considered before “the nonconsumptive value people attribute” to marine mammal or endangered species
protection. And yet nowhere in the survey do you introduce any information to

5

provide respondents with the context of ESA mandates or policy prescriptions.
While you do provide a great many apparently biased (and often misleading)
statements on the economics costs or conflicts associated with endangered species
or Steller sea lion protection, you do not balance that information with contextual
statements on, for example, the purpose and requirements of the ESA or the
judicial history of ESA review that so strongly supports protection over economic
considerations (e.g., the landmark case of TVA v. Hill, 437 US 153 [1978]). It is
unclear why you do not include information that might provide respondents with a
more realistic understanding of the ESA policy arena, especially when you do
include information that is misleading and provides a false idea of the type of
economic conflicts that are created by species protection under the law.
•

In your justification for the survey, you prominently note “the non-consumptive
value people attribute to…protection” and how “little is known about these
values.” And yet, in the survey, you devote substantial time and space to
exploring the potential economic costs of protecting Steller sea lions and next to
none to exploring the many social, economic, psychological, and experiential
benefits of Steller sea lion (and other species) protection. There is a voluminous
literature on the benefits of species protection, as there is on the economic costs of
species protection. NMFS staff who are involved in writing and implementing a
survey such as this ought to be at least conversant if not expert in the literatures
associated with the fields of study of the benefits and costs of endangered species
protection. If they were – conversant or expert – they would recognize that (1) the
survey is severely lacking in its attention to the benefits of endangered species
protection and (2) most of the literature on economic conflicts with endangered
species protection betrays the bias of your survey and repudiates the false
assumptions it seems to promote about the economic costs of protection Steller
sea lions.

NMFS should recognize that presenting a survey that is both lacking a balanced approach
to the costs and benefits of species protection and apparently biased toward a view that
species protection carries serious economic costs is at its core a biased instrument. The
only explanation for the use of such a biased instrument would be to collect data for
which the agency wants to control the outcome – that is, a biased survey designed to
provide specific information desired by the agency under the guise of objectivity. I am
not cynical enough to believe that NMFS is interested in such a transparently crass
political goal as this, but I do hope that you will recognize that the survey as written will
not help to promote a clear and effective decision making process for Steller sea lion
protection. I would strongly recommend dropping the survey entirely due to its deeply
flawed nature and lack of overall utility to Steller sea lion protection and the goals of the
ESA.
Thank you for the opportunity to comment. Please don’t hesitate to contact me if you
have any questions.
Sincerely,

6

Richard L. Wallace, Ph.D.
Chair, Environmental Studies
Ursinus College
601 E. Main Street
Collegeville, PA 19426 USA
(610) 409-3730
(610) 409-3660 fax
rwallace@ursinus.edu

cc:

Timothy Ragen, Ph.D.
Acting Executive Director
Marine Mammal Commission

7

Response to Comments
Submitted from Richard L. Wallace, Ph.D., Ursinus College
on October 16, 2006
Overview
Dr. Wallace’s comments primarily fall into four general categories as discussed in this section.
Detailed comments are discussed in the next section.
1. Use of economics in ESA program evaluation. One thread underlying the comments is a
concern that economics should not be used in Endangered Species Act (ESA) program
evaluation. For Steller sea lions, we are not suggesting that economics will or should be the
deciding factor in program evaluation, but rather that economic costs and benefits (public
preferences) are among many useful sources of information that can be used in the evaluation
of alternative protection programs. In deciding between the available management actions to
protect Steller sea lions, policy makers must balance the ESA and Marine Mammal
Protection Act (MMPA) goals of protecting Steller sea lions from further declines with
providing for sustainable and economically viable fisheries mandated under the MagnusonStevens Fishery Conservation Act (P.L. 94-265). Since Steller sea lion protection is often
linked to fishery regulations, decision makers must comply with several federal laws and
executive orders in addition to the ESA and MMPA, including Executive Order 12866 (58
FR 51735), which requires regulatory agencies to consider costs and benefits in deciding
among alternative management actions, including changes to fishery management plans
made to protect Steller sea lions. Thus, under this executive order, decision makers need to
consider both the benefits and the costs associated with proposed actions, but are not
required to base their decisions on these considerations. This survey is being conducted to
provide information on the economic benefits associated with protecting Steller sea lions,
which is currently unavailable. Also, in contrast to Dr. Wallace’s comments, it should be
noted that economic considerations are explicitly included in the ESA. As noted by Gardner
Brown and Jason F. Shogren, under Section 4 of the ESA: 1
The Secretary of the Interior may “take into consideration the economic impact, and any other relevant
impact, of specifying any particular area as critical habitat” for a threatened or endangered species, and can
exclude an area from critical habitat designation if the benefits of exclusion outweigh the benefits to
specifying the critical habitat, “unless failure to designate leads to extinction.” (Brown and Shogren, 1998,
8)

2. Application of Economic Methods. Many of the comments are assertions that do not reflect
the abundant economics literature and applications of survey-based non-market valuation
methods. The surveys were developed (a) by published experts in the field following
standard methods, (b) reviewed in detail by nationally recognized non-market valuation
economic experts and by nationally recognized survey design experts who have worked with
non-market valuation methods, and (c) with multiple focus groups, one-on-one interviews,
and a formal pretest that were conducted to obtain and evaluate public input.
1

Brown, G. and J.F. Shogren (1998). “Economics of the Endangered Species Act,” Journal of Economic
Perspectives, 12(3): 3-20.

1

3. Design biases. Many comments express concerns that the specific survey content and
wording may bias the results toward lower values for alternative policies to protect Steller
sea lions. Several general points are important in response to these concerns:
a. To limit factual error, the content of the surveys were reviewed by NOAA program
scientists and managers for technical accuracy.
b. To eliminate presentation biases, the survey was reviewed by non-market valuation
economic experts and survey research experts.
c. Many of the comments about bias concern inclusion of material on the social or
financial costs of the ESA or of Steller sea lions protection, specifically impacts to
commercial fishing income and jobs (Page 3 of comments: “I hope that NMFS would
not want to bias respondents to favor economic concerns over species protection”).
While ideally we seek to measure only the preferences and benefits of protection (not
the benefits net of social impacts and costs, which may be evaluated separately),
entirely omitting or underplaying these potential impacts and costs: (i) does not make
these issues disappear for respondents who are concerned about them (in the design
testing, respondents would bring it up if we did not mention it), (ii) can create a
perception by respondents that the survey is not neutral in obtaining public
preferences and thus is biased in the opposite direction, (iii) can cripple the
investigators’ ability to detect differences in respondent attitudes about these impacts
and to account for co-mingled benefits and costs in the estimation of benefits, and (iv)
precludes the development of a realistic valuation scenario and mechanism for
respondents to pay for additional protection. The selected content on social impacts
and costs was included after careful attention to the matter in our review of the
literature, and following input from focus groups and one-on-one interviews with
members of the public.
d. An important design consideration in stated preference non-market valuation surveys
is that the survey should be cautious to avoid inflated values and should even error
toward understated values. Dr. Wallace’s suggestions to spend more time on ESA
mandates and benefits, and omitting any consideration of impacts and costs would
both be clearly upwardly biased, and would preclude economic measurement of
public preferences. Simply asking for respondents to indicate support for or against a
program without having realistic personal consequences, such as specified additional
cost to the household, has little defensibility within this framework.
4. Use of the information collected generally. Several comments surround the issue of how
information from the survey will be used. Specifically, Dr. Wallace raised concerns about
how information that appears to indicate a lack of concern or support for Steller sea lions or
similar sentiments will influence policy decisions. As suggested by the discussion in
Overview item #1 above, the information collected in the survey will be made available to
decision makers as an additional source of information that may possibly be used in the
evaluation of alternative protection programs. However, it is important to note that the
reporting of the survey results will provide any necessary caveats concerning the nature and
intent of asking the questions. In many cases, the questions that Dr. Wallace expressed
concern about are asked as cognitive prompts that aid respondents to process and review the
material they have been presented. These are critical for ensuring respondents read and

2

understand the content of the survey. Many of these questions also act as internal
consistency checks to ensure that a respondent’s responses to the valuation questions are
consistent with the attitudes and preferences they indicate in these questions. At the same
time, it is important to acknowledge that the nature of the actual responses (e.g., whether the
results indicate the public supports or does not support additional protection efforts) does not
affect the validity of the results. In our view, it is our responsibility to develop a valid survey
instrument for the purpose of estimating public preferences and values associated with Steller
sea lion protection, implement it in a way that is consistent with state-of-the-art methods in a
scientifically-defensible manner, and convey the results in a way that makes transparent any
assumptions and issues that would affect the interpretation of the results. Then, it is up to the
decision makers to decide whether or not and how to use the results, if at all, within the
confines of applicable laws and regulations.

Detailed Discussion
Below, we identify the location of specific comments in Dr. Wallace’s letter (denoted by italics
below) and briefly state the core of the comment and our response. 2
1. Page 1, paragraph 3, bullet 1. Dr. Wallace disputes the use of the term “costs” on survey
page 2, line 5. Replacing the word “cost” with “impacts” (as has been done in subsequent
edits) resolves the item. The comment that the original wording is “likely to bias the results”
is unsupported.
2. Page 2, bullet 1. This comment is about identifying a potential trade-off between species
protection and jobs in Q3 of the survey, which he suggests “is not well established” and that
the issue “must not be a concern of NMFS when the agency makes decisions about actions
under the ESA.” Irregardless of whether such a conflict exists or not (it is documented for
Steller sea lions) or is a factor in ESA decisions, understanding a respondent’s views on this
are beneficial in the non-market valuation (see the response under overview item #3c and
item #4). Respondents are asked this question for a couple reasons. First, it provides a
neutral perspective by acknowledging the issues many in the public raise themselves and lets
respondents express views on the issue early in the survey. And second, it is one of several
items used to help identify “protest” respondents who may mix protection concerns and
concerns about impacts and costs.
3. Page 2, bullet 2. This comment concerns the summary chart of seals and sea lions on survey
page 3. Dr. Wallace suggests the population numbers do not communicate the trend and
status of these populations. In fact, the chart and subsequent page do provide information on
both population trends and the threatened and endangered status for these species. This
presentation has well-grounded non-market valuation design objectives. The first is to put
the Steller sea lion population in perspective – it is not the only seal or sea lion, and it is not
the only one listed as threatened or endangered. Whether one agrees or not, for some
members of the public this is important information in forming their preferences about
additional Steller sea lion protection and without this information the survey would be
2

The numbering of comments is ours to facilitate review.

3

compromised as potentially overstating the importance of Steller sea lions. Second, the
presentation is also clear that Steller sea lions are the only seal or seal lion species presently
being actively evaluated for new protection actions, which lays the basis for why respondents
are asked to focus on this species.
4. Page 3, bullet 1. This comment regards the second bullet after Q4 on survey page 4 and
repeats Dr. Wallace’s concerns with the potential that the public may be concerned about
jobs as well as threatened and endangered species. Omitting this information does not make
it go away in the minds of respondents who are concerned with this issue (they bring it to the
survey on their own). This information is important as it develops the real-life link between
fish stocks and Steller sea lion stocks, and thus why actions to regulate fishing have occurred.
It also contributes to a realistic non-market valuation scenario of additional fishing
restrictions for additional protection, with additional costs paid by respondents.
5. Page 3, bullet 2. This comment concerns Q7 in the survey that asks about the respondent’s
level of concern for each Steller sea lion stock. Dr. Wallace states he does not know the use
of this question (given his view of statutory mandates), and expresses concern that
respondents may say they are “not at all concerned” about the Steller sea lion stocks. 3 This
question is used to support the respondent’s cognitive process of reviewing and evaluating
the survey material provided. It also provides a consistency check on subsequent valuation
responses (i.e., Are the valuation responses consistent with other attitudes in the survey?).
Thus, this question falls under the category of questions discussed in overview item #4
above.
6. Page 3, bullet 3. This comment is about Q8 in the survey, which asks for the respondent’s
level of concern about the impact on fishing. As in item 5 above, Dr. Wallace expresses
concern about the potential for undesirable results (Page 3 of comments: “…what will you
make of the data if the responses come back overwhelmingly concerned with the loss of jobs
due to Steller sea lion protection?”). The same responses apply here as in overview item #4
and detailed items #3, #4, and #5 above. That is, the survey is being neutral, it is setting up a
realistic valuation scenario, and NMFS is not seeking or expecting any specific type of
response from respondents beyond valid ones. The responses, whatever they may be, are
pieces of information that may be useful for decision makers to have available when
evaluating alternative protection actions.
7. Page 3, bullet 4. This comment concerns statements on page 8 of the survey. Dr. Wallace
expresses concern with the economic non-market valuation scenario developed because “the
survey is unmistakably preoccupied with economic concerns and the economic and
(ostensibly) other costs of protecting Steller sea lions”. This appears to repeat the general
concern discussed above in this response (overview item #1), and thus the same responses
apply. He further states the survey “fails to address in even the mildest possible fashion the
benefits of protecting Steller sea lions,” without acknowledging the discussion of benefits on
the top of page 2 in the survey, and that generally respondents bring to the survey a strong
sense of concern for ecologic protection.
3

As an aside, the pretest and interviews suggest that a substantial majority of the public’s responses do not support
Dr. Wallace’s concerns expressed about Q7, or later for Q8, Q9, or Q10.

4

8. Page 3, bullet 5. This comment repeats concerns about the purpose and use of the results for
Q9 in the survey, which asks whether respondents believe more should be done to protect the
Eastern and Western stocks (Page 3 of comments: “what will you make of the data if the
responses come back overwhelmingly against spending more money to protect the western
stock of Steller sea lions?”). This question and Dr. Wallace’s concern about it are addressed
by overview item #4 above.
9. Page 4, bullet 1. This comment repeats concerns about the purpose and use of Q10 of the
survey that deals with protection of the Western stock in some versus all habitat areas. Based
on pretest results suggesting limited relative importance of this characteristic vis-à-vis other
protection program characteristics, this aspect (and Q10) of the survey has been removed.
10. Page 4, bullet 2. This comment expresses concern that the stated choice (conjoint) questions
are too complex. It should be noted that the questions have been significantly simplified by
the elimination of the Western stock area attribute (row 3 under the Western stock in the
choice questions). 4 That said, the stated choice questions are certainly non-trivial, which is
why the scenarios are carefully developed on earlier pages, and the choice questions are
specifically explained on page 9. The stated choice methods are accepted and frequently
applied in market research, transportation choice, non-market valuation, and other
applications. The specific stated choice questions in this survey (including multiple
variations) are now generally less complex than in typical applications, and have been
thoroughly tested in the design process.
11. Page 4, bullets 3 and 4. Dr. Wallace takes issue with several aspects of Q15 in the survey.
He dislikes the phrase “some people” used in Q15. However, this wording was specifically
recommended to us by a survey design expert and acknowledges that different people have
different opinions, and has been successfully tested and used in this and other surveys. Dr.
Wallace then expresses a dislike for the response categories to the question. The categories
reflect traditional items included in stated preference follow-up evaluation, which are often
required by OMB in the survey approval process and are needed to evaluate valuation
responses 5 . The specific question items help identify the potential for rejecting the valuation
scenario (1st, 4th, and 5th items, used in conjunction with other survey data), poorly formed
responses (2nd item, used in conjunction with other survey data and response variance
analysis), and part-whole embedding bias (3rd item).
12. Page 5, middle to the end. The remainder of the comments expresses concern that the survey
may not be sufficiently extensive to fully educate the public about the ESA and may bias
values downward.

4

This attribute was removed in favor of applying study resources to the more important question of varying future
baselines with the current program actions (decreasing, stable, increasing population levels). The future baseline is
varied across respondents, not within a survey for the individual respondent, thus adding richness to the overall
investigation without adding complication to the respondents.
5
For the formal pretest conducted for this study, OMB specifically suggested these types of items be included in the
survey.

5

a. Dr. Wallace indicates “nowhere in the survey do you introduce any information to
provide respondents with the context of ESA mandates or policy prescriptions”
(pages 5-6 of comments). On the contrary, page 1 of the survey provides information
about the ESA mandate that “requires the federal government to take reasonable
actions to protect threatened and endangered species…” Going into more detail was
specifically addressed in focus group discussions and was ruled out as it
overburdened respondents without purpose.
b. He repeats concerns about the attention in the survey to the impacts and economic
costs associated with Steller sea lion protection. As noted above, this material is
necessary because we learned in focus groups and in one-on-one interviews that the
public will introduce it (explicitly or implicitly) in their responses with or without the
researcher raising the topics. It is better for the researcher to understand the
respondent’s views as part of the process of evaluating the valuation responses. Also,
as noted above, this information is part of the non-market scenario development
required for valuation.

6

Response to Comments
Submitted from the Humane Society of the United States
on October 16, 2006
Overview
The comments received from the Humane Society of the U.S. (HSUS) fall into four main
categories as delineated in the original comments and as discussed below.
1. The Stated Goals of the Survey are Inconsistent with the Language of the Questions Posed in
the Survey. The HSUS’ claim that the goals of the survey has changed is incorrect. The
goals of the survey have always been the same—to collect stated preference economic
information about respondents’ preferences and values related to outcomes of protection
actions on Steller sea lions. This information would then be made available to decision
makers as an additional source of information that may possibly be used in the evaluation of
alternative protection programs to supplement other information already available; although
whether or not the information is used, or is a factor in any decisions made, is solely up to the
decision makers. The original Federal Register (FR) notice (71 FR 47177) was not clear on
these points and was misconstrued by several readers to imply different goals. As a result,
the FR correction (71 FR 54472) was developed to clarify this point.
2. The Mandate to Recover Endangered Species is Independent of Cost. The HSUS expresses
concern that the results of the survey should not affect the mandates to protect Steller sea
lions. It should be made clear that the results are not intended to determine whether or not to
protect Steller sea lions at all. In fact, the information collected in the survey is intended to
supplement other information on the costs and impacts of the variety of protection
alternatives to help decision makers better evaluate the available options for protecting
Steller sea lions. Furthermore, we are not suggesting that economic considerations will or
should be the deciding factor in any Steller sea lion protection program evaluation, but rather
that economic costs and benefits (public preferences) are among many useful sources of
information that can be used in the evaluation of alternative protection programs. In deciding
between the available management actions to protect Steller sea lions, policy makers must
balance the Endangered Species Act (ESA) and Marine Mammal Protection Act (MMPA)
goals of protecting Steller sea lions from further declines with providing for sustainable and
economically viable fisheries mandated under the Magnuson-Stevens Fishery Conservation
Act (P.L. 94-265). Since Steller sea lion protection is often linked to fishery regulations,
decision makers must comply with several federal laws and executive orders in addition to
the ESA and MMPA, including Executive Order 12866 (58 FR 51735), which requires
regulatory agencies to consider costs and benefits in deciding among alternative management
actions, including changes to fishery management plans made to protect Steller sea lions.
Thus, under this executive order, decision makers need to consider both the benefits and the
costs associated with proposed actions, but are not required to base their decisions on these
considerations. This survey then is being conducted to provide information on the economic
benefits associated with protecting Steller sea lions, which is currently unavailable.

1

3. Inaccurate cost estimate. The HSUS objects to the reporting of $0 as the annual total cost
burden to the public in the FR notice. However, this is the correct cost burden to report in
the context of the FR notice. The notice was published as part of the Paperwork Reduction
Act (PL 104-13) process, which requires we notify the public of the cost burden on
respondents to the survey. Thus, the reported cost estimate does not refer to the costs of
conducting the study, which HSUS correctly points out are not $0, but rather to the monetary
cost completing the survey would have on respondents to the survey. Given the survey
would be mailed with a self-addressed and stamped return envelope, there are no out-ofpocket costs to the respondent. For more details of this reporting requirement, please see the
OMB Paperwork Reduction Act website:
http://www.whitehouse.gov/omb/inforeg/infocoll.html#PRA.
4. Comments on the Quality, Utility, and Clarity of the Information to be Collected. Many of
the comments under this heading are assertions that do not reflect the abundant economics
literature and applications of survey-based non-market valuation methods. The surveys were
developed (a) by published experts in the field following standard methods, (b) reviewed in
detail by nationally recognized non-market valuation economic experts and by nationally
recognized survey design experts who have worked with non-market valuation methods, and
(c) with multiple focus groups, one-on-one interviews, and a formal pretest that were
conducted to obtain and evaluate public input.
Some comments express concerns that the specific survey content and wording may bias the
results toward lower values for alternative policies to protect Steller sea lions due to an
emphasis on economic considerations. Several general points are important in response to
these design bias concerns:
a. To limit factual error, the content of the surveys were reviewed by NOAA program
scientists and managers for technical accuracy.
b. To eliminate presentation biases, the survey was reviewed by non-market valuation
economic experts and survey research experts.
c. Some of the comments about bias concern inclusion of material on the social or
financial costs of the ESA or of Steller sea lions protection. While ideally we seek to
measure only the preferences and benefits of protection (not the benefits net of social
impacts and costs, which may be evaluated separately), entirely omitting or
underplaying these potential impacts and costs: (i) does not make these issues
disappear for respondents who are concerned about them (in the design testing,
respondents would bring it up if we did not mention it), (ii) can create a perception by
respondents that the survey is not neutral in obtaining public preferences and thus is
biased in the opposite direction, (iii) can cripple the investigators’ ability to detect
differences in respondent attitudes about these impacts and to account for co-mingled
benefits and costs in the estimation of benefits, and (iv) precludes the development of
a realistic valuation scenario and mechanism for respondents to pay for additional
protection. The selected content on social impacts and costs was included after
careful attention to the matter in our review of the literature, and following input from
focus groups and one-on-one interviews with members of the public.

2

d. An important design consideration in stated preference non-market valuation surveys
is that the survey should be cautious to avoid inflated values and should even error
toward understated values. HSUS’ suggestions to spend more time on ESA mandates
and benefits, include “facts that are likely to endear these animals to the public such
as maternal care and defense” (page 4 of comments), and omitting any consideration
of impacts and costs would both be clearly upwardly biased, and would preclude
economic measurement of public preferences. Simply asking for respondents to
indicate support for or against a program without having realistic personal
consequences, such as specified additional cost to the household, has little
defensibility within this framework.
Below, we consider specific comments not covered by the above general response to design
bias concerns. We identify the location of specific comments made by HSUS (denoted by
italics below) and briefly state the core of the comment and our response.
a. Page 3, second to last paragraph. HSUS states the survey “seems designed to
assess the public’s attitudes toward paying to protect sea lions rather than the
newly stated purpose of evaluating ‘active use values associated with being able
to view Steller sea lions and passive use values unrelated to direct human use.’”
This concern may have arisen due to unfamiliarity with how information from
questions in non-market valuation surveys generally, and this survey specifically,
are used to estimate total economic values that embody viewing benefits and
nonuse benefits. Responses to the stated choice questions (Q11, Q13, and Q14)
provide information about each respondent’s preferences with respect to different
goals of Steller sea lion protection, such as increasing the Western stock
population size versus increasing the Eastern stock population size. These
responses are analyzed using econometric models that describe the choices that
are observed and result in a valuation function that reflects the public’s
preferences and can be used to estimate economic values, such as those described
above, and to evaluate trade-offs between competing protection objectives (e.g.,
preferences for increasing Western stock abundance versus increasing Eastern
stock abundance).
b. Page 3, bottom. The HSUS appears concerned that no mention was made about
the Marine Mammal Protection Act (MMPA) and its mandates or additional
details about the ESA. The design bias concerns response (d) above addresses
this issue. Also, going into more detail about these laws was specifically
addressed in focus group discussions and was ruled out as it overburdened
respondents without purpose.
c. Page 4, first full paragraph. The HSUS expresses concern that the facts provided
about Steller sea lions will bias respondents by only providing facts that “portray
Steller sea lions as an economic nuisance”. The design bias concern response (d)
above addresses this issue.
d. Page 4, second full paragraph. HSUS is concerned that in the survey version
they reviewed, the projected population trajectory for the Western stock is stable,
which may be misleading as it “misrepresents the current situation and will
inappropriately reduce respondents’ concern about the viability of the species. In

3

e.

f.

g.

h.

i.

turn, it will be all too easy to misinterpret the findings as a lack of public
concern.” We agree that this is but one possible realization of the future
abundance of the Western stock. However, as noted in the e-mail accompanying
the draft survey, there are several other survey versions that differ in the
trajectories that are presented from the one discussed by HSUS. One presents a
declining population trend, and the other an increasing population trend.
Individuals in the sample will receive one of these three survey versions. By
accounting for the uncertainty associated with future abundance estimates of the
Western stock in different survey versions, we can explicitly account for this
uncertainty in the model framework, thus adding richness to the overall
investigation without adding complication to the respondents.
Page 4, third full paragraph. The HSUS expresses concern that information on
page 7 of the survey is “designed to elicit responses that will express sympathy
for the cost to fisheries over the damaging effects to the species.” The design bias
concern response (c) above addresses this issue.
Page 4, last paragraph. This comment is about objections to underlining of
“higher prices for fish and fish products you buy” and “increases in your federal
taxes” on page 9 of the survey. As implied by responses (c) and (d) of the design
bias concerns response, a key component of stated preference survey design is
developing a realistic valuation scenario and mechanism for respondents to pay
for additional protection. The underlining is used to ensure respondents pay
attention to the means through which new protection actions would have personal
consequences on them, which was found beneficial to respondents in focus
groups.
Page 5, first full paragraph. The HSUS’ concern that the stated preference
questions in the survey do “not allow the respondent to separate the cost of the
unspecified recovery efforts for Eastern Steller sea lions (which they are told are
increasing) against the cost for recovery efforts focused on the endangered
Western stock” is again based on unfamiliarity with how responses to these
questions are analyzed. Response (a) in this section addresses this issue.
Page 5, second full paragraph. HSUS mentions that the presented future
trajectories for the abundances of the two stocks are uncertain. As noted in
response (d) in this section above, this uncertainty is handled through different
survey versions that portray differing assumptions about future stock abundance.
This allows uncertainty to be explicitly incorporated in the model framework.
Page 5, above the Conclusion. HSUS questions the utility and function of Q16,
which asks respondents for their confidence in answering the stated preference
questions, calling the question “nonsensical”. In fact, Q16 has a very specific and
important function. Combined with other answers in the survey, we will use
answers to this question to evaluate the impact of uncertainty on valuation results,
such as the mean and variance of estimated values. Results can also be reported
with and without respondents who self-report that they are “not at all confident”
in their answers.

4

Marine Mammal Commission
4340 East-West Highway, Room 905
Bethesda, MD 20814

30 October 2006
Ms. Diana Hynek
Departmental Paperwork Clearance Officer
Department of Commerce
14th and Constitution Avenue, NW
Room 6625
Washington, DC 20230
Dear Ms. Hynek:
The Marine Mammal Commission, in consultation with its Committee of Scientific Advisors
on Marine Mammals, has reviewed the National Marine Fisheries Service’s proposed survey for
measuring the preferences of U.S. residents regarding programs for the conservation and recovery of
the Steller sea lion as described in the Service’s 16 August 2006 Federal Register notice. The
Commission appreciates the importance of generating useful information for decision-makers but
has several concerns about the design of the proposed survey. We provide the following general and
specific comments that we hope will be helpful.
GENERAL COMMENTS
First and foremost, we question whether the data generated by the survey will be useful.
Because participants in the survey are being selected randomly, many, if not most, will be unfamiliar
with the mandates of the Endangered Species Act and the specifics of the situation involving Steller
sea lions. Thus, the survey results will likely reflect the first impressions of relatively uninformed
citizens based on limited, and sometimes misleading, information involving fairly complicated issues.
We also question the utility of seeking public opinion, not on the general mandates and goals
of a broadly applicable statute such as the Endangered Species Act, but on whether or how that Act
should be implemented on a species-specific basis. Should the Service, for example, base its
decisions on what is needed to achieve the recovery goals of the Act and its allocation of endangered
species recovery funds on the popularity of the various species?
In some respects, the survey is designed to be a referendum on the mandates of the
Endangered Species Act and the priorities placed on achieving its goals. In others, the questions are
designed more to elicit what the average citizen is willing to spend (or forego) in furtherance of
conserving Steller sea lions. By intermingling these objectives, the survey does not do a very good
job of achieving either. If, in fact, the survey is intended to provide public opinion on the general
directives of the Act, additional information on its provisions and rationale need to be provided. In
addition, such questions should be couched in terms of protecting endangered species and
ecosystems generally and should not be raised in the context of a single species. If, on the other
hand, the survey is intended to apply only to Steller sea lions, it should begin by setting forth the

Ms. Diana Hynek
30 October 2006
Page 2
mandates of the Act, and the questions should be geared toward eliciting views on the best ways to
achieve those goals.
In the Commission’s view, the survey seems inconsistent with the broad and farsighted
findings and purposes of the Endangered Species Act. The survey presents inaccurate and
insufficient information; seems inappropriately to lead respondents to particular conclusions;
misrepresents the complexity of the issues involving Steller sea lion status and conservation; appears
to assume that the only values of consequence are short-term economic ones; proposes actions that
appear inconsistent with or contrary to the provisions of the Endangered Species Act and the
Marine Mammal Protection Act; and promotes a utilitarian perspective rather than a rigorous
science-based search for solutions to difficult conservation problems. The superficiality and
inaccuracy of the survey design seem likely to produce responses that could detrimentally affect the
conservation and management of fisheries and marine ecosystems. More significantly, the survey
seems to challenge the fundamental premises of major federal statutes, including the MagnusonStevens Fishery Conservation and Management Act, the Endangered Species Act, the Marine
Mammal Protection Act, and the National Environmental Policy Act.
We also question the scale of the suggested economic consequences in several of the
questions and the level of detail in how those impacts might affect particular households. Also, the
survey does not indicate what is being spent on Steller sea lion conservation under the current
program. This would be useful background information for assessing the various alternatives.
Over the past several years, funding for the National Marine Fisheries Service’s Steller sea
lion program has varied considerably, from about $3 million in 1998 to more than $40 million in
2001. Similarly, expenditures that the Coast Guard attributes to enforcement related to Steller sea
lions have varied considerably from year to year, reaching a high of just under $40 million in 2003.
Total federal and state expenditures on Steller sea lion programs peaked in 2002 at about $56
million. Current census figures indicate that there are about 109 million households in the United
States. As such, the maximum amount spent in any year on Steller sea lion conservation has been on
the order of 50 cents per household. Against this background, it seems incongruous to be asking
survey participants whether they would be willing to spend an additional 10, 40, or 80 dollars per
year on sea lion recovery. Is the Service truly suggesting that optimal Steller sea lion recovery
programs will cost $8.7 billion per year over the next 20 years? If so, additional justification for the
amounts suggested and the predicted outcomes is needed. Presumably, you would elicit a very
different response if you asked participants whether they would be willing to spend an additional 50
cents a year on Steller sea lion conservation. This amount would about double the maximum
amount of federal and state expenditures in any one year over the past decade.
The questions related to costs seem to assume that potential increases in the cost of fish will
be shared equally among consumers. It should be recognized that these costs would vary regionally
and among households. In fact, much of the fish caught in the area inhabited by the western stock
of Steller sea lions is marketed overseas. As such, it might be more appropriate to ask if the
participant cares whether a Japanese consumer has to pay more for fish from the United States if

Ms. Diana Hynek
30 October 2006
Page 3
any such cost has only a small impact on the income of U.S. fishermen and contributes to the
conservation of endangered and threatened species.
SPECIFIC COMMENTS
Question 1 appears to be a referendum on the Endangered Species Act. As noted above, if
this is one purpose of the survey, the background information provided is insufficient to generate
informed opinions. Further, although it is reasonable to assume that respondents will have various
opinions regarding the merits of the Act, it is not clear to us why the National Marine Fisheries
Service would be seeking such information as a basis for determining whether it should carry out its
responsibilities under the Act.
Question 2 suggests that agencies responsible for implementing the Endangered Species Act
should determine the rigor with which they do so based on other considerations, such as the state of
highways and roads, etc. The implication appears to be that laws should be rigorously implemented
only if the issues they address are rated as high priority or—conversely—those deemed of lesser
priority do not warrant implementation. Also, if this question is retained, it could be made more
useful by asking related questions as to what the participants think the United States is currently
doing to protect endangered and threatened species. It is of little value that someone thinks we
should be doing more, less, or the same if they do not know what we are doing now.
Question 3 suggests that whether or not our conservation laws are implemented by the
responsible agencies is simply a matter of the effects on jobs—and that protection of threatened and
endangered species is always a trade-off resulting in a reduction of jobs. This seems a great
oversimplification that may be true in some cases but certainly not true in all cases.
Question 4 is preceded by information including estimates of sea and sea lion abundance,
but the information is incorrect and highlights seal species with large abundances, perhaps giving a
false impression regarding overall status of seals and sea lions. The information is incorrect with
regard to the trend in Hawaiian monk seals and the combined abundance of other seals. Further, the
Steller sea lion is not the only seal or sea lion for which new protection efforts are being considered.
As the Service should be fully aware, new protection measures are being considered for the
Hawaiian monk seal (which, contrary to the information in the survey, is continuing to decline).
Question 7, and the information preceding it, give one possible future scenario for Steller sea
lions, but there are others that may be equally likely. Scientists have documented, but cannot explain,
an 80 percent decline in the western stock of Steller sea lions over the past three or four decades. To
suggest that they have a reliable basis for projecting the trend in sea lions over the next 35 years
presents a misleading representation of our understanding of sea lion status.
In the information preceding question 8, the second bullet suggests that fishing is not
considered a major problem in the area where the eastern stock occurs. Is it that fishing occurs in a
manner similar to that in the Gulf of Alaska and Bering Sea and doesn’t have effects, or is
commercial fishing in the southeast not comparable to that in the areas occupied by the western

Ms. Diana Hynek
30 October 2006
Page 4
stock? These alternative explanations could result in misinterpretation and misunderstanding by
persons taking the survey.
Question 8 states that fishing restrictions to help conserve Steller sea lions have made fishing
more costly. Again, this is not always the case. When measures were imposed on the Bering Sea
pollock fishery to spread fishing effort over time and space, some of the large factory trawlers
formed a cooperative that (1) established a joint strategy for dividing their catch allocation, (2) ended
the race for fish (thereby making fishing safer), (3) distributed their fishing effort over time on a
more rational basis (allowing fishing to occur when the target fish stocks were in the best condition),
and (4) experienced a year of fishing that was profitable well beyond their expectations. So it is not
always true that fishing costs more because of Steller sea lion measures. Also, as indicated above,
those costs may not be borne by all consumers equally or, for that matter, even by U.S. consumers.
This should be explained.
Question 8 presents costs only for conservation measures and therefore seems entirely onesided in its perspective. Those purported costs are oversimplified, not necessarily true, and should
be backed up by analysis and verification. To be well balanced, the question might also have
included benefits of conservation measures, such as the likelihood of a more stable, functioning
ecosystem, opportunities for tourism, and a decreased probability of further decline or extinction of
sea lions.
The information preceding question 9 is also misleading. It states that scientists believe that
protection, enforcement, and monitoring actions will have little impact on other species. First, some
substantiation of that claim seems necessary. Second, it seems clear that the potential effects of
oceanographic regime shifts, fisheries, and killer whale predation—hypotheses raised to explain the
Steller sea lion decline—all may have bearing on the status of northern fur seals, which are
continuing to decline in the same region, may be subject to similar risk factors, and may experience
some benefit from suitable measures to protect sea lions.
Question 9 seems to suggest that, despite the directives of the Endangered Species Act and
the Marine Mammal Protection Act, we ought to be able to pick and choose which populations to
protect and which to ignore into extinction. The implication violates not only the spirit of
conservation generally but the statutory requirements developed and enacted by Congress to guide
domestic conservation programs.
Question 10 again suggests that there is some background analysis, rather than mere
speculation, that costs of protection will be greater in the Aleutian Islands and that the purported
difference in cost is a basis for dismissing protection and conservation measures in that region. This
question suggests that the Service is considering dismissing the requirements of the Endangered
Species Act and the Marine Mammal Protection Act and allowing the decline and extirpation of
Steller sea lions throughout the Aleutian Islands. Furthermore, the question does not, but should,
explain that there are potentially significant conservation benefits that arise from retaining Steller sea
lions throughout their existing range.

Ms. Diana Hynek
30 October 2006
Page 5
Questions 11, 13, and 14 suggest a set of alternative choices that link costs to the number of
sea lions. We know of no bases for these cost estimates and their linkages to the number of sea
lions. They appear to be entirely hypothetical or speculative. These questions imply a degree of
management control that is entirely inconsistent with our past experience with this conservation
challenge. Moreover, as discussed above, these estimates appear to be orders of magnitude higher
than seem warranted in light of recent costs of Steller sea lion conservation programs.
The information leading to questions 11, 13, and 14 is, again, simplistic and biased. That
information states that the survey respondent should “[r]emember, if you spend money for [sea lion
conservation], it won’t be available to buy other things.” Might it also remind readers that if they are
willing to support conservation measures for sea lions, their contribution might help to conserve
functioning ecosystems and thereby provide a more sustainable world for future generations? Recall
that the Endangered Species Act states that there are numerous values associated with effective
conservation.
We know that, as the lead agency responsible for recovery of the Steller sea lion, the Service
is faced with a great many challenges and difficult choices. However, we also expect that the
information provided and the choices made will reflect an appropriately broad perspective that is
based on the best available information and that reflects a clear focus on the responsibilities
entrusted to the Service by the Endangered Species Act and the Marine Mammal Protection Act. We
question whether the survey as currently designed is likely to obtain the information necessary to
further the goals of these statutes.
Sincerely,

Timothy J. Ragen, Ph.D.
Executive Director
cc:

Douglas P. DeMaster, Ph.D.
William T. Hogarth, Ph.D.
Daniel K. Lew

Response to Comments
Submitted from the Marine Mammal Commission
on October 31, 2006
Overview
Comments received from the Marine Mammal Commission (MMC) primarily fall into three
general categories as discussed in this section. Detailed comments are discussed in the next
section.
1. Use of the information collected generally. Several comments surround the issue of how
information from the survey will be used. Specifically, the MMC raises concerns about how
information from the survey will be used to influence policy decisions. For Steller sea lions,
we are not suggesting data collected in the survey or estimated from the data will or should
be the deciding factor in program evaluation, but rather that economic costs and benefits
(public preferences) are among many useful sources of information that can be used in the
evaluation of alternative protection programs. In deciding between the available
management actions to protect Steller sea lions, policy makers must balance the ESA and
Marine Mammal Protection Act (MMPA) goals of protecting Steller sea lions from further
declines with providing for sustainable and economically viable fisheries mandated under the
Magnuson-Stevens Fishery Conservation Act (P.L. 94-265). Since Steller sea lion protection
is often linked to fishery regulations, decision makers must comply with several federal laws
and executive orders in addition to the ESA and MMPA, including Executive Order 12866
(58 FR 51735), which requires regulatory agencies to consider costs and benefits in deciding
among alternative management actions, including changes to fishery management plans
made to protect Steller sea lions. Thus, under this executive order, decision makers need to
consider both the benefits and the costs associated with proposed actions, but are not
required to base their decisions on these considerations.
As described in the Federal Register correction (71 FR 54472), the goal of the survey is to
collect stated preference economic information about respondents’ preferences and values
related to outcomes of protection actions on Steller sea lions sufficient to “estimate the nonconsumptive benefits associated with the results of protection actions on Steller sea lions.”
This information would then be made available to decision makers as an additional source of
information that may possibly be used in the evaluation of alternative protection programs to
supplement other information already available; although whether or not the information is
used, or is a factor in any decisions made, is solely up to the decision makers. The original
Federal Register (FR) notice (71 FR 47177) was not clear on these points and was
misconstrued by several readers to imply different goals. As a result, the FR correction was
developed to clarify this point.
It is important to note that the reporting of the survey results will provide any necessary
caveats concerning the nature and intent of asking the questions. In many cases, the
questions that the MMC expressed concern about are asked as cognitive prompts that aid
respondents to process and review the material they have been presented. These are critical
for ensuring respondents read and understand the content of the survey. Many of these

1

questions also act as internal consistency checks to ensure that a respondent’s responses to
the valuation questions are consistent with the attitudes and preferences they indicate in these
questions. At the same time, it is important to acknowledge that the nature of the actual
responses (e.g., whether the results indicate the public supports or does not support additional
protection efforts) does not affect the validity of the results. In our view, it is our
responsibility to develop a valid survey instrument for the purpose of estimating public
preferences and values associated with Steller sea lion protection, implement it in a way that
is consistent with state-of-the-art methods in a scientifically-defensible manner, and convey
the results in a way that makes transparent any assumptions and issues that would affect the
interpretation of the results. Then, it is up to the decision makers to decide whether or not
and how to use the results, if at all, within the confines of applicable laws and regulations.
2. Application of economic methods. In general, a number of the comments are assertions that
do not reflect the abundant economics literature and applications of survey-based non- market
valuation methods. The surveys were developed (a) by published experts in the field
following standard methods, (b) reviewed in detail by nationally recognized non- market
valuation economic experts and by nationally recognized survey design experts who have
worked with non- market valuation methods, and (c) with multiple focus groups, one-on-one
interviews, and a formal pretest that were conducted to obtain and evaluate public input. In
this survey, we employ stated preference choice methods to elicit economic preference, or
value, information from respondents. The methods are accepted and frequently applied in
market research, transportation choice, non- market valuation, and other applications.
Responses to the stated choice questions (Q11, Q13, and Q14) provide information about
each respondent’s preferences with respect to different goals of Steller sea lion protection,
such as increasing the Western stock population size versus increasing the Eastern stock
population size. These responses are analyzed using econometric models that describe the
choices that are observed and result in a valuation function that reflects the public’s
preferences and can be used to estimate economic values, such as those described above, and
to evaluate trade-offs between competing protection objectives (e.g., preferences for
increasing Western stock abundance versus increasing Eastern stock abundance).
3. Design biases. Several comments express concerns that the specific survey content and
wording may bias the results or be misleading. Several general points are important in
response to these concerns:
a. To limit factual error, the content of the surveys was reviewed by NOAA program
scientists and managers for technical accuracy.
b. To eliminate presentation biases, the survey was reviewed by non-market valuation
economic experts and survey research experts.
c. Many of the comments about bias concern inclusion of material on the social or financial
costs of the ESA or of Steller sea lions protection, specifically impacts to commercial
fishing income and jobs (Page 3 of comments: “Question 8 presents costs only for
conservation measures and therefore seems entirely one-sided in its perspective.”).
While ideally we seek to measure only the preferences and benefits of protection (not the
benefits net of social impacts and costs, which may be evaluated separately), entirely
omitting or underplaying these potential impacts and costs: (i) does not make these

2

issues disappear for respondents who are concerned about them (in the design testing,
respondents would bring it up if we did not mention it), (ii) can create a perception by
respondents that the survey is not neutral in obtaining public preferences and thus is
biased in the opposite direction, (iii) can cripple the investigators’ ability to detect
differences in respondent attitudes about these impacts and to account for co- mingled
benefits and costs in the estimation of benefits, and (iv) precludes the development of a
realistic valuation scenario and mechanism for respondents to pay for additional
protection. The selected content on social impacts and costs was included after careful
attention to the matter in our review of the literature, and following input from focus
groups and one-on-one interviews with members of the public.
d. An important design consideration in stated preference non- market valuation surveys is
that the survey should be cautious to avoid inflated values and should even error toward
understated values. The MMC’s suggestions to spend more time on ESA mandates and
benefits, and downplaying considerations of impacts and costs would both be clearly
upwardly biased, and would preclude economic measurement of public preferences.
Simply asking for respondents to indicate support for or against a program without
having realistic personal consequences, such as a specified additional cost to the
household, has little defensibility within this framework.

Detailed Discussion
Below, we identify the location of specific comments (denoted by italics below) and briefly state
the core of the comment and our response. 1
1. Page 2, comment about including costs of protection. The MMC suggests including
information about how much has been spent on Steller sea lion protection. The survey is
aimed at understanding how much the public values protection of Steller sea lions
independent of the costs of that protection. Thus, inclusion of specific protection costs would
likely bias responses to the valuation questions and preclude measurement of public
preferences. 2
2. Page 2, last paragraph. The MMC comments that “the questions related to costs seem to
assume that potential increases in the cost of fish will be shared equally among consumers.”
The survey is set up so that respondents can interpret the cost to them as a personal
household cost that may be different from another household’s since the payment mechanism
is described as a combination of increased taxes and higher prices for fish and fish-related
items they buy. Thus, there is an explicit recognition of differing costs among households.
3. Page 3, comment on Q1. The MMC is unclear about why a question that asks for how
people view the Endangered Species Act (ESA) is being asked and expresses concern that the
responses will be used “as a basis for determining whether [the Agency] should carry out its
1

The numbering of comments is ours to facilitate review.
When presented program costs, respondents often “cost-calculate” an average household share. So long as the
cost-calculated amount is less than or equal to their WTP, they then anchor on this value for reported values rather
than revealing their WTP, thus typically biasing values downward.
2

3

responsibilities under the Act”. Q1 is used to identify respondents’ general feelings toward
endangered species protection. It provides an easy start to the process of thinking about
threatened and endangered species, and it sets a tone of neutrality by allowing positive and
negative reactions right from the start. In initial testing and from the pretest implementation
results, responses to this question were good predictors of how respondents would answer the
stated preference valuation questions. It thus provides a consistency check on subsequent
valuation responses (i.e., Are the valuation responses cons istent with other attitudes in the
survey?). As noted above in overview item 1, the information collected in the survey is not
intended to determine whether or not to carry out its responsibilities under the Act, but rather
to supplement other information available to decision makers who must evaluate available
protection actions.
4. Page 3, comment on Q2. The MMC appears concerned that the purpose of Q2 is to
“determine the rigor with which” agencies implement laws, particularly the ESA, based on
the prioritization implied by responses to this question. In fact, the question is asked to put
the issue of protecting threatened and endangered species in the context that there are many
social issues (each with costs), and thus to reduce survey “importance bias” and the resultant
inflating of stated values (as discussed above). This type of bias is prevalent in non- market
valuation surveys that do not provide sufficient context or reminders for respondents that
there are other issues that may be important to them.
5. Page 3, comment on Q3. This comment expresses concern over asking respondents about
their opinions regarding a trade-off between protecting threatened and endangered species
and job losses. As discussed in the overview item 1 above, omitting this is sue does not make
it go away in the minds of respondents who are concerned with it (they bring it to the survey
on their own). It is better for the researcher to understand the respondent’s views as part of
the process of evaluating the valuation responses. Additionally, it provides a neutral
perspective by acknowledging the issues many in the public raise themselves and lets
respondents express views on the issue early in the survey. And, it is one of several items
used to help identify “protest” respondents who may mix protection concerns and concerns
about impacts and costs.
6. Page 3, comment on information preceding Q4. The MMC appears concerned that the
population numbers on page 3 of the survey for seals and sea lions are inaccurate. The seal
and sea lion population estimates in the survey are based on the latest stock assessment
reports. Still, we would appreciate MMC’s input on what the appropriate number is for the
aggregated “Other” seal and sea lions on page 3, which is a very conservative estimate that
omits speculation about species with unknown population sizes. The MMC’s concern about
the use of the term “new protection efforts” as applied to only Steller sea lions is noted;
however, as used in the survey, the term does not apply to implementation of existing
protection actions, such as those being implemented to protect the Hawaiian monk seal. Note
that this presentation (on pages 3 and 4 of the survey) has well- grounded non- market
valuation design objectives. The first is to put the Steller sea lion population in perspective –
it is not the only seal or sea lion, and it is not the only one listed as threatened or endangered.
Whether one agrees or not, for some members of the public this is important information in
forming their preferences about additional Steller sea lion protection and without this

4

information the survey would be compromised as potentially overstating the importance of
Steller sea lions. Second, the statement that Steller sea lions are the only seal or seal lion
species presently being actively evaluated for new protection actions lays the basis for why
respondents are asked to focus on this species. In addition, MMC is concerned that the
survey portrays the Hawaiian monk seal population as stable or increasing when in fact they
continue to decline. However, the wording in the survey related to the Hawaiian monk seal
population states that it “is small and decreasing”.
7. Page 3, comment on Q7. MMC is concerned that in the survey version they reviewed, only
one possible future population trajectory for the Western stock is presented. We agree that
this is but one possible realization of the future abundance of the Western stock. However,
as noted in the e- mail accompanying the draft survey, there are several other sur vey versions
that differ in the trajectories that are presented from the one seen in the survey reviewed by
MMC. One presents a declining population trend, and the other an increasing population
trend. Individuals in the sample will receive one of these three survey versions. By
accounting for the uncertainty associated with future abundance estimates of the Western
stock in different survey versions, we can explicitly account for this uncertainty in the model
framework, thus adding richness to the overall investigation without adding complication to
the respondents.
8. Page 3, comment on information preceding Q8. This comment concerns the second bullet
statement that reads in part: “commercial fishing is not considered a major problem where
the Eastern stock lives”. The MMC suggests inclusion of more details about this statement
(“Is it that fishing occurs in a manner similar to that in the Gulf of Alaska and Bering Sea and
doesn’t have effects, or is commercial fishing in the southeast not comparable to that in the
areas occupied by the western stock?”), and expresses concern that the current wording will
“result in misinterpretation and misunderstanding” by respondents. The statement is made to
let respondents know that fishing activities have not been identified as a threat to Steller sea
lions in the Eastern stock habitat, which is consistent with the most current information (e.g.,
the draft SSL recovery plan). It is not clear how more information along the lines MMC
suggests would be beneficial to respondents, particularly in light of the fact that
overburdening respondents with information generally leads to lower response rates and
lower response quality. The information presented was selected based on focus group
investigations, with the study objectives in mind.
9. Page 4, first comment on Q8. MMC states that “it is not always true that fishing costs more
because of Steller sea lion measures.” While the MMC comment is true, it misses the point
of the valuation scenario design, which is to establish credible payment scenarios with
respondent responsibility. The information presented is consistent with reported estimates.
Additionally, suggesting impacts to commercial fishing may or may not occur adds
uncertainty to the scenario, which can be expected to compromise the valuation; some
respondents may choose to report $0 or low values because it is not certain that it will (or
should) cost them anything, thus again compromising the realism and personal responsibility
elements of the valuation scenario.

5

10. Page 4, second comment on Q8. MMC suggests the presentation of “costs only for
conservation measures…seems entirely one-sided”. This concern is discussed in overview
item 3 above.
11. Page 4, comment on information preceding Q9. MMC is concerned that the statement that
“scientists believe the actions being considered will have little impact (good or bad) on other
species” is misleading. In focus groups, respondents sometimes wondered whether changes
in Steller sea lion populations would cause a chain reaction in the food chain leading to
ecosystem collapse. The bulleted statement is made to address the concern of whether or not
SSLs are a keystone species. The statement allows respondents to focus on assessing their
feelings about changes in SSL populations without thinking that changes in SSL populations
will cause the ecosystem to fundamentally change. Note that the statement does not preclude
impacts associated with protection actions on other species, but does suggest that the current
thinking is the effects would be small.
12. Page 4, comment on Q9. This comment repeats concerns about the purpose and use of the
results for Q9 in the survey, which asks whether respondents believe more should be done to
protect the Eastern and Western stocks. This concern is addressed by overview item #1
above, as this question is primarily used to check for consistency of attitudes expressed in the
survey with the responses to stated preference choice questions.
13. Page 4, comment on Q10. This comment expresses concerns about the purpose and use of
Q10 of the survey that deals with protection of the Western stock in some versus all habitat
areas. Based on pretest results suggesting limited relative importance of this characteristic
vis-à-vis other protection program characteristics, this aspect (and Q10) of the survey has
been removed.
14. Page 5, comments on Q11, Q13, and Q14. MMC is critical of the set of choices, particularly
cost estimates, included in the version of the survey they reviewed, stating, “We know of no
bases for these cost estimates…” As described in the overview items above, stated
preference choice questions are intended to measure a respondent’s preferences (i.e.,
economic benefits) associated with protecting Steller sea lions. The public benefits
associated with protection of Steller sea lions are independent of the costs of that protection.
Thus, how much individuals are willing to pay for such protection is not bound by the actual
costs, but is bound by their ability to pay (i.e., their income). As a result, cost amounts in
stated preference surveys are determined based on the likely distribution of the economic
benefits. To estimate values across the population, the amounts presented must vary widely
across respondents. The cost amounts presented in the survey are based on pretesting results.
15. Page 5, comments on instructions for Q11, Q13, and Q14. The MMC considers the budget
reminder statement (“Remember, if you spend money on this, it won’t be available to buy
other things.”) “biased” and “simplistic”. In stated preference valuation surveys, budget
reminders are standard elements in the design, are included to ensure respondents consider
the personal consequences of their choices, and are required by OMB in the survey approval
process (see OMB survey guidance document accessible from
http://www.cio.noaa.gov/itmanagement/pra.htm).

6