Justification for Non-Substantive Change

Justification for Non-Substantive Change

EPA ICR 2456.02, OMB 2010-0043

Willingness to Pay Survey for Improved Water Quality in the Chesapeake Bay

EPA submitted an Information Collection Request (ICR), titled “Willingness to Pay for Water Quality Improvements in the Chesapeake Bay” for review and approval to OMB according to the procedures prescribed in 5 CFR 1320.12. OMB approved a pretest of the survey with a limited sample size and requested that EPA report the results of the pretest to OMB prior to receiving approval for the implementation of the main survey. EPA concluded the pretest on February 14, 2014 and subsequently conveyed the results to OMB demonstrating the validity of the survey. EPA requests approval from OMB to implement the main survey with the following non-substantive changes to the survey instrument and experimental design.

1. Omit the 2040 time horizon

Before conducting the pretest EPA conducted extensive outreach, publishing a Federal Register notice announcing the information collection request and soliciting public comments on the stated preference study. One of the comments received pointed out that some of the improvements described on the survey would occur farther in the future than the year 2025. While the EPA concurs that is the case it is widely accepted practice in the economics literature on stated preference to describe improvements happening at a specific time in the relatively near future, and to discount willingness to pay estimates to reflect time lags (e.g., Alberini, et al. 2004; Banzhaf et al. 2006; Cameron and DeShazo 2013). As a survey design issue this approach has been shown to minimize respondent scenario rejection (i.e. disbelief of the conditions described in the choice question) that is associated with long time horizons (Banzhaf, et al. 2006). Further, because the researcher does not know how long time horizons affect each respondent’s willingness to pay responses (via discounting, scenario rejection, or scenario adjustment), estimates generated from near-future scenarios are more generalizable and useful for benefits transfer. EPA would like to ensure that this survey has utility beyond the immediate analysis and can be applied to other water quality benefits assessments within the Chesapeake Bay watershed. At the same time, EPA recognizes that benefits estimated from this study should not be transferred to sites outside the Chesapeake Bay watershed.

However, in order to be responsive to public comments and explore this issue completely, the pretest employed a split sample design to examine how respondents reacted to different time horizons when making tradeoffs between income and environmental improvements. Half of the pretest sample received surveys that described water quality conditions in the year 2025 and the other received surveys that described conditions in the year 2040.

EPA included a debriefing question designed to explore the impact of the time horizon on validity of the survey instrument. In the pre-test respondents were asked how much they agreed with the statement: I voted as if the programs would actually achieve the results shown by 2025/2040. Disagreeing with such a statement is problematic for stated preference benefits analysis because the validity of WTP responses depends on respondents making the tradeoffs put before them in the choice questions rather than adjusting the tradeoffs based on their own subjective information or rejecting the tradeoffs outright (Cameron, DeShazo, and Johnson 2011). Researchers use questions like these to identify respondents that are adjusting or rejecting the tradeoffs presented and control for them econometrically or remove such respondents from the analysis. Without additional information available to determine the basis from which the votes were cast by these respondents, their responses become difficult to use.

On a 5-point Likert scale 7.5% of respondents to the 2040 version of the survey said they did not vote as if the programs would achieve the results shown in the choice questions. Among the respondents to the 2025 version, 4.7% did not vote as if the programs would actually achieve the results shown. While the difference between these results is not quite statistically significant (two-sample t-test p-value = 0.108), it is consistent with general findings in the literature that participants are less likely to believe predictions for longer time horizons.

Analysis of pretest data show that willingness to pay (WTP) for water clarity is generally lower in the 2040 sample than in the 2025 sample, which is consistent with a positive rate of time preference and subjective discounting of benefits. We find that the present value of the 2040 WTP estimates are approximately equal to the present value of 2025 WTP estimates at a discount rate of 2.8%, very close to the lower estimate of 3% in the range recommended in OMB’s Circular A-4 and EPA’s Guidelines for Preparing Economic Analyses (U.S. EPA 2010). Further, if we apply a rate of 7% to compare these WTP estimates we find that, though the means differ, we cannot reject a conventional discounting model given the confidence intervals of the estimates.

These results suggest that differences in WTP between 2025 and 2040 are consistent with conventional discounting methods and that only one time horizon is necessary for the main survey. Given the larger incidence of scenario rejection in the subsample that received the 2040 time horizon, EPA prefers to implement the 2025 time horizon in the full scale administration of the survey.

In summary, the pretest results on the alternative time horizons suggest (a) a larger proportion of respondents accept the tradeoffs in the choice questions in the 2025 version of the survey; and (b) respondents discount future benefits at rates consistent with standard practice in benefit-cost analysis and WTP for benefits occurring in other time periods can be inferred using conventional discounting procedures. Based on these results, the body of existing literature, and EPA’s interest in ensuring the utility of this survey for other applications in the Chesapeake Bay watershed, EPA requests that OMB approve use of the 2025 time horizon only in the main survey. EPA will follow precedent established in peer reviewed literature and discount benefits that occur in alternative time horizons.

2. Omit Improving Baseline in Watershed and Other East Coast States

In addition to splitting the sample according to different time horizons as discussed above, the pretest divided the sample according to different assumptions about future baseline conditions due to uncertainty surrounding water quality conditions in the Chesapeake Bay and Watershed lakes in the absence of the TMDL. To guard against this uncertainty in the pretest EPA fielded three versions of the survey (each split between 2025 and 2040 time horizons): one in which respondents are told conditions would improve in the absence of further action, one in which they are told conditions would stay constant, and one in which respondents are told conditions would decline in the absence of further action.

Some of the original uncertainty surrounding baseline conditions has been resolved and EPA has been able to rule out the improving baseline scenario. While it is true that water quality conditions in the Chesapeake Bay showed a generally improving trend in the 25 years preceding the implementation of the Chesapeake Bay TMDL in 2010, the question relevant to the correct baseline is whether or not those improvements would continue in the future.

One of the most compelling answers to that question is provided by a study conducted by U.S. Geological Survey (Hirsch et. al 2013, attached). The study examines, in part, changes in yields of total nitrogen, total phosphorus and suspended sediment among the nine USGS River Input Monitoring stations (RIMs) for two time periods, long-term (1985–2010) and short-term (2001–2010), both of which are prior to the establishment of the Chesapeake Bay TMDL. The results of the study show that the improving trend observed over the long term (1985-2010) levels out, and by some measures begins to decline, in the more recent time period (2001-2010). Based on these results we can conclude that water quality in the Chesapeake Bay would not improve in the future without additional measures such as those required under the TMDL.

The table below summarizes the results of Hirsch et al. Comparing the long-term to the short-term periods among the constituents (total nitrogen, total phosphorus and suspended sediment) and among all monitored sites, there are no instances of better loading trends in the short-term. Trends either remain the same or are worse from long-term to short-term. During the short term period only one of the nine RIMs showed an improving trend for total nitrogen. Likewise, only one RIM showed an improving trend in the short term for suspended sediment. The short term trend for total phosphorus is either flat or getting worse in all nine RIMs. These results strongly suggest that the practices that were in place before the TMDL will not continue to provide loading reductions to the Chesapeake Bay in the future.

Comparison of long-term and short-term trends across nine River Input Monitoring Stations for three constituent pollutants (reproduced from Hirsch et. al 2013)

	Total Nitrogen		Total Phosphorus		Suspended Sediment
	1985-2010	2001-2010	1985-2010	2001-2010	1985-2010	2001-2010
Susquehanna	Improving	Minimal Change	Minimal Change	Degrading	Degrading	Degrading
Potomac	Improving	Minimal Change	Improving	Minimal Change	Degrading	Degrading
James	Minimal Change	Minimal Change	Minimal Change	Degrading	NA	Degrading
Rappahannock	Minimal Change	Minimal Change	Degrading	Degrading	NA	Minimal Change
Appomattox	Minimal Change	Minimal Change	Degrading	Degrading	NA	Minimal Change
Pamunkey	Minimal Change	Minimal Change	Degrading	Degrading	NA	Degrading
Mattaponi	Minimal Change	Minimal Change	Minimal Change	Minimal Change	NA	Improving
Patuxent	Improving	Improving	Improving	Minimal Change	Improving	Degrading
Choptank	Minimal Change	Minimal Change	Degrading	Degrading	Improving	Degrading

However, pretest results show that baseline conditions do affect willingness to pay estimates. We are requesting OMB approval of the non-substantive change of administering the improving baseline version of the survey in just one of the three geographic strata. This will improve the utility of the survey for future applications to the Chesapeake Bay in which an improving baseline may be relevant. The Bay States stratum had the highest response rate in the pretest and therefore provides the most robust geographic area for collecting additional data.

3. Changes in Agency Cost and Respondent Burden Estimates

Removing the 2040 version completely and the improving baseline version from two geographic strata reduces EPA’s costs of conducting the main survey by over 60%. Those changes will also affect the estimated respondent burden in the following way.

Change in Burden and Cost Estimates for Survey and Non-Response Bias Questionnaire

	Completed Surveys		Burden (hours)*		Cost (2012$)**
Main Survey	5,184	1,848	1,555	554	$36,283	$12,934
Non-response Bias Study Respondents	900	350	75	29	$1,743	$868

* Burden estimates of 18 minutes per completed survey are based on observations from cognitive interviews during survey development.

**Cost estimates are based on an hourly wage rate of $23.33 (U.S. Department of Labor, Bureau of Labor Statistics. (2012). Table 1: Civilian workers, by major occupational and industry group. May 2012. http://www.bls.gov/news.release/pdf/realer.pdf.)

The sample size reductions shown above will not affect EPA’s ability to conduct statistical analysis on survey responses. The original sample size was developed to enable an independent analysis for each survey version in each geographic stratum. Removing survey versions completely or in certain strata allows EPA to reduce the sample size proportionally without affecting the robustness of the statistical analysis. Therefore the power analysis presented in the supporting statement of the ICR is still applicable to the modified experimental design and no additional power analysis is needed.

4. Minor changes to description of the Lakes Condition attribute

We also request that OMB approve minor text changes to the survey to clarify the description of the Lakes attribute and evaluate respondents’ preference for “low algae lakes” (see the attachment below that shows the requested changes to page 4 of the survey in RED). Full versions of the Declining, Constant, and Improving Baseline Surveys reflecting this revision are also attached as separate PDFs.

References

Alberini, A., M. Cropper, A. Krupnick, and N.B. Simon (2004). “Does the value of a statistical life vary with age and health status? Evidence form the US and Canada,” Journal of Environmental Economics and Management 48: 769-792.

Banzhaf, H. Spencer, Dallas Burtraw, David Evans, and Alan Krupnick. "Valuation of natural resource improvements in the Adirondacks." Land Economics 82.3 (2006): 445-464.

Cameron, Trudy Ann, J. R. DeShazo, and Erica H. Johnson. "Scenario adjustment in stated preference research." Journal of Choice Modelling 4.1 (2011): 9-43.

Hirsch, Robert M., Douglas L. Moyer, and Scott W. Phillips. "Determining Nutrient and Sediment Loads and Trends in the Chesapeake Bay Watershed by Using an Enhanced Statistical Technique." (2013).

OMB. 2003. Circular A-4, Regulatory Analysis, September 17, 2003. Available at: http://www.whitehouse.gov/omb/circulars_a004_a-4/

U.S. EPA. Guidelines for Preparing Economic Analysis. Washington, DC. (2010).

Attachments

1. Changes to description of Lakes Condition attribute (below)

2. Revised Declining Baseline version of the survey

3. Revised Constant Baseline version of the survey

4. Revised Improving Baseline version of the survey

5. Hirsch et. al (2013) "Determining Nutrient and Sediment Loads and Trends in the Chesapeake Bay Watershed by Using an Enhanced Statistical Technique."

6. Pretest summary slides

Attachment: Changes to page 4 of the survey