Supporting-statement-A_11192014

ECONOMIC RESEARCH SERVICE OMB CLEARANCE PACKAGE

SECTION A

for

GENERIC CLEARANCE TO CONDUCT EXPERIMENTAL ECONOMIC RESEARCH FROM FY2015 THROUGH FY2016

(OMB Control No. 0536-0070)

Prepared by

The Economic Research Service

November 19, 2014

Supporting Statement – Part A

The Economic Research Service (ERS) of the United States Department of Agriculture (USDA) requests renewal from the Office of Management and Budget (OMB) of an existing generic clearance that will allow ERS to conduct experimental research into the cost-effectiveness of alternative policy mechanisms. ERS offers policy-relevant research to its stakeholders, including other agencies of the USDA, but does not offer recommendations or make policy decisions itself. Research conducted under the expected clearance will be used to assess features of alternative policy mechanisms, but will not be used for the purposes making regulatory decisions; ERS does not intend to use the information collected under this approval for the purposes of developing or evaluating specific policies. ERS will not conduct any experiments which alter a government program. ERS has no regulatory or program authority with which to conduct such an experiment.

The primary mission of ERS is to provide economic and other social science information and analysis for public and private decisions on agriculture, food, natural resources, and rural America.¹ ERS has constructed a set of key strategic goals in support of this mission. ² The anticipated generic clearance will authorize research in furtherance of an ongoing initiative to use insights from behavioral economics to provide economic intelligence, research, and analysis to inform agricultural resource and conservation policies, including those related to development of markets and incentives for environmental services, reduced greenhouse gas emissions and renewable energy production, and to improve food choices and weight outcomes, particularly among children and low income adults. The requested renewal is being sought as a pilot of the concept of using a generic approval mechanism for the types of experiments described in this document. As such, experiments will be limited to two topic areas that encompass the example topics just mentioned: conservation and nutrition.

The specific purpose of this generic clearance is to allow ERS to develop and implement state-of-the-art research methodologies to better inform and advance scientific understanding for its customers³ in response to both specific requests and in anticipation of future need.

The expected clearance will allow ERS to conduct only those experiments that are voluntary and low-burden. Experiments will be used only when extant data necessary to answer the relevant research question do not exist. ERS seeks to conduct market experiments, i.e. experiments in which the structure of a market is manipulated in order to study market outcomes.

No experiments will put participants at risk of physical, monetary, or psychological harm.

Section A. Justification

1. Circumstances making the collection of information necessary

The primary function of the Economic Research Service is to provide economic and social science research, analysis, and to disseminate data under the authority of 7 U.S.C. 2204 (a) (Attachment B) and 7 U.S.C. 2026 (a)(1) (Attachment C).

ERS would like to renew the existing generic clearance mechanism for the purpose of methodological research in agricultural behavioral economics. Generic clearance is a good fit for this type of information collection since the research is intended to be formative in nature (not of policy sensitive impact); the studies will be iterative, as variation in technique or research question are proposed, evaluated, and retested; the core methodology and target populations will be consistent; and burden caps and incentive structure are proposed in this request. If approved, the requests submitted through this generic clearance will benefit from consultation with professional groups and stakeholders. ERS will apply internal quality control monitoring, through its PRA liaison and program staff, in the development and management of requests submitted through the generic clearance mechanism. This formative research will inform future ERS information collections and advance the scientific literature, resulting in more efficient and robust subsequent studies.

2. Purpose and use of the information collection

Information obtained from randomized experiments will be used to develop models of behavior under alternative stylized policy mechanisms. ERS uses behavioral models to estimate a variety of policy outcomes, for instance the level of farmer participation in voluntary conservation programs under alternative contract terms or changes in the nutritional quality of meals chosen when healthy foods are promoted by the government. Variation in behavioral response can have important implications for performance measures such as economic efficiency and effectiveness, and can help predict unintended consequences of policy-design options. ERS proposes to engage in methodological research to better understand how variation in behavioral response may affect economic policies relevant to the Department of Agriculture. The results of these methodological studies will be made available to the public, labeled as exploratory in nature, and will also be used internally at ERS and within USDA to inform subsequent information collection requests regarding behavioral economic policy. Prior to each research project under the anticipated clearance ERS will provide OMB with a copy of the research instrument (if one is used) and all other materials describing the project.

In the social sciences a “laboratory” typically refers to a computer lab or other meeting room to which participants are invited. A synthetic economy is created inside the laboratory, wherein voluntary participants interact in a controlled environment, making economic decisions.⁴ The synthetic economy consists of a series of voluntary participants and specific rules of interaction between the participants.⁵ A protocol containing instructions governing interaction in the synthetic economic environment is used to guide these sessions, which are administered by a trained monitor. All laboratory experiments will be administered on a computer if possible. The use of computers is helpful to encourage easy access to instructions, to aid comprehension, and to help participants make calculations to support their economic decisions.

In the social sciences the “field” is a natural setting, i.e. a setting outside of a university laboratory where economic activity takes place.⁶ Because they are less costly to administer, ERS plans to conduct laboratory experiments when possible. In many cases a laboratory provides a credible economic environment where the results of the experiment can be useful for predicting behavior outside the lab. For instance, two pricing mechanisms could be compared in a laboratory experiment – there is little reason to suspect that purchases under a fixed-rate pricing scheme would be different if they took place outside a lab rather than inside. In some settings, however, laboratory experiments will be inappropriate and field experiments will be preferable. For instance, because it is difficult to assemble a sample of farmers in a laboratory, it might be preferable to collect a convenience sample of farmers in a setting where farmers have already congregated, such as an equipment auction or trade show. Although more costly, the use of farmers as subjects will be preferred in some cases, such as when testing the generalizability of lab experiments to farmer decision making.

Over the last 18 months, no data was collected under the generic clearance. Instead, time was spent evaluating how Behavioral Experiments could be conducted under this clearance framework.

3. Use of improved information technology and burden reduction

ERS will employ information technology as appropriate to reduce the burden of respondents who agree to participate in its research.

ERS plans to use a single general method of information collection, in two environments, each with a different type of target population: in-person group experiment activities will be held either at (1) university computer labs or similar with university students or (2) public spaces, such as a library or meeting hall with agricultural stakeholders, such as farm operators. Sometimes when interaction between subjects is not necessary it will be possible to conduct experiments through email or mail, or by interaction through a computer interface such that the subjects can participate from home. Computer assisted participation will be used when possible; else, paper and pencil will be used.

4. Efforts to identify duplication; use of similar information

Laboratory and field studies will not be undertaken unless they are necessary to answer questions that have not yet been satisfactorily addressed in the literature. In some cases it will be appropriate to replicate previous laboratory and field studies with a new population, as much of the economic and social science literature does not address the specific target populations for ERS’s research, such as farmers or participants in USDA nutrition programs. In these cases, supporting statements for specific information collection proposals will describe how extant studies to be replicated have informed, and would be extended by, ERS’ proposed formative research.

5. Impact on small businesses or other small entities

The impact on small businesses or other small entities will be kept to a minimum. No more than 5% of the total number of respondents is expected to be small businesses.

6. Consequences of not conducting data collection, or of collecting information less frequently

The proposed renewal will allow the development of more robust and efficient measures regarding agricultural behavioral economics, with minimal burden, that will benefit subsequent ERS and USDA information collections.

The quality of research that ERS can provide to its stakeholders will be increased if ERS is able to utilize state-of-the art experimental research mechanisms. The quality of quantitative research and its contribution to prospective policy will especially benefit under the proposed generic clearance. Experimental studies are often the only empirical tool that can be used to evaluate economic mechanisms that do not exist in the real world.

7. Special circumstances that would cause an information collection to be conducted so as to require respondents to report information to the agency more often than quarterly

There are no special circumstances associated with this information collection. Most information collections under this generic clearance will require only a single interaction between the agency and respondents.

8. Comments in response to the Federal Register Notice and efforts to consult outside the agency

We received one comment in response to the 60-day federal register notice: Federal Register Vol. 79, No. 178, Monday, September 15, 2014: 54955-54956 (Attachment D). The comment was not specifically about experimentation and was not relevant to the current request (Attachment E).

All data collections under the clearance will be pilot-tested to confirm that instruments are clear; outside experts have been sought out by the agency and will continue to be consulted on projects under the generic clearance.

Because no data was collected under the generic during the initial phase, no consultation with respondents has been conducted.

9. Explanation of any payment or gift to respondents

Incentives will not be provided to increase participation or to reduce nonresponse bias. Incentives will not be used in order to encourage the participation of hard-to-find populations, nor will incentives be used as rewards for highly burdensome activities or sensitive questions.

Incentives will, however, be made as an integral part of the research methodology. Often used in experimental and market research, we propose to vary the amount of incentive payments across specific formative research studies, depending on the outcomes of the experiments and the decisions each individual makes during an experiment. These payments will be used only for the purpose of eliminating hypothetical bias, a well-known phenomenon in economics. Payments will be retained by respondents (not returned to the agency at the completion of the experiment).

Hypothetical bias refers to the difference between intentions and actions in an economic environment.⁷ Substantial literature finds that subjects overstate their value for items when they are faced with a hypothetical choice (such as a survey question). For instance, a subject might state on a survey that they would be willing to pay $5 for a coffee mug, but might not purchase the same mug for $5 if given the opportunity. Hypothetical bias refers to this difference between what the subject proclaimed (a value for the good of $5) and what the subject revealed through their action (a value of less than $5). The NOAA Blue Ribbon Panel on contingent valuation concluded “that hypothetical markets tend to overstate willingness to pay for private as well as public goods.”⁸

Because individuals tend to act differently when faced with hypothetical choices, the prevailing practice in the field of experimental economics is to incentivize experiments. Experiments are said to be incentivized when actions taken by experimental subjects have monetary consequences. For instance, rather than ask a subject how much they would be willing to pay for a coffee mug, an experimenter might give a subject the choice between a coffee mug and a five dollar bill, only one of which will be theirs to keep. If the subject chooses the five dollar bill, the subject has revealed through their actions that they prefer $5 to the mug. When revealed through action rather than intention, this data point reflects the subject’s true value for the mug.

The practice of incentivizing experiments is nearly universal.⁹ The overwhelming majority of experiments carried out by economists are incentivized using real-money payments. Every experimental paper published in the American Economic Review – the preeminent professional publication in economics – between 1970 and 1997 was incentivized (Camerer and Hogarth, Journal of Risk and Uncertainty, 1999). Glenn Harrison and Elisabet Rutstrom list the debate about the use of hypothetical or incentivized experiments as a “closed” argument in 2008.¹⁰ Failure to properly incentivize experiments would place ERS outside the mainstream, and could harm the reliability of collected data.

In order to incentivize each experiment under the proposed generic clearance, respondents will receive payments that are tied to the decisions or choices they make during the experiment. Expected payments vary depending upon the respondent pool, the task, and the hypothesis to be tested. Typical payments are lowest for experiments conducted in a laboratory, where payments average between $15 and $30. Laboratory tasks tend to be simple, straightforward, and thus relatively inexpensive to incentivize.¹¹ Tasks carried out in the field with farmers, on the other hand, are more costly to incentivize.

In order to incentivize action in a more complex environment, and in a natural context, a higher amount of incentive is likely needed. For example, potential payment of $15¹² might be a sufficient incentive for undergraduate students to bid sincerely in an auction for a small good, but might be insufficient to incentivize farmers to bid sincerely in an auction for farm equipment. If equipment auctions are the target environment for the hypothesis to be tested, substantial incentives would need to be offered.

Because experiments in the field are less uniform than are experiments in the laboratory, it is more difficult to predict the payment for experiments to be carried out in the field under the proposed generic clearance. ERS requests the ability to provide monetary incentives greater than $30 per subject in some cases. Accordingly, supporting statements accompanying each specific formative research request will describe the proposed incentive payment, and the rationale for the particular amount proposed. Each experiment submitted to OMB for clearance under the generic will contain a statement justifying the chosen level of funding.

The decision to incentivize respondents by paying them does not place respondents at risk, nor does it increase burden, as respondents volunteer to participate in the studies.

10. Assurance of confidentiality provided to respondents

Respondent data will be protected by the Privacy Act of 1974 (5 USC 552a) and the E-Government Act of 2002 (44 U.S.C. Ch 36).

Consistent with the Privacy Act and the E-Government Act, a Systems of Records Notice (SORN) and a Privacy Impact Assessment (PIA) will be submitted for approval, as appropriate. The SORN and PIA will document the ways in which participant personally identifiable information will be collected, stored, and accessed. Data will be managed for research purposes only. Specific details regarding information handling will be specified in individual submissions under this generic clearance, but will conform to these broad guidelines.

ERS has decided not to invoke the Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA). The complexity and cost necessary to invoke CIPSEA is not justified given the nature of the collection; the collections would include a very limited amount of personally-identifiable information (PII), and would generally be designed to be hosted in university computer labs, where CIPSEA compliance could not be assured.

11. Justification for sensitive questions

No questions of a sensitive nature are anticipated in work conducted under this generic clearance. At most, simple demographic questions may be asked as part of a study in order to control for income effects, gender effects, and other effects that the researchers believe may reasonably influence outcomes. Respondent participation and all activities within the laboratory study are voluntary; subjects will be made aware of this fact.

All respondents are free to opt-out of a data collection at any time, and for any reason.

12. Estimates of hour-burden including hourly costs

Table 1. Estimated Total Response Burden for the Proposed 18 Months Pilot Study Period

Type of research instrument	Estimated number of respondents	Estimated number of responses per respondent	Average burden-hours per response	Estimated total burden hours requested
Laboratory study	4,240	1	1.5 (90 mins)	6,360
Field study	2,660	1	≤1 (15 mins)	665
Total	6,900			7,025

Estimated burden costs:

ERS expects to use 6,900 unique respondents over the period of the generic clearance.

An average hourly salary of approximately $24.54 is assumed for all respondents, based on the Bureau of Labor Statistics Establishment Data¹³. With a total respondent burden of 7,025 hours, the total combined cost to the public for all studies under this request is estimated to be a maximum of $172,393.50 (7,025 hrs. X $24.54).

The table above refers to the estimated burden hours over the entire 18 months period of the proposed generic clearance.

13. Estimate of other total annual cost burden to respondent or recordkeepers

There will be no capital, operating, or maintenance costs to the respondent as the result of participation in an information collection under this generic clearance.

14. Estimate of costs to the Federal Government

The cost to the Federal Government generated by research projects covered under this generic clearance is approximately $270,000. This assumes a GS-13 (step 5) annual salary of $101,914 for an ERS PhD Economist to guide the design and evaluation, and a GS-09 (step 5) annual salary of $59,098 for administrative support in processing individual research instruments (support in preparation of travel, preparation of materials, and processing instruments through clearance).¹⁴ Details are provided in the table below.

Activity	Description	Period	Cost
Administration of research	ERS PhD Economist, 50% FTE @ $101,914	18 months	$76,435.50
	ERS support staff, 15% FTE @ $ 59,098	18 months	$13,297.05
Cooperative research and inter-agency agreements	Collaboration with academic experts, graduate student support and use of academic laboratory facilities; collaboration with program agencies; experimental payments		$180,000
Total			$269,732.55

15. Changes in burden hour

There are no changes to the information collection since the last OMB approval..

16. Plans for tabulation, publication, and project time schedule

ERS research leads to methodological improvements to future research. Methodological improvements will be published as technical articles in peer reviewed journals and as articles in ERS’s two in-house research publication series that are freely available to the public and widely disseminated in print and on ERS’s website: ERS Economic Research Reports and ERS Economic Information Bulletins. Findings published as technical articles are regularly distilled and combined with policy research into ERS Economic Briefs and Amber Waves articles.

Data collection, analysis, and publication will span the entire period of the anticipated generic clearance. ERS plans to conduct an ongoing program of research and therefore plans to apply for an extension to the anticipated generic clearance when it expires.

Because of “publication-lag” a typical academic journal article is published anywhere from one to two years after initial submission. ¹⁵ The publication process for ERS in-house products can be quicker than that for external academic publications, but because in-house publications undergo a peer review process similar to that used by professional journals, it typically takes no less than six months.

Data collection for a professional publication based on laboratory experimentation typically takes three to four months, or a single academic semester.¹⁶ Data analysis and article preparation typically lasts another three to four months. Thus professional articles in a peer-reviewed outlet of any kind are estimated to be published from 12 months to 30 months from the time that data collection can begin.

17. Reasons display of OMB expiration date is inappropriate

No exemption is requested.

18. Exceptions to certification for paperwork reduction act submissions

This data collection has been designed in accordance with the requirements specified in Item 19 of the OMB 83-I. No exceptions to certification are requested.

1 Code of Federal Regulations, Chapter XXXVII, Part 3700.1 (Attachment A).

2 Economic Research Service Strategic Plan for 2013-2018. Available at: http://www.ers.usda.gov/about-ers/strategic-plan.aspx#.VCHf6xa41Jk

3 ERS’s primary customers are: USDA policy officials and program administrators, the Office of the White House, Congress, and agricultural, environmental, consumer, and rural public interest groups, including farm groups and industry. CFR, op. cit.

4 Economic decisions can include the decision to trade a good provided as part of the synthetic economy, the price at which to purchase such a good, or the bargaining strategy employed in a trade.

5 One simple set of rules might stipulate that two participants act respectively as buyer and seller of a “ticket” or “chit.” The buyer has a value of 10 for the item while the seller has a value of 5. The participants playing the role of buyer and seller then must decide at which price to trade the chit. An iconic example of laboratory techniques is given by Chamberlin. Chamberlin, E. H. (1948). "An Experimental Imperfect Market." The Journal of Political Economy 56(2): 95-108. Chamberlin created an economy inside of a classroom, in which student participants assumed the role of buyers and sellers for an artificial good. The study provided an early illustration of the process of what is now called “price discovery,” i.e. the determination of an equilibrium price of an asset through continued interaction of buyers and sellers. More modern experiments in the same vein are being used to create and study asset bubbles in laboratories.

6 A very early example of an experiment in the field is given by Bohm. Bohm, P. (1972). "Estimating demand for public goods: An experiment." European Economic Review 3(2): 111-130. Bohm’s 1972 study addressed a classic problem in economics: how a group of people determine a price for a good they will collectively purchase and consume. Rather than use an artificial good, Bohm used a real good: access to a closed-circuit showing of a comedy sketch featuring two popular Swedish comedians.

7 John A. List (2001). “Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards,” American Economic Review, 91(5): 1498-1507.

8 “Report of the NOAA Panel on Contingent Valuation,” January 11, 1993, page 43. Available at http://www.darrp.noaa.gov/library/pdf/cvblue.pdf.

9 See Vernon L. Smith (1976). “Experimental Economics: Induced Value Theory,” American Economic Review, 66(2): 274-279; and Vernon L. Smith (1982). “Microeconomic Systems as an Experimental Science,” American Economic Review, 72(5): 923-955.

10 The subject of the article was risk elicitation, which is a popular and relevant topic of study by experimental economists, as well as a regular component of many experiments on other topics.

11 Vernon L. Smith (1976), op. cit., page 277.

12 Potential earnings reflect a possible outcome of an entire experiment. For any given choice, earnings may be larger or smaller. When we refer to potential earnings, we refer to the total earnings from participation in an experiment, which may involve many choices.

13 Average hourly earnings in private industry, August 2014.

14 Based on 2014 Office of Personnel Management salary tables.

15 The most recently available data indicate that average time to publication for the leading economics journals are approximately 19 months. A leading journal in the field appropriate for research on issues of conservation and environmental issues in agriculture is the Journal of Environmental Economics and Management (JEEM). JEEM has an average time to publication of approximately 13 months. Reference: Heintzelman, Martin and Nocetti, Diego (2009) "Where Should We Submit Our Manuscript? An Analysis of Journal Submission Strategies," The B.E. Journal of Economic Analysis & Policy: Vol. 9 : Iss. 1 (Advances), Article 39. Available at: http://www.bepress.com/bejeap/vol9/iss1/art39.

16 In the case of cooperative agreements with academic professionals, most research is done during one of the two semester periods in the academic year.

5