Mini-supporting-statement-UMD-II-Part-B-Final

Mini-Supporting Statement for

Alternative tools for improving CRP cost-effectiveness

Section B

OMB Control No. 0536-0070

B. COLLECTION OF INFORMATION USING STATISTICAL METHODS

1. Universe and Sampling

The data collected will be used for research activities which address an important question in conservation program design by comparing alternative mechanisms for competitive purchase (auctions), rather than to produce estimates about the population. The study uses a convenience sample of 192 students recruited from a respondent pool of 4,320 students at UMD, College Park and/or UC, Santa Cruz who have expressed interest in participating in economics experiments and who have shared their email addresses with the respective economics departments. Participants will be randomly assigned to 12 experimental sessions to take part in three of five experimental conditions (tight bid cap, loose cap, reference price, endogenous reference price, and grouping). Per the generic clearance, results of the study are not meant to be used to make policy decisions, and the sample is not meant to be nationally representative.

2. Procedures for Collecting Information

Recruitment:

Potential participants will be solicited by an email (Attachment-A: Email for recruitment) from a respondent pool that the experimental economics labs at UMD, College Park and UC, Santa Cruz maintain. The respondent pool is a database of students who have expressed interest in participating in economics experiments and who have shared their email addresses with the economics department. Because it is common to get a number of no-shows and last-minute cancellations for a given experiment session, and because it is important to have a particular number of participants in each session, each session will also be “overbooked” by 2 potential participants as backups to ensure there will be the right number of individuals to run the experiment. In cases where more people show up than can be accommodated in a session, any extras who show up before the scheduled start time will receive a $7 payment for time and travel and can reschedule for another session. This is common practice among experimentalists.

Prior to giving their consent, potential participants at the beginning of each experimental session will be provided with: (1) a Consent Form (Attachment F) that contains a description of the study, including details of study purpose, data collection methodology, and burden estimate, as well as the use of the information collected and the voluntary nature of the study; and (2) a copy of the data protection disclaimer (Attachment G).

Experiment:

The goal of this experiment is to investigate the performance of alternative auction mechanisms designed to limit information rents. The performances of four alternative auction designs (aka treatments): loose cap, reference price, endogenous reference price, and grouping will be tested against that of a baseline design (tight bid cap). The experiment is composed of twelve 90-minute sessions (Table 1 – Experimental Design). Participants will be randomly assigned to each experimental session to take part in three of five experimental conditions (reverse auctions) as a bidder. Their earnings will be paid to them in cash at the end of the experiment. The treatment order within each session is random. Each treatment will be used for an average for 12 rounds. But subjects in different sessions often proceed at different speeds, making 15 rounds possible in some circumstances; in all cases each session will maintain a 90 minute maximum experiment time. More details for the experimental design are presented in the Attachment B: Experimental Design.

Once an experimental session is scheduled, it is the responsibility of the respondents to travel to the experiment site.

The auctions are conducted via a Z-tree interface (a specialized computer program) with internet administration. At the beginning of each session and each treatment, participants are given detailed explanations and instructions for both general experimental and specific treatment procedures (Attachment C: Instructions). A short demographic questionnaire (Attachment D: Questionnaire) will also be given to participants during the experiment, for the purposes of collecting control variables useful in basic regression analysis on laboratory experimental data.

Table 1. Experimental Design
Session	Treatment*	Average # of rounds per treatment	Max # of rounds per treatment	Time (in minutes)	# of participants
1	1,2,3	12	15	90	16
2	1,2,4	12	15	90	16
3	1,2,5	12	15	90	16
4	1,3,4	12	15	90	16
5	1,3,5	12	15	90	16
6	1,4,5	12	15	90	16
7	1,2,3	12	15	90	16
8	1,2,4	12	15	90	16
9	1,2,5	12	15	90	16
10	1,3,4	12	15	90	16
11	1,3,5	12	15	90	16
12	1,4,5	12	15	90	16
* Random order of treatment within session.

Debriefing and Payment:

At the end of each session, all respondents will be debriefed. This gives an opportunity to address respondents' questions regarding the study.

Each participant will also receive a cash payment at the end of each 90 minute session. The cash payment will be of uncertain value before the experiments take place, the minimum payment will be $7.00 for “showing up”. Depending on their own strategic behavior, they will be paid based on whether their bid is accepted given the equilibrium in the experimental market. We do not expect any payments in excess of $50 based on a similar experiment conducted under clearance titled “Alternative tools for improving CRP cost-effectiveness” (hereafter “CRP Study 1”); average payments for that experiment were approximately $15.

The payments listed here are for the entire 90 minute session, i.e. all auctions participated in by a given individual. Although individuals participate in many auctions for each auction type, individuals are paid at the end of the 90 minutes based on 2 randomly-drawn auctions for each auction type (for example, in a session with 1 auction treatment that includes 12 auctions, experimentalists will draw two auctions at random. Payment is based on the outcomes within these randomly chosen rounds to prevent any wealth effects from distorting the findings of the experiment¹). This practice is standard in the literature.² Therefore, each participant’s payment will be the sum of 6 randomly drawn rounds: 2 per each auction type. If they earned no money in these 6 rounds, they will receive a 7 USD show up fee. More detailed explanation for payment method and justifications are given in A.9 of this Mini-supporting Statement.

Power Analysis:

The primary hypotheses to be tested will be univariate comparisons (t-tests and non-parametric tests) of expected cost between each of the four alternative CPR enrollment mechanisms (loose cap, reference price, endogenous reference price, and selection by grouping) and under the baseline auction mechanism (tight bid cap). We seek to detect a minimum detectable effect (MDE) of 40 ECUs (Experimental Currency Units). Our simulations suggest that this MDE is reasonable to expect. Lower effects (effects smaller than 40 ECUs) would certainly be useful to know about and potentially cost-effective to implement in a given practical application. We do not, however, want to pre-commit to running more sessions than are necessary in order to detect a very small effect if, as is the case here, we expect a larger effect.

We used simulations programmed in the computer programming language R to determine the expected cost of procurement for the baseline auction. Simulation results indicate an estimated average of 241.1 ECUs and a standard deviation of 65 for the baseline treatment.

We are interested in the total procurement cost for each auction treatment; we obtain one (non-independent) observation of a given auction outcome each round. Based on our proposed experimental design, each experiment will yield an average of 12 rounds of data per treatment and a total of 36 rounds of data per session (three treatments X 12 rounds = 36). Because the same subjects participate in each of the rounds, the 36 observations generated in each session are not treated as independent. If we have 12 sessions and each session yields 36 observations, it gives us a total of 12*36 = 432 observations, or an average of 86 (rounded down) observations per auction treatment. Given this plan, the proposed experimental design will support a powerful test (α =.05 and1- β = 0.8) to detect a minimum detectable effect (MDE) of 40.2 for all planned univariate comparisons.

3. Methods to Increase or Maximize Response Rates

Potential participants are sent recruitment emails asking them to take part in an experiment. Many participants will have participated in similar economic experiments conducted on the campus of the University of Maryland or UC, Santa Cruz. Typically, a single solicitation email is sufficient to recruit enough subjects to complete an experimental session.

Because it is common to get a number of no-shows and last-minute cancellations for a given experiment session, and because it is important to have a particular number of participants in each session, each session will also be “overbooked” by 2 potential participants as backups to ensure there will be the right number of individuals to run the experiment. In cases where more people will show up than can be accommodated for a given session , any extras who show up before the scheduled start time will receive a $7 payment for time and travel and can reschedule for another session. This is common practice among experimentalists.

4. Tests of Procedures or Methods

The methods and procedures in use in this experiment have been used widely in the literature. Procedures for creating a market in a laboratory and carrying out a sealed-bid auction have been known and accepted practice in the experimental economics community for decades (See Davis and Holt 1992, e.g.).

A pre-test was conducted on 9 students in the Experimental Economics Laboratory at UMD, College Park, on 30 June 2014 to test the auction software, its interface, the timing of the experiment and to observe auction ranges and averages. Information on the specific areas of interests being tested, respective testing results, and/or actions taken (if applicable) was summarized in the Part B submission to CRP Study 1. Subsequently, 12 experimental sessions were held using the software and no additional issues were discovered.

5. Identification of Consultation

The contact individual for this research is Nathaniel Higgins, Ph.D., Economic Research Service, USDA, telephone (202) 694-5602.

The research is being conducted under a cooperative agreement between the Economic Research Service and the University of Maryland. Data will be collected, entered, and provided to ERS by Professor Peter Cramton at the University of Maryland, telephone (240) 479-9345.

REFERENCES

Davis, Douglas D. and Charles A. Holt (1992). Experimental Economics, Princeton University Press, Princeton, NJ.

1 Wealth effects are the theoretical changes in behavior that occur after a given individuals’ wealth increases. Since the CRP is a “one-shot” auction – there is only one CRP auction conducted at a moment in time, not a series of CRP auctions – it is necessary to eliminate wealth effects.

2 See “Incentives in Experiments: A Theoretical Analysis” by Azrieli, Chambers, and Healy. http://www.econ.ucsb.edu/about_us/events/seminar_papers/Healy.pdf.

6