EIA-914 Supporting Statement B

­­­

Supporting Statement for Survey Clearance 1905-0205

1. Part B: Collections of Information Employing Statistical Methods

Form EIA-914 Monthly Crude Oil and Lease Condensate, and Natural Gas Production Report

OMB No. 1905-0205

September 2020

Independent Statistics & Analysis

www.eia.gov

U.S. Department of Energy

Washington, DC 20585

Table of Contents

Part B: Collections of Information Employing Statistical Methods i

B.1. Respondent Universe 1

B.2. Statistical Methods 1

B.3. Maximizing Response Rates 8

B.4. Test Procedures and Form Consultations 8

B.5. Statistical Consultations 9

B.1. Respondent Universe

The universe for Form EIA-914 consists of all operators of wells in the lower 48 United States that are producing natural gas, crude oil and/or lease condensate, including federal and state offshore wells. This is roughly 15,000 operators. A well operator producing natural gas, crude oil and/or lease condensate forms the responding unit.

A monthly cut-off sample of well operators is designed to give the targeted relative standard errors in all published geographic areas with the minimum number of operators while reducing sample turnover from month to month. Currently, EIA met this quality goal with about 400 respondent companies. This number fluctuates from month to month.

The cut-off sample is designed to provide about 90 percent coverage at the lower-48 level for natural gas, and crude oil and lease condensate. This ensures adequate coverage in each geographic area for which production data are collected.

The cut-off sample operators are selected based on their natural gas and oil production obtained from a current commercially available data source, “Enverus”, supplemented with data collected on the Form EIA-23L, Annual Survey of Domestic Oil and Gas Reserves. These sources provide readily consolidated data that is recent and accurate. These two data sources constitute the frame file for the EIA-914.

B.2. Statistical Methods

B.2.1 Sampling

The Enverus database supplemented with Form EIA-23 data is used for both the sampling and estimation processes. Enverus acquires well or lease level oil and natural gas information from state agencies, places it in their own database format, and performs some data standardization and validation. A new Enverus database is transmitted to EIA at the end of every month.

Enverus data for the most recent reporting months are usually incomplete. The months used for sampling are determined by how much of a lag there is in the Enverus data before they become complete. A different lag time is used for every state/area and for oil and gas. The process used to determine the lag time to complete data is described in further detail in the EIA-914 Monthly Crude Oil and Lease Condensate, and Natural Gas Production Report Methodology. Table 1 shows lag times by state and commodity.

Table 1: State lags, August 2020

State	Oil Lags (number of months)	Gas Lags (number of months)
Arkansas	6	3
California	10	30
Colorado	13	13
Federal Gulf of Mexico	5	4
Kansas	4	4
Louisiana	3	5
Montana	8	7
New Mexico	4	6
North Dakota	6	5
Ohio	18	18
Oklahoma	18	18
Pennsylvania	6	6
Texas	12	14
Utah	6	6
West Virginia	18	18
Wyoming	10	6

For operators not in the sample, the most recent four months of Enverus production is examined for each of them. If a non-sampled operator is observed to produce above the state cutoff for either crude oil or natural gas for four consecutive months in any state, that company is added to the sample for all the states in which they produce. The production cutoff varies by each state because state production levels differ. If a sampled company reports production on the Form EIA-914 below the cutoff in every area in which it produces, for six consecutive months, for both crude oil and natural gas, it is dropped from the sample. These procedures maintain high sample coverage of the population, and minimize frequent adding and dropping of operators.

B.2.2 Rationale for 85 percent sample coverage target

The 85 percent target was selected to be consistent with the current sampling rate for the EIA-914 survey and to maintain consistency of data series, and to guard against inaccurate or biased model estimates. In general, relative standard errors are within acceptable ranges at an 85 percent sample volume coverage. Of more practical concern is that in some specific cases, large operators can behave very differently from small operators. This occurred in Louisiana during 2009 when the Haynesville Shale was developed. When the sampled companies do not represent the non-sampled companies the model estimates will not accurately reflect production activity in the state. By using a high sample coverage rate, data quality issues arising from model failure is mitigated.

Enverus data for four of the smaller producing states are missing or inadequate. For Illinois, Indiana, Kentucky, and Tennessee, annual production data from the Form EIA-23 (Annual Oil and Gas Reserves Report) survey are used to supplement the Enverus database. Hereafter, references to Enverus data include supplemental data from the Form EIA-23 survey for these four states. These data are used to determine a different cutoff sample for each state/area designed to achieve a roughly 85 percent sample coverage of that area’s oil and natural gas production volumes. The cutoff production rates will be re-determined, annually, to maintain the target of 85 percent coverage of state level production. Although a target of 85 percent sample coverage is used to design the sample, in practice this typically results in coverage closer to 90 percent because when an operator is selected to report their production for one state, they also are required to report their production for all states in which they operate.

The problem of inaccurate or biased model estimates can normally be solved by combining a certainty stratum with a probability-proportional-to-size sampling or other random sampling scheme of the smaller companies. Because the sampling frame is in a constant state of flux due to operator acquisitions and mergers, sample weights calculated on a monthly basis are prone to error. Using the classical ratio estimator, this sort of behavior causes much less fluctuation in estimates.

Another way of solving the problem of inaccurate or biased model estimates is through stratification of variables based on well characteristics, such as formation type (shale, conventional, etc.). While the frame has such information, it is often reported inconsistently and sometimes incorrectly. Also, to be effective, the respondent would have to provide this information in their survey response since an individual operator may produce large volumes from multiple wells in multiple formations. Not only would this increase burden on the respondent, in EIA’s experience with the EIA-23, respondents do not provide this information reliably enough to clean the data within the necessary turnaround time.

States such as Kansas and Arkansas have higher numbers of small oil producers than other major states. States composed primarily of small operators and without potential for dramatic growth are generally much less likely to suffer from inaccurate or biased model estimates. This is because the sampled companies accurately reflect the behavior of the non-sampled companies. As a result, the below 85 percent sample coverage shown below in Table 2 is acceptable.

B.2.3 Rationale for 500 barrel per day minimum threshold

In addition to the 85 percent sample coverage criteria used to determine the oil and gas production cutoff values, a minimum crude oil cutoff rate of 500 barrels per day (bpd) is used as an additional constraint to limit the number of smaller operators sampled. This is needed because some liquids producing states, such as Kansas, are composed of an inordinate number of small operators, and achieving an 85 percent sample coverage would create a much larger sample in a region that EIA does not expect model failure to be an issue. Application of the 500 bpd minimum oil cutoff rate will yield less than 85 percent coverage in some states with small production volumes and also minimizes the approximate sample size to 400 operators. Sample size can vary due to external economic triggers. The triggers can include significant price movements or merger and acquisitions activity. The application of this lower bound on the oil cutoff production rate creates an acceptable balance between sample coverage and reduced burden on smaller operators.

EIA considered what impact various minimum cutoffs would have on the sample and resulting data quality. Sample coverage declines at increasing cutoff levels (see Table 2). With no minimum cutoff, coverage across all states meets the EIA goal of 85 percent coverage in each state, but sample size increases considerably, thereby increasing burden. Raising the minimum threshold to 500 bpd, 1,000 bpd, and finally 5,000 bpd succeeds in lowering the sample size, but generates progressively worse coverage and less reliable estimates. The minimum cutoff rate of 500 b/d was selected in part because this was used effectively on Form EIA-23 to estimate annual crude oil production.

Table 2: Percent Coverage by State for Sampled Oil Production
State	Minimum Cutoff Applied (bpd)
	0	500	1,000	5,000
Arkansas	85%	44%	26%	20%
California	90%	90%	88%	88%
Colorado	92%	92%	91%	91%
Federal Gulf of Mexico	92%	92%	92%	90%
Kansas	85%	57%	48%	29%
Louisiana	86%	79%	71%	39%
Montana	88%	88%	87%	70%
North Dakota	87%	87%	87%	87%
New Mexico	86%	86%	86%	86%
Ohio	86%	85%	85%	72%
Oklahoma	85%	85%	79%	68%
Other States	86%	67%	60%	31%
Pennsylvania	86%	70%	70%	55%
Texas	86%	86%	86%	86%
Utah	87%	87%	87%	76%
West Virginia	87%	87%	87%	56%
Wyoming	85%	85%	85%	64%
Lower 48	87%	87%	86%	83%
Sample Size	643	380	297	254

Note: This table is the result of a simulation based on some simplifying assumptions that are more conservative than reality.
It illustrates that there is a significant reduction in sample size with the application of minimum cutoffs. Actual coverages are slightly higher.

Form EIA-914 will utilize 34 cutoff samples, one for each of the 17 states/areas for crude oil and lease condensate production, and another 17 cutoff samples for each of the states/areas for natural gas production. If an operator is sampled for one state/area for either crude oil or natural gas, then it must report natural gas, and crude oil and lease condensate for all states/areas where it has production, regardless of volume in those areas.

Form EIA-914 utilizes samples for the 16 states/areas for which crude oil and lease condensate, and natural gas production respondents must provide information separately for each state. The remaining 19 states/areas (the “Other States” group) that have crude oil and lease condensate, or natural gas production will also be sampled, as a single target variable, collectively rather than individually by state.

B.2.4 Estimating

The target population to report on Form EIA-914 is all active crude oil and lease condensate, and natural gas producers. Members of the target population not sampled must be estimated for.

Estimation will be performed separately for natural gas and liquids production for each of the 16 states/areas using the Classical Ratio Estimator (CRE) method, which multiplies the current survey reported production by a ratio (the Other States group is excluded from this calculation and its method will be described below). The CRE ratio is an area’s total production in an earlier 6-month period as found in the Enverus database, to the area’s production from the current sample of operators in the same period, also found in the Enverus database. The six month period is dependent on the reporting lag time for each state. A six-month ratio is used to avoid potential volatility from using a ratio of a single month’s production. The Enverus data are typically incomplete for more recent months. The months used in calculating the ratio are determined by how much of a lag there is in the Enverus data before they become sufficiently complete. A different lag is used for each state/area and for liquids and natural gas. Estimates for the survey month are computed using the following formula for each area:

where:

Estimated total production for the state/area in month ‘m’

Total survey reported production for the state/area in month ‘m’

L = The Enverus lag for the particular state/area (negative value indicating going back in time)

Total production for the area in month ‘i’ from the Enverus database

Production of the current sample of operators in month ‘i’ from the Enverus database

Errors can occur in the estimation process when producing properties are transferred between operators, and this transfer is not reflected in both the Enverus database and the Form EIA-914. This can be caused by mergers and acquisitions. EIA monitors these events by following trade journals and news releases and by following up with respondents whose reported production is substantially different from what is expected given their production history. When transfers and other changes occur, EIA adjusts either the Enverus data or the survey data, as appropriate, to synchronize reporting until the two are in sync again.

Some responses can be far outside the anticipated range if the company is experiencing extreme growth or decline, or some of the production reported on the survey is not represented in the Enverus data (this can occur for various reasons). In such cases, the company is treated as an outlier/add-on. This means that the company production is excluded from the sums in the ratio equation above, and its reported production is then added to the total estimate.

All estimates will be based on the CRE method with the exception of “Other States,” which is currently determined by applying the ratio of total gross natural gas, and crude oil and condensate production published in the Natural Gas Annual and Petroleum Supply Annual (NGA and PSA) to the EIA-914 surveyed production for the most recent available year. The numbers from the NGA and PSA come from state agencies. This is used instead of the CRE because the frame for the “Other States” group is less complete. If the frame coverage in the other states improves it could be estimated using the CRE method as well, but this is unlikely. The “Other States” group is roughly 1 percent or less for both oil and gas production in the lower 48 states.

B.2.5 Imputation for Item Non-Response

In the event that item level non-responses cannot be resolved within a survey cycle, EIA uses the average of the previous three months’ data to impute current month data for a non-respondent. If three months of survey data are unavailable, then Enverus data is used in the imputation formula. EIA will replace the imputed data with actual data, once the respondent files the report with EIA. If any of the three previous months’ survey data are deemed not analogous to the current missing reporting month’s data (i.e., being influenced by maintenance, weather event, power outage, etc.) that previous month’s data would not be used in the imputation process.

B.2.6 Frame Maintenance

Since the inception of the Form EIA-914 in 2005, EIA has maintained a natural gas operator frame of the nation’s natural gas producers. EIA uses various sources of information to maintain the completeness of the frame, including trade press, other data providers, the EIA-23 frame file, and state regulatory agencies. EIA maintains the natural gas, and crude oil and lease condensate frame by following similar practices.

EIA continuously reviews all available information to adjust the survey frame for the Form EIA-914 data collection for births, deaths, mergers, and company information changes.

B.2.7 Efforts to Reduce Total Survey Error

Frame Coverage Errors

In states with stricter reporting requirements, Enverus obtains more complete data on a timely monthly basis and frame coverage error is expected to be negligible for those states. These states make up the majority of U.S. production. However, in other states, data are less timely and less complete, requiring continuing effort on the part of EIA to research companies that may be operating in these areas to ensure the best possible frame coverage.

Reporting Errors and Data Processing Errors

Form EIA-914 questionnaire was carefully developed and tested with potential respondents to estimate reporting burden and respondents’ ability to report the requested information within the 40 day reporting period to minimize the amount on imputation in the published estimates. The survey instrument includes a detailed set of instructions for filing data, subject to a common set of definitions similar to those already used by the industry. EIA continually enhances data validation/editing software (programmed edits checks) to detect probable reporting errors and flag them for resolution by analysts, either through confirmation of the data by the respondent or submission of amendments to the previously-filed data.

Data processing is automated to minimize the possibility of data entry errors by EIA staff. Data processing errors are further minimized through the use of data quality metrics on the aggregate data and outlier analysis performed on the micro data, which can help identify both reporting and data processing errors.

Estimation Errors by Respondents

EIA uses agency-developed software to detect Form EIA-914 responses that appear out of the ordinary and are candidates for further investigation. Software is used to flag data items that are inconsistent or out of their normal range, which are further investigated by EIA analysts. When necessary, EIA analysts contact the respondents to resolve related questions.

Revisions

Revisions to aggregated data from respondents submitting revised data for prior months or late data for the current month are published according to the Form EIA-914 Revision Policy. This revision policy is available here. Publication revision thresholds have been determined for each state/area and for oil and gas. In addition, crude oil and lease condensate (combined), and natural gas production estimates will be replaced with state reported volumes when state reported volumes are deemed complete.

Respondents are required to resubmit data when their production estimates change by more than +/- 1,000 barrels of crude oil and lease condensate in a state, or by more than +/- 150 million cubic feet of natural gas in a state, and this requirement is detailed in the survey instructions.

B.3. Maximizing Response Rates

EIA will use standard procedures to administer the data collections for Form EIA-914. An introductory letter signed by the responsible EIA official is sent to each company that will be a respondent to the Form EIA-914. Follow-up procedures for non-response consist of an email message or a reminder phone call (for those not using email) to all companies that do not return a completed survey form by the due date. This initial re-contact is followed by repeated email messages, letters, and phone calls until an appropriate response is received.

In the event that item level non-responses cannot be resolved within a survey cycle, EIA uses the average of the previous three months’ data to impute current month data for a non-respondent. If three months of survey data are unavailable, then Enverus data is used in the imputation formula. EIA will replace the imputed data with actual data, once the respondent files the report to EIA. If any of the three previous months’ survey data are deemed not analogous to the current missing reporting month’s data (i.e., being influenced by maintenance, weather event, power outage, etc.) that previous month’s data would not be used in the imputation process.

B.4. Test Procedures and Form Consultations

A team of EIA crude oil and lease condensate, and natural gas production analysts and survey experts collaborated to develop and test the current Form EIA-914. EIA conducted two rounds of cognitive testing of Form EIA-914 with respondents. This research involved verifying the reporting burden and respondents’ ability to report crude oil and natural gas production during the Fall of 2016 and Spring of 2017.

B.5. Statistical Consultations

For additional information concerning this data collection, please contact Jeff Little at (202) 586-6284, or jeffrey.little@eia.gov.