updatedSupporting_State12_20_12

Docket No. AD12-8

FERC-922, OMB Control No.: to be determined

Commission Staff Report Issued October 15, 2012

(updated 12/20/2012)

Supporting Statement for

FERC‑922, Performance Metrics in Regions outside ISOs and RTOs

Docket No. AD12-8-000

(Commission Staff Report Issued October 15, 2012)

The Federal Energy Regulatory Commission (Commission or FERC) requests Office of Management and Budget (OMB) review of a new collection, FERC‑922, Performance Metrics in Regions outside ISOs and RTOs, as contained in the Commission issuance in Docket No. AD12-8-000.

A. JUSTIFICATION

1. Circumstances That Make the Collection of Information Necessary

In September 2008, the United States Government Accountability Office (GAO) issued a report titled “Electricity Restructuring: FERC Could Take Additional Steps to Analyze Regional Transmission Organizations’ Benefits and Performance,” GAO-08-987. In its report, the GAO notes that “[t]he efficient and reliable operation of the electricity industry is critical to the health of the U.S. economy and the well-being of Americans.”¹ Pointing out that it has been over ten years since the advent of electricity restructuring and the emergence of ISOs and RTOs, the GAO noted that there is little agreement whether this has been good for consumers, what the impact has been on electricity prices and whether ISOs and RTOs have produced the benefits that the Commission envisioned.”² The GAO report further criticized the Commission’s existing measures of RTO performance because, among other things, such measures “do not compare performance between RTO and non-RTO regions.”³ Thus, the GAO report recommended that the Chairman of the Commission, among other actions, work with regional transmission organizations (RTO), independent system operators (ISO), stakeholders and other experts to develop standardized measures that track the performance of ISO/RTO operations and markets and report the results to Congress and the public annually, including providing: (1) an interpretation of what the measures indicate about the benefits of ISOs and RTOs; and (2) where appropriate, changes that need to be made to address any performance concerns.

Consistent with the goals outlined in GAO’s report, the Commission’s Strategic Plan for Fiscal Years 2009-2014 outlined a multi-year process for developing and implementing a common set of performance measures for markets both within and outside of ISOs and RTOs. The Commission considers it important to compare the performance of ISOs/RTOs with non-ISO/RTO regions because large portions of the country, notably the Pacific Northwest and the Southeast, have not engaged in restructuring and remain outside of ISOs/RTOs. In the Commission’s view, the benefits of ISOs/RTOs cannot be assessed in isolation, but are best considered in comparison with non-restructured regions. Indeed, the GAO and experts reached the same conclusion and recommend such comparison.⁴ Furthermore, as the metrics developed by Commission Staff seek to glean information in various categories, including reliability and congestion management, for example, the Commission aims to assess whether certain particular features of ISOs/RTOs demonstrate superior performance and/or certain (other) features of non-ISO/RTO regions demonstrate superior performance, with an eye towards improving the performance of each type of electricity market. This information will ultimately benefit electricity consumers.

Consequently, as recommended by the GAO, Commission Staff worked with representatives from all of the jurisdictional ISOs and RTOs to develop a set of performance metrics. Commission Staff and ISO/RTO representatives met with interested stakeholders to solicit their perspectives and comments on the proposed performance metrics. Commission Staff then released the proposed metrics for public comment in Docket No. AD10-5-000. In October 2010, Commission Staff issued a report addressing the comments received and recommending a final list of metrics for ISOs and RTOs. In December 2010, the ISOs and RTOs submitted information for the 2005-2009 period addressing the final metrics developed by Commission Staff. This information, along with a Commission Staff analysis, was included in a report sent to Congress in April 2011. The ISOs and RTOs subsequently submitted a report providing data for the 2006-2010 period.

In recognition of the finding in the GAO report that the Commission’s data gathering to date has not been comprehensive since, among other items, it does not compare performance between RTO and non-RTO regions,⁵ Commission Staff has developed metrics to measure performance in regions outside of ISOs and RTOs. Consistent with the process used in developing metrics for ISO/RTO markets, Commission Staff has worked with the Edison Electric Institute and its members to develop a set of performance metrics for regions outside of ISOs and RTOs. Commission Staff, along with the Edison Electric Institute and its members, met with interested stakeholders to solicit their perspectives and comments on the proposed performance metrics. These metrics are based on the metrics previously selected in Docket No. AD10-5, but have been tailored to fit markets outside of ISOs and RTOs.

As stated in the report to Congress in April 2011, the long-term purpose of developing and analyzing performance metrics is to assist the utility industry, stakeholders, and the Commission in evaluating industry trends and best practices. This stems from GAO’s recommendation to measure market performance in ISOs and RTOs and to recommend changes to enhance performance. Commission Staff has already worked to develop metrics to track performance in ISO and RTO markets. Utilizing metrics for regions outside of ISOs and RTOs will assist the Commission in evaluating market performance both within ISOs and RTOs and outside of such regions, identifying challenges, and developing best practices.

Commission Staff notes that the Commission is not requiring entities to report on the metrics. On the contrary, like in the ISO and RTO context, Commission Staff is engaged in a voluntary and collaborative process with utilities and other stakeholders to report on the metrics.

2. How, By Whom, And For What Purpose the Information Is To Be Used and the Consequences of Not Collecting the Information

The performance metrics for regions outside of ISOs and RTOs will be used by Commission Staff and the public to evaluate performance in markets both outside of ISOs and RTOs and in such regions. If Commission Staff did not use these metrics, it would not have the information necessary for a comprehensive assessment of the benefits of ISOs and RTOs and to recommend changes to address performance concerns, as directed by the GAO report referenced above. Without information allowing for a comparison of performance between ISOs/RTOs and regions outside of ISOs/RTOs, the Commission will not have the information necessary to identify the benefits of ISOs/RTO based on actual performance of ISOs/RTOs and regions outside of ISOs/RTOs over the same time periods. Furthermore, as noted above, the GAO report specifically criticized the Commission’s prior measures for evaluating ISO/RTO performance as “not comprehensive” because they “do not compare performance between RTO and non-RTO regions.”⁶ Consequently, the performance metrics in regions outside of ISOs/RTOs that were issued in October 2012 seek information that would address the GAO’s concerns.

3. Describe Any Consideration Of The Use Of Improved Technology To Reduce Burden And Technical Or Legal Obstacles To Reducing Burden.

Commission Staff expects to receive the data in this collection via the Commission’s electronic filing system (eFiling). By eFiling materials, respondents file the necessary information more efficiently than if they filed on paper. Further, Commission Staff is able to process the information more efficiently when the filings are submitted electronically versus in paper form.

4. Describe Efforts To Identify Duplication And Show Specifically Why Any Similar Information Already Available Cannot Be Used Or Modified For Use For The Purpose(S) Described In Instruction No. 2

While some of the information that will be reported in response to the metrics is available publicly or already reported to the Commission, the information collection at issue here is not intended to be a rote recitation of previously reported data. Instead, we expect that those entities that decide to provide data in response to the metrics will provide an analysis of what this information means. For example, where there are aberrations in the data or trends are apparent, we expect those that decide to report on the metrics will explain what caused these changes and their implications for performance. In this regard, Commission Staff anticipates that the metrics will result in a unique work product; one that will allow both the Commission and the public at large to look beyond the raw data and identify those factors that drive market outcomes and impact performance.

5. Methods Used To Minimize Burden in Collection of Information Involving Small Entities

This collection of information is voluntary. Small entities that do not want to participate because of the burden in the collection of information are not required to do so. Further, the Commission anticipates that the average burden of 245 hours per response is not significant, even for smaller entities.

6. Consequence to Federal Program If Collection Were Conducted Less Frequently

Commission Staff expects to release a report every other year using data for rolling five- year periods. Less frequent data collection would forestall dissemination of information on industry performance, delaying knowledge and ultimately industry-wide adoption of industry best practices and the benefits that result from their implementation.

7. Explain Any Special Circumstances Relating To the Information Collection

5 C.F.R. 1320.5(d) (2012) specifies that an agency should explain when a collection of information calls for any record-keeping requirement beyond three years. This collection does not explicitly require entities to keep data beyond three years, but does ask for data covering a five-year period. The Chairman believes it is in the respondent’s best interest to provide data over the full five-year period as this provides a longer average data period for analysis (smoothing over any anomalies).

The above referenced C.F.R. section also specifies that an agency should provide at least 30 days to respond to a collection of information. Depending on when OMB decides on this collection, the respondents may have less than 30 days in which to provide the data. The Commission does not consider this to impose any extra burden on the respondents, since the process to develop these metrics has been a collaborative one.

8. Describe Efforts to Consult Outside the Agency: Summarize Public Comments and the Agency's Response to These Comments

Commission Staff, along with the Edison Electric Institute and its members, met with interested stakeholders to solicit their perspectives and comments on the proposed performance metrics. Following this outreach, Commission Staff compiled proposed metrics (see attachment to this package) and the Commission issued a notice for publication in the Federal Register soliciting comment on the proposed metrics (76 Fed. Reg. 12,832, March 2, 2012). Comments were filed by Edison Electric Institute, Electric Power Supply Association), Joint Commenters,⁷ Multiple TDUs,⁸ and Northwest & Intermountain Power Producers Coalition. Edison Electric Institute filed reply comments.

Specific Comments

In general, commenters were supportive of the proposed metrics. However, Joint Commenters argue that the process used for developing performance metrics outside of ISOs and RTOs is fundamentally flawed. They claim that the metrics developed for ISOs and RTOs do not adequately measure performance because the Commission relied on the regulated ISOs and RTOs themselves to develop measures of their own performance. Thus, they maintain that any attempt to develop comparable metrics for public utilities outside of ISOs and RTOs is a fruitless endeavor. Moreover, they state that the Commission is making the same mistake here by allowing those entities that will eventually report under the metrics to drive their development. While they acknowledge that regulated entities have expertise that can inform the development of the metrics, they object to having regulated entities develop the metrics without the benefit of what Joint Commenters consider to be an open and transparent public process.⁹

In reply, Edison Electric Institute argues that the Joint Commenters overlook the fact that the metrics are the product of a collaborative process. In this regard, Edison Electric Institute notes that it and its members participated in Commission-led outreach sessions to discuss the proposed metrics and solicit feedback from stakeholders, which was taken into account before the metrics were issued for public comment. Edison Electric Institute notes that the Joint Commenters fail to provide reasons why the metrics will not be useful and cautions the Commission against ignoring the benefits of the metrics in favor of accepting the Joint Commenters’ unsupported claims.¹⁰

FERC Response

Commission Staff disagrees with Joint Commenters’ characterization of the process used to develop the metrics for regions outside of ISOs and RTOs as “fundamentally flawed.” Commission Staff used a process similar to the process that was used to develop metrics for ISOs/RTOs. Commission Staff initiated a process with Edison Electric Institute, its members, and other interested stakeholders to develop performance metrics tailored to regions outside of ISOs and RTOs. Since the goal is to develop metrics that are comparable for ISOs/RTOs and non-ISOs/RTOs, Commission Staff began by assessing which ISO/RTO metrics should apply to non-ISOs/RTOs, and tailored these metrics to the non-ISO/RTO context. Commission Staff met with representatives from various stakeholder groups and solicited comments prior to issuing the metrics for public comment. Commission Staff then provided an opportunity for public comment and, as further discussed below, Commission Staff has taken these comments into account when crafting a final list of metrics. Thus, Commission Staff concludes that the process was sufficiently interactive and transparent. Moreover, Commission Staff concludes that any benefits to be gained from restarting the process would not justify the attendant delay in using the draft metrics to gather performance data. Therefore, just as similar procedural criticisms were considered in the ISO/RTO metrics report, we also dismiss them here.

Comment on Burden Estimate

In its solicitation of comments, Commission Staff estimated the public reporting burden for participating utilities to be approximately 140 hours per respondent for each report.

Edison Electric Institute asserts that the response time could be as high as 300-400 hours.

FERC Response

Commission Staff will adjust the burden estimate based on Edison Electric Institute’s high estimate of 300-400 hours. Commission Staff considers Edison Electric Institute’s estimate to reflect the most time that it would take an entity to respond to the metrics. While Commission Staff recognizes that this report requires additional metrics and narrative discussions, Commission Staff nevertheless continues to conclude that 140 hours still represents a reasonable estimate of the burden, since much of the data required should be readily available to the responding utilities. However, in recognition of the fact that the burden will vary from entity to entity, we will revise our estimate to 245 hours per respondent¹¹, which is the mid-point between these estimates.

Other comments

Commenters also submitted comments regarding individual metrics. In response, Commission Staff have created additional metrics and there are more narrative discussion responses contemplated than what were initially proposed. These details, along with a summary of all other public comments, are described in the Commission Staff report attached in “supplemental documents” to this package.

9. Explain Any Payment or Gifts to Respondents

No payments or gifts have been made to respondents.

10. Describe Any Assurance of Confidentiality Provided To Respondents

Commission Staff generally does not consider the data to be confidential.

11. Provide Additional Justification For Any Questions Of A Sensitive Nature That Are Considered Private.

There are no questions of a sensitive nature that are considered private.

12. Estimated Burden of Collection of Information

The estimated reporting burden for the reporting requirements is as follows.

FERC-922 Requirements	Number of Respondents Annually (1)	Number of Responses per Respondent (2)	Average Burden Hours per Response (3)	Total Annual Burden Hours (1)x(2)x(3)
Metrics Data Collection	11	1	140	1,540
Write Performance Analysis			85	935
Management Review			20	220
Total			245	2,695

Respondents are asked to submit data every other year. Since FERC is requesting a three-year clearance from OMB for this collection the actual burden hours annually (averaged over the next three years) are 1,797 [(2,695 hours * 2)/3 years = 1,797 hours]. This three- year average figure will be submitted to OMB for this collection.

The Chairman has projected the cost of compliance to be $184,460 per each year the respondents submit data (every other year).

Technical Expertise = $168,300 (1,540 hours data collection + 935 hours report completion @ $68 per hour)

Management Review = $17,160 (220 hours report review @ $78 per hour)

Cost per hour figures are calculated using Bureau of Labor Statistics (BLS) data.¹² The technical expertise category factors in the median wage for an engineer, analyst, attorney and economist. The management category factors in the median wage for general and operations managers. Based on BLS data,¹³ both cost figures have been adjusted to include benefits (benefits represent 29.5% of the total hourly figure).

13. Estimate of Total Annual Cost Burden to Respondents

There are no non-labor start-up costs or other capital costs associated with this collection.

14. Estimated Annualized Cost to Federal Government

The estimated annualized cost to the Federal Government related to the data collections are shown below:

	Number of Employees (FTEs)	Estimated Annual Federal Cost
Analysis and Processing of Filings	0.04	$5,742
PRA Administrative Cost14		$1,588
FERC Total		$7,330

15. Reasons for Changes in Burden Including The Need For Any Increase

The estimated 1,797 burden hours annually are due to the time it will require to compile, review and submit the information requested. This is a new collection of information and there is no increase or decrease from a previously approved amount. The burden is necessary in order to collect the information and to be able to compare the performance of non-ISOs/RTOs and with the performance of RTOs/ISOs and to assess the performace of the RTOs/ISOs. Please note that the collection is done every two years and that the total burden and number of responses have been averaged over the next three years to correspond to the clearance that FERC is requesting for this collection.

The following table shows the total burden of the new collection of information. The format, labels, and definitions of the table follow the submission system’s “Information Collection Request Summary of Burden” for the meta-data.

FERC-922	Total Request	Previously Approved	Change due to Adjustment in Estimate	Change Due to Agency Discretion
Annual Number of Responses	7	Not applicable	-	7
Annual Time Burden (Hr)	1,797	Not applicable	-	1,797
Annual Cost Burden ($)	0	Not applicable	-	-

16. Time Schedule for Publication of Data

Commission Staff issued a report with recommended metrics concurrent with this submission. Commission Staff will make use the data as it sees fit.

17. Display of the Expiration Date

Once an expiration date and OMB Control Number are available FERC will include these on the final metrics sheet for this collection.

18. Exceptions to the Certification Statement

There are exceptions to the Paperwork Reduction Act Submission Certification. The Commission does not use statistical methodology for information filed under the new FERC-922. Also, this collection does not have any explicitly stated record retention requirements (see item 7 above for more detail).

1 GAO Report at 1.

2 Id. at 58.

3 Id. at 56.

4 Id. at 56, 57.

5 Id. at 56.

6 Id.

7 Joint Protesters are: AARP, American Public Power Association, Citizen Power, Electricity Consumers Research Council, and Virginia Citizens Consumer Council.

8 Multiple TDUs are: Public Works Commission of the City of Fayetteville, North Carolina, Lafayette Utilities System, and the City of Orangeburg, South Carolina.

9 Joint Commenters Comments at 2-3.

10 EEI Reply Comments at 2-3.

11 140 hours + 350 hours = 245 hours

12 See http://bls.gov/oes/current/naics4_221100.htm#(3)

13 See http://www.bls.gov/news.release/ecec.nr0.htm. The BLS news release indicates a current factor of 29.6% while we are using 29.5% for this submission.

14 Paperwork Reduction Act of 1995 (PRA). The amount of $1,588 is an internal estimate based on 24 hours of Commission Staff time per collection.

9