Voluntary Reporting of Performance Indicators

NRC INSPECTION MANUAL

DIRS

Manual Chapter 0608

PERFORMANCE INDICATOR PROGRAM

0608-01

PURPOSE

This Inspection Manual Chapter (IMC) provides guidance on the implementation of the
operating Reactor Oversight Process (ROP) Performance Indicator (PI) Program. It
includes guidance on the process for modifying existing PIs and for developing
additional PIs for use in the oversight process.

0608-02

OBJECTIVE

02.01 To provide policy and guidance regarding implementation of the ROP PI
Program, including data submission, verification, and posting of data and frequently
asked questions (FAQs) on the internal and external web.
02.02 To establish a formal process for responding to questions related to
interpretation of PI reporting guidance and for developing and implementing changes to
the PI Program, including creating new PIs and making changes to existing PIs or
thresholds.
02.03 To provide information to licensees that is required by the Paperwork Reduction
Act regarding licensee submission of PI data to the NRC.

0608-03

APPLICABILITY

This manual chapter applies to all operating commercial nuclear power reactors.

0608-04

DEFINITIONS

04.01 NEI 99-02 A guidance document used by licensees, which is jointly agreed
upon by the NRC and the Nuclear Energy Institute (NEI), published by NEI, and entitled
“Regulatory Assessment Performance Indicator Guideline.“
04.02 Feedback Form A form contained in IMC 0801, “Program Feedback,” used by
NRC staff to submit a request to the Performance Assessment Branch (IPAB) of the
0608
-1Issue Date: 02/27/07

Office of Nuclear Reactor Regulation (NRR) for clarification of the PI guidance
document.
04.03 Frequently Asked Question (FAQ) A question from an external stakeholder
regarding the PI Program or its implementation. All FAQs submitted to the NRC by
external stakeholders will be available on the NRC’s internal and external web sites and
will be discussed in public meetings. The web sites are periodically updated to include
draft FAQs (i.e., FAQs for which the response has not yet been approved), tentatively
approved FAQs, and FAQs that have been approved for use. FAQs can be viewed by
cornerstone, PI, posting date, or identification number.
04.04 ROP Working Group A group composed of NRC staff and licensee
representatives who meet typically once every month in an open public meeting to
discuss FAQs and other issues related to the ROP PI Program.
04.05 Self-Assessment An annual report to the Commission on the Reactor Oversight
Process.
04.06 Extended Shutdown For the purposes of the PI Program, a plant is considered
to be in extended shutdown when the reactor has been subcritical for at least six
months.

0608-05

RESPONSIBILITIES AND AUTHORITIES

05.01 Director, Office of Nuclear Reactor Regulation (NRR)
a.

Provides overall policy direction for the PI Program.

b.

Directs the development and implementation of policies, programs, and
procedures for the PI Program and oversight of program effectiveness and
implementation.

05.02 Director, Division of Inspection and Regional Support (DIRS). Manages PI
Program development and implementation within NRR and oversees program
implementation and effectiveness.
05.03 Chief, Performance Assessment Branch
a.

Develops policy, programs, and procedures for implementation of the PI
Program.

b.

Receives and posts PI data and FAQs on the internal and external web.

c.

Manages and implements the process for responding to questions related to
interpretation of PI reporting guidance and develops and implements changes to
the PI Program, including creating new PIs and making changes to existing PIs
or thresholds.

0608

-2-

Issue Date: 02/27/07

d.

Assesses PI Program effectiveness and implementation.

05.04 Regional Administrator. Manages regional implementation of the PI Program in
accordance with the requirements of this IMC, Management Directive (MD) 8.13,
“Reactor Oversight Process,” Inspection Procedure (IP) 71151, “Performance Indicator
Verification,” and IP 71150, “Discrepant or Unreported Performance Indicator Data.”
0608-06
BACKGROUND
06.01 Framework
The ROP is built upon a framework directly linked to the Agency’s mission. That
framework includes cornerstones of safety that focus on the licensee’s ability to (1) limit
the frequency of initiating events; (2) ensure the availability, reliability, and capability of
mitigating systems; (3) ensure the integrity of the fuel cladding, the reactor coolant
system, and containment; (4) ensure the adequacy of the emergency preparedness
functions; (5) protect the public from exposure to radioactive material releases; (6)
protect nuclear plant workers from exposure to radiation; and (7) provide assurance that
the physical protection system can protect against the design-basis threat of radiological
sabotage.
Within each cornerstone, a broad sample of data on which to assess licensee
performance in risk-significant areas is gathered from PI data submitted by licensees
and from the NRC’s risk-informed baseline inspections. The PIs are not intended to
provide complete coverage of every aspect of plant design and operation, but they are
intended to be indicative of performance within the related cornerstone.
Data submitted by each licensee are used to calculate PI values. These values are
then compared to objective thresholds to determine the performance band associated
with those values. The bands are color coded. Plant data for a PI that falls within the
“green” band indicates licensee performance is within the nominal, expected range.
The “white” band indicates that performance is outside of the nominal, expected range
and can be characterized as of low to moderate safety significance, but performance
remains acceptable. Performance in the “yellow” band indicates a more significant
decline in performance and can be characterized as being of substantial significance.
Performance is considered acceptable, but a reduction in safety margin exists.
Performance in the “red” band indicates a very significant decline in performance.
Changes can be characterized as being of high safety significance. Performance may
be acceptable with a significant reduction in safety margin or may be unacceptable.
06.02 Performance Indicators
PIs are a means of obtaining information related to the performance of certain key
attributes in each of the cornerstone areas. They provide indication of problems that, if
uncorrected, may increase the probability and/or the consequences of an off-normal
event. Since not all aspects of licensee performance can be monitored by PIs, safety
significant areas not covered by PIs will be assessed through inspection.
a.

For the reactor safety strategic performance area, the objectives of the
cornerstones and PIs are as follows:

Issue Date: 02/27/07

-3-

0608

1.

Initiating Events - this cornerstone is intended to limit the frequency of those
events that upset plant stability and challenge critical safety functions during
power operations. Such events include a reactor trip due to a turbine trip,
loss of feedwater, loss of off-site power, and other reactor transients. The
following indicators are provided in this cornerstone:
•
Unplanned scrams (automatic and manual) per 7,000 critical hours
•
Scrams with loss of normal heat removal
•
Unplanned power changes per 7,000 critical hours

2. Mitigating Systems - this cornerstone is intended to ensure the availability,
reliability, and capability of systems that mitigate initiating events to prevent
reactor accidents. Mitigating systems include those associated with safety
injection, residual heat removal, and their support systems, such as
emergency AC power. The following indicators are provided in this
cornerstone:
•

Safety System Functional Failures - this PI monitors the readiness of the
most important systems to perform their safety function(s).

•

Mitigating System Performance Index - this PI is calculated separately
for each of the following five systems for each reactor type:

BWRs
- emergency AC power systems
- high pressure injection systems (high pressure coolant injection, high
pressure core spray, or feedwater coolant injection)
- reactor core isolation cooling and/or isolation condenser systems
- residual heat removal systems (or the equivalent function)
- cooling water support systems for the above systems
PWRs
- emergency AC power systems
- high pressure safety injection systems
- auxiliary feedwater systems
- residual heat removal systems (or the equivalent function)
- cooling water support systems for the above systems
3.

Barrier Integrity - this cornerstone is intended to ensure the integrity of the
physical barriers designed to protect the public from radionuclide releases
caused by accidents. These barriers are the fuel cladding, reactor coolant
system boundary, and containment. The following indicators are provided in
this cornerstone:
•
•

0608

Reactor Coolant System (RCS) Specific Activity
RCS Identified (or total) Leak Rate

-4-

Issue Date: 02/27/07

4.

Emergency Preparedness - this cornerstone is intended to ensure that
actions taken in accordance with the emergency plan provide adequate
protection of public health and safety during a radiological emergency. The
cornerstone does not include off-site actions, which are covered by the
Federal Emergency Management Agency. The following indicators are
provided in this cornerstone:
•
•
•

Drill/Exercise Performance
Emergency Response Organization Drill Participation
Alert and Notification System Reliability

b. For the radiation safety area, the cornerstones and PIs are as follows:
1.

Occupational Radiation Safety - this cornerstone is intended to ensure
adequate protection of worker health and safety from exposure to radiation
and radioactive materials during routine civilian nuclear reactor operations.
The following indicator is provided in this cornerstone:
•

2.

Occupational Exposure Control Effectiveness

Public Radiation Safety - this cornerstone is intended to ensure adequate
protection of public health and safety from exposure to radiation and
radioactive materials released into the public domain as a result of routine
civilian nuclear reactor operations. These releases include routine gaseous
and liquid radioactive effluent discharges, the inadvertent release of solid
contaminated materials, and the offsite transport of radioactive materials and
wastes. The following indicator is provided in this cornerstone:
•

Radiological Effluent Technical Specifications (RETS)/Offsite Dose
Calculation Manual (ODCM) Radiological Effluent Occurrences

c. For the safeguards area, the cornerstone and PIs are as follows:
1.

Physical Protection - this cornerstone is intended to provide assurance that
the physical protection system can protect against the design basis threat of
radiological sabotage. The threat could come from either external or internal
sources. Although the NRC is actively overseeing the Physical Protection
Cornerstone, the Commission has decided that the related performance
indicator, inspection, and assessment information will not be publicly
available.

0608-07 PI DATA SUBMISSION
07.01 Reporting of PI Data
The information in this paragraph is required by the Paperwork Reduction Act of 1995
(44 U.S.C. 3501 et seq.). Licensee submission of PI data constitutes an information
collection, and therefore, must comply with requirements in the Paperwork Reduction
Issue Date: 02/27/07

-5-

0608

Act . The information collections contained in this manual chapter were approved by
the Office of Management and Budget, approval number 3150-0195, which expires
11/30/08. The burden to the public for these voluntary information collections is
estimated to be 200 hours per response, and 40 hours annually per utility to maintain
records. This information is used by NRC to evaluate licensee performance as part of
the reactor oversight process. Send comments regarding this burden estimate to the
Records and FOIA/Privacy Services Branch (T-5 F52), U.S. Nuclear Regulatory
Commission, Washington, DC 20555-0001, or by email to
INFOCOLLECTS.RESOURCE@NRC.GOV; and to the Desk Officer, Office of
Information and Regulatory Affairs, NEOB-10202, (3150-0195), Office of Management
and Budget, Washington, DC 20503. Additionally, the NRC may not conduct or
sponsor, and a person is not required to respond to, a request for information or an
information collection requirement unless the requesting document displays a currently
valid OMB control number.
Reporting of PI data to the NRC is a voluntary program in which all licensees of
operating reactor plants participate. To submit PI data, licensees send a delimited text
file to the NRC at pidata@nrc.gov. Hard copy submissions, in accordance with 10 CFR
50.4 “Written Communications,” are not required, except in the event that the email
submission is unsuccessful. Within two business days of receipt of the PI data, the
NRC will send each licensee a return email to confirm and authenticate receipt of the
data. Licensees have four business days from receipt of the NRC’s email to report any
transmission problems to the NRC.
Once the data are confirmed by the NRC, they are entered into the Reactor Program
System database to calculate the indicator values. Within five business days from
receipt of the licensees’ data transmissions, the NRC will post the data, the indicator
values, and associated graphs on the NRC’s internal web site. The regions will be
notified when the PIs are available on the internal web site. This is to allow the regions
an opportunity to become familiar with the PIs and to identify any obvious
inconsistencies prior to public release. Within 10 business days of receipt of the
licensees’ data transmittals, the NRC will place the PIs on the NRC’s external web site
to make them available to external stakeholders.
07.02

PI Submission For Plants In Extended Shutdown

An operating commercial nuclear power plant with significant performance or equipment
problems may be shut down for an extended period of time for a variety of reasons.
Licensees may voluntarily or involuntarily shut down the plant due to significantly
degraded performance, major equipment failures, or a significant plant event. In these
cases, the NRC may make the decision to place the plant under the process described
in Inspection Manual Chapter 0350, “Oversight of Reactor Facilities in a Shutdown
Condition Due to Significant Performance and/or Operational concerns.” Plants in
extended shutdown should report PIs in accordance with the guidance provided in the
current version of NEI 99-02.

0608-08 PI VERIFICATION
0608

-6-

Issue Date: 02/27/07

08.01 Because of the importance of PIs in the ROP as a source of information regarding
performance upon which agency actions will be based, PI data must be reported
accurately. Inspection Procedure 71151, “Performance Indicator Verification,” shall be
conducted to review licensees’ PI data collection and reporting activities for adherence
to pertinent guidance. It is expected that licensees will make reasonable, good faith
efforts to comply with the guidance in NEI 99-02. This includes taking appropriate and
timely action to identify and report performance issues captured by the indicators. It
may be necessary for inspectors to exercise some judgment on the adequacy of
licensee actions to make a reasonable, good faith effort to comply with the guidance.
Discrepancies with the performance indicator data collection and reporting or with the
actual data should be documented in accordance with IP 71151 and the requirements of
Inspection Manual Chapter 0612, “Power Reactor Inspection Reports.”
Enforcement action will be taken for inaccurate PI reporting in accordance with the
“General Statement of Policy and Procedure for NRC Enforcement Actions.”
08.02 Discrepant or Unreported PIs
In the event the NRC determines that major discrepancies exist in the PI data submitted
by a licensee that causes the Agency to lose confidence in the licensee’s ability to
collect and report PI data accurately, the affected PI(s) will be classified as discrepant.
Examples of situations in which a PI would be considered to be discrepant may include
but are not limited to the following: (1) recurring errors in the reported data; (2) recurring
instances of incorrect interpretations of NEI 99-02; or (3) inadequate documentation of
PI data.
When PI data has been determined to be discrepant or is not being reported by the
licensee, IP 71150, “Discrepant or Unreported Performance Indicator Data,” will be
conducted. IP 71150 provides for the performance of selected inspection activities to
compensate for the discrepant or unreported PI data. Regional management should
coordinate activities in this area with IPAB. The selected inspections will be performed
in addition to the baseline inspection. Once the licensee has corrected the root
cause(s) of the discrepant or unreported data and the NRC has verified that the
licensee can collect and report PI data accurately, oversight of PI reporting in
accordance with IP 71151 will resume.
When a plant has been in an extended shutdown, some PIs may not provide a
meaningful indication of plant performance in the areas they are intended to monitor. In
these situations, the guidance provided in IP 71150 should be followed to obtain
sufficient performance information via the inspection program until the plant has
restarted and sufficient PI data has been collected.

0608-09 QUESTIONS AND FEEDBACK
The NRC has received many questions and comments regarding the PIs over the
years. The staff expects that changes to existing PIs and thresholds as well as
development of new PIs will occur. Therefore the NRC has established a formal
Issue Date: 02/27/07

-7-

0608

process to address questions and feedback from internal and external stakeholders,
make changes to existing PIs and thresholds based on lessons learned, and develop
new PIs and associated thresholds. This formal process is provided in Exhibit 1, “PI
Process For Addressing Feedback and Questions.” The process consists of the
following four major components: input, evaluation, resolution, and closure.
The remainder of this IMC describes the formal process. Exhibit 1 (flowchart) may be
referred to at any point hereafter to gain an understanding of the four phases of the
formal process. The section numbers of this IMC are included at corresponding points
in the flowchart to provide easy reference to the write-up of the respective section.
09.01 Input
NRC staff, industry, or the public may raise questions or provide feedback regarding an
individual PI. Questions from the NRC staff should be documented in a Feedback Form
and submitted to IPAB in accordance with IMC 0801. Questions raised by industry
personnel should be documented in an FAQ and submitted to an industry member of
the ROP Working Group. These questions will be provided to the NRC at periodic
public meetings of the ROP Working Group. Questions raised by the public or other
stakeholders should be submitted via email to the Office of Public Affairs at
opa@nrc.gov. Alternatively, questions from the public can be submitted in writing to the
United States Nuclear Regulatory Commission, Office of Public Affairs, Washington
D.C. 20555. Regardless of their origin, all questions will be addressed in accordance
with the process described below.
An NRC staff member with a question about a PI should first look in NEI 99-02 for
guidance. If further clarification is required, he or she should search the FAQs on the
NRC’s internal or external web sites. If neither of these provides sufficient clarification,
he or she should fill out a feedback form from the internal ROP web page. The
inspector may also provide feedback on issues related to PI guidance or
implementation, including observed or perceived instances of unintended
consequences. Feedback forms will be forwarded to regional management for review
and response, as appropriate. The purpose of the regional review is to allow regional
management to be aware of questions or feedback being submitted and to provide an
opportunity for the regions to resolve the issue for those they believe do not warrant
headquarters review. All forms, including those to which the region has responded, will
then be forwarded to the PIPBCAL email mailbox or mailed to the Chief, IPAB (mail stop
O7H4).
Upon receipt, IPAB will perform an initial screening of each question and feedback.
IPAB will assign a lead reviewer from within IPAB, and one from the appropriate
technical branch, if necessary. The lead reviewer will forward a reply to the originator
within 14 business days to acknowledge receipt of the form and to inform the originator
of the Feedback Form tracking number. Similarly, NRC will acknowledge receipt of
questions and feedback provided directly to the NRC from members of the public or
from members of industry. This response will be in the form of written correspondence.
All follow-up questions should be directed to the lead reviewer.

0608

-8-

Issue Date: 02/27/07

Differences in interpretation of the PI guidance between the inspector and the licensee
are the only issues to be entered, via feedback form, into the PI feedback process.
Issues involving technical differences should follow already established agency
processes such as a task interface agreement, or a conference call with the NRR
technical reviewer, the licensee, and regional staff.
09.02 Evaluation of Questions or Feedback
Issues that only require an explanation of the existing guidance will be promptly
resolved. The lead reviewer will provide the originator with the explanation and the
issue will be closed out in accordance with “Closure” (Section 09.05).
Questions or feedback that require modification to the guidance to clarify meaning or
intent will be addressed in accordance with “Resolution of Questions and Feedback not
Requiring a PI Change” (Section 09.03).
Questions or feedback in which the resolution would require a new PI or a change to an
existing PI or threshold will be addressed in accordance with “Resolution of Questions
and Feedback Requiring a PI Change” (Section 09.04) and subsequent steps.
09.03 Resolution of Questions or Feedback Not Requiring a PI Change
The following steps will be performed to resolve questions or feedback that do not
require a PI or threshold change:
a. The ROP Working Group will review the question and develop a proposed
response. DIRS staff will involve the regions and NRR technical staff when
necessary in developing the response.
b. The issue will be discussed at a public meeting of the ROP Working Group to
arrive at tentative approval of the question and its proposed response. Although
it is desirable that tentative approval be achieved by the close of the meeting in
which the issue is first discussed, this portion of the process is iterative and could
take several working meetings. In the event NRC and its stakeholders are unable
to reach alignment on the issue being discussed, the DIRS Director will make the
final decision. Regardless of whether or not tentative approval is achieved by the
conclusion of the meeting, NEI will enter the new FAQs into a log that contains
draft FAQs and will provide a copy of the electronic file to the NRC. The NRC will
make the FAQs available to the public, industry and other stakeholders on the
ROP internal and external web pages.
c. Following tentative approval, the FAQ will be held for a waiting period – normally
until the next regularly scheduled meeting – to allow a final opportunity for all
stakeholders to review the proposed FAQ and provide any input. Stakeholders
should forward any feedback that impacts the resolution of the issue to the
assigned lead reviewer for resolution prior to the next scheduled public meeting.
The schedule for upcoming meetings is posted on the ROP web-page.

Issue Date: 02/27/07

-9-

0608

d. At the conclusion of the waiting period, the ROP Working Group will consider any
additional input and will issue its final conclusion. IPAB will then place the
approved FAQ on the internal and external web pages and will notify appropriate
internal stakeholders of the resolution. NEI will notify licensees of the updated
FAQ.
e. NEI 99-02 will be updated periodically, as appropriate, to clarify the PI reporting
guidance based on insights from the resolution of the FAQs.
09.04 Resolution of Question/Feedback Requiring A Change
Questions or feedback that raise issues which require more than clarification of
reporting guidance or policy will be addressed as described below. Resolution may
involve creating a new PI, changing an existing PI, changing a threshold for an existing
PI, or changing an existing PI to reflect a unique plant design features. Each of the
processes share common steps, but will be discussed separately.
Developing new PIs or making changes to existing PIs or thresholds can require
significant NRC resources. Prior to expending those resources, the DIRS Director will
determine whether the proposed change appears to be feasible and is therefore
justified. For those changes that would clearly not be feasible, the DIRS Director will
suspend consideration of the change and provide a response to the originator that
includes a rationale for not proceeding. The issue will then be closed out.
If a change appears to be feasible, one of the four steps described below will be
followed.
a. New PI
When an existing PI is ineffective, consistently generates many FAQs, or has the
potential to be misleading or to create unintended consequences, there may be a
need to develop a new PI. A proposed new PI should provide indication of
licensee performance in the same cornerstone(s) as the existing PI.
Once the need for a new PI has been determined and the scope of the
information the PI will cover has been identified, the ROP Working Group will
propose a definition for the PI, including draft reporting criteria. NRC will consider
previous lessons learned and any stakeholder feedback in developing the
proposed definition. The proposed PI will be discussed at a public meeting of the
ROP Working Group to develop a mutually agreed upon definition. The proposed
PI will be made available to internal and external stakeholders for comment via
the NRC ROP web site. Following the comment period, NRC and the Industry
ROP Working Group will review the comments provided and make appropriate
changes to the PI as necessary.
Early consideration should be given to the potential need for a revised OMB
Clearance for the new PI to ensure clearance processing will not adversely impact
final PI implementation.
0608

-10-

Issue Date: 02/27/07

Following the development of the final proposed PI definition and reporting
guidance, the NRC must determine the efficacy of the PI. The PI must be
benchmarked against past industry performance data to determine whether the
results obtained from the PI would be indicative of current plant performance. If
historical data are available, the ROP Working Group will collect the data to
determine if the PI can identify declining performance in a timely manner so that
increased regulatory attention can be applied before performance becomes
unacceptable. In the event that historical data is not available, NRC and the
Industry ROP Working Group will use the best information available.
If the proposed PI cannot identify declining performance in a timely manner, the PI
must either be revised prior to proceeding or development efforts discontinued.
Once the PI has been successfully benchmarked, the ROP Working Group will
consider whether the PI will provide information that is not currently being
collected and adds benefits commensurate with the reporting burden. In the
event the PI does not provide information that would make its continued
development and implementation beneficial, it must be revised or it will be
discontinued.
The ROP Working Group will conduct a pilot test or a tabletop exercise using a
representative sample of plants to collect data for the proposed PI, in addition to
continuing to collect data on the existing PIs. The staff must obtain a new OMB
clearance approval before the pilot testing can begin if the number of plants in the
pilot exceeds nine. The purpose of this pilot or tabletop exercise is to conduct a
real-time test of the proposed guidance, to establish thresholds, and to determine
the effectiveness of the proposed PI. When the pilot or the tabletop exercise has
been completed, NRC will provide an opportunity for the industry, public, and
other stakeholders to provide feedback. This feedback, along with lessons
learned from the pilot, will be evaluated by the staff and may be used to modify
the proposed PI and/or its thresholds, as necessary.
In conjunction with adding a PI, NRC will consider whether changes to the
baseline Inspection Program are warranted to eliminate potential overlap or
ensure coverage of key attributes.
After the ROP Working Group has agreed on final changes to the proposed PI
and thresholds, NEI will, in collaboration with the NRC staff, revise NEI 99-02 .
IPAB will update the web page as appropriate to include the new PI data. The
ROP Working Group may conduct training, as deemed necessary. IPAB will
issue a Regulatory Information Summary (RIS) to inform stakeholders of the new
PI and its reporting criteria. The RIS will also be placed in NRC’s Public
Document Room and on the external web-site at
http://nrr10.nrc.gov/NRR/OVERSIGHT/ASSESS/INDEX.html, which can be
accessed from the Inspection Manual of Agency Wide Applications. IP 71151 will
be revised to reflect the new PI.
b. Changes To An Existing PI
The process for making a significant change to an existing PI is similar to creating
a new PI. Like the initial steps in creating a new PI, NRC must ensure that the
Issue Date: 02/27/07
-110608

revised PI will provide an indication of licensee performance in the key attributes
of the cornerstone for which the existing PI was intended. The ROP Working
Group will conduct public meetings with all stakeholders to discuss and reach
agreement on the proposed change, including the PI definition and reporting
criteria. The proposed change will be made available to internal and external
stakeholders for comment via the NRC ROP web site. Following the comment
period, NRC and NEI will review comments provided and make changes to the PI
as appropriate. This process is iterative and allows all stakeholders an
opportunity to contribute to the process.
Early consideration should be given to the need for a revised OMB Clearance for
the modified PI to ensure clearance processing will not adversely impact final PI
implementation.
If the proposed change is approved, the NRC and NEI will identify a
representative sample of plants that are willing to pilot test the proposed change
by collecting data for the modified PIs while continuing to provide data on the
existing PIs. The staff must obtain a new OMB clearance approval before the
pilot testing can begin if the number of plants in the pilot exceeds nine. The
purpose of a pilot program is to conduct a real-time test of the proposed guidance,
validate the thresholds, and ensure the effectiveness of the PI. When the pilot
has been completed, NRC will provide an opportunity for the industry, public, and
other stakeholders to provide feedback. This feedback, along with lessons
learned from the pilot, may be used to modify the proposed PI as appropriate.
After the ROP Working Group has agreed on final changes to the PI, NEI will
revise NEI 99-02 accordingly and IPAB will update the web page as appropriate to
reflect the changes. The ROP Working Group may conduct training as deemed
necessary. IPAB will issue a RIS to inform stakeholders of the PI change and
approve the use of the new PI. The RIS will be placed in the NRC’s Public
Document Room and on its external web site at
http://nrr10.nrc.gov/NRR/OVERSIGHT/ASSESS/INDEX.html, which can be
accessed from the Inspection Manual of Agency Wide Applications. Additionally,
IP 71151 will be revised to reflect the new PI.
c. Change In Threshold(s)
Thresholds may need to be adjusted based on lessons learned from experience
with individual PIs. Such adjustments is are not intended to continually raise
licensee performance expectations, but rather they are intended to ensure that the
initial thresholds, some of which were established without the benefit of actual
industry performance data, are performing as intended.
Once the need for a threshold change has been identified, the NRC and Industry
ROP Working Group will meet in a public forum to discuss and reach agreement
on the proposed threshold change. The proposed change will be made available
to internal and external stakeholders for comment via the NRC ROP web site.
Following the comment period, the ROP Working Group will review the comments
in a public meeting. If the Working Group is unable to reach agreement on a new
0608

-12-

Issue Date: 02/27/07

threshold, the issue will be elevated to higher levels of NRC management to make
the final decision.
IPAB will issue a RIS to inform stakeholders of the threshold change and will
modify the web page accordingly. The RIS will provide guidance on when the
revised threshold will become effective. The RIS will be forwarded to the regional
Directors of Reactor Projects, Reactor Safety, and Plant Support; inspectors; and
NEI. Additionally, the RIS will be placed in NRC’s Public Document Room and on
the external web-site,
http://nrr10.nrc.gov/NRR/OVERSIGHT/ASSESS/INDEX.html., which can be
accessed from the Inspection Manual of Agency Wide Applications.
d. Unique PI
With multiple reactor designs, plants may have unique design features that make
compliance with the data reporting criteria established in NEI 99-02 impossible,
impractical, or ineffective.
In such cases, the ROP Working Group will form a sub-group that includes
representatives of the affected licensees to develop unique criteria to
accommodate plant-type differences. If historical data are available, it will be
collected and used in this effort. When historical data are unavailable, an expert
panel will be assembled to identify appropriate thresholds based on experience.
The NRC will then follow the remainder of the guidance outlined in Section C,
Change In Threshold(s), to complete this process.
09.05 Closure. Once an issue has been resolved, the lead reviewer will notify the
originator of the final response. This notification will normally occur via email and within
14 business days after NRC has reached a resolution. The completion date will be
entered into the Feedback Form tracking system and the issue will closed out.
If a licensee disagrees with the resolution documented on a feedback form, the licensee
should submit an FAQ to the ROP Working Group to present at the next ROP Working
Group meeting. The FAQ process outlined in section 9.03 will be followed.

0608-10 PI REFERENCES
Management Directive 8.13, “Reactor Oversight Process”
SECY-99-007, “Recommendations For Reactor Oversight Process Improvements”
SECY-99-007A, “Recommendations For Reactor Oversight Process Improvements
(Follow-up to SECY-99-007)”
SECY-00-049, “Results Of The Revised Reactor Oversight Process Pilot Program”
Temporary Instruction 2515/144, “Performance Indicator Data Collecting and Reporting
Process Review”
Issue Date: 02/27/07

-13-

0608

Inspection Procedure 71151, “Performance Indicator Verification”
Inspection Procedure 71150, “Discrepant or Unreported Performance Indicator Data”
NEI 99-02, “Regulatory Assessment Performance Guideline,” (Revision 5)
Regulatory Information Summary 99-06, “Voluntary Submission Of Performance
Indicator Data” (collecting and reporting historical data)
Regulatory Information Summary 2000-08, “Voluntary Submission Of Performance
Indicator Data” (collecting and reporting data reflecting plant performance during full
implementation of revised reactor oversight process)
General Statement of Policy and Procedure for NRC Enforcement Actions
Manual Chapter 0350, “Oversight of Reactor Facilities in a Shutdown Condition Due to
Significant Performance and/or Operational Concerns”
Web-site For Frequently Asked Questions:
http://NRR/OVERSIGHT/ACCESS/FAQs_by_pi_pdf
ROP Web-site: http://nrr10.nrc.gov/NRR/ROP_DIGITAL_CITY/ROP_digital_city.html

END

0608

-14-

Issue Date: 02/27/07

PERFORMANCE INDICATORS
Process for Addressing Questions and Feedback
FAQ’s

NEI
Tracking
System

9.01

9.02

ROP
Monthly
Meeting

Major Change
to Existing PI or
New PI?

9.04

Discuss and
Evaluate at ROP
Meeting

YES

Public Input
Feedback Forms
Self-Assessment

NRC
Tracking
System

NO

Lessons Learned

Respond and
close out

END

Existing
Policy
or
Previously
Addressed

YES

Practical ?

C

NO

D

NO
Respond to
Initiator and
close out

9.03

Develop proposed
Reponse
A
B

TO NRC
Division
Director

YES

Discuss at ROP Meeting
END

Third ROP
Meeting?

Agreement
&
Tentative
approval

NO

NO

YES
NO

Issue
approved
response

B

Waiting period for
stakeholder comments

A

A

YES

Changes?

NRC Update
Web
NEI Update
FAQ log

Update
guidance as
necessary
Document in
next revision

END

Exhibit 1
Part 1

Issue Date: 02/27/07

-15-

0608

PERFORMANCE INDICATORS
Process for Addressing Questions and Feedback
(Continued)
Sort Issues
a.
New PI
or
Change to
Existing PI

C

Define PI
y Identify Key Attributes
y Update OMB Clearance

Solicit
Input from
Stakeholders

Historical
Data
Available?

Yes

Collect
Data

Establish
Threshold

No
Use Best
Data Available
or Assemble
Expert Panel

Is PI
Giving
Information
Not Already
Known?

Benchmark
Results

No

Review
Feedback and
Lessons
Learned

Yes

Does
PI Addition
or Change Add
Enough Value to
Justify Burden
Resources?

Yes

Pilot Program or
Table Top
Exercise

Consider
Adjustments to
Baseline
Inspection
Program

Solicit Public
comment

No

D

D

Training

Update
NRC Bases
documents
NEI 99-02 and
Webpage
As Appropriate

IPAB
Issues
Regulatory
Issue Summary
(RIS)

END

E

b.
Threshold
Change

c.
Unique
(Plant Design)
PI

Establish
Proposed
Threshold(s)

Form
AdHoc Group
Representing
Owners Group
of Unique
Design

Benchmark
Results

Define PI
y Verify Key
Attributes
Captured

Incorporate
Results as
Appropriate

Solicit
Input from
stake holders

Historical
Data
Available?

Yes

Update
NRC Bases
Documents
NEI 99-02 and
Web Page
As Appropriate

Collect
Data

IPAB
Issues
Regulatory
Issue
Summary
(RIS)

Establish
Threshold(s)

END

E

No
Use Best
Data Available
or Assemble
Expert Panel

Exhibit 1 - Continued
Part 2

Issue Date: 02/27/07

Att A1-1

ATTACHMENT 1
Revision History for IMC 0608

Commitment
Tracking
Number
N/A

Issue Date

02/27/07
CN 07-007

Issue Date: 02/27/07

Description of Change

IMC0608 Delete SSU, add MSPI;
update flow charts; add definitions

Training Required

Training
Completion Date

Comment Resolution
Accession Number

None

N/A

N/A

Att A1-1

0608 Att 1