Download:
pdf |
pdfMAJOR FACILITIES GUIDE
Prepared by the Large Facilities Office in the Budget,
Finance, and Award Management Office (BFA-LFO)
Draft for OMB Approval
NSF 21-107
July 2021
Major Facilities Guide: NSF 21-107 (July 2021)
Summary of Significant Changes
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
SUMMARY OF SIGNIFICANT CHANGES
This document has been cleared for publication under OMB approval number 1315-0239, which
expires 9/30/2022. The purpose of this revision is to update the material and to improve the
clarity and legibility of the guide for the targeted audience of users both inside and outside NSF.
A summary of the changes for this revision is given below. Footers within each section of this
document indicate the last revision date of the content in that section, while all page headers in
the document include the date and NSF number of the current version of the Major Facilities
Guide.
1.
In Section 1.4, Applicable Legislation and NSF Policy, revisions were made in response to
new legislation regarding Congressional notification of total project cost increases and
planned divestments and the amended definition of a major multi-user research facility
project.
2.
In Section 1.4.2, MREFC Threshold, updated the threshold for MREFC account eligibility.
3.
In Section 2.1.3, The Major Facility Life Cycle, revisions were made to clarify the
description of each life cycle stage and the timing of strategic assessment of projects.
4.
In Section 2.2, Development Stage, revisions were made to clarify definition of the
Development Stage.
5.
In Section 2.3, Design Stage – Conceptual, Preliminary, and Final Design Phases, revisions
were made to clarify the design requirements at the individual phases and the
requirements to transition into and out of the Design Stage and between phases and the
timing of strategic assessment of projects.
6.
In Section 2.4.1, Construction Award Management and Oversight, revision was made to
clarify the frequency of post-award reviews.
7.
In Section 2.6, Divestment Stage, revisions were made to clarify the definition of the
Divestment Stage.
8.
In Subsection 3.4.2.15, Commissioning, provided more detailed guidance on Segregation
of Funding Plans.
9.
In Subsection 4.2.5.1, Implementation of NSF’s No Cost Overrun Policy, added the
congressional notification requirement regarding Total Project Cost (TPC) increases of
10% or more.
10. Various edits were made to List of Acronyms, Lexicon, and Subsections 4.2.5.7, 4.2.5.8,
6.2.8.1, 6.2.11.1, and 6.2.11.4 to clarify the definitions of estimate at completion (EAC),
estimate to complete (ETC), risk exposure, risk-adjusted estimate at completion (RAEAC),
and liens list.
Section Revision:
July 26, 2021
i
Major Facilities Guide: NSF 21-107 (July 2021)
Summary of Significant Changes
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
11. New material was added to the reserved Section 4.3 Schedule Development, Estimating,
and Analysis.
12. In Section 4.6.2, Recipient Performance Reports, revisions were made to clarify the risk
management reporting in the construction monthly report.
13. In Sections 4.6.3.4 Incurred Cost Audits and 4.6.3.5 Accounting System Review or Audits,
revisions were made to clarify requirements for consistency with practices.
14. Section 4.6.6, Project Personnel and Competencies was added to establish criteria for key
personnel and project teams for major facility projects.
15. In Section 4.7.1, Partnerships Overview, revisions were made to clarify the notification
requirements regarding foreign collaborations.
16. In Section 5, Guidance for Mid-Scale Research Infrastructure Projects, revisions were
made to clarify that the Project Execution Plan (PEP) is for the construction stage and
that the requirement is for “performance measurement and management” not “earned
value management (EVM)”. EVM is a technique for performance measurement and
management.
17. In Section 6.3, Guidelines for CyberSecurity of NSF’s Major Facilities, add new material
regarding the fourth pillar, Mission Alignment, of information security programs and
updated website addresses.
18. In Section 6.6, Guidelines for Property Management, clarified terminology and
expectations associated with property management terms and conditions.
19. In Section 6.8, Guidelines for Earned Value Management Systems (EVMS), clarified the
difference between tailored and scaled EVMS and provided guidance on application of
scaled EVMS for mid-scale projects.
20. In Section 8, List of Acronyms, added new acronyms introduced in Section 4.3, Schedule
Development, Estimating, and Analysis.
21. In Section 9.2, Terms and Definitions, clarified the definitions for Re-Baselining and RePlanning and added definitions associated with schedule development and property
management.
Section Revision:
July 26, 2021
ii
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1 INTRODUCTION............................................................................................................. 1.1-1
1.1 Purpose and Scope ...................................................................................................... 1.1-1
1.2 Precedence.................................................................................................................. 1.2-1
1.3 Document Structure .................................................................................................... 1.3-1
1.4 Applicable Legislation and NSF Policy.......................................................................... 1.4-1
1.4.1 Research Infrastructure ................................................................................... 1.4-1
1.4.2 MREFC Threshold ............................................................................................ 1.4-1
1.4.3 Major Multi-User Research Facility Project (Major Facility) ............................ 1.4-1
1.4.4 Mid-Scale Project and Mid-scale Research Infrastructure ............................... 1.4-3
1.4.5 National Science Board Policy on Recompetition ............................................ 1.4-3
1.4.6 NSF “No Cost Overrun” Policy ......................................................................... 1.4-3
1.4.7 Legislation on Congressional Notification of Total Project Cost Increases....... 1.4-4
1.4.8 Legislation on Congressional Notification of Divestments of NSF-owned Facilities
or Capital Assets .............................................................................................. 1.4-4
2 MAJOR FACILITY LIFE CYCLE AND MAJOR FACILITY OVERSIGHT.................................. 2.1-1
2.1 Process Introduction ................................................................................................... 2.1-1
2.1.1 Definition of the MREFC Account ................................................................. 2.1.1-1
2.1.2 Eligibility for MREFC Funding ........................................................................ 2.1.2-1
2.1.3 The Major Facility Life Cycle ......................................................................... 2.1.3-1
2.1.4 Summary of the Major Facility Implementation Process .............................. 2.1.4-1
2.1.5 Timeline and Flowcharts for the Major Facilities Approval Process ............. 2.1.5-1
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major
Facilities ........................................................................................................ 2.1.6-1
2.1.6.1 Overview........................................................................................ 2.1.6-1
2.1.6.2 Main Participants ........................................................................... 2.1.6-8
2.1.6.3 Coordinating and Advisory Bodies ............................................... 2.1.6-17
2.1.6.4 Governing Bodies......................................................................... 2.1.6-20
2.2 Development Stage .................................................................................................. 2.2.2-1
2.2.1 Initiation of a Potential Major Facility Project .............................................. 2.2.1-1
2.2.2 Exit from Development to Design Stage ....................................................... 2.2.2-1
2.3 Design Stage – Conceptual, Preliminary, and Final Design Phases ........................... 2.3.1-1
2.3.1 Conceptual Design Phase ............................................................................. 2.3.1-1
2.3.1.1 Introduction – Conceptual Design Phase ....................................... 2.3.1-1
2.3.1.2 Conceptual Design Phase Activities ............................................... 2.3.1-2
2.3.1.3 Conceptual Design Review (CDR) ................................................... 2.3.1-4
2.3.1.4 Exit from the Conceptual Design Phase ......................................... 2.3.1-5
2.3.2 Preliminary Design Phase ............................................................................. 2.3.2-1
2.3.2.1 Introduction – Preliminary Design Phase ....................................... 2.3.2-1
2.3.2.2 Preliminary Design Phase Activities ............................................... 2.3.2-2
2.3.2.3 Preliminary Design Review (PDR)................................................... 2.3.2-3
2.3.2.4 Exit from Preliminary Design Phase ............................................... 2.3.2-4
2.3.2.5 NSF Director’s Recommendation for Advancement to Final Design 2.3.24
2.3.2.6 National Science Board Authorization following PDR .................... 2.3.2-5
Section Revision:
July 26, 2021
iii
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3.2.7 Inclusion in an NSF Budget Request............................................... 2.3.2-6
2.3.3 Final Design Phase ........................................................................................ 2.3.3-1
2.3.3.1 Introduction – Final Design Phase.................................................. 2.3.3-1
2.3.3.2 Final Design Review (FDR) ............................................................. 2.3.3-1
2.3.3.3 Exit from the Final Design Phase.................................................... 2.3.3-3
2.3.3.4 NSF Director’s Recommendation for Advancement to Construction
Stage .............................................................................................. 2.3.3-3
2.3.3.5 National Science Board Authorization for Construction ................ 2.3.3-3
2.4 Construction Stage ................................................................................................... 2.4.1-1
2.4.1 Construction Award Management and Oversight ........................................ 2.4.1-1
2.4.2 Construction Award Close-out...................................................................... 2.4.2-1
2.4.2.1 Project Close-out Process .............................................................. 2.4.2-1
2.4.2.2 Request for No-Cost Extension ...................................................... 2.4.2-1
2.5 Operations Stage ...................................................................................................... 2.5.1-1
2.5.1 Operations Management and Oversight ...................................................... 2.5.1-1
2.5.2 Renewal/Competition .................................................................................. 2.5.2-1
2.6 Divestment Stage ........................................................................................................ 2.6-1
3 FACILITY LIFE CYCLE MANAGEMENT PLANS FOR MAJOR FACILITIES .............................. 3.1-1
3.1 Introduction ................................................................................................................ 3.1-1
3.2 NSF Facility Plan [Reserved]......................................................................................... 3.2-1
3.3 NSF Internal Management Plans for the Major Facility Life Cycle ............................... 3.3-1
3.4 Project Execution Plan..............................................................................................3.4.1-1
3.4.1 Components of a Project Execution Plan ...................................................... 3.4.1-1
3.4.2 Detailed Guidelines for Project Execution Plans ........................................... 3.4.2-1
3.4.2.1 Introduction [Reserved] ................................................................. 3.4.2-1
3.4.2.2 Organization [Reserved] .............................................................. ..3.4.2-1
3.4.2.3 Design and Development [Reserved]........................................... ..3.4.2-1
3.4.2.4 Construction Project Definition ..................................................... 3.4.2-1
3.4.2.5 Staffing [Reserved] ........................................................................ 3.4.2-1
3.4.2.6 Risk and Opportunity Management............................................... 3.4.2-1
3.4.2.7 Systems Engineering [Reserved] .................................................... 3.4.2-1
3.4.2.8 Configuration Control .................................................................... 3.4.2-1
3.4.2.9 Acquisitions [Reserved] .................................................................. 3.4.2-1
3.4.2.10 Project Management Controls ....................................................... 3.4.2-1
3.4.2.11 Site and Environment [Reserved]................................................... 3.4.2-1
3.4.2.12 Cyber-Infrastructure ...................................................................... 3.4.2-1
3.4.2.13 Environmental, Safety and Health [Reserved] ............................... 3.4.2-1
3.4.2.14 Review and Reporting .................................................................... 3.4.2-2
3.4.2.15 Commissioning .............................................................................. 3.4.2-2
3.4.2.16 Project Close-out Plan ................................................................... 3.4.2-4
3.5 Operations Planning ................................................................................................. 3.5.1-1
3.5.1 Preparation of Proposals for Operations and Management ......................... 3.5.1-1
3.5.2 Procedures for Renewal or Competition of an Operating Major Facility ...... 3.5.2-1
Section Revision:
July 26, 2021
iv
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.5.3 Detailed Guidelines for Oversight of Operations .......................................... 3.5.3-1
3.6 Facility Divestment Plan .............................................................................................. 3.6-1
4 KEY MANAGEMENT PRINCIPLES AND REQUIREMENTS FOR MAJOR FACILITIES..............4.1-1
4.1 Introduction ................................................................................................................ 4.1-1
4.2 Cost Estimating and Analysis .................................................................................... 4.2.1-1
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
..................................................................................................................... 4.2.1-1
4.2.2 Elements of Both Construction and Operations Estimates ........................... 4.2.2-1
4.2.2.1 Cost Estimating Plan ...................................................................... 4.2.2-1
4.2.2.2 Estimate Formats ........................................................................... 4.2.2-4
4.2.2.3 Application of GAO Cost Guidance to Major Facilities ................... 4.2.2-6
4.2.2.4 Supplementary Guidance for NSF Budget Categories from the PAPPG
..................................................................................................... 4.2.2-10
4.2.2.5 Fee ............................................................................................... 4.2.2-15
4.2.2.6 Escalation..................................................................................... 4.2.2-16
4.2.2.7 Work Breakdown Structure (WBS) .............................................. 4.2.2-16
4.2.3 Additional Guidance for Construction Estimates .......................................... 4.2.3-1
4.2.3.1 Purpose and Process...................................................................... 4.2.3-1
4.2.3.2 Construction Cost Book – Introduction and Executive Summary ... 4.2.3-1
4.2.3.3 Construction Cost Book – Format .................................................. 4.2.3-2
4.2.3.4 Construction Cost Book – Detail .................................................... 4.2.3-3
4.2.4 Additional Guidance for Operations Estimates............................................. 4.2.4-1
4.2.4.1 Purpose and Process...................................................................... 4.2.4-1
4.2.4.2 Operations Awards Proposals – Overview ..................................... 4.2.4-1
4.2.4.3 Operations Awards Proposals – Format ........................................ 4.2.4-2
4.2.4.4 Operations Awards Proposals – Detail ........................................... 4.2.4-4
4.2.5 Risk Planning for the Construction Stage ...................................................... 4.2.5-1
4.2.5.1 Implementation of NSF’s No Cost Overrun Policy.......................... 4.2.5-1
4.2.5.2 Introduction to Budget Contingency.............................................. 4.2.5-2
4.2.5.3 Contingency Planning and Assessment during Conceptual Design 4.2.5-5
4.2.5.4 Contingency Planning and Assessment during Preliminary Design 4.2.5-6
4.2.5.5 Development of the Contingency Use Process .............................. 4.2.5-8
4.2.5.6 Contingency Planning and Assessment during Final Design .......... 4.2.5-8
4.2.5.7 Contingency Use and NSF Oversight during Construction ............. 4.2.5-9
4.2.5.8 Reporting Requirements .............................................................. 4.2.5-10
4.2.5.9 Partnership Considerations ......................................................... 4.2.5-11
4.2.6 Budget Contingency Planning during the Operations Stage ......................... 4.2.6-1
4.3 Schedule Development, Estimating, and Analysis .................................................... 4.3.1-1
4.3.1 Schedule - Introduction ................................................................................ 4.3.1-1
4.3.2 Characteristics of a Reliable Schedule .......................................................... 4.3.2-1
4.3.2.1 Comprehensive .............................................................................. 4.3.2-1
4.3.2.2 Well-Constructed ........................................................................... 4.3.2-2
4.3.2.3 Credible ......................................................................................... 4.3.2-3
Section Revision:
July 26, 2021
v
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.3.2.4 Controlled ...................................................................................... 4.3.2-4
4.3.3 Developing and Estimating a Baseline Schedule ........................................... 4.3.3-1
4.3.3.1 Steps .............................................................................................. 4.3.3-1
4.3.3.2 Schedule Documentation .............................................................. 4.3.3-3
4.3.4 Schedule Maintenance During Construction Stage ...................................... 4.3.4-1
4.3.4.1 Baseline Schedule or PMB ............................................................. 4.3.4-1
4.3.4.2 Progress Schedule.......................................................................... 4.3.4-1
4.3.5 NSF Analysis of Construction Stage Resource-Loaded Schedules ................. 4.3.5-1
4.3.5.1 Schedule Review Component of Stage-Gate Reviews (CDR, PDR, FDR)
....................................................................................................... 4.3.5-1
4.3.5.2 Schedule Review Component of Independent Cost Estimate Reviews
....................................................................................................... 4.3.5-2
4.3.5.3 Schedule Review Component of NSF EVMS Verification Review ... 4.3.5-2
4.3.5.4 Schedule Review Component of NSF Cost Analysis ....................... 4.3.5-3
4.4 System Integration, Commissioning, Testing and Acceptance .................................... 4.4-1
4.5 Documentation Requirements .................................................................................... 4.5-1
4.6 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting ..... 4.6.1-1
4.6.1 Introduction to Oversight, Reviews, Audits, and Reporting .......................... 4.6.1-1
4.6.2 Recipient Performance Reports .................................................................... 4.6.2-1
4.6.3 Recipient Performance Reviews and Audits ................................................. 4.6.3-1
4.6.3.1 Recipient Internal Reviews ............................................................ 4.6.3-1
4.6.3.2 NSF External Reviews..................................................................... 4.6.3-1
4.6.3.3 Business Systems Reviews (BSR) .................................................... 4.6.3-2
4.6.3.4 Incurred Cost Audits ...................................................................... 4.6.3-3
4.6.3.5 Accounting System Review or Audits............................................. 4.6.3-3
4.6.3.6 Earned Value Management Verification, Acceptance, and Surveillance
....................................................................................................... 4.6.3-4
4.6.4 NSF Performance Metric for Construction ................................................... 4.6.4-1
4.6.5 Re-Baselining ................................................................................................ 4.6.5-1
4.6.6 Project Personnel and Competencies ........................................................... 4.6.6-1
4.6.6.1 Key Personnel ................................................................................ 4.6.6-1
4.6.6.2 Project Team.................................................................................. 4.6.6-2
4.6.6.3 Competency Requirements for Major Facility Management ......... 4.6.6-3
4.7 Partnerships ............................................................................................................. 4.7.1-1
4.7.1 Partnerships Overview ................................................................................. 4.7.1-1
4.7.2 Partnership Funding ..................................................................................... 4.7.2-1
4.7.3 Memorandum of Understanding (MOU) with NSF ....................................... 4.7.3-1
5 GUIDANCE FOR MID-SCALE RESEARCH INFRASTRUCTURE PROJECTS ............................... 5-1
6 SPECIAL TOPICS AND SUPPLEMENTARY MATERIALS ..................................................... 6.1-1
6.1 Introduction ................................................................................................................ 6.1-1
6.2 Risk Management Guidelines For Construction Stage .............................................. 6.2.1-1
6.2.1 Introduction.................................................................................................. 6.2.1-1
6.2.2 Definition of Project Risk and Risk Exposure ................................................ 6.2.2-1
Section Revision:
July 26, 2021
vi
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.3 Definition of Allowable Contingencies .......................................................... 6.2.3-1
6.2.3.1 Allowable Contingency .................................................................. 6.2.3-1
6.2.3.2 Contingency Definitions ................................................................. 6.2.3-1
6.2.4 Risk Management Steps and Methodology .................................................. 6.2.4-1
6.2.5 Risk Management Planning .......................................................................... 6.2.5-1
6.2.5.1 Risk Management Plan (RMP) ....................................................... 6.2.5-2
6.2.5.2 Roles and Responsibilities.............................................................. 6.2.5-4
6.2.6 Risk Identification ......................................................................................... 6.2.6-1
6.2.6.1 Risk Identification Process ............................................................. 6.2.6-1
6.2.6.2 Risk Identification and the Risk Register ........................................ 6.2.6-2
6.2.6.3 Risk Description ............................................................................. 6.2.6-3
6.2.6.4 Risk Identification Concerns .......................................................... 6.2.6-4
6.2.7 Qualitative Risk Analysis – Risk Register Ranking ......................................... 6.2.7-1
6.2.7.1 Purpose of Qualitative Risk Analysis .............................................. 6.2.7-1
6.2.7.2 Considerations When Performing Qualitative Risk Analysis .......... 6.2.7-1
6.2.7.3 Limitations of Qualitative Analysis ................................................. 6.2.7-2
6.2.7.4 Qualitative Risk Analysis – Probability and Impact Assessment ..... 6.2.7-3
6.2.7.5 Alternative Approach to Qualitative Risk Impact Analysis ‒ Maxwell
....................................................................................................... 6.2.7-7
6.2.7.6 Qualitative Risk Analysis – Risk Level Matrix.................................. 6.2.7-8
6.2.7.7 Risk Level Input to the Risk Register ............................................ 6.2.7-11
6.2.7.8 Other Qualitative Risk Analysis Methods ..................................... 6.2.7-14
6.2.8 Quantitative Risk Analysis – Estimating Contingency ................................... 6.2.8-1
6.2.8.1 The Purpose of Quantitative Risk Analysis ..................................... 6.2.8-1
6.2.8.2 Key Elements in Quantitative Risk Analysis.................................... 6.2.8-4
6.2.8.3 Platforms for a Project Quantitative Risk Analysis ......................... 6.2.8-6
6.2.8.4 Case Study: Quantitative Risk Analysis Exercise............................. 6.2.8-8
6.2.8.5 Schedule Risk Analysis ‒ Uncertainty ............................................. 6.2.8-9
6.2.8.6 Schedule Risk Analysis ‒ Discrete Risks Added as Drivers ............ 6.2.8-12
6.2.8.7 Cost Risk Analysis ‒ Uncertainty and Discrete Risk Drivers .......... 6.2.8-14
6.2.8.8 Handling Inflation ........................................................................ 6.2.8-17
6.2.8.9 Prioritizing the Discrete Risks ‒ Risk Mitigation Workshop .......... 6.2.8-18
6.2.9 Risk Response Planning ................................................................................ 6.2.9-1
6.2.10 Risk Monitoring and Control ....................................................................... 6.2.10-1
6.2.11 Contingency Management for Risk Mitigation ........................................... 6.2.11-1
6.2.11.1 Contingency Budget Timeline ...................................................... 6.2.11-1
6.2.11.2 Change Control for Contingency Adjustments ............................. 6.2.11-1
6.2.11.3 Liens List: Forecasting and Opportunity Management ................ 6.2.11-2
6.2.11.4 Updates of the Estimate at Completion and Risk Exposure ......... 6.2.11-3
6.2.11.5 Contingency Use and NSF Oversight during Construction ........... 6.2.11-4
6.2.11.6 Documentation and Reporting of Contingency Use..................... 6.2.11-4
6.2.12 Partnership Considerations for Contingency Management ........................ 6.2.12-1
6.3 Guidelines for CyberSecurity of NSF’s Major Facilities ........................................... ..6.3.1-1
Section Revision:
July 26, 2021
vii
Major Facilities Guide: NSF 21-107 (July 2021)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.3.1 Introduction.................................................................................................. 6.3.1-1
6.3.2 Major Facility Cybersecurity Program........................................................... 6.3.2-1
6.3.3 Mission Alignment ........................................................................................ 6.3.3-1
6.3.3.1 Stakeholders & Obligations ........................................................... 6.3.3-1
6.3.3.2 Information Asset Inventory .......................................................... 6.3.3-1
6.3.3.3 Information Classification .............................................................. 6.3.3-2
6.3.4 Governance .................................................................................................. 6.3.4-1
6.3.4.1 Roles and Responsibilities.............................................................. 6.3.4-1
6.3.4.2 Policies ........................................................................................... 6.3.4-1
6.3.4.3 Risk Management and Acceptance ................................................ 6.3.4-2
6.3.4.4 Evaluation ...................................................................................... 6.3.4-2
6.3.5 Resources ..................................................................................................... 6.3.5-1
6.3.5.1 Budget ........................................................................................... 6.3.5-1
6.3.5.2 Personnel ....................................................................................... 6.3.5-1
6.3.6 Controls ........................................................................................................ 6.3.6-1
6.3.6.1 Control Set ..................................................................................... 6.3.6-1
6.4 Guidelines for Planning and Executing External Reviews of NSF's Major Facilities...... 6.4-1
6.5 Environmental Considerations in Major Facility Planning ........................................... 6.5-1
6.6 Guidelines for Property Management ......................................................................... 6.6-1
6.7 Guidelines for Financial Management ......................................................................... 6.7-1
6.8 Guidelines for Earned Value Management Systems ................................................. 6.8.1-1
6.8.1 EVMS Requirements ..................................................................................... 6.8.1-1
6.8.1.1 Seven Basic Principles of EVMS...................................................... 6.8.1-1
6.8.1.2 Guidelines for NSF’s Verification and Acceptance of EVMS ........... 6.8.1-1
6.8.2 Tailored Implementation of EVMS ............................................................... 6.8.2-1
6.8.3 Guidelines for Establishing an EVMS ............................................................ 6.8.3-1
6.8.4 NSF Scaled EVMS .......................................................................................... 6.8.4-1
6.8.5 Practice Guide to Establish Scaled EVMS ...................................................... 6.8.5-1
6.8.5.1 Process 1: Define and organize the project (Principle 1 and 2) ..... 6.8.5-1
6.8.5.2 Process 2: Establish project cost, schedule, and contingencies (Principle
3) ................................................................................................... 6.8.5-1
6.8.5.3 Process 3: Progress and performance monitoring (Principles 4, 5, 7)
....................................................................................................... 6.8.5-2
6.8.5.4 Process 4: Management analysis and control (Principles 6 and 7) 6.8.5-3
7 REFERENCES ..................................................................................................................... 7-1
8 LIST OF ACRONYMS.......................................................................................................... 8-1
9 LEXICON ........................................................................................................................ 9.1-1
9.1 Lexicon Preface ........................................................................................................... 9.1-1
9.2 Terms and Definitions ................................................................................................. 9.2-1
10 APPENDICES ..................................................................................................................... A-1
Section Revision:
July 26, 2021
viii
Major Facilities Guide: NSF 21-107 (July 2021)
List of Figures
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
LIST OF FIGURES
Figure 2.1.3-1
Figure 2.1.3-2
Figure 2.1.4-1
Figure 2.1.4-2
Figure 2.1.4-3
Figure 2.1.4-4
Figure 2.1.6-1
Figure 2.1.6-2
Figure 2.1.6-3
Figure 2.1.6-4
Figure 4.2.1-1
Figure 4.2.2-1
Figure 4.2.2-2
Figure 4.2.3-1
Figure 4.2.3-2
Figure 4.2.4-1
Figure 4.2.5-1
Figure 6.2.4-1
Figure 6.2.6-1
Figure 6.2.6-2
Figure 6.2.6-3
Figure 6.2.7-1
Figure 6.2.7-2
Figure 6.2.7-3
Section Revision:
May 30, 2019
Progressive steps in the facility life cycle, showing the high-level review
and decision points for exit and entry into each stage. The Design Stage is
further broken down into phases. ......................................................... 2.1.3-1
Progressive Phases within the Design Stage, showing review and decision
points for advancement to the next phase and NSB authorization for
budgeting and award. ............................................................................ 2.1.3-3
Summary Timeline for Major Facility Projects (Development and
Design) ................................................................................................... 2.1.4-2
Summary Timeline for Major Facility Projects (Construction, Operations,
and Divestment)..................................................................................... 2.1.4-3
Linear Design-Construct Process is Classic "Waterfall Model"
Implementation ..................................................................................... 2.1.4-4
Spiral Development Process................................................................... 2.1.4-5
NSF organization chart highlighting staff who have primary oversight and
management roles and responsibilities for major facilities and mid-scale
research infrastructure. ......................................................................... 2.1.6-2
NSF organization chart showing coordinating and advisory bodies for
major facilities and mid-scale research infrastructure. .......................... 2.1.6-3
NSF organization chart showing policy and approval bodies for major
facilities and mid-scale research infrastructure. .................................... 2.1.6-4
An Integrated Project Team (IPT), chaired by the Program Officer, is
composed of three subgroups, with appointed Award Management
Group members from BFA, Science and Technology Group members from
the sponsoring program offices, and Strategic Group members from the
Office of the Director. .......................................................................... 2.1.6-18
NSF Cost Analysis Process ...................................................................... 4.2.1-4
Sample Project Control Systems Relationship Diagram ......................... 4.2.2-3
NSF Budget Categories Sample Format .................................................. 4.2.2-5
Construction WBS and Cost Book Sample Format ................................. 4.2.3-3
Construction Cost Book Sheet Sample Format ....................................... 4.2.3-7
Operations WBS and Budget Sample Format ......................................... 4.2.4-3
Sample of a Change Control Request Form, with instructions for filling out
the various sections ............................................................................. 4.2.5-12
Picture of Six Risk Management Processes (According to PMI).............. 6.2.4-1
Risk Identification Process...................................................................... 6.2.6-1
Typical Risk Breakdown Structure (RBS)................................................. 6.2.6-5
OMB Risk Categories: to be used as a starting point for projects to select
their own categories .............................................................................. 6.2.6-6
Qualitative Risk Analysis Process............................................................ 6.2.7-1
Overlap in Risk Probability of Occurring When Descriptors Are Used .... 6.2.7-6
Symmetrical Risk Level Matrix ............................................................... 6.2.7-9
ix
Major Facilities Guide: NSF 21-107 (July 2021)
List of Tables
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.7-4
Figure 6.2.7-5
Figure 6.2.7-6
Figure 6.2.7-7
Figure 6.2.8-1
Figure 6.2.8-2
Figure 6.2.8-3
Figure 6.2.8-4
Figure 6.2.8-5
Figure 6.2.8-6
Figure 6.2.8-7
Figure 6.2.8-8
Figure 6.2.8-9
Figure 6.2.8-10
Figure 6.2.8-11
Figure 6.2.8-12
Figure 6.2.8-13
Figure 6.2.9-1
Figure 6.2.10-1
Figure 6.8.4-1
Asymmetrical Risk Level Matrix ............................................................. 6.2.7-9
Probability and Impact Matrix including Threats and Opportunities ... 6.2.7-10
Sample Risk Register with Risk ID Number, Associated WBS Identification,
Qualitative Probability and Impact for Initial and Post-mitigation States,
and Mitigation Actions ......................................................................... 6.2.7-12
Sample Top Risk Matrix and Status Report, showing list of project risks
selected as most significant to monitor on a frequent basis, with ranking
and trend data ..................................................................................... 6.2.7-13
Quantitative Risk Analysis Process ......................................................... 6.2.8-1
Typical Result of a Quantitative Schedule Risk Analysis ......................... 6.2.8-3
Time-Cost Scatter Diagram: Each data point represents one realization of
the simulation ........................................................................................ 6.2.8-4
Resource-Loaded Schedule Used for a Simple Case Study of an Integrated
Cost-Schedule Risk Analysis for Design, Fabrication, Testing, and Delivery
of a Space Vehicle .................................................................................. 6.2.8-8
End Date Results for Schedule Duration Uncertainties ........................ 6.2.8-10
Cost Result for Schedule Duration Uncertainties ................................. 6.2.8-11
Total Cost and End Date Scatterplot for Schedule Uncertainties ......... 6.2.8-12
Schedule Risk Drivers – Organizational Risk ......................................... 6.2.8-13
Total Cost and End Date Scatterplot Showing Greater Correlation of Time
and Cost Risk ........................................................................................ 6.2.8-14
Uncertainty in the Burn Rate and Total Cost ........................................ 6.2.8-15
Screenshot of Risk Driver Editor ........................................................... 6.2.8-16
Scatterplot Showing Less Connection between Time and Cost ............ 6.2.8-17
Scatterplot with Addition of Cost Inflation Factor ................................ 6.2.8-18
Risk Response Planning Process ............................................................. 6.2.9-1
Risk Monitoring and Control Process ................................................... 6.2.10-1
Relationship between Scaled EVMS and Full Implementation ............... 6.8.4-1
LIST OF TABLES
Table 2.1.6-1
Table 2.4.2-1
Table 3.4.1-1
Table 4.6.6.3-1
Table 4.6.6.3-2
Table 4.6.6.3-3
Table 6.2.5-1
Table 6.2.5-2
Table 6.2.7-1
Table 6.2.7-2
Section Revision:
May 30, 2019
Summary of Principal Roles and Responsibilities of the core members of
the IPT (PO, G/AO or CO, and LFO) Liaison by Facility Life Cycle Stage .. 2.1.6-5
Sample of a No-Cost Extension Tasks Table ........................................... 2.4.2-4
List of the Typical Components of a Project Execution Plan, with SubTopics and Descriptions ......................................................................... 3.4.1-2
PMIAA Areas of Program Management Standards and Principles ......... 4.6.6-3
Competency Resource Assignment Requirements.................................4.6.6-5
Competency Descriptions ...................................................................... 4.6.6-5
Sample Format for a Risk Management Plan ......................................... 6.2.5-2
Example of a Risk Management Roles and Responsibilities Table.......... 6.2.5-4
Sample Risk Probabilities Table .............................................................. 6.2.7-5
Sample Risk Consequences Table........................................................... 6.2.7-5
x
Major Facilities Guide: NSF 21-107 (July 2021)
List of Tables
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 6.2.7-3
Table 6.2.8-1
Table 6.2.8-2
Table 6.2.8-3
Table 6.2.8-4
Table 6.2.8-5
Table 6.2.8-6
Table 6.2.9-1
Section Revision:
May 30, 2019
Maxwell Risk Driver Assessment Framework ......................................... 6.2.7-7
Resources for Quantitative Risk Analysis Example ................................. 6.2.8-9
Schedule Duration Risk Reference Ranges ............................................. 6.2.8-9
Results with Schedule Uncertainties Assigned ..................................... 6.2.8-11
Results with Schedule Uncertainties and Discrete Risks Assigned ....... 6.2.8-14
Results with Uncertainties and Cost Risks Assigned ............................. 6.2.8-16
Savings and Days Saved........................................................................ 6.2.8-19
Impact of Risk Handling on Project Cost ................................................ 6.2.9-3
xi
Major Facilities Guide: NSF 21-107 (July 2021)
1.1 Purpose and Scope
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1
INTRODUCTION
1.1
PURPOSE AND SCOPE
A major responsibility of the National Science Foundation (NSF) is the support of scientific
facilities as an essential part of science and engineering enterprise. Facilities are defined as
shared-use infrastructure, instrumentation and equipment that are accessible to a broad
community of researchers and/or educators. These facilities are generally intended to serve the
science community that is critical to supporting innovation across the nation. Facilities
supported by NSF may be centralized or may consist of distributed-but-integrated installations.
They may incorporate large-scale networking or computational infrastructure, multi-user
instruments or networks of such instruments, or other infrastructure, instrumentation, and
equipment having a major impact on a broad segment of a scientific or engineering discipline.
Historically, NSF has supported such diverse projects as particle accelerators, telescopes,
remote research stations, research vessels, aircraft, and geographically distributed but
networked observatory systems.
In general, NSF does not directly construct or operate the facilities it supports. The National
Science Foundation Act of 1950 (“Organic Act,” Public Law 81-507) establishes that the
“principal purpose” of NSF’s relationship with award Recipients is to fund and facilitate
scientific and engineering research and education programs, and to appraise the impact of
research upon industrial development and upon the general welfare. It states that NSF “shall
not, itself, operate any laboratories or pilot plants”. NSF makes awards to external Recipients
that include nonprofit organizations, universities, and private sector (industry) to undertake
construction, management, and operation of facilities. Such awards frequently take the form of
cooperative agreements but may also be made in the form of contracts. The reasons underlying
the selection of the cooperative agreement as the preferred award instrument are:
•
Scientific justifications, design and specifications for facilities are prepared by science
and engineering communities, and management and operations are conducted on their
behalf;
•
The facilities do not support NSF nor does NSF permanently station government
personnel on-site;
•
NSF involvement is to assure sufficiency of progress to justify continued sponsorship,
and its award administration and oversight activities are not conducted for purposes of
inspection or acceptance; and
•
NSF does not maintain the unilateral right to change or redirect work under the
agreement.
However, NSF’s responsibility is for overseeing the Recipient’s development and management
of the facility as well as assuring the successful performance of the funded activities. The
Recipient is responsible for the day-to-day management of the facility.
Section Revision:
May 30, 2019
1.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
1.1 Purpose and Scope
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Cooperative agreement is the legal award instrument that reflects the above-described
relationship. Federal Grant and Cooperative Agreement Act (“Grant Act,” Public Law 95-224)
requires that executive agencies use cooperative agreements when the “principal purpose” of
the relationship between the agency and a non-federal entity is to “transfer a thing of value” to
the non-federal entity “to carry out a public purpose of support or stimulation authorized by a
law of the United States,” and “substantial involvement is expected” between the agency and
the non-federal entity in carrying out the activity contemplated by the agreement.
NSF uses cooperative agreements (CAs) to fund the construction and operation and
maintenance (O&M) of large-scale research facilities. Cooperative agreements with universities,
consortia of universities or non-profit organizations are governed by OMB Uniform Guidance. 1
Under the Uniform Guidance, cooperative agreements structure allows for additional oversight
and accountability mechanism to be built into the agreements. Cooperative agreements also
afford flexibility to tailor project-specific requirements and performance metrics. Unlike a
contract, these can be readily adjusted as needed to ensure the appropriate rigor in oversight
with relatively minimum administrative and time burdens.
Many major facility awards, including those for NSF-supported Federally Funded Research and
Development Centers (FFRDCs), consist of a cooperative agreement as an umbrella award,
establishing the overall basic provisions of the award, and separate cooperative support
agreements. The cooperative support agreements contain specific terms and conditions for
construction activities, management and operations, research activities that are co-sponsored
by other agencies, and any other focused activities that NSF needs to monitor separately from
the overall objectives of the cooperative agreement.
Procurement contracts could be used in circumstances where the agency “decides in a specific
instance that use of a procurement contract is appropriate.” Federal Acquisition Regulation
(FAR) states that “contracts shall be used only when the principal purpose is the acquisition of
supplies or services for the direct benefit or use of the Federal Government”. The policies and
procedures in this Guide apply to research infrastructure projects regardless of the award
instrument employed. When using contracts, the FAR will take precedence in event of conflict. 2
2 Code of Federal Regulation (CFR) § 200. The Office of Management and Budget's (OMB) Uniform Administrative
Requirements, Cost Principles, and Audit Requirements for Federal Awards (commonly called "Uniform Guidance") was
officially implemented in December 2014 by the Council on Financial Assistance Reform (COFAR). The Uniform Guidance – a
"government-wide framework for grants management" – synthesizes and supersedes guidance from earlier OMB circulars
1
2
See Guide to the NSF Contracting Process for information related to NSF contracts.
Section Revision:
May 30, 2019
1.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
1.1 Purpose and Scope
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The Major Facilities Guide (MFG) contains NSF policy on the planning and management of
major facilities through their full life cycle. 1 The purpose of the Guide is to:
•
Provide guidance to NSF staff on conducting oversight of major facilities and to
Recipients in carrying out effective project planning and management, and
•
Clearly state the required policies and procedures as well as pertinent guidance and
practices at each stage of a facility’s life cycle.
NSF typically supports facility construction from two appropriations accounts: the Major
Research Equipment and Facility Construction (MREFC) Account and the Research and Related
Activities (R&RA) Account, but additional support may come from Education and Human
Resources (EHR) Accounts. The MREFC Account was created in 1995 to fund the acquisition,
construction, commissioning, and upgrading of major science and engineering infrastructure
projects that could not be otherwise supported by Directorate level budgets without a severe
negative impact on funded science. MREFC projects generally range in cost from seventy million
to several hundred million dollars expended over a multi-year period. The R&RA account is used
to support other activities involving a major facility that the MREFC Account cannot support,
including planning and development, design, operations and maintenance, and scientific
research. Construction and acquisition projects at a smaller scale, usually of a scale ranging
from millions to tens of millions of dollars, are also normally supported from the R&RA Account.
The provisions and principles in the Major Facilities Guide should also be applied to these
smaller-scale facilities funded through the R&RA Account, but procedures should be modified
appropriately to fit the needs of each facility (see Section 5).
The policies in the Major Facilities Guide apply to the full life cycle of all major facility projects
funded by NSF. The policies in this Guide are also tailored for mid-scale research infrastructure
projects as described in Section 5.
If, on a case-by-case basis, departures from the policies in this Guide are considered necessary
or prudent, the Recipient must provide a written justification and discuss proposed deviations
with the Program Officer, Large Facilities Officer (LFO) Liaison, and Grants and Agreements
Officer or Contracting Officer as early as possible. Agreed upon deviations should be
documented as part of the NSF Internal Management Plan (IMP) or the Recipient’s Project
Execution Plan (PEP), as appropriate.
There are five stages in a facility’s life cycle – development, design, construction, operations, and divestment. Section 2 of this
Guide describes each of these stages in detail.
1
Section Revision:
May 30, 2019
1.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
1.2 Precedence
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1.2
PRECEDENCE
The Major Facilities Guide (MFG) comprises Chapter II.E.11 of the Proposal and Award Policies
and Procedures Guide (PAPPG) and published as a public document under separate title. The
MFG is managed by the Office of Budget, Finance and Award Management’s (BFA) Large
Facilities Office (LFO) and available on the LFO public website
(https://www.nsf.gov/bfa/lfo/index.jsp) as well as through the internal LFO website. This
version replaces the Large Facilities Manual, NSF 17-066, published in 2017, and incorporates
changes in organization and content intended to clarify the policies and procedures by which
Major Facility candidate projects are identified, developed, prioritized, and selected. 1
The MFG requirements flow from other NSF policies and statutory requirements. The hierarchy
of documentation 2, in order of precedence, is as follows:
1. 2 CFR, part 200: Uniform Administrative Requirements, Cost Principles, and Audit
Requirement for Federal Awards (Uniform Guidance).
2. The Solicitation and subsequent Award Terms and Conditions
3. NSF Proposal & Award Policies and Procedures Guide (PAPPG): The PAPPG is comprised
of documents relating to the Foundation's proposal and award process for the
assistance programs of NSF. The PAPPG, in conjunction with the applicable award terms
and conditions, serves as the Foundation’s implementation of the Uniform Guidance.
4. Major Facilities Guide (MFG): as referenced in the PAPPG
5. Business Systems Review (BSR) Guide
The MFG does not replace existing formal procedures required for all NSF awards, which are
described in the publically available Proposal and Award Policies and Procedures Guide
(PAPPG). Instead, it draws upon and supplements them for the purpose of providing detailed
guidance regarding NSF management and oversight of facilities projects.
All facilities projects require merit review, programmatic/technical review, and a substantial
approval process. This level of review and approval differs substantially from standard grants,
as does the level of oversight needed to ensure appropriate and proper accountability for
federal funds. The policies, requirements, recommended procedures, and good practices
presented herein apply to any facility large enough to require interaction with the NSB or any
facility so designated by the Director, the Deputy Director, or the Assistant Director/Office Head
of the Sponsoring Organization(s). 3 For all other facilities, NSF staff members should use their
See the Joint National Science Board —National Science Foundation Management Report: Setting Priorities for Large Facility
Projects Supported by the National Science Foundation (NSB-05-77); September 2005
1
Assumes assistance awards, contract awards are governed by the Federal Acquisition Regulation (FAR) and requirements will
be tailored as applicable to FAR.
2
See Section 2.1.6 for definition of this and other key terms. It also describes the NSF organizations and officers that are
involved throughout the initiation, development, approval and implementation of a major facility project. Readers not familiar
with NSF and its processes should review this material before proceeding.
3
Section Revision:
September 20, 2019
1.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
1.2 Precedence
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
judgment in proportionately scaling the requirements and recommended procedures for
specific projects.
This Guide will be updated periodically to reflect changes in requirements and/or policies. As
part of the NSF Major Facilities Knowledge Management program, NSF will continue to identify
and adopt good practices aimed at improving agency oversight and Recipient management of
major facility projects and at enabling the most efficient and cost-effective delivery of tools to
the research and education communities.
Section Revision:
September 20, 2019
1.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
1.3 Document Structure
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1.3
DOCUMENT STRUCTURE
This Guide is organized as follows:
•
Section 1 introduces the purpose, scope, and historical perspective of this document.
•
Section 2 describes the life cycle stages and the process and principles NSF uses to plan,
construct and operate major facilities. The steps for approval and execution of major
research facility projects and the roles and responsibilities of NSF staff are detailed.
•
Section 3 describes the requirements for preparing and following the various detailed
management plans required during the life cycle of a major facility, including Recipient’s
plans and guidance for NSF’s Internal Management Plans (IMPs).
•
Section 4 is an expanded compendium of several NSF key requirements and principles
listed in Sections 2 and 3. It includes detailed descriptions of processes used to plan,
acquire, and manage major facilities.
•
Section 5 is guidance on scaling the Major Facilities Guide’s principles to mid-scale
projects.
•
Section 5 contains extensive supplementary information on specific topics concerning
NSF’s role in the planning, oversight, and assurance of major facility projects. It consists
of sections containing important explanatory and procedural information and pointers
to separate documents (or modules) with similar information. The information in the
documents is presented in a tutorial format that should be of particular benefit to
individuals who are newly involved with major facility projects.
•
Sections 7, 8, and 9 contain reference material: document References, List of Acronyms,
and a Lexicon.
•
Section 10 contains appendices contain other information relevant to construction
projects and major facilities.
This Guide is intended for use by NSF staff and by external proponents of major facility projects
for use in planning. However, there are occasional references to materials, such as the NSF
Proposal and Award Manual 1 (PAM) and internal operating guidance documents, which are
available only internally to NSF staff and refer to details of NSF administrative practices and
procedures that are not relevant to external project proponents. Wherever these internal
references are included, they are clearly noted as such. Any questions about the content of
internal NSF documents by external proponents or Recipients should be addressed to the
appropriate Program Officer.
Owing to the rigor of merit and programmatic review, constraints on funding, changing
priorities and competing interests of NSF and the research community, only a limited number
of projects will proceed successfully through all stages described herein. To improve the
possibility of success, facility advocates should be thoroughly familiar with the entire contents
1
The NSF Proposal and Award Manual is a compendium of internal policies and procedures.
Section Revision:
November 25, 2016
1.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
1.3 Document Structure
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
of this Guide even if the proposed project is in the earliest stages of formulation. Anticipating
downstream requirements will dramatically improve the efficiency of the process.
Section Revision:
November 25, 2016
1.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
1.4 Applicable Legislation and NSF Policy
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1.4
APPLICABLE LEGISLATION AND NSF POLICY
1.4.1 Research Infrastructure
NSF defines Research Infrastructure (RI) as any combination of facilities, equipment,
instrumentation, computational hardware and software, and the necessary human capital in
support of the same. Major facilities and mid-scale projects are subsets of research
infrastructure. NSF's Research Infrastructure investments are described in the agency's annual
budget request to Congress.
1.4.2 MREFC Threshold
NSF Director Memo dated October 26, 2020 reduced the Total Project Cost (TPC) threshold for
MREFC account eligibility to $20 million. This modification is consistent with the appropriations
action by Congress and supersedes previous MREFC threshold limits.
1.4.3 Major Multi-User Research Facility Project (Major Facility)
1.4.3.1 Definition
Per Section 110 of the 2017 American Innovation and Competitiveness Act (AICA), a major
multi-user research facility project is a science and engineering facility project that:
(A) exceeds the lesser of (i) 10 percent of a Directorate’s annual budget; or (ii)
$100,000,000 in total project costs; or
(B) is funded by the major research equipment and facilities construction account, or
any successor account.
This language was subsequently amended by Section 267 of the National Defense Authorization
Act (NDAA) of FY 2021 by striking the text in (A) and (B) above and inserting the following:
‘‘(2) MAJOR MULTI-USER RESEARCH FACILITY PROJECT. The term ‘major multi-user
research facility project’ means a science and engineering facility project that exceeds
$100,000,000 in total construction, acquisition, or upgrade costs to the Foundation.’’
NSF interprets the above to mean the Total Project Costs (TPC) as defined by the investment in
construction or acquisition, not the operations or associated science program costs. If the TPC
for research infrastructure is above the Major Facility project threshold as defined by statute, it
is considered a Major Facility throughout its full life cycle.
For the purposes of this Guide, the term Major Facility is used throughout to equate to the
Congressional term Major Multi-User Research Facility Project.
1.4.3.2 Oversight Requirements
The policies and procedures established in this Guide and supporting internal NSF guidance
documents fulfill the oversight requirements in Section 110 of AICA, as listed below:
Section Revision:
July 26, 2021
1.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
1.4 Applicable Legislation and NSF Policy
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
(A) prioritize the scientific outcomes of a major multi-user research facility project and the
internal management and financial oversight of the major multi-user research facility
project;
(B) clarify the roles and responsibilities of all organizations, including offices, panels,
committees, and directorates, involved in supporting a major multi-user research facility
project, including the role of the Major Research Equipment and Facilities Construction
Panel 1;
(C) establish policies and procedures for the planning, management, and oversight of a
major multi-user research facility project at each phase of the life-cycle of the major multiuser research facility project;
(D) ensure that policies for estimating and managing costs and schedules are consistent
with the best practices described in the Government Accountability Office Cost Estimating
and Assessment Guide, the Government Accountability Office Schedule Assessment Guide,
and the Office of Management and Budget Uniform Guidance (2 C.F.R. Part 200);
(E) establish the appropriate project management and financial management expertise
required for Foundation staff to oversee each major multi-user research facility project
effectively, including by improving project management training and certification;
(F) coordinate the sharing of the best management practices and lessons learned from each
major multi-user research facility project;
(G) continue to maintain a Large Facilities Office to support the research directorates in the
development, implementation, and oversight of each major multi-user research facility
project, including by—
(i) serving as the Foundation’s primary resource for all policy or process issues related to
the development, implementation, and oversight of a major multiuser research facility
project;
(ii) serving as a Foundation-wide resource on project management, including providing
expert assistance on nonscientific and nontechnical aspects of project planning,
budgeting, implementation, management, and oversight;
(iii) coordinating and collaborating with research directorates to share best
management practices and lessons learned from prior major multi-user research facility
projects; and
(iv) assessing each major multi-user research facility project for cost and schedule risk;
and
The MREFC Panel has been superseded with the Facilities Readiness Panel and the Facilities Governance Board. See Section
2.1.6 of this Guide for the roles and responsibilities of these Governing Bodies.
1
Section Revision:
July 26, 2021
1.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
1.4 Applicable Legislation and NSF Policy
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
(H) appoint a senior agency official 1 whose responsibility is oversight of the development,
construction, and operations of major multi-user research facilities across the Foundation.
1.4.4 Mid-Scale Project and Mid-scale Research Infrastructure
Per Section 109 of AICA, a mid-scale project means research instrumentation, equipment, and
upgrades to major research facilities or other research infrastructure investments that exceeds
the maximum funded by the Major Research Instrumentation program (MRI) and are below
that of a major multi-user research facility project (Major Facility).
Like Major Facilities, NSF interprets the above to mean the Total Project Cost (TPC) as defined
by the investment in construction, implementation, or acquisition, not the design, operations or
associated science program costs. If the TPC for research infrastructure is within the mid-scale
project range as defined by statue, it is considered mid-scale research infrastructure
throughout its full life cycle. Refer to Section 5 of this Guide for planning and oversight
requirements of mid-scale projects.
1.4.5 National Science Board Policy on Recompetition
NSB statement 2015-45 and resolution 2015-46 address competition, renewal, and divestment
of major facilities. The NSB issued a statement that the question of whether to recompete or
not should be assessed at the time of every potential renewal. Competitions would be launched
when the NSF, in consultation with the NSB, judges that it is necessary to ensure the optimum
scientific impact and the most effective use of taxpayer dollars. 2
1.4.6 NSF “No Cost Overrun” Policy
NSF’s “No Cost Overrun” policy was originally codified for major facility projects in the Fiscal
Year (FY) 2009 Budget Request to Congress 3 which reads:
NSF is implementing a ‘no cost overrun’ policy, which will require that the cost estimate
developed at the Preliminary Design Stage have adequate contingency to cover all
foreseeable risks, and that any cost increases not covered by contingency be accommodated
by reductions in scope. NSF senior management is developing procedures to assure that the
cost tracking and management processes are robust and that the project management
oversight has sufficient authority to meet this objective. As project estimates for the current
slate of projects are revised, NSF will identify potential mechanisms for offsetting any cost
increases in accordance with this policy.
1
Chief Officer for Research Facilities, see Section 2.1.6 of this Guide.
See NSB Statement on Recompetition of Major Facilities NSB-2015-45, https://www.nsf.gov/nsb/publications/2015/NSBStatementRecompetitionFacilities_2015-11-19.pdf, and
NSB Resolution on Recompetition of Ongoing Facilities NSB-2015-46, https://www.nsf.gov/nsb/publications/2015/NSBResolutionRecompetitionFacilities_2015-11-19.pdf
3 See the MREFC Section of the NSF’s 2009 Budget Request to Congress, page 3, available online.
2
Section Revision:
July 26, 2021
1.4-3
Major Facilities Guide: NSF 21-107 (July 2021)
1.4 Applicable Legislation and NSF Policy
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The policy has been continually reinforced in subsequent budget requests to Congress for the
purpose of instilling diligence and rigor in establishing the Total Project Cost (TPC) at award and
a strong NSF oversight position for major facility projects. The implementation of this No-Cost
Overrun Policy is defined fully in Sections 4.2.5.1 and 4.2.5.2.
1.4.7 Legislation on Congressional Notification of Total Project Cost Increases
Congressional notification is required when there is reason to believe the Construction Stage
Total Project Cost increase by 10% or more. Public Law 116-93, Section 518 reads:
If at any time during any quarter, the program manager of a project within the jurisdiction
of the Departments of Commerce or Justice, the National Aeronautics and Space
Administration, or the National Science Foundation totaling more than $75,000,000 has
reasonable cause to believe that the total program cost has increased by 10 percent or
more, the program manager shall immediately inform the respective Secretary,
Administrator, or Director. The Secretary, Administrator, or Director shall notify the House
and Senate Committees on Appropriations within 30 days in writing of such increase, and
shall include in such notice: the date on which such determination was made; a statement of
the reasons for such increases; the action taken and proposed to be taken to control future
cost growth of the project; changes made in the performance or schedule milestones and
the degree to which such changes have contributed to the increase in total program costs or
procurement costs; new estimates of the total project or procurement costs; and a
statement validating that the project’s management structure is adequate to control total
project or procurement costs.
1.4.8 Legislation on Congressional Notification of Divestments of NSF-owned Facilities or
Capital Assets
The Science Appropriations Act of 2019 includes the following under NSF’s Administrative
Provisions:
The Director of the National Science Foundation (NSF) shall notify the Committees on
Appropriations of the House of Representatives and the Senate at least 30 days in advance
of any planned divestment through transfer, decommissioning, termination, or
deconstruction of any NSF-owned facilities or any NSF capital assets (including land,
structures, and equipment) valued greater than $2,500,000.
Sections 2.6 and 3.6 of this Guide discuss the Divestment Stage of the major facility lifecycle and
provides guidance and procedures associated with the divestment of NSF-owned facilities covered by
this legislative language. The divestment of NSF capital assets valued greater than $2,500,000 is
governed by the Federal property management requirements and award terms and conditions.
Section Revision:
July 26, 2021
1.4-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.1 Process Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2
2.1
MAJOR FACILITY LIFE CYCLE AND MAJOR FACILITY OVERSIGHT
PROCESS INTRODUCTION
National Science Foundation (NSF) investments through the Major Research Equipment and
Facility Construction (MREFC) Account provide state-of-the-art infrastructure for research and
education, such as laboratory and field instrumentation and equipment, multi-user research
facilities, remote research stations, distributed instrumentation networks and arrays, and
mobile research platforms. In addition, investment is increasing in highly sophisticated
information technology (IT)-based infrastructure, including distributed sensor networks,
extensive data-storage and transmission capabilities, advanced computing resources, and
Internet-based distributed user facilities. 1
This section describes the overall major facility life cycle as well as the roles and responsibilities
of the various participants for oversight. It provides guidelines for planning and managing major
research infrastructure facilities. Because each facility has unique aspects, each project
necessarily requires adaptation of general principles. NSF promotes flexibility in the application
of these guidelines but requires justification and substantiation for the specific approach taken
in each case. That is accomplished through the processes of formal planning, documentation,
and review.
1
These resources, many of which are now in development, are collectively known as “cyber infrastructure.”
Section Revision:
November 25, 2016
2.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.1 Definition of the MREFC Account
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.1 Definition of the MREFC Account
The MREFC Account is an agency-wide capital account, created in 1995 with Congressional
approval, which provides funding to establish major science and engineering infrastructure
projects. Specifically, the MREFC Account is intended to:
•
In accordance with legislation, provide a special account specifically for acquisition,
construction and commissioning of major facilities and other infrastructure projects,
including major upgrades;
•
Prevent large periodic obligations from distorting the budgets of NSF Directorates and
program offices; and
•
Ensure availability of resources to complete large projects that are funded over several
years. 1
The MREFC Account funding is specifically for the Construction Stage. It cannot be used to
support other activities related to the Development, Design, Operations or Divestment Stages
as defined in other sections of this Guide.
The MREFC threshold is set by internal NSF Policy (See Section 1.4.2 of this Guide).
Reliable long-term funding commitments are essential to maintaining partnerships and for preventing cost overruns due to
schedule delays.
1
Section Revision:
November 25, 2016
2.1.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.2 Eligibility for MREFC Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.2 Eligibility for MREFC Funding
To be eligible for consideration for MREFC funding, each candidate project should represent an
outstanding opportunity to enable research and innovation, as well as education and broader
societal impacts. Each project should offer the possibility of transformative knowledge and the
potential to shift existing paradigms in scientific understanding, engineering processes and/or
infrastructure technology. Moreover, each should serve an urgent contemporary research and
education need that will persist for years beyond the often-lengthy process of planning and
development.
In addition, a candidate project should:
•
Be consistent with the goals, strategies, and priorities of the NSF;
•
Establish a long-term tools capability accessible to an appropriately broad community of
users on the basis of merit;
•
Require large investments for construction/ acquisition, over a limited period of time,
such that the project cannot be supported within one or more NSF Directorate(s)/
Office(s) without severe financial disruption of their portfolios of activities;
•
Have received strong endorsement of the appropriate science and engineering
communities, based upon a thorough external review, including an assessment of
(1) scientific and engineering research merit, (2) broader societal impacts,
(3) importance and priority within the relevant Science and Engineering communities,
(4) technical and engineering feasibility, and (5) management, cost, and schedule issues;
•
Be of sufficient importance that the Sponsoring NSF Organization 1 is prepared to fully
fund the costs of pre-construction planning, design and development, operation, and
maintenance, and associated programmatic activities (with full awareness that, for a
long-lived facility, operations costs may ultimately amount to many times the
construction costs); and
•
Have been coordinated with other organizations, agencies, and countries to ensure
complementarity and integration of objectives and potential opportunities for
collaboration and sharing of costs.
See Section 2.1.6 for definition of this and other key terms. It describes the NSF organizations and officers that are involved
throughout the conception, development, approval and implementation of a major facility project.
1
Section Revision:
July 26, 2021
2.1.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.3 The Major Facility Life Cycle
A facility’s life cycle is characterized by the following five stages:
1. Development 1
2. Design
3. Construction
4. Operation
5. Divestment
Each life cycle stage involves different activities as well as certain actions by NSF and the
Recipient that are necessary to advance the project to the next stage. These activities include
reviews and approvals needed to obtain NSF funding, and the creation of budgets and NSF
awards to support these activities. Entry and exit from each life cycle stage are defined in this
Guide, including the required documents and deliverables. A high-level graphic of the
progression through the stages is given below in Figure 2.1.3-1.
Figure 2.1.3-1
Progressive steps in the facility life cycle, showing the high-level review and decision points for
exit and entry into each stage. The Design Stage is further broken down into phases.
Descriptions for each stage are given below. See Sections 2.2 to 2.6 for detailed discussions of
the various procedures and deliverables for progression through each stage in the facility life
cycle.
Development Stage
The Development Stage is where initial ideas emerge, and a broad consensus is built for the
potential long-term needs, priorities, and general requirements for Research Infrastructure (RI)
A project in the Development Stage may be referred to as a “Horizon” or a Conceptual Development project in earlier NSF
documents and references.
1
Section Revision:
July 26, 2021
2.1.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
of interest to NSF. Investments in development by NSF, other government agencies, or private
interests can be focused or sporadic, but these annual investments are generally smaller than in
the Design Stage. Investments are typically focused on the high-level ideas, building community
consensus on requirements, establishing partnerships, setting priorities across a broad
landscape of potential needs, and developing rough order of magnitude cost estimates and
schedules. This stage can last 10 years or more and consequently the cumulative investment
over this period can be quite substantial. Next to transitioning to the Divestment Stage,
transition from the Development Stage to the Design Stage is often the most challenging to
navigate depending upon how federal agencies strategically prioritize projects and how science
communities are organized. To exit from this Stage, the Sponsoring Organization sends a
memorandum to the Chief Officer for Research Facilities (CORF) recommending that a project is
ready to enter Design Stage, normally at the beginning of the Conceptual Design Phase. If
entrance is proposed to a later phase in the Design Stage, the recommendation should be made
prior to the stage-gate review that aligns with project technical readiness so that the review can
be officially conducted. Depending on the point of entry, the CORF may conduct a senior
leadership review focusing on strategic agency and science community issues followed by
making a recommendation to the NSF Director. The NSF Director may elect to consult the
National Science Board prior to acting on a recommendation. Approval of transition to the
Design Stage does not imply a commitment to advance the project to the Construction Stage.
Design Stage
The Design Stage is where detailed, construction-ready budget estimates, schedules, technical
specifications and drawings, and management processes are developed by the Recipient. This is
also the Stage where the project is formally approved by NSF as a candidate for a future budget
request and potential obligation of construction funding. Entrance into the Design Stage occurs
with approval from the NSF Director and the Sponsoring Organization obligates the necessary
funding to advance refinement of the scope and the estimated cost and schedule. This Stage
generally lasts 3-5 years and can cost 10% or more of the estimated construction cost
depending on the nature of the project. It is also the Stage where estimated budgets are
presented to Congress and where partnerships are generally formalized.
The Design Stage is divided into three phases – Conceptual Design, Preliminary Design, and
Final Design; each with a formal and rigorous NSF review of the Project Execution Plan at the
end of each phase to show readiness for advancement to the next design phase or Construction
Stage, as shown in Figure 2.1.3-2 below. The proposal submitted for each stage-gate review
(CDR, PDR, and FDR) should include a funding request to support the next phase of Design and
the latest update of the Project Execution Plan for NSF to assess technical readiness.
Advancement to the next phase is based on successful completion of the current phase by the
Recipient and is not guaranteed. Review at the end of each phase is a potential off-ramp for the
project.
Section Revision:
July 26, 2021
2.1.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 2.1.3-2
Progressive Phases within the Design Stage, showing review and decision points for
advancement to the next phase and NSB authorization for budgeting and award.
Conceptual Design Phase: This phase advances the approximate definition of the cost,
scope, and technical requirements from the Development Stage, determines feasibility
(often through the development and testing of prototypes), and produces updated
drafts of most elements of the Project Execution Plan, including parametric cost
estimates, notional integrated schedule, and a preliminary risk analysis.
Preliminary Design Phase: This phase further advances the project definition and the
Project Execution Plan. It produces a bottom-up cost estimate, a near-final proposed
scope and robust schedule (i.e., the project definition), and risk analysis of sufficient
maturity to allow determination of the Total Project Cost and overall duration for a
given Fiscal Year start and to establish the budget request to Congress. The Preliminary
Design Phase ends with a thorough review of the design, the Preliminary Design Review
(PDR), and NSF approval to advance to the Final Design Phase, and NSB authorization for
possible inclusion in future budget request. The strategic assessment of a project’s
priority relative to other opportunities is made before NSF considers a request to NSB
for inclusion in a future budget request. The CORF’s recommendation and the Director’s
strategic decision are made separately from the assessment of technical readiness made
by Program and BFA.
Section Revision:
July 26, 2021
2.1.3-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Final Design Phase: This phase further refines the project definition and the Project
Execution Plan and demonstrates that project planning and management meet
requirements for readiness to receive funding and begin construction. This phase can
also incorporate events or conditions that were unforeseen when the PDR was
conducted. The Final Design Phase ends with a Final Design Review (FDR) and an
obligation of construction funds following NSB authorization for the Director to make an
award.
The stage-gate reviews (CDR, PDR, FDR; see Sections 2.3 of this Guide) are conducted via
external panels consisting of scientific, technical, and project management experts. The
purpose of these reviews is to evaluate the sufficiency of progress made during the respective
phase and the technical readiness to advance to the next phase, including project management
capabilities. NSF uses the findings and recommendations from the external review, together
with in-house financial and business systems analyses, as appropriate to the phase, as input to
an internal review by a Facility Readiness Panel to determine readiness for advancement.
For projects that have received previous development and design funding from NSF, other
agencies, or private sources, a Sponsoring Organization can propose entrance to the Design
Stage at the Conceptual Design Review (bypassing the Conceptual Design Phase) or the
Preliminary Design Review (bypassing the Preliminary Design Phase) based on the technical
readiness of the project. The Preliminary Design Review is the latest point at which a project
can be considered a candidate for major facility funding since this phase is tied to the budget
request. The Final Design Phase must always be conducted.
Construction Stage
The Construction Stage begins when funds are obligated for the acquisition and/or construction
of the research infrastructure in accordance with the terms and conditions set forth in an award
instrument between NSF and the Recipient(s). Depending on the technical nature and scale of
the project, the Construction Stage typically lasts 4-10 years and costs range from $100M to
$800M. Particularly complex projects can cost significantly more and take 10 or more years to
complete. This Stage has the most stringent requirements for overseeing Recipient
performance in managing the scope, cost, and schedule against plan, for reporting progress,
and for formality of oversight and assurance by NSF. Progress is reported against the approved
Performance Measurement Baseline (PMB) in the Recipient’s Project Execution Plan (PEP). The
project status is reviewed periodically to assess whether the project is capable of finishing
within budget and schedule and what corrective actions (if any) might need to be taken. The
Construction Stage normally includes activities to transition the facility to operations. This Stage
ends after final delivery and acceptance of the defined scope of work and an assessment of
facility performance per the terms of the award instrument.
Section Revision:
July 26, 2021
2.1.3-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Operations Stage
The Operations Stage includes the day-to-day activities needed to operate and maintain the
infrastructure and to support scientific research. During this Stage, the facility is actively
collecting and distributing data for use by the science community. Operations may include
activities necessary to complete the transition from construction to full operational capability
(depending on the technical nature of the facility and how the construction scope is defined)
and, during the lifetime of the facility, routine refurbishment activities, and major upgrade
project development. The Operations Stage may also include activities that support transition
to the Divestment Stage. The Operations Stage typically lasts 20-40 years, the total cost of
which often greatly exceeds the cost of construction. It normally includes a series of periodic
status reviews that assess performance. These reviews may be accompanied by decisions on
continued investment, competition, or divestment. The Concept of Operations Plan refined
during the Construction Stage (including robust operations and maintenance cost estimates and
agreements between parties for funding, data sharing, etc.) should be finalized in preparation
for entering this Stage. The decision to divest is generally made when NSF, with input from the
scientific community, determines that the facility is no longer considered an operational priority
with regard to advancing science. This final decision is often the most challenging.
Divestment Stage
The decision to divest is generally made when NSF, with input from the scientific community,
determines that the facility is no longer considered an operational priority for the Foundation.
However, divestment does not necessarily lead to a reduction in project scope or in the
performance or scientific output of a facility. As part of the divestment process, additional
support is often sought from other agencies or non-governmental entities, such as Universities,
state-run programs, or charitable foundations. This decision to divest is often the most
challenging step in the Operations Stage.
The decision to divest may be made at any time during the Operations Stage, though it is
expected to occur after a project’s primary science goals have been achieved (usually after
many years of operations). Divestment options may include partial or complete transfer of a
facility to another entity’s operational and financial control (with or without reduction in
project scope), “moth-balling” the facility so that operations can be restarted at a later date, or
decommissioning. This last option may include complete removal of the infrastructure and site
restoration. The cost of decommissioning can be substantial and must be thoroughly
researched. The decommissioning process may also be very complex, and must include careful
assessment of the risks, benefits, and environmental impacts (in the form of an environmental
impact statement). Entrance into the Divestment Stage occurs when an award is made to cover
the costs of decommissioning or transitioning the facility to its new role. This generally takes
the form of an award that ramps down NSF’s investment over the award duration with the
expectation by all parties that no further operations award from NSF will be forthcoming, other
than for potential scientific use through individual investigator awards.
Section Revision:
July 26, 2021
2.1.3-5
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.3 The Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
It is important that NSF devise plans to address the specific issues that arise as part of the
divestment of a facility. It is recommended that the Sponsoring Organization develop a plan
that follows NSF policy on competition, renewal, and divestment decisions, engages the science
community for the anticipated divestment of the facility, and includes the estimated costs and
associated legal requirements. The first version of this plan should be developed as part of the
construction Project Execution Plan. Periodic review of an evolving plan for the
decommissioning of the facility, disposal of assets and other environmental obligations of the
Government should be conducted during the Operations Stage. While not part of the annual
budgeting process, this plan is part of the longer‐term planning for the Sponsoring
Organization.
Section Revision:
July 26, 2021
2.1.3-6
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.4 Summary of the Major Facility Implementation Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.4 Summary of the Major Facility Implementation Process
Major facility projects cover a wide range of disciplines and activities in science and
engineering, so they can require different approaches to the development and implementation.
The approach described in this Guide is derived largely from experience with large acquisition
and construction projects and operations defined by the following characteristics:
•
They serve a relatively broad and substantial community or collaboration; whose
members have self-organized and agree on the basic parameters of the project
•
They result from proposals to NSF, either solicited through a targeted NSF program or
unsolicited, for the design and construction of research infrastructure
•
Operation of research infrastructure may be carried out by the construction
organization or, in some cases, by another organization
As the diagrams in Figure 2.1.4-1 and Figure 2.1.4-2 indicate, the typical process for preconstruction development and design for a candidate major facility project progresses through
a sequence of stage-gates with increasing investment, planning, assessment, oversight, and
assurance. These stages help ensure that the technical evolution of a candidate project is
coordinated with science community needs and NSF requirements; increasing the likelihood
that it will be able to qualify for funding of continued planning and eventual construction.
NSF supports scientific investigation at the frontiers of human knowledge where the necessary
technologies and methodologies are often not firmly established. The agency is also responsible
for nurturing the various science and engineering disciplines that it supports. As a result, the
various project life cycle stages may best be achieved through the expertise of different
organizations such as educational institutions, non-profit, or the private sector (industry)
depending upon the technical nature of the facility or infrastructure. For example, NSF may
provide researchers the funding sufficient to develop compelling research agendas, to refine
and prioritize their technical requirements, and to complete research and development on
prototypes and other needed technologies, without assuming those researchers will have a
direct role in managing either construction or operations. Following successful research and
development by scientists and engineers in an educational institution, the entire project may
then be further designed and constructed by an award made directly to a competent managing
organization, including industry.
Section Revision:
December 14, 2018
2.1.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.4 Summary of the Major Facility Implementation Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Program evolution
Budget evolution
Development
Initial ideas emerge
Broad science
community
consensus built for
potential long-term
needs, priorities,
and general
requirements
High level concept
developed
Figure 2.1.4-1
Summary Timeline for Major Facility Projects (Development and Design)
Conceptual Design Phase
Preliminary Design Phase
Final Design Phase
Preconstruction Planning Funded via R&RA and EHR funds
Develop construction budget based on
conceptual design
Develop budget requirements for
advanced planning
Estimate operations costs
Formulate science goals: define
requirements, prioritize, review
Develop conceptual design, identify
critical technologies, high risk items
Formulate initial risk assessment
Develop top-down parametric cost and
contingency estimates
Initial proposal submission to NSF
Initial Project Execution Plan (PEP)
Expend ~5-25% of construction cost on planning & design activities
Construction estimate based on preliminary design
Update operations cost estimate
Final design over approximately 2 years
Construction-ready budget & contingency estimates
Update operations cost estimate
Proponents development strategy defined in Project Development Plan
Develop site-specific preliminary design, environmental
assessments/ impacts (NEPA)
Develop enabling technologies
Bottom-up cost and contingency estimates, updated risk
analysis
Develop Project Management Control System
Develop preliminary operations cost estimate
Update PEP
Develop final construction-ready design & PEP
Verify key technologies are ready for production or detailed
production design
Refine bottom-up cost and contingency estimates
Finalize Risk Assessment & Mitigation, Management Plans
Complete key staff recruitment
Internal review
regarding
advancement to
design stage
NSF Director
approval to start
Conceptual Design
Section Revision:
December 14, 2018
Integrated Project Team (IPT)
organized
Develop Internal Management Plan
(IMP), estimate PD costs, timeline
Establish interim review plan and
competition milestones
Forecast international and interagency
participation, issues
Initial analysis of NSF opportunities,
risks
Conceptual Design Review (CDR) –
external panel review and internal
review
CDR Cost Analysis
Merit review, apply 1st ranking criteria
Facilities Readiness Panel Review
NSF Director approval for advancement
to Preliminary Design
Approve Project Development Plan (PDP) & budget
Forecast external partner decision milestones
Preliminary Design Review (PDR) – external panel
review and internal review
Establish target total project cost (TPC)
PDR Cost Analysis
Project Definition established – cost, scope,
schedule, plans, risks, & contingency
Apply 2nd and 3rd ranking criteria
Facilities Readiness Panel Review
DRB Review
NSF Director requests NSB approval for MREFC
request & approves advancement to Final Design
NSB authorization for inclusion in MREFC Budget
Request & to proceed with final design
NSF approves submission to NSB
Interface with the
research
community to
nurture concepts
for development
Recommends to the
NSF Director that a
project advance to
Conceptual Design
NSF Director approves PD phase
FRP, DRB, OD & NSB
Project oversight
NSF oversight defined in Internal Management Plan (IMP) updated at each development phase.
OMB/Congress negotiations on proposed project and
budget profile
Evaluate design costs, schedules; and operations cost
estimate.
Semi-annual assessment of baseline and projected
operations budget for projects not in construction
Finalization of interagency and international requirements,
agreements
Final Design Review (FDR) - external panel review and
internal review
FDR Cost Analysis - informed by an Independent Cost
Estimate (ICE) if not done earlier
EVMS Acceptance
Establish project construction baseline
Facilities Readiness Panel Review
DRB Review
NSF Director approves advancement to construction stage
NSB authorizes NSF Director to make a construction award
2.1.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.4 Summary of the Major Facility Implementation Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 2.1.4-2
Project evolution
Budget evolution
Construction
Summary Timeline for Major Facility Projects (Construction, Operations, and Divestment)
MREFC or R&RA funds
Operations
Divestment
R&RA, EHR funds
R&RA, EHR funds
Expend budget & contingency per baseline, PEP, and Award
document
Refine operations budget
Congress appropriates MREFC funds
Yearly budgets with out-year projects
Transition from operations budget to decommissioning
and disposal costs
Construction per baseline and PEP
Annual Work Plans with goal setting
Annual Reports that track progress relative to goals
Update Facility Divestment Plan developed as part of
the construction PEP.
Develop transition plan for transfer and/or
decommissioning and disposal
Execute the Divestment Plan
Congress appropriates funds
FRP, DRB, OD & NSB
Program oversight
NSF oversight defined in Internal Management Plan (IMP), updated at each lifecycle stage.
Review monthly project performance reports
Review periodic financial reports
Annual external panel reviews and internal assessment
Site visit and assessment
Approves usage of contingency above established
thresholds
Allocation of contingency funds
Review commissioning plans
EVMS Surveillance, generally annually
Accounting System Review, as determined
Business Systems Review, as determined
Cost Incurred Audits, as determined
Annual Operations Review
Preparation for facility re-competition, renewal or
disposal/divestment
Award Proposal Cost Analysis – informed by an
independent cost assessment
Accounting System Review, as determined
Business Systems Review, as determined
Cost Incurred Audits, as determined
Identify and devise plans for divestment of a facility at
the end of its scientifically competitive life.
Annual review of evolving plan for decommissioning
and/or disposal of facility assets and environmental
obligations
External panel reviews as appropriate
Cost analysis as appropriate
Recommends to the NSF Director that a facility move
into the Divestment stage.
Review bi-monthly facilities report
NSF Director “deep-dive” reviews, as appropriate
NSB Facilities Portal with bi-monthly facilities report and
EVM trend data
NSB authorization for re-baselining that exceed
thresholds
DRB review of Renewal and Competition packages
NSF Director approval for award
NSB authorization of renewals and re-competitions
NSB authorization for awards that exceed thresholds
NSF Director authorizes transition into Divestment
Section Revision:
December 14, 2018
2.1.4-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.4 Summary of the Major Facility Implementation Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Although all major facility projects progress though the five life cycle stages described in
Section 2.1.3 above, there are appropriate alternate approaches to the Development and
Design Stages, as well as alternate approaches to upgrade during the Operations Stage.
Facilities at the leading-edge of the scientific endeavor never remain stagnant. It is not
uncommon for major facilities and smaller research infrastructure to be in an almost
continuous state of upgrade following transition to operations. Therefore, the more linear
“waterfall” method described above is not always the most appropriate process to follow,
particularly in fields where the technologies are unproven or changing rapidly.
When proposing to NSF, candidate projects should consider whether a “spiral
development” model is more appropriate than the classic “waterfall” method as shown in
Figure 2.1.4-3. Spiral development refers to the process of designing, building, testing and
using a technology to increase understanding and reduce risk; and then repeating the
process again. Although almost all facilities use spiral development for various components
and sub-systems during development, design and, at times, construction, the process
described here is intentionally planned for and executed at the macro scale, with each spiral
having a discrete total project cost (TPC). Figure 2.1.4-4 illustrates this concept of one
project leading into follow-on projects.
The duration of the spirals can be relatively short (2 years) or quite long (a decade or more)
depending on technical maturity and the rate of technological change. Risk is reduced
following the completion of each spiral to improve confidence in the ability to meet the
technical objectives of the next spiral within budget. A spiral development approach is
generally embedded within the Operations Stage and may combine aspects of the Design
and Construction Stages. NSF oversight is based on the TPC and associated authorization
thresholds.
Figure 2.1.4-3
Section Revision:
December 14, 2018
Linear Design-Construct Process is Classic "Waterfall Model" Implementation
2.1.4-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.4 Summary of the Major Facility Implementation Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 2.1.4-4
Spiral Development Process
In all cases, NSF is committed to the principle that flexibility does not preclude rigor. Every
candidate major facility project – including those that call for novel treatment – is subject to
the highest standards of merit review and technical evaluation. The approach used should
be identified early in either the project Development Stage or Design Stage and
documented as part of the managing organization’s proposal and eventually the Project
Execution Plan (PEP), as well as NSF’s Internal Management Plan (IMP). Proposing
organizations should discuss the approach envisioned with the cognizant NSF Program
Officer.
Section Revision:
December 14, 2018
2.1.4-5
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.5 Timeline and Flowcharts for the Major Facilities Approval Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.5 Timeline and Flowcharts for the Major Facilities Approval Process
This section, to be written, will illustrate when various preconstruction planning activities
should be completed in order to commence construction in a particular future fiscal year.
Although the majority of those activities proceed at a pace specific to the needs of an individual
project, late-stage planning activities following completion of a project’s Preliminary Design are
paced by the process for developing NSF’s annual Budget Request to Congress. This section will
also explain key features of that process that are of particular interest to those involved with
major facility projects.
Section Revision:
June 17, 2015
2.1.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major
Facilities
2.1.6.1 Overview
The Major Facilities Guide (MFG) describes the actions NSF takes to carry out its oversight and
assurance responsibilities for major facility projects. One key element is the definition of the
roles and responsibilities of the NSF participants who carry out those actions. The participants
with primary oversight and management roles and responsibilities are listed below and
highlighted in the NSF organizational chart in Figure 2.1.6-1:
•
Program Officer (PO) – A scientist or engineer having primary oversight responsibility
within NSF for all aspects of the project. 1
•
Sponsoring Organization – The NSF Division, Directorate, or Office which proposes
projects for funding through the MREFC Account or other funding source and is
committed to pre-construction planning activities and eventual facility operation and
use.
•
Senior Management of the Sponsoring Organization – The leadership individuals who
utilize community inputs, discipline-specific studies, advisory committee
recommendations and internal NSF considerations to prioritize the opportunities
represented by the candidate project relative to competing opportunities and demands
for available resources.
•
Grants and Agreements Officer (G/AO) – NSF Grants and Agreements Officer who has
legal responsibility and authority for the business and financial management of grants
and cooperative agreements.
•
Contracting Officer (CO) – NSF Contracting Officer which has legal responsibility and
authority for the business and financial management of award contracts.
•
Cost Analyst – NSF staff from the Cost Analysis and Pre-Award Branch (CAP) Branch of
the Division of Institution and Award Support (DIAS), which perform cost assurance
reviews of proposals and monitor Recipient financial practices.
•
Chief Officer for Research Facilities (CORF) – The senior official who advises the NSF
Director on all aspects of the agency's support for major and mid-scale research facilities
throughout their life cycle and collaborates with NSF employees involved in oversight
and assistance of the NSF multi-user research facilities portfolio.
•
Head, Large Facilities Office (HLFO) – The individual who heads the Large Facilities
Office (LFO). The LFO provides an NSF-wide resource for assistance with project
oversight and assurance that agency policies and procedures are followed. The LFO is in
The PO may have a title such as Program Manager or Program Director. The PO is administratively part of a Directorate or
Office, comprised of Divisions, which serves a range of research disciplines. These are referred to as the “Sponsoring
Organization” in this document.
1
Section Revision:
July 26, 2021
2.1.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
the Office of Budget, Finance, and Award (BFA) and reports to the Chief Financial
Officer.
•
Large Facilities Office Liaison – The designated project management advisor from the
LFO, who is assigned as project liaison by the HLFO. This individual is the PO’s primary
resource for assistance with all policy, process, and procedural issues related to the
development, implementation, and oversight of major facility projects.
Figure 2.1.6-1
NSF organization chart highlighting staff who have primary oversight and management roles
and responsibilities for major facilities and mid-scale research infrastructure.
As shown in Figure 2.1.6-2, various bodies within NSF provide coordination, assistance,
assurance, and advice to the main participants and to the agency as a whole:
•
Integrated Project Team (IPT) – Three primary NSF groups comprise the IPT and
represent the major oversight organizations within NSF: Science and Technical, Award
Management, and Strategic. The Strategic group includes representatives from the
Office of the Director (OD), Office of the General Counsel (OGC), the Office of Legislative
and Public Affairs (OLPA), the Office of International Science and Engineering (OISE), and
other offices under OD as deemed appropriate. The composition and size of each IPT
depends on the risks, scope, and complexity of the project. The IPT is a coordinating
body that provides internal agency assurance and guidance to the PO in the planning,
review, and oversight of that project. The members of the IPT are selected by the
management of the cognizant directorates and offices, in consultation with the PO, at
the beginning of the Conceptual Design Phase. The IPT is chaired by the PO.
Section Revision:
July 26, 2021
2.1.6-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Major Facilities Working Group (MFWG) – The purpose of the Major Facilities Working
Group (MFWG) is to assure the uniform and effective programmatic oversight of major
and mid-scale research infrastructure of the National Science Foundation throughout
their entire life cycles. Specifically, the MFWG provides input to the Facilities
Governance Board regarding all strategy, governance, and implementation issues under
consideration by that Board, establishes and maintains a list of NSF’s major research
infrastructure at all life cycle stages, supports the Head of the Large Facilities Office in
reviewing the Major Facilities Guide, internal operating guidance, and procedures for
NSF facility oversight, advises the Facilities Governance Board on the sufficiency and
appropriateness of these documents, and shares good practices for the oversight of
facilities across the science and engineering directorates and offices.
•
Advisory Committee of the Sponsoring Organization – Comprised of researchers from
the community (external to NSF), it advises the sponsoring Directorate or Office in a
wide variety of programmatic areas, including major facilities.
Figure 2.1.6-2
NSF organization chart showing coordinating and advisory bodies for major facilities and midscale research infrastructure.
There are also planning and assurance bodies, shown in Figure 2.1.6-3, that review and make
recommendations on the suitability and readiness as well as on the allocation of resources for
the development, funding, and operation of major facility projects, according to the NSF
strategic objectives:
•
Facilities Readiness Panel (FRP) - advises the Director on Recipient and Programmatic
readiness to advance major facilities projects within the formal Design Stage as
described in NSF’s Major Facilities Guide (MFG); this includes the transition from Final
Section Revision:
July 26, 2021
2.1.6-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Design to Construction. Decisions on readiness to enter the Design Stage and whether
or not to include a project in a future budget request are made separately.
•
Facilities Governance Board (FGB) - Oversees and makes recommendations on all
aspects of governance of major multi-user research facilities and mid-scale research
infrastructure of the National Science Foundation.
•
Director’s Review Board – Comprised of Senior Management Representatives from the
Directorates and Offices of NSF, it reviews and approves the package of materials
associated with all topics to be submitted to the National Science Board (NSB) for
information or action, including major facility projects.
Finally, there are entities also shown in Figure 2.1.6-3 that set NSF policy and that approve the
advancement, funding requests, and obligation of funds for the development, construction, and
operation of major facility projects:
•
NSF Director – Responsible for the implementation of NSF policies and practice for
agency oversight of major facilities, and for proposing new major facility projects to the
NSB, the Office of Management and Budget (OMB), and Congress.
•
NSB – Establishes agency policy for major facilities, and reviews and authorizes the
advancement of major facility projects including budget requests and Construction
Stage awards. The Board also authorizes Operations Stage awards that are above
certain thresholds. By statute, all projects funded from the MREFC account require
Board authorization.
Figure 2.1.6-3
NSF organization chart showing policy and approval bodies for major facilities 1 and mid-scale
research infrastructure.
1 Refer to Figures Figure 2.1.4-1 and Figure 2.1.4-2 for a mapping of the Panels and Boards to the major facility life cycle stage
and NSF oversight responsibilities.
Section Revision:
July 26, 2021
2.1.6-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The PO, G/AO or CO, and LFO staff members are the individuals that interact most frequently to
carry out NSF’s oversight and assurance role for major facility projects. Their roles and
responsibilities are summarized, by life cycle stage, in Table 2.1.6-1. Fuller descriptions of their
roles (and those of senior management in the sponsoring Directorate or Office, and the
support, advisory, policy making, and approving entities) are provided in individual sections of
this document following Table 2.1.6-1.
Table 2.1.6-1
Summary of Principal Roles and Responsibilities of the core members of the IPT (PO, G/AO or
CO, and LFO) Liaison by Facility Life Cycle Stage
Program Officer (PO)
Grants and Agreements (G/AO)
or Contracts Officer (CO)
LFO Liaison
• Primary representative of the
NSF in all business dealings with
the Recipient
• Assigned to a project on a longterm basis
• Experienced with Federal
regulations and unique NSF
requirements needed for
adequate NSF oversight of major
facility projects
• Program’s primary resource
for all policy or process issues
related to the development,
implementation, and
oversight of major facility
projects
• Experienced and trained in
project management of
major facilities.
• Advises POs on project
management issues during
project development and
oversight
• Becomes acquainted with the
anticipated scope of the
proposed project
• In collaboration with PO,
plans CDR
Summary
• Primary responsibility for all
oversight aspects of a major facility
project
• Experienced or trained in
management of projects.
• Appointed by the Division Director
(DD) or Section Head
• Must not be a temporary
employee of the NSF
Conceptual Design Phase
• Determines the importance and
research priority to the affected
research community of the science
objectives motivating
consideration of a future major
facility
• Works with the research
community to develop an overall
scope for a major facility project.
• Develops the IMP
• Organizes and chairs the IPT
• Formulates a plan for divestment
of the facility
• Devises and carries out strategies
for renewal or closeout strategy
that implements competition of
the operating award wherever
feasible
Section Revision:
July 26, 2021
• Participates in planning
meetings to work out details of
partnerships, international or
multi-agency agreements,
property issues, etc.
• Participates in the development
of the IMP
• Serves on the IPT throughout
the project to expedite financial
and administrative actions and
decisions concerning the project
• Independently assesses the
CDR outcome for the LFO
• Serves on the IPT throughout
the project to advise on
management, business, and
administrative issues
• Participates in the
development of the IMP
2.1.6-5
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Program Officer (PO)
Preliminary Design Phase
• Creates solicitations for any
enabling research, workshop,
summer study, or other activity of
the research community that
supports proposal development
• Works with the research
community to develop a proposal
that includes a preliminary Project
Execution Plan (PEP)
• Arranges external peer review of
the proposal
• Presents the proposed project to
the Facilities Readiness Panel
• Updates the IMP
• Continues to meet with the IPT
Grants and Agreements (G/AO)
or Contracts Officer (CO)
LFO Liaison
• Advises PO on creation of
solicitations for any enabling
research, workshop, summer
study, or other activity of the
research community that
supports proposal development
• Advises PO
• Responsible for the business
aspects of the proposal review
and cost analysis and mentoring
of the proposing institutions
• Participates in preparation of
materials for the FRP Review
and Director’s Review Board
(DRB)
• Serves on the IPT
• Reports monthly to HLFO on
project’s technical and financial
status
Final Design Phase
• Continues to monitor project in
accordance with the IMP
• Provides monthly project status
updates to the HLFO
• Organizes periodic cost update
reviews
• Organizes the Final Design Review
(FDR)
• In collaboration with PO,
plans Preliminary Design
Review (PDR)
• Independently assesses
outcome of PDR for the LFO
• Receives monthly reports on
project development from
PO, and provides
independent assessment to
the Head, LFO
• Contributes to business
aspects of the proposal
review and cost analysis and
in surveillance or mentoring
of the proposing institutions
• Serves on the IPT
• Instigates as required proposal
review, cost analysis, and
mentoring necessary to ensure
that the Recipient follows NSF
business and budgeting policies
and requirements
• Participates in periodic cost
update reviews.
• Participates in preparation of
materials for the FRP Review
and DRB
• Serves on the IPT
• Continues to monitor project
• Receives monthly project
status updates from the PO,
adds comments and
evaluation
• Aids the PO with the
organization of the periodic
cost update reviews in
interval between PDR and
FDR.
• In collaboration with PO,
plans FDR and independently
assesses outcome
• Contributes to business
aspects of the proposal
review and cost analysis and
in surveillance or mentoring
of the proposing institutions
• Serves on the IPT
Section Revision:
July 26, 2021
2.1.6-6
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Grants and Agreements (G/AO)
or Contracts Officer (CO)
LFO Liaison
• Works with the G/AO to develop
the award agreement (Cooperative
Agreement (CA) or contract
agreement)
• Works with the PO to develop
the award agreement
(Cooperative Agreement (CA) or
contract agreement)
• Advises PO
• Approves the establishment of a
project baseline scope, cost, and
schedule and other updates to the
PEP
• Approves submittals from
Recipient
Program Officer (PO)
Construction/Implementation
Stage
• Approves significant changes to
the project baseline
• Receives monthly financial and
technical status reports, quarterly
and annual progress reports
• Reports monthly to HLFO on
project’s technical and financial
status
• Conducts periodic reviews of
project progress using an external
ad hoc panel
• Arranges internal review of
Memorandums of Understanding
(MOUs)
• Reviews the scope of activities
associated with each award to
ensure that the financial and
administrative framework aligns
with NSF’s expectations for
stewardship and reporting.
• Receives and provides approval
to the Recipient on award
documents
• Participates in baseline review
and subsequent periodic
reviews as necessary to assure
the NSF that the Recipient
follows agency financial policies
• In collaboration with PO,
plans construction reviews
and independently assesses
outcome
• Receives monthly project
status reports from the PO
• Visits the project site
periodically in coordination
with PO
• Participates in baseline
review and subsequent
periodic reviews as necessary
to assure the NSF that the
Recipient follows agency
major facility management
policies
• Serves on the IPT
• Serves on the IPT
• Regularly visits the project
• Updates the IMP
• Ensures compliance with
Government Performance and
Results Modernization Act
(GPRAMA)
Section Revision:
July 26, 2021
2.1.6-7
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Program Officer (PO)
Operations Stage
• Prepares and participates in
solicitation of award for Operations
and Maintenance (O&M) CA or
contract agreement
• Ensures compliance with GPRAMA
• Approves the Annual Work Plan
(which includes high level
performance goals) developed by
the Recipient
• Reviews and approves the Annual
Report
• Develops budgets that operate and
maintain facilities
• Obtains Condition Assessment
reports
• Monitors planning for IT and
property security, and validates
through periodic review
• Organizes and participates in
periodic reviews of the facility
including annual operations reviews
• Formulates a plan for divestment of
the facility
• Devises and carries out a renewal
or competition and closeout
strategy of the operating award
Grants and Agreements
(G/AO) or Contracts Officer
(CO)
• Advises the PO in development
of solicitation for O&M award
(shared responsibility with PO)
• Creates special terms and
conditions in the CA or contract
agreement to capture
requirements for annual
performance goals (shared
responsibility with the PO)
• Defines business practices for
renewal, competition, closeout,
or termination of Award
• Attends periodic reviews
including operations and
business systems reviews
(BSRs) as appropriate
• Assists in developing financial
strategy, as appropriate, to
budget for facility maintenance
and replacement or
refurbishment of long-lived
capital-assets (shared
responsibility with PO)
• Prepares Decision Memo and
performs independent cost
analyses as required
LFO Liaison
• Advises PO and G/AO or CO
on effective operational
oversight strategies, renewal
and competition strategies,
closeout, or termination
• Periodically visits operating
facilities in coordination with
PO
• In collaboration with PO and
G/AO or CO, insures
implementation of
performance measures
within the CA for operation
• Assists with organizing and
evaluating the results of
operational reviews of major
facilities
• Advises PO and G/AO or CO
on related project
management issues in the
event of competition of
award for facility operation
• Serves on the IPT
• Serves on the IPT
• Updates the IMP
Divestment Stage
Reserved for future content
2.1.6.2 Main Participants
Program Officer (PO)
The PO is the research community’s primary interface to the NSF. The PO’s responsibilities are
substantial, and crucial to NSF’s success. Examples of these responsibilities are listed below: 1
•
They are typically the main contact a principal investigator (PI) has with NSF.
Paraphrased from National Science Foundation: Governance and Management for the Future, a report by a panel of the
National Academy of Public Administration, April 2004. pp. 10-11.
1
Section Revision:
July 26, 2021
2.1.6-8
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
•
•
•
•
They are the link between what is happening in the research community and the
appropriately responsive program solicitation from NSF.
They are the catalysts for the increasing amount of research that crosses traditional
single-discipline boundaries.
They are the coaches and encouragers for proposals from less experienced researchers
– particularly ones with innovative ideas – as well as those from underrepresented
segments of the research community.
They are the recruiters and managers of a peer review process that involves numerous
experts from the research community to assess the intellectual merit and broader
impacts of proposals from the community for new research.
They are the post-award managers and monitors for awarded research.
NSF’s Authorization Act of 2002, 42 U.S.C.1862n-4I, signed into law on December 19, 2002,
restricts the choice of POs (also referred to within the NSF as Program Directors or Program
Managers) to be regular employees of the NSF. The statutory language of the Act states:
“PROJECT MANAGEMENT. No national research facility project funded under the major
research equipment and facilities construction account shall be managed by an individual
whose appointment to NSF is temporary.”
Administratively, the PO is part of a Directorate or Office that provides supervisory oversight
and the budgetary authority to fund PO actions. Depending on the administrative structure of
the Sponsoring Organization, a Section Head, Division Director, Assistant Director (AD), or
Office Head may assign a PO (or POs) 1 to oversee a particular facility-related initiative and will
directly or indirectly oversee and guide the activities of the PO. Actions of the PO described
here implicitly recognize the authority of the individuals within this supervisory structure to
appropriately guide, direct, and approve the actions of the PO.
The PO exercises primary responsibility within NSF for all aspects of a major facility project,
including:
•
Project planning, both internally and in coordination with the relevant research
community;
•
Serving as the NSF interface with the research community to nurture concepts for
development and utilization by the community of a facility;
•
Formulating an IMP that defines NSF strategy for conducting project oversight,
managing NSF risk, and providing project funding;
•
Coordinating contact between the project proponents and other NSF staff members
that may need to have direct contact with the project or that the project may wish to
contact;
In some cases, more than one individual will be designated as a PO for a facility related initiative. Wherever the PO is
referenced in this guide, it should be understood that the reference is to all the relevant assigned POs.
1
Section Revision:
July 26, 2021
2.1.6-9
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Chairing the IPT;
•
Conducting merit and programmatic/technical reviews of proposals for development,
implementation, operation, and utilization of a facility (Conceptual Design Review (CDR),
Preliminary Design Review (PDR), Final Design Review (FDR), construction and
operational reviews);
•
Preparing all required documentation for internal project review and approval within
the NSF;
•
Participating in developing the estimated costs of planning, construction, operations,
maintenance and related programmatic activities, and, under management direction of
the Sponsoring Organization, assigns budgets to these tasks; and
•
Overseeing implementation, operation, and divestment and eventual closeout of NSF
support for the project.
Senior Management of the Sponsoring Division, Directorate, or Office Assistant Director or
Office Head
Assistant Directors (ADs) and Office Heads lead Directorates or Offices, and by extension their
Divisions or Sections, which propose projects for funding through the MREFC Account or other
funding source.
The AD (or Office Head) of the Sponsoring Organization utilizes community inputs, disciplinespecific studies, advisory committee recommendations and internal NSF considerations to
prioritize the opportunities represented by the candidate project relative to competing
opportunities and demands for NSF resources. The AD determines that the scientific merit and
relative importance of the proposed facility are sufficiently strong to justify advancement of the
project to readiness stage (i.e., ready to begin Preliminary Design activities), and authorizes the
PO to proceed with organizing the development and external review of a Project Execution Plan
and with updating the IMP to explain how NSF will oversee and fund further development. The
AD reviews and approves the IMP. The AD determines whether to propose a project to the
Facilities Readiness Panel as a candidate for future construction funding, based on the project’s
relative scientific importance and on the Sponsoring Organization’s commitment to preconstruction planning activities and eventual facility operation and use. The AD is regularly
updated by the PO on the status of the project throughout the remainder of its life cycle phases
and brings critical issues to the attention of the NSF Office of the Director (OD) and NSB as
appropriate.
The AD has overall responsibility for advancing prospective projects for consideration of
construction funding. In this capacity, the AD formulates strategic planning and budget
development within the sponsoring Directorate or Office. This strategic planning includes
prioritizing across the research objectives of the range of disciplines served by the Directorate
or Office. The AD oversees and monitors development of NSF’s project planning, with the
assistance of supporting staff, advisory committees, and direct interactions with the broader
community affected by the facility.
Section Revision:
July 26, 2021
2.1.6-10
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Unless delegated to a lower level, the AD oversees development of MOUs with other agencies,
international partners, private foundations, and other entities and, with the approval of the NSF
OD, enters into negotiations with those parties and either signs or delegates signature authority
for these agreements on behalf of NSF when authority to do so is delegated by NSF OD.
Throughout a project’s life, the AD has a primary responsibility to keep all major stakeholders in
the project informed. Interested parties include policy stakeholders (the NSF, OD); funding
stakeholders (OMB, Congress); and community stakeholders (scientific organizations and the
relevant research community).
At each stage of project development, the AD has the responsibility for making key decisions
within the Sponsoring Organization that advance a project or remove it from consideration for
further development.
Specific responsibilities include, but are not limited to:
•
Approving the IMP at the Directorate level;
•
Ensuring that the qualifications of the relevant Division Directors reflect the
requirements and expectations of the MFG and NSF policy, and the necessity to provide
an environment of open communication and transparency in the management of
research infrastructure;
•
Assuring the evaluation and endorsement of a candidate project by the Directorate or
Office advisory committee prior to submission of the project to the Facilities Readiness
Panel for entry into the readiness stage;
•
Overseeing the Division’s organization of all design reviews including appointment of
review panels, charges to the panels, and Directorate responses to review panel
recommendations;
•
Reviewing and approving all Director's Review Board packages and organizing
representation of the project before NSF internal approval bodies, i.e., FRP, DRB, and
the NSB;
•
Representing the sponsoring Directorate or Office in decisions to compete management
of an operating facility, terminate support, admit new partners, and other major
decisions affecting the facility;
•
Selecting members of Directorate Office staff to serve as representatives on an IPT; and
•
Establishing appropriate Delegation of Authority for awards following NSB action.
Division Director
The Division Director (DD), assisted by Divisional Staff, has primary responsibility for overseeing
planning, review, oversight and funding of major facilities. This responsibility includes
coordination of planning; serving as the interface with relevant scientific and engineering
communities; preparing all required documentation for project consideration and approval;
Section Revision:
July 26, 2021
2.1.6-11
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
conducting merit review of proposals; fully funding costs of operations, maintenance and
relevant programmatic activities; and overseeing the project.
Administratively, a major facility in planning, construction, or operation, is under the purview of
a Sponsoring1 Organization, a Directorate, Division, or Office. The Sponsoring Organization
provides supervisory oversight and budgetary authority. Depending on the administrative
structure of the Sponsoring Organization, the cognizant PO is usually selected by the Divisional
management (e.g., Section Head and DD collaborate in the selection) with concurrence of the
AD. The PO’s superiors directly or indirectly oversee and guide the activities of the PO.
The DD has overall responsibility for the conduct of programs in a related range of disciplines
within NSF, and for the NSF interfaces between these programs and the scientific communities
in these disciplines. For major facility projects, the DD:
•
Evaluates and maintains, through appropriate mechanisms, the proper balance between
the totality of life cycle costs for major facilities and the rest of the division’s activity;
•
Establishes and continually examines, through appropriate mechanisms and forums, the
priorities among candidate projects within the discipline (those in development, under
construction, and in operation);
•
Appoints a cognizant PO for each project;
•
Ensures that the program officer has the requisite experience and/or training to respond
to the responsibilities of the position;
•
Ensures that the cognizant PO follows appropriate good practices;
•
Ensures that the PO is responding appropriately to the requirements of the Major
Facilities Guide and other NSF policies and practices;
•
Ensures that the PO is managing interfaces with other NSF units effectively and
productively;
•
Ensures that the performance plan of the program officer reflects the requirements and
expectations of the MFG and other NSF policy statements; and
•
Facilitates the flow of information at an appropriate level of detail and timescale to keep
all NSF stakeholders appropriately informed of project progress, status, and problems.
Grants and Agreements Officer
The Grants and Agreements Officer (G/AO) has authority, subject to statutory limitations, to
award and administer cooperative agreements (CA). The G/AO holds a cooperative agreements
warrant and is the only individual authorized to obligate or de-obligate Federal funds. The
G/AO, through their warrant, has the sole authority to award and administer the construction
agreements(s) used in support of major facility projects. The G/AO is administratively part of
Division of Acquisition and Cooperative Support (DACS) in BFA, except for mid-scale projects
1
This is the “lead organization” in the case where more than one Division participates in sponsoring a project.
Section Revision:
July 26, 2021
2.1.6-12
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
where they may be part of DGA (See Section 5 of this Guide). The timing of this assignment is at
the discretion of the DGA or DACS DD but should be early enough in the planning stage of a
major facility project to allow the participation of the G/AO in the strategic planning and
development of the IMP for a major facility project (i.e., during the Conceptual Design Phase
when NSF begins to consider strategies for the business aspects of managing oversight of the
proposed project).
The G/AO is an integral member of the IPT for a facility project when the award instrument is a
cooperative agreement, in order to expedite NSF action on business and administrative issues
related to the project. The G/AO participates in management reviews, risk assessments and
issues affecting the management of the award. The G/AO plans and coordinates development
of award instruments from early planning stages through award administration and closeout.
The G/AO negotiates terms and conditions, interprets Federal and NSF policy, and reviews
business proposals and budgets, subawards1, MOUs, and partnership agreements. The G/AO
also monitors awards for compliance with the most current NSF financial and administrative
policies and procedures.
The G/AO is the primary point of contact at the NSF with the Recipient institution for all
business and financial matters. The G/AO represents the NSF in conducting all of the financial
and administrative business-related oversight of the Recipient, including:
•
Providing approval or authorization for all financial transactions,
•
Ensuring compliance with financial and administrative award terms and conditions,
•
Accepting submittals or reports from the Recipient,
•
Leading the cost analysis process of proposals and negotiating the budget, and
•
Negotiating any specific terms and conditions which define the conduct and execution
of a project, such as CAs and subsequent amendments, MOUs, property leases and
insurance, etc.
The G/AO is responsible for oversight of the financial and administrative terms and conditions
of the assistance award,2 just as the PO is responsible for scientific and technical oversight.
Unlike the PO, he/she holds the warrant to obligate Federal funds. The G/AO and the PO jointly
share the principal technical and financial responsibilities for the oversight and assurance of a
major facility project. In this capacity, the G/AO is jointly responsible with the PO for the
success of a project.
Except for the purchase of materials and supplies, equipment or general support services allowable under the award, no
portion of the proposed activity may be subawarded to another organization without written prior NSF authorization. All
proposing organizations are required to make a case-by-case determination regarding the role of a Subrecipient versus
contractor for each agreement it makes. See PAPPG II.C.2g (vi) e for further guidance.
1
An assistance award is a grant or cooperative agreement (CA) to a non-Federal organization with fiduciary responsibility for
the project or facility.
2
Section Revision:
July 26, 2021
2.1.6-13
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The G/AO confers with the PO and other relevant offices to ensure that the NSF’s technical and
administrative oversight activities are well coordinated. The G/AO and the PO collaborate on
the preparation of solicitations and the proposal and award process. The G/AO has individual
responsibility for developing and overseeing the implementation of financial and administrative
aspects of the award process, and joint responsibility with the PO for competition planning and
execution and for award termination or closeout.
The G/AO develops the CAs that establish a business relationship between the NSF and the
Recipient. Consequently, the G/AO has an oversight responsibility that extends to the business
practices of that Recipient, in addition to the specific business operations and oversight
practices of the particular project that may be based with that Recipient.1
The G/AO, with the assistance of BFA resources, establishes that the financial stewardship and
reporting practices of the Recipient institution, as they pertain to NSF instruments, are
consistent with NSF requirements, 2 CFR § 200 (Uniform Administrative Requirements, Cost
Principles, and Audit Requirements for Federal Awards), or Federal Acquisition Rules, as
appropriate.2
Contracting Officer
The Contracting Officer (CO) has authority, subject to statutory limitations, to award and
administer contracts for the construction and operations of facilities that are managed through
contract rather than Cooperative Agreements (CAs). The CO is appointed by the agency Senior
Procurement Executive and is administratively part of the Division of Acquisition and
Cooperative Support within BFA. The CO is solely responsible for oversight of the terms and
conditions of the contractual agreement.
The CO holds the warrant and is the only individual authorized to obligate or de-obligate
Federal funds. The CO, through their warrant, has the sole authority to award and administer
the prime construction contract(s) used in support of major facility projects.
The CO is an integral member of the IPT for a facility project when the award instrument is a
contract, in order to expedite NSF action on business and administrative issues related to the
project.
Cost Analyst
The G/AO or CO requests assistance from an NSF Cost Analyst from the Cost Analysis and PreAward (CAP) Branch of the Division of Institution and Award Support (DIAS), located within BFA,
when cumulative or individual awards exceed certain thresholds or for Recipients with
previously identified risks. The PO, G/AO or CO, and Cost Analyst all review proposed budgets
Refer to the Business Systems Review (BSR) Guide described in Section 4.6.3.3 for discussion on this point. When NSF is not the
cognizant audit agency for the Recipient institution, its oversight of Recipient business practices is narrowly defined.
1
2
Refer to the Business Systems Review (BSR) Guides for more details on the criteria and processes for this assessment.
Section Revision:
July 26, 2021
2.1.6-14
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
to help determine if they are allowable, allocable, reasonable, and realistic for the scope of
work. However, the primary purpose of the NSF Cost Analyst’s budgetary review is to support
the G/AO or CO to ensure that the Recipient has properly estimated and calculated costs and
that they are supported and documented with sufficient rigor. The Cost Analyst provides a
written recommendation to the G/AO or CO stating whether costs are supported or
unsupported. The recommendation may include advice on award terms and conditions, or
limitations or other concerns identified.
The Cost Analyst may also help determine if the Recipient has adequate business and
accounting systems in place, assess a Recipient’s financial capability and viability, validate
indirect cost rates, or assist in other areas of concern as identified by the requesting G/AO or
CO.
While the G/AO or CO is the primary point of contact with the Recipient for all award and cost
analysis issues, this should not inhibit direct communications between the Cost Analyst and
Recipient when necessary. Cost analysis communications with the Recipient should include the
Cost Analyst, G/AO or CO, and PO to help ensure efficient resolution, close collaboration, and
clear and consistent direction.
Chief Officer for Research Facilities (CORF)
The position of Chief Officer for Research Facilities (CORF) resides within the Office of the
Director, reports directly to the Director, and has full life cycle oversight responsibility for NSF
major research facilities. The CORF advises the NSF Director on all aspects of NSF major and
mid-scale facilities throughout their life-cycles and collaborates with all at NSF who are involved
in oversight and assistance for the NSF research facilities portfolio. The CORF chairs the
Facilities Readiness Panel, the Major Facilities Working Group, and the Facilities Governance
Board.
This position also fills the previous statutory requirement for NSF to have a Deputy Director for
Large Facility Projects.
Head, Large Facilities Office (HLFO) and BFA’s Large Facilities Office
The NSF’s Head, Large Facilities Office (HLFO), and the LFO supporting staff are the NSF’s
primary resource for all policies or processes related to the development, implementation, and
oversight of research infrastructure. LFO is the Foundation’s primary resource for all oversight
practices related to major facility projects and is the NSF-wide resource on project management
good practices. The LFO has the institutional authority and resources to effectively develop
mandatory policies, practices and procedures, which are approved by senior management, for
all stages of the facility life-cycle. The LFO works closely with the BFA and NSF Senior
Management Officers, providing expert assistance on non-scientific and non-technical aspects
of project planning, budgeting, and implementation for major facilities. It also provides
assurance that all applicable requirements are followed in order to give credence to NSF’s
oversight capabilities. The LFO also facilitates the use of good practices by fostering
Section Revision:
July 26, 2021
2.1.6-15
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
coordination and collaboration throughout NSF to share application of lessons learned from
prior major facility projects.
The LFO develops and implements processes for ensuring that all facility award instruments
include, at a minimum, four performance evaluation and measurement components:
1. Clear and agreed-upon goals and objectives;
2. Performance measures and, where appropriate, performance targets;
3. Periodic reporting; and
4. Evaluation and feedback to assess progress.
Prior to NSF requesting NSB authorization to include a proposed project in a future budget
request, the HLFO contributes to agency assurance that the project plans are construction
ready, and that the construction and operations budgets are satisfactorily justified.1 This
assurance comes through assignment of the LFO Liaison to the IPT and membership (as
assigned) on various governance bodies such as the Facilities Readiness Panel and the Director’s
Review Board.
The HLFO prepares a periodic status report for NSF Leadership on all ongoing major facility
projects, candidate projects in planning, and other major facility projects designated by the
Sponsoring Organization. Inputs to the monthly report are provided by each cognizant PO and
their associated Directorate/Division. The Recipient submits a monthly report to the PO that
summarizes the technical and financial status of the project, pending near-term milestones, and
any other issues that should be brought to the attention of the LFO. The PO reviews the report
and prepares a written response to the monthly report that is uploaded into eJacket. The HLFO
combines all of these inputs into a single report, summarizes the key technical and financial
status information, and provides an independent commentary on project management issues,
as necessary.
Under the direction of the NSF Senior Management, the HLFO prepares and presents a variety
of information to the National Science Board (NSB) related to the status and plans for the
portfolio of major multi-user facility projects that are either receiving or are candidates for
receiving MREFC funds. This information supplements information contained in the NSF’s
annual Budget Request to Congress.
LFO Liaison
For each major facility project, the HLFO designates an LFO Liaison to work closely with the PO
and the G/AO or CO, providing expert assistance on non-scientific and non-technical aspects of
project planning, budgeting, implementation, and management to further strengthen the
oversight capabilities of NSF. The LFO Liaison participates in each project IPT and also advises
the cognizant PO of mitigating steps when project management challenges arise. The LFO
See “Priority Setting for Large Facility Projects” (NSB-04-96), National Science Board White Paper, May 2004, Attachment 5 to
NSB Meeting Report, https://www.nsf.gov/nsb/meetings/2004/may_srprt.doc.
1
Section Revision:
July 26, 2021
2.1.6-16
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Liaison works with the PO and the G/AO or CO, not directly with the Recipient or their project
staff.
The LFO Liaison also collaborates with the PO and G/AO or CO to plan and carry out key project
reviews including CDR, PDR, FDR, operations reviews, and other ad hoc project reviews in all life
cycle stages as appropriate. While the PO is responsible for planning, carrying out, and
assessing the full range of topics addressed in the review, LFO Liaison focuses on project
management, business, and administrative issues, and assists the PO and G/AO or CO in these
areas. The LFO Liaison independently assesses and reports to the HLFO on the outcome of
these reviews with respect to project management issues.
The LFO Liaison participates in site visits in coordination with the PO and Sponsoring
Organization, to strengthen project management and affirm aspects of NSF’s oversight and
assurance role. During these interactions, the PO is the single point of contact with the project
for all programmatic issues, and the G/AO or CO is the point of contact with the Recipient
institution for administrative issues. Any project-specific communications between the LFO
Liaison and the project is coordinated through the respective PO, G/AO, or CO, and generally as
part of the IPT process.
LFO also carries out BSRs of Recipient business systems for major facilities in design,
construction or operation based on a regular review cycle and other potential risks, such as
building institutional capacity in advance of a construction award. BSRs may also be conducted
at smaller scale facilities at the request of NSF Leadership or the Sponsoring Organization. BSR
objectives and processes are described in detail in NSF’s Business Systems Review (BSR) Guide,
described in Section 4.6.3.3.
2.1.6.3 Coordinating and Advisory Bodies
The Integrated Project Team
The Integrated Project Team (IPT) serves as a formal internal NSF coordinating body for major
facilities oversight throughout the Design, Construction, and Operations Stages. The IPT consists
of three primary sub-groups:
1. Science and Technical Group led by Program with primary responsibility for project
oversight. This group may include other Staff from the Division and/or Directorate as
deemed appropriate by Program (budget, science program, etc.).
2. Award Management Group comprised of various Offices and Divisions within the BFA.
This group is primary responsible for assurance. The linkage with the Science and
Technical Group is with the review and monitoring of cost, scope and schedule as well
as the Project Execution Plan and Recipient performance. The linkage with the Strategic
Group is related to internal NSF processes and procedures.
3. Strategic Group comprised of various offices within the OD. This group’s role is primarily
with assessing risk. The linkage with the Science and Technical Group is with
communication with external stakeholders.
Section Revision:
July 26, 2021
2.1.6-17
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The IPT is chaired by the PO, see Figure 2.1.6-4. Members are selected by the DDs, ADs or Office
Heads, in consultation with the PO. The PO will convene the IPT at least quarterly to address
any project-related issues.
Figure 2.1.6-4
An Integrated Project Team (IPT), chaired by the Program Officer, is composed of three
subgroups, with appointed Award Management Group members from BFA, Science and
Technology Group members from the sponsoring program offices, and Strategic Group
members from the Office of the Director.
Major Facilities Working Group
The Major Facilities Working Group (MFWG) assures the uniform and effective programmatic
oversight of major and mid-scale research infrastructure of the National Science Foundation
throughout their entire life cycles.
The MFWG is chaired by the Chief Officer for Research Facilities (CORF). The MFWG meets
approximately monthly and at other times as required and carries out the following duties:
•
Provide input to the Facilities Governance Board regarding all strategy, governance, and
implementation issues under consideration by that Board.
•
Establish and maintain a list of NSF’s major research infrastructure at all life cycle stages,
development through divestment, and the major upcoming decision points for those
facilities.
Section Revision:
July 26, 2021
2.1.6-18
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Support the Head of the Large Facilities Office in reviewing the Major Facilities Guide
(MFG), Standard Operating Guidance (SOG), and Standard Operating Procedures for NSF
facility oversight, and advise the Facilities Governance Board on the sufficiency and
appropriateness of these documents.
•
Provide concurrence on a bi-monthly report produced by the Large Facilities Office
summarizing the status of all major research infrastructure facilities in their Operation
and Divestment Stages.
•
Provide input, as appropriate, for the Large Facilities Office bi-monthly report
summarizing the status of major facilities and related projects at stages ranging from
development through the completion of construction.
•
Maintain situational awareness of each relevant major research infrastructure in their
home directorate and communicate important information via the CORF and the
cognizant Assistant Directors in a timely way.
•
Share good practices for the proper oversight of major research infrastructure, and work
with the cognizant Assistant Directors to implement good practices across their
directorates.
The MFWG membership consists of the following members:
•
•
•
•
•
•
•
•
Chief Officer for Research Facilities (Chair);
Head, Large Facilities Office (Vice-Chair);
Accountable Directorate Representative (ADR), Directorate for Mathematical and
Physical Sciences (MPS);
Accountable Directorate Representative, Directorate for Geosciences (GEO);
Accountable Directorate Representative, Directorate for Biological Science (BIO);
Accountable Directorate Representative, Directorate for Computer and Information
Science and Engineering (CISE);
Accountable Directorate Representative, Directorate for Engineering (ENG), and
Executive Secretary
Advisory Committee of the Sponsoring Organization
The Advisory Committee of the Sponsoring Organization provides input to the NSF AD, or Office
Head of the Sponsoring Organization concerning priorities among and between projects and
other activities sponsored by the Directorate. The NSF Director requires the endorsement of
the Advisory Committee of the Sponsoring Organization prior to requesting NSB action
authorizing a project’s inclusion (at the Director’s discretion) in a future NSF budget request to
Congress.
Section Revision:
July 26, 2021
2.1.6-19
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.1.6.4 Governing Bodies
Facilities Readiness Panel
The Facilities Readiness Panel (FRP) advises the Director on Recipient and Programmatic
readiness to advance major facilities projects within the formal Design Stage as described in
NSF’s Major Facilities Guide (MFG); this includes the transition from Final Design to
Construction. Decisions on readiness to enter the Design Stage and whether or not to include in
a project in a budget request are made separately. Projects include the major multi-user
research facilities as defined in the Section 1.4.3.1. Members of the FRP include:
•
Chief Officer for Research Facilities (Chair);
•
Head, Large Facilities Office, HLFO;
•
Head, Office of General Counsel (or Designee);
•
Division Director, Division of Acquisition & Cooperative Support (or Designee); and
•
At least 4 senior Program Officers, Section Heads, Deputy Division Directors or Division
Directors (at least 3 from MPS, GEO, BIO, CISE, or ENG)
•
Executive Secretary
The primary duties include:
•
Assess overall project readiness to advance based on the requirements and guidelines in
the MFG and other internal NSF policies and procedures. This includes technical
readiness of the project itself, business system and management readiness of the
Recipient, and programmatic readiness with regard to adequate oversight.
•
Assess whether or not agency risks, including significant project risks managed by the
Recipient that may impact the agency, have been identified and properly considered by
the Sponsoring Organization in developing the Internal Management Plan.
Facilities Governance Board
The Facilities Governance Board (FGB) oversees and makes recommendations on all aspects of
governance of major multi-user research facilities and mid-scale research infrastructure of the
National Science Foundation.
Members of the Board are:
•
Chief Officer for Research Facilities (Chair);
•
Assistant Directors for MPS, GEO, BIO, CISE, ENG;
•
Chief Financial Officer; and
•
Executive Secretary
Section Revision:
July 26, 2021
2.1.6-20
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The primary duties include:
•
Advise the Director on all aspects of strategy, governance, and implementation of major
multi-user research facilities and mid-scale research infrastructure.
•
Approve the NSF Major Facilities Guide (MFG) and all agency-wide Standard Operating
Guidance (SOG) and supporting Standard Operating Procedures for implementation of
facilities oversight, with input from the Major Facilities Working Group.
•
Maintain situational awareness for major multi-user research facilities and mid-scale
research infrastructure at all life cycle stages, from development through divestment,
and advise the Chief Officer for Research Facilities on oversight issues.
•
Recommend to Director on renewal, competition, or divestment of major multi-user
research facilities, based on the Guidelines for Competition of Major Research Facilities
and subsequent Standard Operating Guidance.
Director’s Review Board
The purpose of the Director’s Review Board (DRB) is to assure the Director that all
recommendations and proposed action items have undergone thorough review, assessment
and discussion. The DRB reviews proposed actions for adequacy of review and documentation
and for consonance with Foundation policies, procedures and strategies. The DRB also brings to
the Director’s attention any policy issues that have been identified.
The DRB is the Director’s forum for reviewing timely recommendations to the NSB on a variety
of critical NSF awards, actions, and information items, including those related to major facilities.
The DRB reviews for responsiveness to questions that may be raised by the NSB.
Members of the DRB may include:
•
•
•
•
•
•
Chairperson (NSF Deputy Director or other);
Three ADs, serving on a rotating basis;
Chief Financial Officer;
Staff Advisor, OD;
Executive Secretary, DRB; and
Such other persons as the Director may designate (i.e., OGCs, Legislative and Public
Affairs, etc.).
Joint meetings between the FRP and DRB may be scheduled as the particular situation warrants
but keeping in mind their distinct roles and responsibilities as described above.
NSF Director
The NSF Director has ultimate responsibility for the approval of the obligation of funds from the
MREFC Account and for proposing new MREFC projects to the NSB, OMB and Congress. The
Director approves all materials submitted to the NSB, OMB or Congress.
Section Revision:
July 26, 2021
2.1.6-21
Major Facilities Guide: NSF 21-107 (July 2021)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
National Science Board
The National Science Board (NSB) establishes policy, reviews, and authorizes Construction Stage
budgets, and reviews and authorizes specific large awards for funding, including major facility
projects. 1 NSB is an independent body established by Congress in 1950 to set policies for NSF.
Along with the Director, the NSB oversees NSF and establishes NSF policies within the
framework of applicable national priorities set forth by the President and the Congress. In this
capacity, the NSB identifies issues that are critical to NSF’s future, authorizes NSF’s strategic
directions, annual budget requests, new major programs, and awards, and provides guidance
on the balance between initiatives, infrastructure investments and core programs. 2
NSB has established processes for reviewing and authorizes recommended actions and funding
requests from NSF regarding major facility projects.3 The NSB performs certain reviews,
including a periodic review of facilities, and prioritizes projects, as necessary. NSB involvement
at each life cycle stage includes:
•
Setting Board-level policies and procedures for overseeing all life cycle stages of NSF’s
major facilities;
•
Being kept apprised of the status of all major facilities funded by NSF through oral and
written information items, particularly projects in the Design and Construction Stages.
•
Authorizes advancement through certain design phases;
•
Authorizes inclusion of a candidate project in a future NSF Budget Request to Congress,
after a PDR and NSF Director approval;
•
If necessary, recommend priorities for construction start among projects;
•
Authorizes the Director to obligate appropriated construction funding to the Recipient;
•
Authorizes award of funds to operate major facilities if above the NSB threshold; and
•
Authorizes competition strategies, divestment, or major reorganization for operations
awards if above the NSB threshold.
NSF policy requires the following items to be submitted to the NSB for authorization: (1) Large Awards. Proposed awards
where the average annual award amount is 1% or more of the awarding Directorate or Office's prior year current plan
(including any funds transferred from other Federal agencies to be awarded through NSF funding actions); (2) Major
Construction Projects. NSB authorization is required when the resulting cost is expected to exceed the percentage threshold for
NSB award authorization; (3) Awards Involving Policy Issues or Unusual Sensitivity. NSB interests may include the establishment
of new centers, institutes, or facilities that have the potential for rapid growth in funding or special budgetary initiatives. (Note:
In determining whether anticipated future commitments beyond an initial award amount for any award instrument meet or
exceed the threshold for NSB authorization, every additional anticipated funding action should be added to the initial award
amount. Awards should be submitted for NSB authorization under this criterion as soon as Program staff anticipate that the
total ultimately to be committed is likely to exceed the threshold established for their Directorate or Office.)
1
2
More about the NSB is available online at https://www.nsf.gov/nsb/
See NSB’s meeting minutes with “Annual Timeline for Integration of Board MREFC Process with NSF Budget Process” (NSB-1066, approved August 2010).
3
Section Revision:
July 26, 2021
2.1.6-22
Major Facilities Guide: NSF 21-107 (July 2021)
2.2.1 Development Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.2
DEVELOPMENT STAGE
2.2.1 Initiation of a Potential Major Facility Project
As in all NSF endeavors, inquiry begins with the research communities, whose members alert
NSF program staff to the most promising and exciting questions and the most important
equipment needed to explore them. NSF POs are attentive to the emergence of breakthrough
concepts and actively encourage discussion and planning. In addition, NSF uses National
Academies’ studies, community workshop reports, professional society activities, Directorate
advisory committees and many other methods to identify opportunities and ensure continuous
community input.
If a Sponsoring Organization intends to eventually propose a project for formal Design, then
there should be sufficient investment during the Development Stage by the Sponsoring
Organization so that the project is reasonably well defined and/or described in preparation for
the more formal Design Stage. Ideas and opportunities identified by the research communities
typically have a 5- to 20-year forward look and are brought to NSF in the form of a submitted
proposal requesting funding for development.
Program management and staff are fully responsible for the lifecycle management of proposals
during the Development Stage, including conducting NSF merit review, recommending, and
decision-making. Recommending and decision-making take into account many factors including
disciplinary trends and identified community priorities, transformative opportunities to
advance science, portfolio balances, directorate and NSF priorities, and available funds.
Section Revision:
December 15, 2020
2.2.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.2.2 Exit from Development to Design Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.2.2 Exit from Development to Design Stage
Regardless of whether the project enters at the beginning of the Conceptual Design Phase or a
more advanced phase of technical readiness, formal start of the Design Stage occurs following a
recommendation by the Chief Officer for Research Facilities (CORF) with input from the
Facilities Governance Board (FGB) and other senior agency officials and written approval by the
NSF Director. This process is initiated by a request from the Sponsoring Organization to the
Director’s Office once a project is determined to be ready. Generally, such a request is made
when the Sponsoring Organization has determined that: (1) the project is a high scientific
priority, (2) the project is eligible for MREFC funding (see criteria in Section 2.1.2 of this Guide)
and the MREFC funding route is preferred, and (3) the Sponsoring Organization is committed to
begin explicit investment in more detailed design activities in the current or upcoming budget
cycle using Directorate or Divisional funding (R&RA).
The CORF’s recommendation (as Chair of the FGB) will focus on providing the Director with
answers to the following questions:
Science
•
Is there a compelling science case, and are the project’s goals well-articulated?
•
Does the project fit within the NSF “mission” and within the strategic plans of the
Sponsoring Organization and their associated research communities? See Appendix A of
this Guide for the first ranking criteria, Scientific and Technical, for prioritizing major
facility projects.
Planning
•
Is the Sponsoring Organization’s plan for stewardship of the Design Stage consistent
with the guidelines set out in the Major Facilities Guide?
•
Does the preliminary timeline for development and implementation include
programmatic, NSB, budget and any necessary partnering milestones, including explicit
project off ramps?
•
Are potential opportunities for internal and or external partnering being considered, if
not already underway?
•
Are there any other major challenges regarding this project that the Director needs to
be aware of?
Based on the CORF’s recommendation and any further examination, the Director then approves
(or disapproves) the project entering the Design Stage as a “candidate” project. Note that no
NSF commitment is implied beyond the Design Phase approved at entrance to the Design Stage.
The CORF or Director might alternatively advise the Sponsoring Organization to look further
into any issues and then return for further consideration.
Section Revision:
December 15, 2020
2.2.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3
DESIGN STAGE – CONCEPTUAL, PRELIMINARY, AND FINAL DESIGN PHASES
2.3.1 Conceptual Design Phase
2.3.1.1 Introduction – Conceptual Design Phase
The goal of this first phase of the major facility Design Stage is the creation of a comprehensive
conceptual design that clearly articulates project elements that NSF will consider, including:
•
Description of the research infrastructure and technical requirements needed to meet
the science, including a definition and relative prioritization of the research objectives
and science questions the proposed facility will address. Technical requirements must
flow down from the science requirements. This description may be site-independent or
site-specific depending on the nature of the project;
•
System-level design, including definition of all functional requirements and major
systems;
•
Concept of operations including an estimate of annual operations and maintenance
costs, staffing levels, and other activities
•
Initial risk analysis and mitigation strategy for construction, identifying enabling
technologies, high-risk or long-lead items, and research and development (R&D) needed
to reduce project risk to acceptable levels;
•
Initial acquisition plans, addressing unique project specific considerations, risks and
uncertainties, such as strategies for evolving technologies or R&D and design efforts
that continue in the Construction Stage.
•
Potential environmental and safety impacts to be considered in site selection (see
“Compliance with Environmental, Cultural and Historical Statues,” at the end of this
section);
•
Description of the proposed construction project definition (scope of work, budget and
schedule) needed to evaluate readiness and continue planning in preparation for the
Preliminary Design Phase. This includes budget and contingency estimates appropriate
to a Conceptual Design 1 level that are based on the initial Risk Analysis and initial
projections for the construction and commissioning schedule;
•
Description of proposed Educational Outreach and Broader Societal Impact, included in
the proposed scope of work, budget and schedule.
Many of these details are included as part of the PEP as described in greater detail in following
sections and in Section 3.4. This Phase may take several years depending on development
activities.
The budget information should be provided using a Work Breakdown Structure (WBS) format, identifying the basis for
estimates and including a WBS dictionary that defines the scope associated with each WBS element. Contingency estimates
should include an explanation of the methodology used to calculate the estimate.
1
Section Revision:
December 15, 2020
2.3.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3.1.2 Conceptual Design Phase Activities
During the Conceptual Design Phase there may be a number of coordinated and complimentary
activities taking place with the various entities involved: (1) community activities, (2) NSF staff
activities, and (3) funding considerations.
(1) Community Activities. Proponents of a project should provide NSF with an early concept
proposal that makes a compelling case for the research that would necessitate development of
a facility, and that describes, in general terms, its essential characteristics if the proposal is
unsolicited. Generally speaking, major facility projects are solicited. In that case, the proposal
must respond to all NSF and programmatic requirements which generally include references to
the Major Facilities Guide if it is already known as a major facility project. These initial proposals
identify what is known at that point in project development, as well as what tasks remain to be
accomplished in order for NSF to consider a project for eventual funding. In the near term, they
also define what work should be done to develop the project to the Conceptual Design level of
maturity.
An NSF PO 1 will be assigned to be the primary point of contact with the Principal Investigator
(PI) and/or Project Manager. The NSF PO conducts a merit and technical/programmatic review
of the proponents’ proposal, and either recommends or declines the request for funding. If
funded, the PO will work with their Directorate and/or Division to organize an Integrated
Project Team to provide coordination on project oversight and assurance.
Proponents should acquaint themselves with NSF’s expectations for the essential elements of a
construction-ready PEP as described in Section 3. Proponents should also develop a skeletal
plan that will result in the future definition of each of these elements, should NSF encourage
further pre-construction planning. The plan should address, even if only in the most cursory
way, each of the essential elements that should be realized in a formal construction-ready PEP.
For example, proponents may wish to develop a “straw man” PEP that contains sections labeled
using each of the entries in Section 3.4, with as much supporting information provided based
on the outputs from the Development Stage (if any) and/or the requirements in the solicitation.
This serves to illustrate an understanding to all parties of the range and magnitude of the tasks
ahead.
Administratively, the Program Officer (PO) is part of a Directorate or Office that provides supervisory oversight and the
budgetary authority to fund PO actions. Actions of the PO described here and in subsequent life cycle stages of facility
development implicitly recognize the authority of the individuals within this supervisory structure to appropriately guide, direct,
and approve the actions of the PO. In particular, when the phrase “PO concurrence” is used in the following text, this assumes
concurrence at whatever management level the AD or Office Head has required. Refer to Section 2.1.6 for a brief description of
the duties of the PO, AD, and others referred to in the Major Facilities Guide.
1
Section Revision:
December 15, 2020
2.3.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
(2) NSF Staff Activities: In response to the development of an early version of a PEP, the PO,
with the advice of the IPT, develops an Internal Management Plan (IMP). 1
This internal document specifies how NSF will conduct its oversight and assurance of the
project, and provides budgetary estimates for developing, constructing and operating the
facility. It also identifies critical issues and risks facing the project (for example: project
management issues, completion of essential R&D activities, partnership agreements, award
closeout or divestment liabilities) and lays out a strategy for financing these activities.
The PO develops the IMP with advice and assistance from the IPT, following internal operating
guidance. The IMP is approved by the cognizant NSF AD after review and approval within the
Sponsoring Organization. The IMP describes the plan for NSF management and funding of the
project to CDR, proposes transitional steps to be taken if the project is admitted to the
Preliminary Design Phase, and lays out NSF’s plan to oversee development of the project
including internal and external review. Each major facility project undertaken by NSF has
unique characteristics. Accordingly, the IMP should be adapted to meet the specific needs of a
particular project. The IMP should state the justification for pursuing alternatives to the
guidelines contained in the Major Facilities Guide.
3) Funding Considerations. During the Conceptual Design Phase, NSF and/or other institutions
and agencies begin to invest research and development funds in design development, and in
efforts that promote community building and planning. Investment in fundamental research
activities, community building, and initial planning activities may occur over many years, and
some are recognized as having contributed to the conceptual design effort only in retrospect. 2
The cumulative pre-construction investment in research, planning and development that occurs
during the Conceptual Design, Preliminary Design, and Final Design Phases may range from five
to 25 percent of total construction cost, depending on the complexity of the project, and
typically amounts to about 10 percent of the construction cost. The technology needed to
construct a facility may be uncertain, unproven or immature, requiring substantial development
over a period of years.
NSF may decide to fund additional planning and development efforts for particular projects
depending on the outcomes of the review and whether or not the Conceptual Design Phase was
funded. 3 Such activities might include workshops in one or more disciplines, National
1
Questions about Internal Management Plans should be addressed to the relevant PO.”
2 Some projects come to NSF very well developed, requiring little in the way of conceptual design phase support. They are
subjected to the same rigorous scrutiny, however, as they are developed by the responsible NSF Directorates or Offices.
3 Relevant program solicitations may be released to announce funding opportunities for these planning and development
efforts.
Section Revision:
December 15, 2020
2.3.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Academies’ studies, and research projects related to the development of new technologies. 1
These activities might be funded as part of the Conceptual Design Phase award, or through a
separate proposal submission.
2.3.1.3 Conceptual Design Review (CDR)
The Conceptual Design Phase is complete when a package containing the Conceptual Design
and funding request leading to a Preliminary Design is received, reviewed, negotiated and
approved for funding. The funding request for the Preliminary Design Phase will generally be
submitted as a supplemental request to the original award. Review of the package associated
with that request is described below.
The package should include the refined PEP and any additional information required by
Program to assess the project readiness and management to-date. Components of the PEP are
given in Section 3.4.
NSF will subject the Conceptual Design package to external review, applying standard NSF
criteria (Intellectual Merit and Broader Impacts) as well as other programmatic and technical
criteria as given in the original solicitation and the panel charge.
The review is organized and conducted by the PO in consultation with the LFO Liaison and
G/AO. The PO has overall responsibility for organizing the review, and throughout the review
process acts as the interface between the NSF and the Recipient. The PO authors the review
charge and organizes the review panel. The LFO Liaison and G/AO strengthen the review
process by specifying language for incorporation within the charge and for aspects of the
review agenda pertaining to project management issues and recommending panelists able to
advise NSF in non-science related areas of the review. The PO, LFO Liaison, and G/AO concur on
the implementation of these recommendations. Following the review, the PO and the LFO
Liaison will each independently assess the review, confer on areas of concern, share their
views, and report their observations through their respective supervisory chains – the PO via
the administrative structure of the Sponsoring Organization and the LFO Liaison via the HLFO.
At this point, the conceptual design baseline is likely to have significant uncertainties. Cost
estimates at CDR are generally parametric in nature. Contingency estimates, representing work
scope not yet defined but nevertheless essential to the completion of the project, will be a
significant fraction of the total project budget estimate. Significant unknowns and uncertainties
often remain to be addressed in more advanced phases of planning and development. The
conceptual design, system requirements, supporting budget estimates, risk analysis, and
forecasts of interagency and international partnerships should be detailed enough for NSF
NSF encourages disciplinary and interdisciplinary science planning by all of the research communities that NSF supports. In
particular, NSF encourages formal planning in fields in which scientists and engineers have traditionally not been organized to
identify major facility projects needed for breakthrough advances.
1
Section Revision:
December 15, 2020
2.3.1-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
program officials to decide whether the project concept warrants further funding for
development.
In conjunction with the CDR, the initial high-level NSF Cost Analysis will be conducted following
NSF internal operating guidance. See also Figure 4.2.1-1 of this Guide. Guidance on refinements
to the Recipient’s Cost Book will be provided as necessary in preparation for the Preliminary
Design Phase. DACS will also conduct the necessary cost analysis of the Preliminary Design
Phase proposal supported by information contained in Section 3.0 of the PEP.
2.3.1.4 Exit from the Conceptual Design Phase
Formal exit from the Conceptual Design Phase entails the following NSF actions:
1. Successful completion of the CDR as described above,
2. Recommendation for advancement by the sponsoring Directorate,
3. Facilities Readiness Panel Review and Recommendation,
4. Approval for advancement to the Preliminary Design Phase by the Director, and
5. Award to support the Preliminary Design Phase.
Recommendation for Advancement by the Sponsoring Directorate
The AD relies on community inputs, discipline-specific studies, advisory committee
recommendations and internal NSF considerations to prioritize the opportunities represented
by the project relative to competing opportunities and demands for resources. If, in the
judgment of the AD, the scientific merit and relative importance of the proposed facility are
sufficiently strong to justify advancement of the project into the Preliminary Design Phase, the
AD will submit a memorandum to the Facilities Readiness Panel recommending the project for
support, that explains how it meets the requirements for construction funding and how it
satisfies the following criteria:
•
The project’s science (research) program addresses one or more science objectives,
clearly demonstrating a compelling need for the project;
•
The project has been reviewed by the research community and by NSF, in consultation
with Directorate Advisory Committees, and has been assigned a high priority; 1 and
•
The project’s CDR indicates that: (1) the engineering design and construction plans are
appropriately defined at the conceptual design level of project maturity and that the
management plans and budget estimates for further planning and development, as well
as constructing and operating the facility are reasonable; (2) the sponsoring Directorate
endorses the IMP and Project Development Plan 2 (PDP) for further development to the
Preliminary Design Phase; (3) the technology to create the facility exists or can exist
shortly, and can be used without excessive risk; (4) other risks to development are
Evaluation by NSF includes external merit review, using the NSF merit review criteria and the 1st ranking Criteria and
evaluation by the Facilities Readiness Panel, using the 2nd ranking Criteria in Appendix A.
1
The Project Development Plan is part of the PEP, providing the plan to develop the project design and definition to readiness
for construction. See Section 3.4 for details.
2
Section Revision:
December 15, 2020
2.3.1-5
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
satisfactorily defined and minimized or otherwise addressed in the IMP, and (5) there
are no better alternatives to the facility (i.e., with a better mix of cost and quality) that
would address the science objectives in a timely manner.
Supporting documentation, including the approved IMP, relevant review evaluations, and any
other supporting information should accompany this memorandum. All materials are
transmitted to the Facilities Readiness Panel (FRP) by the AD or Office Head of the Sponsoring
Organization.
Approval by the Director
The Director evaluates the FRP recommendation and, if satisfied, approves advancement to the
Preliminary Design Phase. The NSF Director may elect to consult the National Science Board
prior to acting on a recommendation. If a project is not approved to advance to the Preliminary
Design Phase, the Director may remand the project to the Conceptual Design Phase for further
work or terminate the project.
More information about the role of the NSB in selecting and prioritizing major facility projects is
available in Section 2.1.6 on Roles and Responsibilities.
Section Revision:
December 15, 2020
2.3.1-6
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3.2 Preliminary Design Phase
2.3.2.1 Introduction – Preliminary Design Phase
The Preliminary Design Phase further develops concepts to a level of maturity in which there
are: a fully elaborated definition of the motivating research questions; a clearly defined sitespecific scope (except mobile platforms/facilities); a PEP and an IMP that address major
anticipated risks in the completion of design and development activities and in the undertaking
of construction; and a realistic (not too low) cost estimate based on known risks that can be
presented with reasonable confidence 1 to the NSF Director, NSB, the Office of Management
and Budget (OMB), and Congress for consideration
“Off-ramps”
for inclusion in a future NSF budget request.
Projects may be removed from the
Preliminary Design Phase by the NSF
To satisfy these requirements, the project is
2
Director due to:
developed to a preliminary design level of maturity.
Results of this development are reflected in a
• Insufficient priority over the long term;
revised and updated PEP. 3 Components of the
• Failure to satisfy milestones or other
updated PEP that deserve particular emphasis at this
criteria defined in the IMP/PEP;
Phase include:
• Eclipse by other projects;
•
Update of the Project Development Plan (PEP
Section 3.0) and timeline, with major
anticipated risks in the completion of design
and development activities to inform the
Final Design Phase award;
•
•
•
•
•
Collapse of major external agreements;
Extensive estimated or actual cost
increases;
Significant changes in schedule for design
readiness or eventual construction;
Unexpected technical challenges;
Changes in the research community that
indicate eroding support for the project;
Any other reason that the Director deems
sufficiently well-founded.
•
Refinement of the research objectives and
priorities of the proposed facility;
•
Update of the description of the required
infrastructure, site-specific design, and
definition of interconnections of all major
subsystems;
•
Environmental Assessments or Environmental Impact Statement (if applicable);
•
Bottom-up budget and contingency estimates for construction, presented using a Work
Breakdown Structure (WBS) structure and supported by a WBS dictionary defining the
scope of individual elements;
•
For guidance on contingency planning refer to Section 4.2.5 of this Guide. Confidence levels must be in the 70-90% range
following PDR depending on the nature of the project.
1
NSF utilizes the conventional definition of preliminary design as used by project managers – a site-specific design defining all
major subsystems and their interconnections, a level of design completeness that allows final construction drawings to
proceed, cost estimation based on construction bidding, and bottom-up estimates of cost and contingency. Preliminary design
usually has a specific meaning within a particular industry or discipline, and NSF adopts the definition most appropriate to each
particular project, as defined in the Project Development Plan part of the PEP.
2
3
See Section 3.4 for a description of the PEP.
Section Revision:
December 15, 2020
2.3.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Scope management plans that include de-scoping options and scope opportunities that
can be implemented depending upon available contingency levels.
•
Updated construction schedule with contingency estimate;
•
Updated Educational Outreach and Broader Societal Impact plan that includes the scope
of work, required budget and schedule to implement the plan, plus the budget and
schedule needed to develop the plan from preliminary design to final design;
•
Implementation of a Project Management Control System (PMCS) 1 and inclusion within
the preliminary design of a resource-loaded schedule;
•
Updated risk analysis, including regulatory issues affecting construction or operation,
and time-dependent factors such as inflation indices, price volatility of commodities,
etc. (The preliminary design budget estimate will be the basis for a future NSF budget
request to Congress if the project successfully emerges from the Preliminary Design
Phase. Costs and risks should be projected forward to the anticipated award date for
construction funds.)
•
Demonstration that key technologies are feasible and can be industrialized if required;
•
Updated acquisition plans and timeline, including clear milestones, justification, and risk
management considerations for transition from R&D to design and procurement.
Updated strategies for evolving technologies.
Plans for management of the project during construction, including preliminary
partnership arrangements and international participation, oversight of major subawards
and contracts, organizational structure, and management of change control; 2 and
•
•
Updated estimates for future operating costs, anticipated future upgrades, or possible
decommissioning costs of the facility at the end of its operating life.
2.3.2.2 Preliminary Design Phase Activities
During the Preliminary Design Phase, the earlier conceptual design evolves into a more mature
plan with respect to the baseline and contingency definitions. The WBS elements and resource
estimates benefit from additional knowledge and planning. Consequently, budget uncertainty
for projected construction is much reduced relative to the earlier conceptual design. At the end
of the Preliminary Design Phase, the resulting total project cost is used to inform the budget
request to Congress. Typically, a significant proportion (often one-third or more) of the total
pre-construction planning budget is expended achieving the preliminary baseline estimate.
The PMCS involves both the software tools for development of the project databases and the processes and procedures
needed to organize and manage the project; schedule and optimize project resources; compute and track Earned Value and
evaluate project risk factors; and manage the change process by evaluating the effects of alterations to the baseline on the
project’s planned budget and schedule. See Figure 4.2.2-1 for examples of project controls systems inputs and outputs.
1
These plans are a preliminary, but relatively mature version of the Project Execution Plan that defines how the project will
conduct itself during the Construction Stage – see Section 3.4.
2
Section Revision:
December 15, 2020
2.3.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Interim reviews 1 during the Preliminary Design Phase will be conducted by NSF as described in
the IMP. This Phase culminates in a Preliminary Design Review (PDR), conducted by NSF, to
ensure that all aspects of the project definition and planning are robust. The results of the PDR
are evaluated by the Facilities Readiness Panel, followed by a recommendation from the CORF
to the Director for decision on forwarding to the NSB for possible inclusion in a future budget
request.
2.3.2.3 Preliminary Design Review (PDR)
NSF conducts a PDR, organized, and led by the PO, to assess the robustness of the technical
design and completeness of the budget and construction planning. Like CDR, the review is
organized and conducted by the PO in consultation with the LFO Liaison and G/AO. The PO has
overall responsibility for organizing the review, and throughout the review process acts as the
interface between the NSF and the Recipient. The PO authors the review charge and organizes
the review panel. The LFO Liaison and G/AO strengthen the review process by specifying
language for incorporation within the charge and for aspects of the review agenda pertaining to
project management issues and recommending panelists able to advise NSF in non-science
related areas of the review. Following the review, the PO and the LFO Liaison will each
independently assess the review, confer on areas of concern, share their views, and report their
observations through their respective supervisory chains – the PO via the administrative
structure of the Sponsoring Organization and the LFO Liaison via the HLFO.
The review scrutinizes the effectiveness of project management through this phase of design,
as well as plans for completion of final design and eventual construction and operation. The
PDR may utilize, as appropriate, external experts, consultants, and outside firms to evaluate
proposed plans and budgets as described in the Project Execution Plan (PEP). The PDR also
examines the management structure and credentials of key staff to assure NSF that an
appropriately skilled management organization is ready to complete final design activities and
execute the construction phase of the project.
Once the project has satisfied any recommendations made by NSF as a result of external
review, and resolved any outstanding issues, the Directorate recommends to the Facilities
Readiness Panel that the project is ready for advancement to the Final Design Phase of
development and is a candidate for NSB authorization for inclusion in a future NSF budget
request for construction funding. At any time, the Facilities Readiness Panel or the OD may
request further external review.
Following the PDR, the PO updates the IMP to describe proposed plans for budgeting and
oversight, and to finalize commitments from interagency and international partners during final
Interim reviews are typically held semi-annually. Exceptions to this, dictated by the needs of a particular project, may be
justified and will be listed in the award terms and conditions.
1
Section Revision:
December 15, 2020
2.3.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
design. The PO directs the Recipient to update the Project Development Plan (PEP 3.0) to lay
out the work scope, budget and schedule necessary to bring the project to Final Design.
In conjunction with the PDR, the second, more detailed NSF Cost Analysis will be conducted to
support the budget request to Congress. The Cost Analysis will be conducted following NSF
internal operating guidance. See also Figure 4.2.1-1 of this Guide. Guidance to the Recipient on
refinements to the Cost Book will be provided as necessary in preparation for the Final Design
Phase. DACS will also conduct the necessary cost analysis of the Final Design Phase proposal
supported by information contained in Section 3.0 of the PEP.
The completion of Final Design should be aligned with the expected time-scale for requesting
and appropriating construction funds. The NSF Budget Division and the CORF are the
coordinators for this critical planning activity, bringing projects forward for construction only if
OMB and Congress are likely to approve the request and appropriation of funds within the time
period in which the Preliminary Design plans and cost estimate remain valid.
2.3.2.4 Exit from Preliminary Design Phase
A candidate project exits from the Preliminary Design Phase and enters the Final Design Phase
after the following have been completed:
1. A successful PDR and subsequent support from the Directorate,
2. A review and recommendation by the Facilities Readiness Panel for advancement to the
Final Design Phase, followed by the Director's approval to advance to the Final Design
Phase,
3. Award to support the Final Design Phase.
Exit from the Preliminary Design Phase may also include:
1. The DRB reviews and recommends to the NSF Director inclusion in a future budget
request,
2. The NSF Director recommends to the NSB inclusion of the project in a future budget
request, and
3. The NSB authorizes inclusion in a future construction budget request.
Generally speaking, the request for inclusion in a future budget request is associated with
advancement to the Final Design Phase. However, advancement to the Final Design Phase may
be granted without proceeding with a budget request based on strategic considerations.
2.3.2.5 NSF Director’s Recommendation for Advancement to Final Design
The Facilities Readiness Panel and the Director should first be satisfied that the following
conditions have been met before making a recommendation to the NSB for authorization:
Section Revision:
December 15, 2020
2.3.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
The AD of the sponsoring Directorate continues to support the high scientific merit and
importance of the project and has a sound financial plan for supporting the remaining
pre-construction planning activities and the future operations and use of the facility.
•
The Preliminary Design Phase PEP has been successfully reviewed by an external panel
of experts in order to obtain the best possible objective advice from authorities in the
fields and disciplines utilized by the project.
•
An appropriate Integrated Project Team (IPT) is in place and has provided assurance that
the Preliminary Design Phase total project cost has been satisfactorily analyzed at a high
degree of confidence to support the budget request.
•
An updated Internal Management Plan (IMP) has been refined by the core members of
the IPT and approved by the Sponsoring Organization.
The Facilities Readiness Panel concurs that the Preliminary Design Phase PEP and IMP
are reasonable and pose an acceptable level of technical/programmatic risk, and that
anticipated costs for construction and operation are sufficiently well known.
•
•
The NSF Director is satisfied that external participation in all phases of the project (other
agencies, international and/or private sector entities, etc.) is well planned.
2.3.2.6 National Science Board Authorization following PDR
Usually, the final steps for exit from the Preliminary Design Phase are review and authorization
by the NSB for inclusion in a future budget request to Congress, which necessitates going
through the Office of Management and Budget. A strategic assessment of a project’s priority
relative to other opportunities is made before NSF considers a request to NSB for inclusion in a
future budget request. Projects are evaluated by applying the second and third ranking criteria
in Appendix A of this Guide.
The Sponsoring Organization is responsible for preparing the documentation needed for the
NSB to review and authorize a proposed major facility project for advancement to Final Design
Phase and inclusion in a future budget request. 1 Prior to NSB submission, the Director’s Review
Board (DRB) 2 reviews the completeness and appropriateness of the documentation supporting
advancement of the project (such as prior phase reviews, committee evaluations, PEP
evaluation and reviewed proposal ratings) to ensure adherence to NSF processes and policies.
As NSB considers projects for advancement to Final Design Phase, NSF makes available to the
NSB, upon request, the PEP and IMP, and the reviews from the community, the Head of LFO,
the Facilities Readiness Panel and other relevant parties. The NSB considers the following
elements, applying the second and third set of ranking criteria identified in Appendix A, as
appropriate:
•
The priority of the project in advancing NSF’s strategic goals;
1
NSF Proposal and Award Manual (PAM) and internal operating guidance documents provide guidance on the DRB package.
2
See Section 2 and Section 2.1.6 on Roles and Responsibilities.
Section Revision:
December 15, 2020
2.3.2-5
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
•
•
•
The research and science enabled by the proposed facility;
Construction readiness and risks to the agency;
Budget justification for construction and operation of the facility; and
The likelihood that funding will be available in the next few years.
As with all NSF proposals, the quality of the Intellectual Merit and Broader Impact activities,
including educational outreach, plays an important role in funding decisions. If NSB authorizes a
project for inclusion in a future budget request, it specifies its priority among all projects in the
approved stage. 1 If a project is not already authorized for the Construction Stage, or if a
project’s plans are no longer deemed to be clearly and fully construction-ready, NSB may
recommend to the Director that the project be remanded to the Preliminary Design Phase or
advanced to the Final Design Phase for further work, or that the project be terminated.
2.3.2.7 Inclusion in an NSF Budget Request
Each year, the NSF Director proposes, in priority order, if necessary, the NSB-authorized
construction-ready projects for the MREFC Account. If an MREFC “new start” is approved for
inclusion in the President’s Budget Request to Congress, then Congress may ask for additional
information through formal hearings and/or informal briefings. Once Congress passes an
appropriations act for NSF and the President signs it into law, NSF may then obligate funds.
1
NSF assigns the very highest priority to projects that are under construction.
Section Revision:
December 15, 2020
2.3.2-6
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3.3 Final Design Phase
2.3.3.1 Introduction – Final Design Phase
The goal of the Final Design Phase is to meet the requirements necessary to advance the
proposed project to the Construction Stage. During this Phase, the Preliminary Design Phase
PEP is refined and may incorporate events, conditions or risks previously unforeseen at the
Preliminary Design Review. Strategic considerations are not generally part of Final Design Phase
since they are considered before inclusion in a future budget request to Congress.
Technical requirements include:
•
To the maximum extent practicable, designs, specifications and work scope that can be
placed for bid to industry;
•
Refined bottom-up cost estimates and contingency estimates;
•
Implementation of a PMCS for project technical and financial status reporting, including
Earned Value Management Systems (EVMS);
•
Completion of recruitment of key staff and control account managers needed to
undertake construction of the project;
•
Industrialization of key technologies needed for construction;
•
Finalization of commitments with interagency and international partners; and
•
Submission to NSF of a PEP 1 for construction.
2.3.3.2 Final Design Review (FDR)
Projects should continue to receive pre-construction funds in order to produce a Final Design,
which includes the following elements, many of which are components of the Construction
Stage PEP:
1
•
A construction-ready design based on the technical nature of the project;
•
Tools and technologies needed to construct the project;
•
Governance of the project, configuration control, EVMS, reporting technical and
financial status, managing sub-Recipients and working with interagency and
international partners;
•
The scope of work captured in detailed WBS format, accompanied by a WBS dictionary
defining the scope of all entries, and a scope management plan including potential
descope options and scope opportunities;
•
Updated budget and schedule, with their respective contingencies, including detailed
risk analysis and methodology, presented in the detailed WBS format;
•
A fully implemented PMCS, including a final version of the resource-loaded schedule and
mechanisms for the project to generate reports – using an NSF verified Earned Value
Refer to Section 3.4 for details of the PEP.
Section Revision:
December 15, 2020
2.3.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Management System (EVMS) 1 – on a monthly basis and use them as a management
tool. Path dependencies, schedule float, and critical path are defined;
•
An updated Educational Outreach and Broader Societal Impact plan (including the scope
of work, budget and schedule) that also includes the capital investment required to
meet the needs of the proposed Educational Outreach and Broader Societal Impact
plan;
•
An appropriate proportion of the budget based on externally provided information such
as vendor estimates or quotes, publicly available supplier prices, and the like;
•
All necessary partnership agreements and MOUs;
•
General arrangements, fit-up and installation details of major components and
commissioning strategy;
•
Plans for Quality Assurance and Safety Management, integrated into the resourceloaded schedule;
•
Updated operating cost estimates;
•
Updated acquisition plans and identification of remaining R&D or design efforts.
The PO is responsible for organizing and leading the FDR. The review is conducted according to
the same standards and with the same respective roles for the PO and LFO Liaison as described
previously for the CDR and PDR. Following the review, the PO and the LFO Liaison will each
independently assess the review, confer on areas of concern, share their views, and report their
observations through their respective supervisory chains.
The scope of the FDR includes assessment of the technical and project-management
components of the proposed project similar to PDR, but at level of readiness suitable for
construction or acquisition based on the PEP. In consultation with the IPT, the IMP should
continue to be assessed periodically by the Program Officer and updated as required to ensure
that the underlying assumptions about the project remain valid.
In the event the project’s construction estimate or funding profile are determined to be
inconsistent with the budget request, NSF will: (1) decrease the scope of the project; (2) justify
the increase to OMB and Congress and request additional funding as part of the budget
process; or (3) cancel the project. If Congress does not appropriate funds as requested, NSF
may conduct annual project status reviews to assure the continued viability of the project’s
plan and budget for construction until such time as funds become available or NSF cancels the
project.
During construction, progress must be tracked and measured using an Earned Value Management System based on EIA-478
criteria (OMB A-11 Capital Programming Guide (2016)). This requirement applies to the award Recipient who acts as a prime
contractor. Secondary contractors to the award Recipient are not required to follow formal EVMS but must be able to provide the
appropriate inputs to the prime for EV reporting.
1
Section Revision:
December 15, 2020
2.3.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.3.3.3 Exit from the Final Design Phase
A candidate project exits from the Final Design Phase and enters the Construction Stage after
the following have been completed:
1. A successful FDR and subsequent support from the Directorate,
2. A review and recommendation by the Facilities Readiness Panel for advancement to the
Construction Stage,
3. A review and recommendation by the DRB for advancement to the Construction Stage,
4. The NSF Director approves advancement and recommends NSB authorization for a
Construction Stage award,
5. The NSB authorizes a Construction Stage award, and
6. NSF makes an award to support the Construction Stage.
2.3.3.4 NSF Director’s Recommendation for Advancement to Construction Stage
The Facilities Readiness Panel and the Director should first be satisfied that the following
conditions have been met before making a recommendation to the NSB for authorization:
•
The AD of the sponsoring Directorate continues to document and support the high
scientific merit and importance of the project and has a sound financial plan for
supporting future operations and use of the facility.
•
The Final Design Phase PEP has been successfully reviewed by an external panel of
experts in order to obtain the best possible objective advice from authorities in the
fields and disciplines utilized by the project.
•
•
Updated Internal Management Plan (IMP) has been approved by the Directorate.
An appropriate Integrated Project Team (IPT) is in place and has provided assurance that
the Final Design Phase total project cost has been satisfactorily analyzed at a high
degree of confidence to support the “No Cost Overrun” policy (NCOP).
•
The Facilities Readiness Panel concurs that the Final Design Phase PEP and IMP are
reasonable and pose an acceptable level of technical/programmatic risk, that
anticipated costs for construction are sufficient to implement the NCOP, and the
updated cost of operations are sufficiently estimated.
•
The NSF Director is satisfied that external participation in construction and operations
(other agencies, international and/or private sector entities, etc.) carries acceptable risk.
2.3.3.5 National Science Board Authorization for Construction
NSB reviews the recommendation and authorizes the NSF Director to obligate funds for the
Construction Stage Award(s). Following the Director’s approval, an award (either a cooperative
agreement or contract) is negotiated between NSF and the Recipient, and construction
activities begin in conformance with the negotiated Performance Measurement Baseline (PMB)
as described in the PEP.
Section Revision:
December 15, 2020
2.3.3-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The NSB authorized Total Project Cost (TPC) following FDR establishes the not-to-exceed cost
under NSF’s “No Cost Overrun” policy. Section 4.2.5.2 describes NSF practices to meet this
policy.
The NSF Cost Analysis conducted in conjunction with the FDR will determine the final
negotiated award amount for the Construction Stage. The analysis will encompass such things
as any bids received that inform the basis of estimate, negotiated subawards and contracts
associated with initiating construction, and negotiation of final indirect cost and labor rates.
The Cost Analysis will be conducted following NSF internal guidelines. See also Figure 4.2.1-1 of
this Guide. If a project is not authorized for a Construction Stage award, or if an approved
project’s plans are no longer deemed to be clearly and fully construction-ready, the NSB may
recommend to the Director that the project be remanded to the Final Design Phase for further
work or that the project be terminated.
Section Revision:
December 15, 2020
2.3.3-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.4
CONSTRUCTION STAGE
2.4.1 Construction Award Management and Oversight
After Congress appropriates funds for the project, NSF proceeds to award the cooperative
agreements (CAs) or contracts for construction, acquisition, and commissioning of the facility.
The policies and procedures in the publicly available NSF Proposal and Awards Policy and
Procedures Guide, apply to major facility projects awarded under a cooperative agreement.
Questions about the internal guidance that covers details of the internal award process from
proposal generation through merit review, DRB and NSB reviews, and final award should be
directed to the PO. The Recipient(s) provides periodic financial and technical status reports to
NSF according to the terms and conditions of the CA or contract. The project is subjected to
periodic post-award reviews that may examine any or all of the following topics: technical
performance, cost, schedule, and management performance. These reviews are typically held
at the facility and are generally conducted annually. More frequent reviews may be scheduled
based on the project’s expenditure rate or due to any other technical or management issues
that arise.
NSF selects the annual review panel members who are typically external experts covering all
aspects of the project, and assess technical progress, cost, schedule, and management
performance. These panels report directly to NSF and provide advice on project direction and
any needed changes. The reviews are organized and conducted by the PO in consultation with
the LFO Liaison and G/AO. The PO has overall responsibility for organizing the review, and
throughout the review process acts as the interface between the NSF and the Recipient. The PO
authors the review charge and organizes the review panel. The LFO Liaison and G/AO
strengthens the review process by specifying language for incorporation within the charge and
for aspects of the review agenda pertaining to project management issues and recommending
panelists able to advise NSF in non-science related areas of the review. (Note: Many projects
invite panels of experts to review and advise on project plans and progress. Such panels report
to the Project Director and are not a substitute for NSF-organized external oversight reviews.)
Because panel recommendations are to NSF and not the Recipient, NSF will generally issue
written guidance to the Recipients for subsequent response and action.
Progress against the project plan is provided through the use of formal Earned Value
Management Systems (EVMS), which provide measurement of ongoing project status, including
deviations or variances from the PMB. The results of the EVM reporting and analysis and any
actions taken, are reported to NSF in the monthly progress report. The Recipient should consult
with the PO and GA/O or CO as necessary on the requirements for the monthly progress
reports. Further information is also provided in Sections 4.2.5.8 and 4.6.2 of this Guide.
Generally, when cost and/or schedule performance begins to deviate from plans, change
control is exercised by the project through a Change Control Board (CCB) 1 action, resulting in
A CCB comprises the senior project managers responsible for defining the project's resource requirements and allocating or
expending those resources. It typically consists of the Project Director, Project Manager, Business Manager, control account
1
Section Revision:
December 15, 2020
2.4.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
modifications to the project’s budget or schedule contingency. It is also normal practice for a
project to update its budget and schedule Estimates to Complete (ETC), which also may result in
PMB changes.
Whenever a project approves a change control action that results in allocating or returning
contingency to the pool of contingency funds, the PMB budget will also change.
Similar change-control actions affect the PMB schedule. They revise the project PMB schedule
and the available schedule contingency or “float” time – that is, the difference between
milestones on the schedule’s critical path and the expected completion dates for activities that
lead to the accomplishment of those milestones.
Modifications to the PMB that are within the defined scope and do not change the total project
duration or Total Project Cost are referred to as “re-planning”. Re-planning may be due to
adjustments or re-organization of the project plan and/or may signify that contingency is being
expended in an expected manner. If the allocations of budget and schedule contingency are
below the budget or schedule thresholds identified in the award instrument (CA or contract
agreement) between NSF and Recipient, the change requests are approved unilaterally by the
project. NSF approval is required when the CCB recommends re-planning change actions that
exceed the budget or schedule thresholds identified in the terms and conditions in the
agreement or contract between NSF and Recipient. Each will have a different threshold for
approval. Approval levels for scope changes are generally outlined in the award instrument.
Minor changes in scope may also fall under re-planning activities. The project maintains a scope
management plan (PEP-4.4) which describes the project process for maintaining control of
scope and outlines scope changes that can be implemented depending upon the project’s
forecast of its ability to complete the project within the approved TPC and duration. The project
can elect to implement minor de-scoping options or to defer scope through the change control
process if it needs to increase the amount of contingency as part of the strategy to prevent
potential cost overruns. It can also elect to implement project enhancements that are within
existing scope of work definitions, following the project change control process and approval
process as set in the award or contract terms and conditions.
It is essential for the project management to respect the project PMB rigorously, maintaining
each adjusted PMB in the project’s database along with the attributed CCB actions. This allows
the project and NSF to systematically track the evolution of the PMB from its initial release
through all subsequent changes.
“Re-baselining” occurs when the changes involve:
1. Increases in the NSB-authorized TPC,
managers of principal work breakdown structure elements, chief scientist and engineer, and systems engineer. It may include
other project staff whose authority pertains to the range of activities considered by the Board.
Section Revision:
December 15, 2020
2.4.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2. An extension beyond the total project duration, and/or
3. Major changes in scope.
When the proposed changes reach the re-baselining level, the approval process involves NSF
and may involve the NSB. Changes in project end date follow NSF’s No-Cost Extension (NCE)
policies. Major changes in scope should be part of the scope management plan and should
follow the project change control process, including NSF approval; if science goals are
significantly impacted, NSB authorization may also be required. An increase in TPC exceeding 20
percent of the NSB-authorized TPC or $10 million (whichever is less) must be reviewed and
authorized by the NSB following a recommendation by the Director. Prior to requesting
approval to re-baseline, a new external baseline review should be conducted to examine the
nature of the problems encountered, and to determine whether de-scoping should be
exercised per the approved scope management plan (PEP-4.4) or, if not, whether the problems
can be solved by use of budget and/or schedule contingency or other means. Upon review and
approval, scope, cost, and schedule are stabilized, and contingency is adjusted to appropriate
levels.
Whenever significant scope, cost, or schedule increases are foreseen, it is most important that
the LFO Liaison is consulted early, concurs with the PO on the details of the Sponsoring
Organization’s plan, and advises and concurs on details of the external re-baselining review.
Similarly, when there are indications that project cost or schedule contingency will fall below
reasonable standards, 1 the PO should discuss plans for dealing with the variance with the
Project Director. This information should be clearly noted in the monthly status report that
goes to the HLFO. The LFO is a resource for helping to deal with such problems and for helping
to identify steps that can be taken to restore adequate contingency.
In addition to supplying regular status reports required in the terms and conditions of the
award instrument, it is essential that project staff inform the PO and/or the G/AO (or CO) in a
timely manner of major issues or significant changes in project status, such as a potential rebaselining, problems with partnerships, or surprising research and development results. NSF
management, the Chief Officer of Research Facilities, and the NSB should in turn be informed of
such developments by the PO. The primary mechanism for coordinating both the transfer of
information and the coordination of any required actions by NSF is through the NSF Integrated
Project Team (IPT).
On rare occasions, major facility projects under construction may encounter unforeseen
budget, schedule, technical, or programmatic challenges that are of a substantial enough level
to be considered grounds for termination or significant modification to the original project
goals. NSF will provide the NSB with appropriate information and a recommendation by the
1
See details in Section 4.6, Requirements for Performance Oversight, Reviews and Reporting.
Section Revision:
December 15, 2020
2.4.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Director. The NSB will decide whether termination or significant modification to the original
project goals is warranted. 1
Joint NSB-NSF Management Report: Setting Priorities for Large Facility Projects Supported by the National Science Foundation
(NSB-05-77); September 2005.
1
Section Revision:
December 15, 2020
2.4.1-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.2 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.4.2 Construction Award Close-out
2.4.2.1 Project Close-out Process
All NSF-funded research awards have final reporting and close-out procedures for the purpose
of ensuring funds have been properly used and to provide the public with information on the
research outcomes (PAM 2017, Chapter XIII). The close-out process for major multi-user
research facility construction awards has the same basic purpose and is governed by the same
NSF policy.
As part of the annual construction review process outlined above in Section 2.4.1 Construction
Award Management and Oversight, at an appropriate time approaching or following
construction completion, NSF will conduct a final construction review. This review is intended
to assess the extent to which the required scope was delivered in accordance with the PEP and
award terms and conditions.
The first step in the formal award-close-out process for major multi-user research facility
construction assistance awards is for the Recipient to submit the Final Project Report and
Project Outcomes Report for the General Public per the reporting requirements detailed in the
Proposal and Award Policies & Procedures Guide (PAPPG). The Final Project Report should
clearly map project accomplishments and deliverables to the Programmatic Terms and
Conditions of the Cooperative Agreement and Cooperative Support Agreement and should be
informed by the final construction review. There may be additional close-out reporting
requirements detailed in the governing Cooperative Agreement (CA) or Cooperative Support
Agreement (CSA). It is the Recipient's responsibility to understand and satisfy all close-out
requirements.
The Program Officer (PO) reviews these reports and distributes, as appropriate, to other NSF
offices involved in award close-outs such as Office of General Counsel, and the Office of Budget,
Finance, and Award Management (BFA). Once the PO is satisfied with the Final Project Report
as submitted, he/she formally approves the report by accepting it in eJacket. Following this
certification all financial transactions with Recipient are closed and the award is closed out.
2.4.2.2 Request for No-Cost Extension
The PO has the authority to recommend approval of the first NSF-Approved No-Cost Extension
(NCE). However, the PO will generally work closely with members of the NSF Integrated Project
Team (IPT) to ensure the request meets the requirements for major facility projects as
described herein. Any subsequent NCEs must be approved by the Grants and Agreements
Officer (G/AO) or Contracting Officer (CO), who is also a core member of the IPT. As the project
nears completion, close-out activities will become a discussion item for the IPT.
Only tasks within the approved project scope may be included in the NCE. As stated in Section
4.2.5, Budget Contingency Planning for the Construction Stage, any unused funds (either
contingency or positive cost variance) must be returned to the agency.
Section Revision:
May 30, 2019
2.4.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.2 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Many intended tasks will already be clearly contained within the approved project scope and
can be directly associated with a particular WBS element. Tasks which cannot be found to fall
within an approved WBS element will be allowed only after they have been reviewed and
approved as new scope through the change and/or configuration control processes contained
in the Project Execution Plan. Depending on the magnitude, this may require very high-level
approvals within the agency. It is highly recommended that the discussion of scope, and the
ability to assign to an approved WBS element, takes place prior to the NCE request.
Good practice suggests that all other project tasks, i.e., those not included in the NCE request,
should be closed out by the original award end date. This means that all risks and liens for those
tasks are also closed out, and that no funds are carried forward for remediation of problems
that arise in the future. The close-out of completed tasks also allows for a more precise
calculation of remaining cost variance and/or contingency which facilitates good decisions
making on the part of the Project and NSF. If any tasks slated for close-out are not completed
by the original award end date, then NSF must be notified that the tasks will be carried over
into the extension period as part of the NCE request.
It is anticipated that the list of tasks to be performed during the extension may change with
time as final negotiations and decisions are made and actual costs are realized. Some tasks may
be held back and subsequently removed as scope contingency options when available
resources or priorities change. Other tasks within the approved scope of the project may be
added (for example, as a result of a reprioritization exercise following final acceptance reviews
or because they are delayed past their close-out dates). Tasks may be added or removed from
the list with adequate justification and with the written approval of NSF. All final close-out
documentation will be saved to the official record by the PO.
Written requests for NCEs should be submitted to the PO and should include the information in
the following list:
1. List of the tasks to be completed during the extension period and justification that they
are within project scope.
a. Link the tasks to the associated WBS element and give a short justification of
how they fit within existing project scope. Risk mitigation effort should be
associated with an identified and documented risk element.
b. Provide the total burdened estimated cost for each task. Detailed cost estimates
do not have to be provided but should be documented and available if
requested.
c. The justification for each task will typically fall into one or more of several
categories: (1) open purchase orders and invoices associated with items whose
delivery is delayed beyond the current period of performance, for example due
to subcontractor performance, (2) rework of existing tasks within the approved
scope, for example due to workmanship or performance issues, (3) existing tasks
within the approved scope that have not yet been completed, and (4) risk
Section Revision:
May 30, 2019
2.4.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.2 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
mitigation to address in-scope performance issues. An example of a task list with
justifications is given in the sample Table 2.4.2-1 on the following page.
2. Indication of which tasks provide de-scoping options 1 if resources (time, staff, budget,
etc.) become limited. Briefly indicate why each task is a candidate for de-scoping and
give any deadlines for exercising the de-scope option. NSF must be notified when and if
the scope contingency option is taken and tasks are removed from scope, including the
impact on project deliverables or performance, if relevant.
3. Description of what funds will be used to cover the proposed tasks – remaining
contingency, unexpended PMB budget, positive Cost Variance, partner funds, etc. Give
the project PMB ETC with all tasks included and remaining risk exposure for comparison
to remaining contingency and TPC. State a confidence level for completing all work
within budget, including the use of any scope contingency options. Indicate if any tasks
involve already obligated funds and give the amount of those funds.
4. Summary schedule or schedule highlights of the extended tasks, including significant
milestones and the new project end date. Provide (BOE) for the new project end date,
including schedule contingency, and give a confidence level for completing by that date.
1
Scope contingency and management is defined in Section 6.2.3.
Section Revision:
May 30, 2019
2.4.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.4.2 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 2.4.2-1
Task #
Task Description
Sample of a No-Cost Extension Tasks Table
Burdened
Subtotals
($K)
WBS
Justification
1
Modifications to
electronics control boards
40.5
3.7 Environmental
Systems ADCs
Rework of existing in-scope
task; technology not
performing as intended
2
Delivery of 3 cryo-pumps
114.9
4.2 Vacuum Systems
Existing in-scope task; Late
delivery on open contract with
obligated funds
3
General purpose utility
carts
25.8
2.4.5 Monitoring
and Maintenance
Equipment
Existing in-scope task; Late
delivery; 1 unit added based on
revised needs estimate
4
Vendor contract to test
relationship of
performance versus
temperature on sample
size widgets
32.4
5.2.3 Sys Eng.
Integrated testing
Risk mitigation added to
address in-scope performance
issues for integrated systems.
Risk Register ID #14-31.
5
Labor extensions for
project management and
business offices
184.2
1.2 Project Controls
Existing in-scope task; revised
effort, salary, and overhead
estimates, including escalation
TOTAL ($K)
Section Revision:
May 30, 2019
$397.8
2.4.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.5
OPERATIONS STAGE
2.5.1 Operations Management and Oversight
Although NSF does not directly manage the operations of the facilities it supports (with the
exception of Arctic and Antarctic activities), the agency engages in oversight and assurance of
facility awards during each stage of the facility’s life cycle. In oversight, NSF employs a teamoriented approach in which scientific and engineering staff work closely with business
operations staff. Additional detail on facility operations may be found in Section 1.1 of this
Guide 1 and among the special topics found in Section 4, Key Management Principles and
Requirements for Major Facilities.
The Recipient responsible for construction or acquisition of a new facility is normally the entity
that submits a proposal for operation of the facility during the Construction Stage. However,
the Operations Stage may be managed by a different entity, depending on circumstances stated
in the IMP.
The operations proposal is merit-reviewed following NSF’s guidelines. Operations activities are
funded through NSF’s R&RA and/or Education and Human Resources (EHR) account. Testing
and acceptance, user training and engineering studies occur as the facility transitions to full
operation. Operations include the day-to-day work required to: support and conduct research
and education activities; ensure that the facility is operating efficiently and cost-effectively; and
provide small- and intermediate-scale technical enhancements when needed to maintain stateof-the-art research capabilities.
The content of the operations proposal will be adapted to the specific nature of the facility, but
at a minimum it should be compatible with the Concept of Operations Plan (PEP-15.3)
established during construction and include:
1
•
A detailed bottom-up cost estimate for operations per Section 4.2 of this Guide.
•
A detailed description of how the operation of the facility will be managed, including the
roles of key staff and use of advisory committees. This could be included as part of the
proposal text or a separate Operations Plan.
•
Description of plan for education and outreach.
•
Description of research program supported through operations, if applicable.
•
A listing of which Environmental, Safety and Health (ES&H) standards will be followed by
the Recipient and a description of how adherence to those standards will be verified. A
policy for reporting to the NSF accidents or environmental releases should also be given.
This may be given as a reference to an existing ES&H plan (PEP-13.1) for the project.
•
A listing of which cyber-security standards will be followed by the Recipient and a
description of how adherence to those standards will be verified. A policy for reporting
Further development of these sections is planned for MFG future version.
Section Revision:
May 30, 2019
2.5.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
to NSF of any breaches of cyber-security should also be given. This may be given as a
reference to an existing Cyber-Security Plan (PEP-12.1) for the project.
•
A discussion of how major overhaul, repair or replacement of long-lived capital assets or
components who useful life extends beyond the duration of the CA will be handled.
•
A discussion and acknowledgement of the current plan for re-competition or
divestment.
•
A set of performance goals and metrics sufficient to establish that the facility is
operating successfully. The Facility’s performance against these metrics will be reported
periodically as required by Program.
•
Discussion on plans or progress toward major upgrades to the Facility to increase
science capabilities.
Given the long Operations Stage of most major facilities, upgrades and refurbishment of
equipment may be required over time in order to stay at the research frontier. In the case of an
observatory, this may include new instruments and cameras. For a sensor network, it may
include the deployment of additional sensors or renewal of cyber-infrastructure. At an
accelerator facility, the upgrades may take the form of higher energy or luminosity or new
detectors. In general, these upgrades and renewals will be funded from R&RA funds, either
from a portion of the operating funds designed for such purposes or from separate equipment
and instrumentation programs. Funding for more significant upgrades that exceed the major
facility threshold 1 require the same approval process that for a new major facility project.
The PO should be closely involved in monitoring and assessing the facility’s evolution and in
supporting advanced R&D planning and budgeting. Evaluation of each NSF major facility, as part
of its yearly operations review, should include a section on the plans for advanced R&D and
should relate these plans to the anticipated evolving mission of the facility. This evaluation
helps guide the PO in formulating a budget strategy for funding advanced R&D efforts.
A Program Officer (PO) may also request a periodic formal Condition Assessment report (an
evaluation of capital assets requiring significant expenditures for periodic replacement or
refurbishment and having a lifetime longer than the usual five-year award cycle), accompanied
by an Asset Management Plan (a strategic plan for dealing with these issues), to inform NSF and
the facility management of anticipated major and infrequent maintenance expenses that cause
a significant departure from the routine funding profile. This allows NSF, as part of its budget
allocation process, to proactively address these issues before they become immediate needs.
Generally speaking, there are three key aspects of NSF oversight and assurance of major facility
operations, which are codified in and required by the CA: (1) Annual Work Plans, (2) Annual
1
Refer to Section 1.4.3 of this Guide for the major facility threshold.
Section Revision:
May 30, 2019
2.5.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Reports, and (3) annual operations reviews. NSF or the cognizant agency may also conduct
periodic audits or Business Systems Reviews.
Annual Work Plan
The Annual Work Plan (AWP) describes what the facility expects to accomplish in the coming
fiscal year. For many facilities, the AWP, annual operations proposal, and Cost Estimating Plan
(per Sections 4.2.2 and 4.2.4) can be combined as one document, so long as all elements are
addressed. The AWP should include a series of high-level performance goals (clear and agreed
upon goals and objectives, performance metrics and, where appropriate, performance targets)
for the coming year. The goals should include both scientific and operations issues (i.e.,
installation of new equipment or commissioning of new buildings, maintenance, Education and
Oversight Training and ES&H). The goals and metrics will naturally vary from facility to facility
and should be agreed upon between the Recipient and the NSF Program Officer (PO). The PO
will review the AWP goals to ensure they are aligned with the long-term scientific objectives of
the facility.
It is the Recipient’s responsibility to manage and maintain the NSF-funded facilities, equipment,
and instrumentation used in the conduct research regardless of ownership. See Section 6.6 of
this Guide. In accordance with federal guidance1 on property that the government owns,
leases, or otherwise manages, Recipients should annually provide a brief discussion, cost
estimate, and actual expenditures at a high level for the following:
•
Recurring routine maintenance and repair.
•
Significant infrastructure changes, including modernization, overhaul, upgrade,
replacement, and/or expansion for science facilities, equipment, utilities, and/or
instrumentation.
•
Utilities (including facility operation and purchase of energy)
•
General support services (such as grounds and waste management).
Annual Report
The annual report describes in detail the activities of the facility in the previous year based on
the award date and the award terms and conditions. This report is required by NSF policy (see
PAPPG) and necessary to review progress on that year’s performance goals as described in the
AWP. Due to changing research priorities or external forces, not all performance goals may be
met each year but an explanation of progress on each goal should be included. The Recipient
should also report all expenditures relative to the planned budget in accordance with award
terms and conditions. The PO reviews and approves the annual report.
1
41 CFR 102-84 “Annual Real Property Inventory”, GSA Guidance for Real Property Inventory Reporting
Section Revision:
May 30, 2019
2.5.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Annual Operations Reviews
In most cases, NSF will annually conduct an operations review of its major facilities, utilizing an
external panel of experts spanning the principal range of functions necessary to sustain facility
operations, or carry out or participate in an alternate activity that accomplishes an equivalent
purpose. When NSF partners with other entities to fund operations, the MOU between the
partners defines the process for monitoring: (1) identification and accomplishment of
programmatic goals; (2) fiscal accountability; (3) stewardship of NSF assets; and (4) compliance
with laws and regulations. Operations reviews should determine the extent to which the facility
is meeting the goals of its Annual Plan, discuss any upcoming challenges for operations, and
highlight good practices that could be applied to other NSF major facilities. Metrics and
performance goals or targets should include objectives related to educational outreach and
broader societal impacts, in addition to research goals of the operating facility. Whenever
possible, the review should be conducted at the facility itself by an external panel comprised of
experts in the operations of similar large scientific facilities and representatives of the user
community served by the facility. The panel should produce a formal written report submitted
to NSF. Results of the review are used by NSF to provide written guidance to the facility
operator in the formulation of goals or targets for the coming year. (The operations review is
not meant to compete with the Business Systems Review 1 (BSR) which looks at business
processes.)
•
The review is organized and conducted by the PO in consultation with the LFO Liaison
and G/AO. The PO has overall responsibility for organizing the review (or representing
NSF’s interests in the case of a partnership), and for acting as the interface between the
NSF and the project’s proponents throughout the review process. The LFO Liaison and
G/AO advises the PO during the planning and execution of the review to ensure that
there is consistent practice across NSF in the formulation of performance goals, that
goals and objectives are clearly stated and represent quantifiable performance
measures or targets where practical, are periodically reported, and that an evaluation
and feedback mechanism is implemented as an essential part of an ongoing program of
continual performance enhancement.
•
Following the review, the PO and the LFO Liaison will share their views and confer on
areas of concern. As a result of internal NSF evaluation of the panel report and other
supporting assessments, the NSF Program Officer should issue clear written guidance to
the Recipient for subsequent response and action.
•
In most cases, observers of the review must include the PO, the G/AO or CO, the LFO
Liaison and other staff from the Large Facilities Office, and possibly other NSF staff from
the Integrated Project Team. Budget considerations, logistical constraints, or alternate
processes for review agreed to by NSF and its funding partners may result in exceptions
to the number and range of NSF staff participating.
See Section 4.6.3.3 for discussion of the BSR process as well as the NSF BSR Guide. To avoid duplication of effort, the scope of
the BSR is adapted to utilize relevant information stemming from other reviews and audits.
1
Section Revision:
May 30, 2019
2.5.1-4
Major Facilities Guide: NSF 21-107 (July 2021)
2.5.2 Renewal/Competition
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.5.2 Renewal/Competition
Prior to the expiration of a major facility operations and maintenance (O&M) award, NSF makes
a determination to either renew an award with the existing managing organization, compete
for a new managing organization, or divest of the facility. 1 Ample time must be allowed to
inform the agency decision. Given that NSF’s statutory language prohibits the agency from
operating “laboratories and pilot plants”, NSF facilities will be operated by a managing
organization through an O&M award. Clear scientific, technical, and business goals, supported
by carefully developed metrics in the existing agreement, will facilitate the programmatic
recommendation. Because the Operations Stage for major facilities can be quite long (some
current facilities have operated in excess of 40 years), competition of managing organization is
appropriate at times. See Section 3.5.2 for procedures for Renewal and Competition.
While renewal can be fairly streamlined when justified, competing the management of a major
facility is a significant undertaking. The goal of competition is to stimulate new approaches on
more effective management to meet scientific and budgetary objectives. Triggers for
competition may include concerns about scientific capabilities based on assessments from the
user community, poor performance by the managing organization, or significant inefficiencies
that have adversely impacted operating costs. Important considerations beyond performance
of the current managing organization include how a competition might affect scientific
productivity and whether a competition would be meaningful. Even in cases where the existing
management has been found to be effective, NSF may still decide to run a management
competition.
1
See Section 1.4.5 of this Guide regarding the National Science Board Policy on Recompetition.
Section Revision:
July 26, 2021
2.5.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
2.6 Divestment Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2.6
DIVESTMENT STAGE
To remain at the research frontier and support new facilities, NSF will consider decreasing
investments in existing facilities when the science they enable is of a lower strategic priority
than science that could be enabled by alternate use of the funds. Such decisions will be difficult
to make, in part because of the number of stakeholders and interested parties, and will require
extensive community consultation and input, which may come from “blue ribbon” panels,
National Academies committees and professional societies. In some cases, in which a facility
can continue to be productive, it may be possible to transfer ownership or stewardship to
another agency, a university or a consortium of universities. It is the responsibility of the
Directorates and Divisions to periodically review their facilities portfolio and to consider which
facilities may have reached an appropriate end of NSF support.
While not part of the annual budgeting process, proposals may be requested to address partial
or full divestment of the facility following the award period, including property divestment,
decommissioning, and disposition costs and other costs related to employee separations.
Periodic reviews of these proposals should create and keep current a plan for the facility’s
divestment and closeout, along with its associated budget liability to inform the longer-term
strategic planning at the NSF Division and Directorate levels.
Sponsoring Directorate should initiate discussions with the Chief Officer for Research Facilities
(CORF) when considering divestment options in accordance internal guidance. As appropriate,
notifications to the National Science Board (NSB) and Office of Management and Budget (OMB)
will be made. When the decision is made to close or transfer ownership or stewardship of a
facility, a detailed transition plan will be developed, which includes all divestment costs and
liabilities, including disposal or transfer of equipment, environmental and site remediation or
restoration, pension, and health care responsibilities, etc.
Guidelines and requirements for creating divestment transition plans are included in Section
3.6, Facility Divestment Plan of this Guide. Since divestment strategies and liabilities may
influence construction strategy, a divestment plan is a necessary element (PEP-1.5) for major
facility projects and thus a draft plan should be created early in the Design Stage planning.
Section Revision:
December 15, 2020
2.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3
3.1
FACILITY LIFE CYCLE MANAGEMENT PLANS FOR MAJOR FACILITIES
INTRODUCTION
Section 3 contains descriptions and guidelines for creating the plans and documents that NSF
and Recipients use in the management and oversight of major facilities. They include two plans
produced by NSF and three plans that are the product of the facility designers, constructors,
and operators.
The NSF Facility Plan, when requested by NSB, is as described in Section 3.2.
Section 3.3 describes an Internal Management Plan (IMP), the NSF document that captures how
NSF will oversee awards for major facilities throughout the life cycle, from candidate facility
projects in design, through construction and operation, and ultimately, through divestment. An
IMP also provides financial strategies for funding given the budgetary estimates. The created
IMPs are internal NSF documents.
The Project Execution Plan (PEP) is produced by the Recipient to detail how management and
execution of design and construction of a major facility will be accomplished. The PEP advances
in maturity from a rudimentary form required at the Conceptual Design Review to a fully
mature document ready to support construction at the Final Design Review. Section 3.4
provides a list of the required components of a PEP and guidelines for creating those
components.
Operations Plans are addressed in Section 3.5, including timelines for submission and review of
operations proposals from prospective Recipients and guidelines for content of proposals and
plans. Operation Plans cover all aspects of operations, maintenance, upgrades, and research
and education programs. Guidelines are also given for the procedures for renewal or
competition of an award for an operating facility.
Guidelines for plans to closeout operations under NSF awards are in Section 3.6, Facility
Divestment Plan, of this Guide and closeout of NSF funding and oversight of a facility may be
accomplished through various options ranging from transfer to another agency or funding
source to decommissioning and removal of the infrastructure and site restoration.
Section Revision:
November 25, 2016
3.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.2 NSF Facility Plan [Reserved]
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.2
NSF FACILITY PLAN [RESERVED]
Section Revision:
December 14, 2018
3.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.3 NSF Internal Management Plans for the Major Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.3
NSF INTERNAL MANAGEMENT PLANS FOR THE MAJOR FACILITY LIFE CYCLE
The Internal Management Plan (IMP) is the primary internal agency document that describes
how NSF will oversee development, construction, operation and eventually divestment. The
requirement to develop an IMP is described in Section 2.3.1 for major facilities. Three primary
purposes are served by development of an IMP:
•
It defines in specific detail how NSF will conduct oversight of a project;
•
It describes plans for managing NSF-specific risks, and
•
It provides budgetary estimates for developing, constructing, and operating the facility,
identifies divestment liabilities, and lays out a strategy for financing these activities as
well as the concomitant NSF oversight requirements.
The IMP should be a living document that is updated at transition points between project life
cycle stages, or as often as needed, to define review criteria, decision points, strategies for
renewal or competition, plan for advanced R&D or technology refresh, upgrades, etc.
Section Revision:
December 14, 2018
3.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.4
PROJECT EXECUTION PLAN
3.4.1 Components of a Project Execution Plan
Typical components of a construction-ready Project Execution Plan (PEP), common to most
plans for construction of major facilities, are listed in Table 3.4.1-1 below, as an example of the
extensive nature of the pre-construction planning that should be conducted prior to expending
construction funds to execute the project. While many of the listed topics cannot be
substantively addressed at the earliest stage of project planning, it is important that project
advocates are aware, at the outset, of the full scope of pre-construction planning activities that
should be undertaken, and the consequent pre-resources required. As the project matures
through the Conceptual, Preliminary and Final Design Phases, these topics become
correspondingly better defined. Some topics will continue to be refined during the Construction
Stage, for example, Commissioning Plans and related sub-plans.
The PEP should ideally contain or reference all project related documents and be the
standalone source explaining how and why the project meets all requirements and should
proceed as planned. Various components of the PEP may often be detailed in separate
documents, especially, living documents for future operations such as cybersecurity and data
management plans. The PEP should reference these separate documents to summarize the
complete scope of the pre-construction planning. In addition to referencing these separate
plans, the PEP should provide a high-level summary, outline the associated goals, and/or
identified responsibility for the specific plan.
It is important for PEP to document all assumptions and boundary conditions driving project
design and implementation. Additions or alterations to the typical PEP components listed below
are likely, due to the unique nature of each specific project. Any special construction elements
should be identified and addressed in the PEP or a separate document such as the work
breakdown structure dictionary or design requirements, drawings, and specification
documents.
The PEP at the end of the Final Design Phase is incorporated as part of the construction award
through reference to define the award scope, schedule, configuration and contingency control,
and project governance.
Section Revision:
December 14, 2018
3.4.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 3.4.1-1
List of the Typical Components of a Project Execution Plan, with Sub-Topics and Descriptions
Component
Sub-Topics
Description of Sub-Section Requirements
1. Introduction
1.1 Scientific Objectives
Description of the research objectives motivating the facility
proposal.
1. Introduction
1.2 Scientific
Requirements
Comprehensive statement of the Requirements Matrix/ Key
Science Requirements to be fulfilled by the proposed facility
(to the extent possible identifying minimum essential as well
as desirable quantitative requirements), which provide a
basis for determining the scope of the associated
infrastructure requirements.
1. Introduction
1.3 Facility /
Infrastructure
Description of the infrastructure necessary to obtain the
research and education objectives.
1. Introduction
1.4 Scientific & Broader
Societal Impacts
Description of the Broader Societal Impacts associated with
the purpose of the facility, including the scope of work,
budget and schedule related to science community or
society related actions or interactions.
1.5 Facility Divestment
Plan
Description of plans and estimate of divestment liabilities at
the end of facility life for transfer, demolition, site
remediation, decontamination, etc., where appropriate.
2. Organization
2.1 Internal
Governance &
Organization and
Communication
Internal Project Governance and Organization Structure with
clear lines of authority, responsibility, and communication
between Internal and institutional governance and oversight
and advisory committees.
2. Organization
2.2 External
Organization and
Communication
External Project Organizational Structure and Governance,
showing clear lines of authority, responsibility, and
communication between NSF, any partners, and the
Recipient.
2. Organization
2.3 Partnerships
Role of interagency or international partners in future
planning and development and/or construction. Plans,
agreements, and commitments for interagency and
international partnerships. Description of the project’s
stakeholders and their roles, responsibilities and meeting
schedules.
2. Organization
2.4 Roles and
Responsibilities
Roles and Responsibilities of key project personnel and
governance groups.
2. Organization
2.5 Community
Relations and Outreach
Community Relations and Outreach plans for building and
maintaining effective relationships with the broader
research community that will eventually utilize the facility to
conduct research and with the public. Description of
scientific and educational outreach programs.
3. Design and
Development
3.1 Project
Development Plan
Description of activities that will be undertaken in order to
achieve readiness for construction, such as design,
prototyping, manufacturing process validation, vendor
qualification, modeling and simulation, creation of required
project management plans, forming partnerships, etc.
Section Revision:
December 14, 2018
3.4.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component
Sub-Topics
Description of Sub-Section Requirements
3. Design and
Development
3.2 Development
Budget and Funding
Sources
Estimate of total budget required to perform Design and
Development, including NSF funding and any contributions
from partners and other outside sources.
3. Design and
Development
3.3 Development
Schedule
Schedule of design and development activities and
milestones, at a level of detail appropriate to the maturity
and complexity of the work.
4. Construction
Project Definition
4.1 Summary of Total
Project Definition
Summary at Work Breakdown Structure (WBS) level II of
total construction project scope, cost, and schedule required
to complete the construction or implementation project,
indicating the baseline (pre-award) or Performance
Measurement Baseline (PMB) (post-award) and
contingencies funded by NSF as well as any associated scope
supported by other funding sources.
4. Construction
Project Definition
4.2 Work Breakdown
Structure (WBS)
WBS contains a product-oriented, hierarchical framework
that organizes and defines the total scope of the project into
individual project component that represent work to be
accomplished, aggregating the smallest levels of detail into a
unified project description. The WBS integrates and relates
all project work (cost, schedule and scope) and is used
throughout the project management to identify and monitor
project progress.
4. Construction
Project Definition
4.3 WBS Dictionary
WBS dictionary defining scope of each WBS element,
through all levels.
4. Construction
Project Definition
4.4 Scope Management
Plan and Scope
Contingency
The plan describes how the scope will be defined,
developed, monitored, controlled, and validated, and how
scoping opportunities and descoping options will be
realized. Scope Contingency compiles savings from potential
de-scoping options, with decision points for exercising
options and time-phased cost and schedule.
4. Construction Project Definition
4.5 Cost Estimating
Plan, Executive
Summary, and Baseline
Budget
A plan to establish and communicate how the
preparation, development, review and approval of the
estimate will be completed. An executive summary
provides a summary of the costs at a high level and an
overall basis of estimate.
4. Construction
Project Definition
4.6 Budget Contingency
Contingency budget and description of method for
calculating contingency, including confidence level for
completing within budget.
4. Construction
Project Definition
4.7 Cost Book, Cost
Model Data Set, and
Basis of Estimate
The Cost Book is the comprehensive and well-documented
compilation of Cost Book Sheets for the total project cost.
The cost model data set is used as input to software tools
and/or project reports to organize, correlate, and calculate
different project management information. The Basis of
Estimate provides supporting documentation outlining the
details used in establishing project estimates such as
assumptions, constraints, and estimating methods, and
referencing the technical information used.
Section Revision:
December 14, 2018
3.4.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component
Sub-Topics
Description of Sub-Section Requirements
4. Construction
Project Definition
4.8 Funding Profile
Show the proposed NSF Funding Profile by year with
baseline commitment and anticipated contingency
allocation profiles. Also provide a total funding profile from
all sources if applicable.
4. Construction
Project Definition
4.9 Baseline Schedule
Basis Document and
Integrated Schedule
Schedule (without contingency) for the overall project and
each major subsystem, including system integration,
commissioning, acceptance, testing and transition activities;
as well as major milestones and milestones for reviews,
critical decisions and deliverables. It uses formal scheduling
programs, is based on the WBS hierarchy, and is resourceloaded before the construction/implementation stage.
Baseline schedule does not include schedule contingency.
4. Construction
Project Definition
4.10 Schedule
Contingency
Schedule contingency amounts and project end date with
contingency; state method of calculating contingency,
including confidence level for meeting project end date.
5. Staffing
5.1 Staffing Plan
Staffing FTE plan, per NSF and other project-specific job
categories, over time. Application of indirect cost rates must
be articulated in Cost Estimating Plan (CEP) and Basis of
Estimate (BOE) per Section 4.2 of this Guide.
5. Staffing
5.2 Hiring and Staff
Transition Plan
Schedule and requirements for hiring and training staff,
including timelines for increasing or decreasing staffing
levels. Required qualifications for key staff.
6. Risk and
Opportunity Mgt
6.1 Risk Management
Plan
Risk Management Plan describes the methodology/process
for identifying, ranking, analyzing, tracking, controlling, and
mitigating risks. Describes both qualitative assessment and
quantitative analysis methods.
6. Risk and
Opportunity Mgt
6.2 Risk Register
A tracking document or tool that provides a ranked list of
identified risks, with risk impact analysis and prioritization,
responsibilities, mitigation plans and opportunities of risk
reduction, and risk status over time. Documents data and
assumptions used in risk analysis.
6. Risk and
Opportunity Mgt
6.3 Contingency
Management Plan
Contingency management plans and approval process using
change control. Describe NSF approval requirements per
cooperative agreements (CAs).
7. Systems
Engineering
7.1 Systems
Engineering Plan
Systems Engineering Management Plan; roles and
responsibilities.
7. Systems
Engineering
7.2 Systems
Engineering
Requirements
System-level design and technical feasibility study, including
definition of all functional requirements and major systems.
Identifies all technical design requirements, drawings, and
specifications.
7. Systems
Engineering
7.3 Interface
Management Plan
Identification of interfaces between major components or
WBS elements and plans for managing communication,
interferences, and interactions. Interface Management Plan
and Documentation.
7. Systems
Engineering
7. 4 QA/QC Plans
Quality assurance and quality control requirements and
description of processes.
Section Revision:
December 14, 2018
3.4.1-4
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component
Sub-Topics
Description of Sub-Section Requirements
8. Configuration
Control
8.1 Configuration
Control Plan
Configuration Control plans.
8. Configuration
Control
8.2 Change Control Plan
Change Control Plan to manage accounting changes and
changes in the baseline or PMB plan: changes in scope,
modifications to budget or schedule, and movement of
contingencies into or out of the PMB. Includes approval and
documentation processes plus roles and responsibilities.
8. Configuration
Control
8.3 Document Control
Plan
Document Control Plan for managing version control, access,
and archiving of project related documentation.
9. Acquisitions
9.1 Acquisition Plans
Describe acquisition plans, processes, subawards, and
contracting strategy, including evolving technologies and
assumptions for design definition. Provide a time-based list
of acquisitions and procurement actions.
9. Acquisitions
9.2 Acquisition
Approval Process
Describe the approval process for acquisitions (NSF,
internal), and create a year by year Acquisition Plan of
actions that are estimated to require NSF approval.
10. Project Mgt.
Controls
10.1 Project Management Control Plan
Description of the project management organization and
processes.
10. Project Mgt.
Controls
10.2 Earned Value
Management System
(EVMS) Plan
Description of the EVMS plans, processes, software, and
tools.
10. Project Mgt.
Controls
10.3 Financial and
Business Controls
Description of Financial and Business processes and controls.
11. Site and
Environment
11.1 Site Selection
Site selection criteria and description of selected site(s).
11. Site and
Environment
11.2 Environmental
Aspects
List need for any Environmental Impact Statements,
permitting, site assessments, etc.
12. CyberInfrastructure
12.1 Cybersecurity Plan
Plan for protecting access, confidentiality, and integrity of
key information assets of the facility.
12. CyberInfrastructure
12.2 Code Development
Plan
Plan to enable critical scientific/engineering capabilities and
data flows within the facility as well as interoperability with
key external collaborators or stakeholders.
12. CyberInfrastructure
12.3 Data Management
Plan
Plans for acquisition and integration of equipment or
services from third parties.
13. Environmental,
Safety and Health
13.1 Environmental,
Safety and Health Plans
Environmental, Safety and Health plans (ES&H).
14. Review and
Reporting
14.1 Reporting
Requirements
Statement of reporting requirements, including notifications
for specific events and periodic reports on progress and
project technical and financial status per NSF contractual
requirements or CAs.
14. Review and
Reporting
14.2 Audits and
Reviews
Statement of the required and proposed reviews, audits,
and assessments for progressing during project life cycle
through project close-out.
Section Revision:
December 14, 2018
3.4.1-5
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.1 Components of a Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component
Sub-Topics
Description of Sub-Section Requirements
15. Commissioning
15.1 Integration and
Testing Plan
Describes the acceptance criteria and technical activities
that should be completed as part of construction to
transition the facility to operations.
15.2 Operational
Readiness Plan
Plan for determining operational readiness; includes
administrative (non-technical) acceptance procedures to
transition the facility from construction to operations such
as conducting the operational readiness review and the
authorities for making the determination(s).
15.3 Concept of
Operations Plan
Plans for, and estimate of, annual operations and
maintenance costs (staffing, services, material/supplies, etc.)
and funding sources that will be needed when the facility
has completed construction and is transitioned to
operations. This plan should include activities to bring the
facility to full science capability after acceptance.
15.4 Segregation of
Funding Plan
Financial accounting procedures for the Recipient to
properly expense the activities between construction and
operations funding per the Plans above.
16.1 Project Close-out
Plan
Procedures and criteria for closing out the project. Includes
acceptance of verification of technical performance as well
as documented completion of all scope contained in the
WBS dictionary. Includes procedures documentation for
closing out all acquisitions and financial accounting.
16. Project
Close-out
Section Revision:
December 14, 2018
3.4.1-6
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.2 Detailed Guidelines for Project Execution Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.4.2 Detailed Guidelines for Project Execution Plans
This section elaborates on the various components outlined in the previous section,
Components of a Project Execution Plan, and offers additional information that should be
helpful to individuals newly involved in planning for construction and future operations. Each of
the sub-sections below are aligned to the PEP Components identified in Section 3.4.1 and
provides some cross-reference to other sections of this Guide.
3.4.2.1 Introduction [Reserved]
3.4.2.2 Organization [Reserved]
3.4.2.3 Design and Development [Reserved]
3.4.2.4 Construction Project Definition
Refer to Section 4.2.2.1 for guidance on the cost estimating plan (CEP). The cost estimate
should include an executive summary of the estimate, including narrative, figures, and tables
per Section 4.2.3.2.
Refer to Section 4.2.6 for guidance on development of construction schedules including the
schedule basis document and NSF expectations associated with the GAO scheduling best
practices.
3.4.2.5 Staffing [Reserved]
3.4.2.6 Risk and Opportunity Management
Refer to Section 6.2 for Risk Management Guidelines.
3.4.2.7 Systems Engineering [Reserved]
3.4.2.8 Configuration Control
Refer to Sections 2.4.1, 4.2.5.5, and 4.6.5 regarding changes to the performance measurement
baselines (PMB) and the use of budget and schedule contingency.
3.4.2.9 Acquisitions [Reserved]
3.4.2.10 Project Management Controls
Refer to Sections 4.6.3.6 and 6.8 for more information and guidelines on earned value
management systems (EVMS).
3.4.2.11 Site and Environment [Reserved]
3.4.2.12 Cyber-Infrastructure
Refer to Section 6.3 for guidelines on cybersecurity.
3.4.2.13 Environmental, Safety and Health [Reserved]
Section Revision:
December 14, 2018
3.4.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.2 Detailed Guidelines for Project Execution Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.4.2.14 Review and Reporting
Refer to Sections 4.2.5.8 and 4.6.2 for more information on reporting to NSF.
3.4.2.15 Commissioning
Every major facility has a unique set of systems and subsystems with associated technical
requirements and interfaces, both internal and external to the facility. Technical requirements
and interface control documentation created during project planning and design assist in
defining the inspection and test regimes necessary for commissioning and acceptance of the
facility. Therefore, the systems engineering documentation indicates the timing of and criteria
for the facility’s transition to operations. These principles should be applied to generate four
plans prior to the start of construction.
Transition from construction to operations could be a single acceptance event or multiple
depending on the nature of the project. Many facility projects require integration and testing,
followed by commissioning activities to bring the facility up to the design level of operation.
Depending upon the complexity and time needed to reach design specifications, commissioning
may be split between the construction effort and operations. Commissioning milestones should
be included in the resource-loaded schedule to identify key elements associated with this
transition. The scope of construction activities is defined in the project’s Integration and Testing
Plan (PEP-15.1) and the Operational Readiness Plan (PEP-15.2) and is included in the initial
construction budget request as part of the baseline. The PEP is included by reference in NSF’s
construction cooperative agreement (CA) or contract with the Recipient institution,
documenting the mutual understanding of the work scope funded under construction.
The Integration and Testing Plan is a comprehensive set of prescribed inspections and tests
within the project technical requirements and provides the means for a process of verification,
throughout commissioning activities, that the facility is complete and ready for operations.
Successful completion of all inspections and tests provides validation that the facility meets
technical requirements and therefore passes all acceptance criteria. These tests should be
included as part of the construction baseline and associated activities included in the resourceloaded schedule.
The Operational Readiness Plan defines the process for acceptance at the end of construction
and determining operational readiness. The Plan should include an overview of the acceptance
inspections and tests that verify and validate technical requirements and interfaces to
transition the facility from construction to operations. Verification is the process of checking
that the construction meets specification as defined in the Integration and Testing Plan.
Validation is the process of checking whether the construction meets the scientific objectives.
Administrative acceptance procedures are to identify the authorities, such as project
management team, review team or independent agents, for making the determination(s).
A Concept of Operations Plan (PEP-15.3), also required by the PEP, defines the resources and
funding needs when the facility has completed construction and is transitioned to operations
Section Revision:
December 14, 2018
3.4.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.2 Detailed Guidelines for Project Execution Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
and is refined during the Construction Stage in preparation for entering the Operation Stage 1.
In some cases, particularly with distributed facility projects, early operations funding begins to
increase as aspects of a facility come on line, although full construction funding may not have
concluded. Although these Stages may overlap in time, the hand-off from construction project
responsibility and funding to operations responsibility and funding must be defined and
managed separately due to segregation of funds requirements.
A Segregation of Funding Plan (PEP-15.4) is intended to establish internal guidelines to be used
by the Recipient and to inform a mutual understanding between NSF and the Recipient of the
Recipient’s practices and responsibilities to determine the appropriate award when allocating
expenses, particularly when construction and operations activities overlap in time. 2 The plan
describes the procedures the Recipient will use to ensure that costs and activities are expensed
to the proper award by clearly defining the separation between the different sources of
funding. Funds used on major facilities often come from sources such as existing ongoing
operations, construction awards, operations start-up awards that include select commissioning
activities, research grants, partner funds, etc. The Segregation of Funding Plan should include
the following:
•
•
•
•
•
•
•
Description of financial controls, including accounting practices, EVMS, business
controls, and/or award management practices
Identification the roles and responsibilities for financial oversight, including decision
authority, of proper allocation of expenditure if a question should arise during execution
Definition of the project scope in terms of deliverables (WBS dictionary, key
performance parameters, etc.)
Description of any contributions to the project from other funding sources and how
these contributions are financially managed (i.e. separate job/cost accounting records)
Description of the hand-off from the project construction to operations
Description of how the guidance in the plan will be articulated to the facility staff
members
Description of materials/services that benefit more than one award (i.e. construction
and operations awards) and methodology used to allocate expenses to the awards.
Various aspects of the Segregation of Funding Plan may be addressed in the Recipient’s internal
policies and procedures or addressed in other parts of the subject PEP. In these cases, the
Segregation of Funding Plan should address these aspects by reference in lieu of duplicating
internal documents or text from other components of the PEP.
The Integration and Testing Plan (PEP-15.1), Operational Readiness Plan (PEP-15.2), Concept of
Operations Plan (PEP-15.3), and Segregation of Funding Plan (PEP-15.4) are to be reviewed
during conceptual, preliminary, and final design reviews. The plans are updated as needed
1
See Section 1.1 for operations planning guidance.
2
2 CFR 200.413 "Direct Costs" describes the criteria recipients must use when direct charging costs against a federal award.
Section Revision:
December 14, 2018
3.4.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
3.4.2 Detailed Guidelines for Project Execution Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
during the Construction Stage. At least one year prior to initial commissioning activities, the
plans must be updated and provided to NSF for review. Commissioning verifies that the
substantially complete facility operates over its full range of capabilities as specified in the final
design documents. Once the commissioning planning is complete, an operations readiness
review may be held to examine and comment on the plan. This can be conducted separately or
as a component of one of the required annual reviews. The review is organized and conducted
by the Program Officer (PO) in consultation with the LFO Liaison and Grants and Agreements
Officer (G/AO) similarly to other reviews.
Refer to Sections 3.5 and 4.4 for more information on operations planning and commissioning.
3.4.2.16 Project Close-out Plan
Refer to Section 2.4.2 for more information on project close-out.
Section Revision:
December 14, 2018
3.4.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
3.5.1 Preparation of Proposals for Operations and Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.5
OPERATIONS PLANNING
3.5.1 Preparation of Proposals for Operations and Management
In order to avoid funding gaps, formal proposals to operate a facility should be prepared well in
advance of the anticipated start date for operations: as much as two years prior to the end of
construction and commissioning activities. Program Officers (PO) and Directorates/Offices are
encouraged to take into account the time needed for internal NSF review, including NSB review,
and offer guidance to the community. Estimates of the funds for operations and maintenance
are provided even in the planning phases of a facility. The potential Recipient and/or the PO
need to establish a dialogue with the user community to determine the resources needed to
fully exploit the facility. In addition, the proposal should include:
•
All costs to operate, maintain and periodically upgrade the facility, its instrumentation
and the IT components, including cost and approximate time of investment (Note: A PO
can expect that IT components will need to be upgraded at least every 3 to 5 years);
•
The costs of an in-house research program (as a separate line item in the budget), if
applicable, including an indication of how the overall research program will be managed
and how research program resources will be allocated;
•
Education and outreach plans and costs;
•
A detailed management plan for operations of the facility, including the roles of key
staff and plans for advisory committees.
Note that Section 4.2 provides requirements for cost estimating. The cost estimating plan may
be incorporated in the annual operations plan.
The review of the proposal includes a realistic assessment of the costs to operate and maintain
the facility in a safe and effective manner. The PO is also responsible for oversight of
operational facilities through the various reviews and reports described in the Internal
Management Plan (IMP) and the terms and conditions of the award instrument. In addition to
following the procedures referenced as appropriate to Chapters V and VI of the Proposal and
Award Manual (PAM), the PO considers (with the assistance of external reviewers with
expertise in managing comparably scaled facilities) these questions:
•
Is the facility ready for reliable operations and is the infrastructure (including personnel
requirements) adequate to execute the proposed work plan?
•
Do the operations and maintenance plans allow for optimal utilization of the facility by
users (e.g., scheduled operating time versus down-time)?
•
Is the data management plan in place and ready to support operations?
•
Is there an appropriate balance between in-house research and research of external
users?
•
Are safety (including cyber-security and security of the physical plant), environmental
and health issues, if any, addressed?
Section Revision:
December 14, 2018
3.5.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.5.1 Preparation of Proposals for Operations and Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Are plans for securing human subjects and/or vertebrate animal clearances included, if
applicable (e.g., assessments of education-related activities)?
•
Are the Educational Outreach and Broader Societal Impact plan and cost reasonable and
include an appropriate strategy to evaluate the outcomes?
•
Have all costs been considered and estimated and is the available funding sufficient, or
is some adjustment needed?
Initial operations awards are generally either a five (5) or ten (10) year duration. Throughout
the Operations Stage, the Recipient operates and maintains the facility in accordance with the
terms and conditions outlined in the cooperative agreement (CA). The PO, together with the
G/AO, drafts the CA that will govern the operational phase of the project in accordance with
the procedures contained in Chapter VIII of the PAM. The CA will include plans for NSF
oversight, reflect the needs of the facility users, and address how the user program will be
managed and how user time will be allocated. The PO provides oversight for all aspects of
operations, maintenance and the research and education program. The PO also maintains an
awareness of emerging technical, managerial, and financial issues through contact with the
facility managers and users, and through oversight, reviews and reports.
Requests for annual funding increments may follow similar review and approval procedures as
initial operations awards depending on the particulars of the Facility and the annual funding
request. For some facilities, the annual report (submitted through Fastlane) will constitute the
funding request/proposal for the next year of funding.
Section Revision:
December 14, 2018
3.5.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
3.5.2 Procedures for Renewal or Competition of an Operating Major Facility
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.5.2 Procedures for Renewal or Competition of an Operating Major Facility
In accordance with internal NSF guidance, the Program Officer (PO) will develop a
programmatic recommendation to their Sponsoring Organization to either renew the
Operations and Maintenance (O&M) award, compete the management of the facility, or divest
of the facility as the end of the award is approaching. In making that recommendation, the PO
will consider the following issues in alignment with principles supported by the National Science
Board:
•
Are there any relevant discipline, community, portfolio balance, or Facility-specific
issues that favor competition or divestment?
•
Does the past performance of the managing organization warrant a competition?
•
Is a change in Recipient through competition feasible?
•
Is there a potential for a meaningful competition?
When the award instrument for operating a major facility is a contract, this evaluation should
be at least every 5 years and Federal Acquisition Regulations (FAR) requirements should be
followed.
The PO will generally conduct annual O&M reviews to assess progress against the Annual Work
Plan (AWP) and, if utilized, facility performance metrics. See Section 2.5.1 of this Guide for
more information on AWP and Annual Operations Reviews. These reviews should be used to
inform the programmatic recommendation on whether to renew, compete, or divest, along
with recommendations from decadal surveys (or similar science community reports) and
advisory committees utilized by the Sponsoring Organization. The PO will prepare a
recommendation to their Division Director (DD) and Assistant Director (AD)/Office Head for
consideration and decision by the Office of the Director.
Federally Funded Research and Development Centers (FFRDCs) funded under contracts follow a
slightly different process as outlined in the FAR Part 35. Like renewal and divestment of NSF
major facilities funded under cooperative agreements, approval to continue or terminate the
sponsorship rests with the head of the sponsoring agency and based upon the results of a
review. NSF FFRDC reviews should consider the steps outlined in FAR Subpart 35.017-4(c).
Section Revision:
July 26, 2021
3.5.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.5.3 Detailed Guidelines for Oversight of Operations
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.5.3 Detailed Guidelines for Oversight of Operations
Please contact the cognizant NSF program officer for additional details regarding NSF’s
oversight of the operational phase of major facilities. Internal operating guidance elaborates on
the principles outlined in the Major Facilities Guide and offers additional information and
examples that should be especially helpful to individuals newly involved in operational
oversight.
Section Revision:
November 25, 2016
3.5.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.6 Facility Divestment Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
3.6
FACILITY DIVESTMENT PLAN
After a decision for divestment is made for the transition or closeout of the facility operation
under a NSF award, the current operations management should start the preparation for the
divestment. The current management should consult stake holders and the program office to
appoint appropriate personnel or management team that will be responsible for managing the
transition activities in the divestment process. The transition team needs to develop a
transition plan and submit to NSF program office. To ensure the smooth and successful
transition, the current operations management should be involved and be an integral part in
the development of the transition plan. The transition plan should first specify the model of
divestment and the final goal of the transition, such as a new operation model under different
funding mechanism, or decommissioning. The following elements should be included in the
plan:
•
Target date for completing the transition;
•
Organizations involved in managing the transition activities
•
Estimated cost of transition, which includes labor and material cost, as well as the
estimated contingency based on the uncertainties and risks
•
Plan for environmental impact analysis;
•
Plan for resolving contractual issues and closing of contracts
•
Any additional costs and responsibilities (e.g., HR and personnel-related costs,
environmental remediation, etc.) associated with divestment/decommissioning should
be noted to the extent possible.”
The plan should identify key steps during the transition period with each step detailed with a
clearly defined goal and target timeline. The plan should identify the organizations that handle
the transition matter at each stage with clearly defined authorities and responsibilities. If the
divestment is accomplished through changing the funding source or a new model of operation,
the transition plan should identify the new management organization and include the following
elements:
•
•
•
Description of the new model of operation and NSF’s role under the new model;
The costs to NSF under the new operation model, and
A hand over procedure to the new management organization.
If the divestment is accomplished through decommissioning, the plan should identify the
equipment or facilities that need to be disposed and include the following elements:
•
•
Cost and procedures for proper disposal of equipment;
Cost and plan for environmental and site remediation.
If there are pension and health care responsibilities after the divestment, the plan should
describe how these responsibilities will be handled and the source of required funding.
Section Revision:
December 14, 2018
3.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
3.6 Facility Divestment Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The transition plan should also include the risk management during the divestment transition
process. This includes list of risks, risk mitigation and management plan.
Section Revision:
December 14, 2018
3.6-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4
4.1
KEY MANAGEMENT PRINCIPLES AND REQUIREMENTS FOR MAJOR FACILITIES
INTRODUCTION
This section provides greater detail about key management, budgeting, and reporting activities
that should be carried out throughout a project’s life cycle stages, for both Major Research
Equipment and Facilities Construction (MREFC) and non-MREFC projects, to ensure adherence
to principles established by National Science Foundation (NSF).
Some of these activities will be funded via MREFC and others via R&RA, depending on life cycle
stage.
Section Revision:
May 2, 2016
4.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2
COST ESTIMATING AND ANALYSIS
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
As noted in Section 1.1, award instruments can take the form of cooperative agreements or
contracts. Unless otherwise noted, the guidance in this section applies to major facility projects
regardless of the award instrument employed. Proposed budgets must comply with the
applicable federal regulations, as implemented by NSF in the MFG, the Proposal and Award
Policies and Procedures Guide (PAPPG) or the Guide to the NSF Contracting Process. Recipients
are required to follow the steps and best practices within the Government Accountability Office
(GAO) Cost Estimating and Assessment Guide 1 , taking into consideration NSF policy and
practice as provided in this Guide. These NSF and GAO Guides are intended for all Stages in a
facility’s life cycle. However, portions of these Guides may be tailored depending on what is
relevant to the particular facility estimate. Accordingly, Recipients must note any departures
from these NSF and GAO Guides and explain their rationale in the Cost Estimating Plan (CEP) 2.
Additional guidance on how to apply the relevant practices from the GAO Cost Guide and
examples of potential deviations are provided in Section 4.2.2.3.
The guidance herein clarifies NSF expectations for the format, content, supporting justification,
and good practices for Recipient cost estimates. This guidance also explains the NSF cost
analysis process and timeline. By following this guidance Recipients should expect a better
estimate and a more efficient review by NSF, facilitating achievement of the science mission.
For existing awards, the Recipient should consult with the PO.
NSF uses internal staff, outside experts, and panel reviews to analyze estimates for construction
and operations awards. The Recipient estimates must meet two sets of criteria that also serve
as the basis of the NSF cost analysis: (1) the cost principles of either 2 CFR § 200, Subpart E for
non-profit entities or the Federal Acquisition Regulation, Part 31 for for-profit entities, and (2)
the GAO Cost Guide. Cooperative support agreement estimates must be allowable 3, allocable,
and reasonable per the 2 CFR §200, Subpart E, and realistic. To be deemed reasonable under
the cost principles, the estimate must be developed in accordance with the best practices and
twelve steps of the GAO Cost Guide to meet the four characteristics of a high-quality estimate
(well-documented, comprehensive, accurate, and credible).
As described in Section 2 of this guide, Recipients must develop estimates for design,
construction, operation, and divestment of facilities. Estimates should be well documented,
1
GAO Cost Estimating and Assessment Guide1: Best Practices for Developing and Managing Program Costs
Definition in Lexicon is adapted from AACE International Recommended Practice No 36R-08, Development of Cost Estimate
Plans – As Applied in Engineering, Procurement, and Construction for the Process Industries, Rev. June 12, 2009.
2
Allowable costs are defined by federal guidelines and relevant cost principles. Allocable costs must be logically related to the
particular award. Reasonable costs are what a prudent individual would pay in a competitive marketplace (i.e., costs are not too
high). Cost realism defines whether the costs are realistic for the work to be performed, reflect a clear understanding of the
requirements, and are consistent with the methods of performance and materials described in the Recipient’s technical
proposal (i.e., costs are not too low).
3
Section Revision:
May 30, 2019
4.2.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
comprehensive, accurate, and credible and should facilitate appropriate analyses from a wide
variety of reviewers at the various life cycle stages. It is understood that cost estimates will
undergo further refinement at each stage-gate review and the materials required herein will
evolve accordingly. NSF will review estimates at an appropriate level as the project advances
through the various facility life cycle stages.
Figure 4.2.1-1 below depicts the general NSF cost analysis process performed for construction
and operations awards. The NSF Program Officer (PO), Grants and Agreements Officer (G/AO)
or Contracting Officer (CO), Large Facilities Office (LFO) Liaison, and Cost Analyst conduct a
detailed analysis of the Recipient cost estimate. NSF may also utilize independent cost
estimates and cost estimate reviews 1 done by external panels and independent contractors or
agencies to inform the analysis. The G/AO or CO and Cost Analyst review includes the detailed
sub-elements, cost categories, and supporting basis of estimate discussed in this section of the
Guide. The PO review includes the technical scope, risks, level of effort, schedule, and
assumptions. The LFO Liaison supports analysis of any risks and proposed contingency budget.
The inputs from the various sources are integrated and addressed with the Recipient, which
could potentially result in a revised cost estimate or additional documentation. The PO
ultimately recommends the budget, funding profile, and internal and external sources of funds
based on the realism of the cost estimate, technical scope of the project, and the availability of
funds. The G/AO or CO approves the Recipients’ cost estimate and ultimately the award of the
proposal and approved budget based on the results of the cost analysis.
For construction awards, the NSF cost analysis is done at the end of each Design Phase, in
conjunction with CDR, PDR, and FDR, to support stage-gate reviews. For operations awards, the
NSF cost analysis is done on operations and management proposals for initial operations,
renewal, and competition of awards. NSF may also perform cost analyses at other times, as
necessary, based on a risk-based assessment. For example, cost analyses may be needed during
construction or operations to support significant changes in scope, schedule, cost, risk or
complexity. These latter types of analysis may only require review of targeted subsets of
information for specific changes. NSF typically requires 90 to 180 calendar days to complete a
full review and detailed cost analysis of a proposal budget prior to proceeding to the next
design phase or prior to award for operations or construction. This time will vary depending on
project scope, cost, risk, complexity, and relative importance. It will also depend upon whether
revisions to the estimate, due to errors or cost re-categorizations, for example, are needed.
During the review time window, the Cost Analysis and Pre-Award (CAP) Cost Analyst may
perform a cost analysis (typically 60 calendar days duration) in parallel with other review
activities to augment the G/AO review and target specific areas noted in Section 2.1.6.2 and
Figure 4.2.1-1.
The definition for Independent Cost Estimate Review in Lexicon is adapted from Table 27 in GAO Cost Estimating and
Assessment Guide.
1
Section Revision:
May 30, 2019
4.2.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
If there are issues with the provided information, the PO, G/AO or CO, LFO, and/or Cost
Analysts may require additional documentation and justification and further interaction with
the Recipient prior to completing the analysis. Communication among all parties as well as a
sound initial basis of estimate are essential for timely and successful completion.
When submitting construction or operations estimates for cost analysis, Recipients must submit
the following as a minimum:
•
•
•
•
•
Cost Estimating Plan per Section 4.2.2.1.
“Cost Model Data Set” per Section 4.2.2.1.
Reports and Proposals per Sections 4.2.2.2 and either 4.2.3.2 or 4.2.4.2.
The Work Breakdown Structure (WBS) per Section 4.2.2.7.
Supporting information forming the Basis of Estimate (BOE) per Sections 4.2.2.3, 4.2.2.4,
4.2.2.5, 4.2.2.6, and either 4.2.3.4 or 4.2.4.4
For proposals that contain subawards 1, each subaward must include a separate budget
justification.
See the Section 9 Lexicon for the difference between a “subaward”, which transfers significant effort from the Recipient to
another entity, and a “contract, which involves the purchase of materials and supplies, equipment or general support services
allowable under the award.
1
Section Revision:
May 30, 2019
4.2.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.1 Overview of Guidance and Process for Both Construction and Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS), & The Large Facilities
Office (BFA-LFO)
Figure 4.2.1-1
Section Revision:
May 30, 2019
NSF Cost Analysis Process
4.2.1-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.2 Elements of Both Construction and Operations Estimates
4.2.2.1 Cost Estimating Plan
For new construction and operations awards, Recipients must develop and submit a Cost
Estimating Plan (CEP) to establish and communicate how the preparation, development, review
and approval of the estimate will be or was completed. For existing awards, the Recipient
should consult with the PO regarding the CEP. Ideally the CEP will be developed and discussed
with NSF far in advance of submission (e.g., one year for major facility awards) to ensure that
that Recipient’s plans are aligned with NSF expectations and requirements outlined herein and
sufficient time is available to collect and package data. The CEP is the cornerstone of the
estimate(s) that come later and, along with the basis of estimate, critically important for
generating a high-quality estimate to facilitate management decisions and NSF cost analysis.
Recipients should contact their NSF PO, G/AO or CO, LFO Liaison, and/or Cost Analyst for more
information or guidance.
The CEP must state the purpose(s) of the estimate and describe how the guidance in Section 4.2
of this Guide, the PAPPG, “2 CFR Part 200, Subpart E – Cost Principles,” and the GAO Cost
Estimating and Assessment Guide will be or has been implemented. Recipients must note any
departures from these NSF and GAO Guides and explain their rationale in the CEP. The CEP
should also state the schedule of specific tasks, due dates, roles and responsibilities, practices,
systems, and calculations used to develop the cost estimate. The CEP should describe the
expected cost estimating methodology, maturity, and, if applicable, accuracy range 1 at each
Stage or Phase (e.g., expert opinion, analogy, parametric, engineering build-up, historical data).
The CEP should also explain any ground rules, assumptions and exclusions that apply broadly to
the estimate, allowances, and other sensitive or significant factors or considerations, including
their rationale and any references. Recipients should also discuss the independent cost
estimates and reviews, if any, they are planning to validate the project estimate.
The CEP should be tailored to the Stage of the facility life cycle and address the most relevant
costs, from Development and Design through Construction, Operation, and Divestment. The
CEP should explain how the cost estimate may evolve over time. For example, the expected
level of funding needed for the Operations Stage should be initially identified by the Conceptual
Design Review. Operating cost estimates will be refined and updated throughout the design
and construction process as further discussed in the Concept of Operations Plan, developed as
part of the PEP described in Section 3.4 of this Guide. The CEP presented in an Operations
Proposal, whether submitted by the Recipient of the construction award or by a separate
entity, should be informed by appropriate excerpts from the Concept of Operations developed
in the PEP, or successor documents.
For example, via classification levels in AACE International Recommended Practice No.18R-97, Cost Estimate Classification
System – As Applied in Engineering, Procurement, and Construction for the Process Industries, Rev. November 29, 2011
1
Section Revision:
September 20, 2019
4.2.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
The CEP should explain how the “Cost Model Data Set” will meet the various needs of the
facility. The “Cost Model Data Set” is the cost data used as input to software tools and/or
project reports to organize, correlate, and calculate different management information. Figure
4.2.2-1 provides an example of a how a “Cost Model Data Set,” Work Breakdown Structure, and
a Recipient’s institutional accounting systems can be used as inputs in conjunction with
scheduling, earned value, and risk analysis tools to generate a variety of output reports for
project purposes. Sections 4.2.2.7, 4.2.3 and 4.2.4 of this Guide refer specifically to the work
breakdown structure, “Cost Model Data Set,” and “Cost Reports” blocks encircled with dashed
lines in Figure 4.2.2-1. The CEP is included as part of the PEP as described in Section 3.4 of this
Guide.
Section Revision:
September 20, 2019
4.2.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS), & The Large Facilities Office
(BFA-LFO)
Figure 4.2.2-1
Section Revision:
September 20, 2019
Sample Project Control Systems Relationship Diagram
4.2.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.2.2 Estimate Formats
The Recipient must be capable of providing the cost estimate in multiple formats and reports,
including but not limited to the following:
•
Reports based on a deliverable-based work breakdown structure (WBS) for construction
and a functional, activity, and/or deliverable based WBS for operations, as further
described in Sections 4.2.2.7, 4.2.3.3, and 4.2.4.3 below. These reports support project
management and execution and detailed cost analysis of sub-elements and are referred
to as Cost Books.
•
Reports based on the standard NSF budget category format 1, depicted in Figure 4.2.2-2
and Section 4.2.2.4 below, per NSF budget and budget justification guidance from the
PAPPG. This format supports cost analysis of NSF budget categories. For contracts, NSF
proposal requests may specify alternate formatting in lieu of the NSF budget categories.
The estimate is built-up from the individual WBS elements and sub-elements. See Section
4.2.2.7 for guidance on work breakdown structures. If the costs associated with each WBS
element are binned into the appropriate NSF budget categories, then both of the above
reporting formats can be readily produced. For example, costs can be coded with NSF budget
format letters (A through I per Figure 4.2.2-2) to populate rolled-up NSF budget format
summaries as well as the Cost Book organized by WBS. The estimate should allow for
mathematical checks of the proposal budget calculations and should contain actual formulas
that allow manipulations to check calculations (i.e., the model should not display just the
results of the application of formulas or be locked such that calculations cannot be verified in
real time).
The cognizant NSF PO and G/AO, or CO can be contacted with questions or for other specific
programmatic requirements.
Projects may choose to use broad, summarized budget categories for internal planning and reporting, but reports with the
detailed breakout into NSF budget categories must be supplied when requested. (Examples: a single combined “E-Travel”
category for internal use rather than “E-1 Domestic travel” and “E-2. Foreign travel” in Figure 4.2.2-2; a single category “Labor”
combining all NSF labor categories A through B-6.
1
Section Revision:
September 20, 2019
4.2.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.2-2
NSF Budget Categories Sample Format
A – Senior Personnel
B – Other Personnel
B.1 – Postdoctoral Scholars
B.2 – Other Professionals (Technicians, Programmers, Etc.)
B.3 – Graduate Students
B.4 – Undergraduate Students
B.5 – Secretarial – Clerical
B.6 – Other
C – Fringe Benefits
D – Equipment
E – Travel
E.1 – Domestic
E.2 – Foreign
F – Participant Support
F.1 – Stipends
F.2 – Travel
F.3 – Subsistence
F.4 – Other
G – Other Direct Costs
G.1 – Materials and Supplies
G.2 – Publication, Documentation, Dissemination
G.3 – Consultant Services
G.4 – Computer Services
G.5 – Subawards
G.6 – Other
H – Total Direct Costs
I – Indirect Costs
Section Revision:
September 20, 2019
4.2.2-5
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.2.3 Application of GAO Cost Guidance to Major Facilities
The MFG is intended to supplement not duplicate the GAO Cost Guide, PAPPG, and industry
good practices and standards. The best practices (twelve steps) of the GAO Cost Estimating and
Assessment Guide are highlighted below to help show how they can be applied or tailored to
NSF major facilities, including potential deviations, and how they should be integrated with NSF
processes. By following the GAO Cost Guide, Recipients should expect a better estimate and a
more efficient review by NSF, facilitating achievement of the science mission. NSF and
independent reviewers use these GAO criteria and other methods when analyzing Recipient
cost estimates to determine whether to make an award. Application of the GAO Schedule
Assessment Guide is discussed further in Section 4.3.
GAO Cost Guide’s Twelve Steps / Best Practices:
1. Define estimate’s purpose:
o The purpose must be clearly defined. There are typically two general purposes:
(1) to help managers evaluate affordability and performance against plans, as
well as the selection of alternative systems and solutions, including value
engineering and scope management, and (2) to support the budget and award
processes by providing estimates of the funding required.
o Defining the purpose helps clarify the intended use and package the estimate to
facilitate review by a range of audiences, including managers and independent
reviewers. Reviewers not familiar with the facility will need a standalone
document with both the appropriate high-level perspective and the detailed CEP,
BOE, and linkages via WBS so that someone unfamiliar with the program can
understand it, recreate it quickly with the same result, and be able to determine
if it meets the GAO’s twelve steps and four characteristics of a high-quality cost
estimate.
o Defining the purpose also helps determine its scope and level of detail, identify
appropriate performance measures for benchmarking progress, address the
benefits it intends to deliver, and link the estimate to NSF’s mission, goals, and
ideas.
o For additional descriptions and guidance on the purpose and context of the
estimate, including why it is developed and how NSF uses the estimate, see MFG
Sections 1.1, 2.3.1, 2.3.2, 2.3.3, 3.4, 4.2.1, 4.2.2.1, 4.2.3.1, 4.2.4.1, 6.2.1, 6.2.8.1,
and Figure 4.2.1-1 and Figure 4.2.2-1.
2. Develop an estimating plan: A CEP must be developed and address the details described
in Section 4.2.2.1.
3. Define program characteristics: Characteristics of the program being estimated must be
defined for construction projects per the Project Execution Plan in Section 3.4 and for
operations awards per the Proposal and Work Plan in Sections 2.5 and 3.5.
4. Determine estimating structure:
Section Revision:
September 20, 2019
4.2.2-6
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
o The estimate must be organized by both the WBS and NSF budget categories as
described further in Sections 4.2.2.2, 4.2.2.7, 4.2.3.3 and 4.2.4.3 and Figure
4.2.3-1 and Figure 4.2.4-1.
o The estimate structure must have clear traceability between WBS, CEP, and
BOEs, correctly roll-up to higher levels, and readily map between the WBS and
NSF Budget Categories.
5. Identify ground rules and assumptions (GR&As):
o The ground rules (a common set of agreed on estimating standards that provide
guidance and minimize conflicts in definitions) and assumptions (a set of
judgments about past, present, or future conditions) must be clearly defined and
documented in the CEP, as described in Section 4.2.2.1.
o The GR&As should be developed by estimators with input from experienced
program and technical personnel, based on information in the technical baseline
and WBS dictionary, vetted and approved by upper management, documented
to include the rationale behind the assumptions and backed up by historical
data.
o GR&As may be global, in which case they apply to the entire estimate and should
be clearly and consistently used throughout the. GR&As may also be programspecific or WBS element-specific, driven by the particular technical
requirements.
o The potential impacts from changing GR&As should be considered when
developing the sensitivity and risk analyses.
o For NSF major facilities, GR&As often include inflation, escalation, indirect rates,
travel, fringe benefits, schedule or budget constraints, acquisition strategy,
participation of other agencies or governments, level of technology maturity and
required research and development. GR&As also often define what is included
and excluded from the estimate, such as use of existing or multi-purpose
equipment and facilities.
6. Obtain data:
o The estimating methods, level of detail, accuracy range, availability of historical
and current cost data will evolve and improve through the design Phases and
Construction and Operations Stages. Current data should be routinely collected,
documented, and included in estimates.
o Data should be collected from multiple sources, normalized, and assessed for
convergence and sensitivity. Cost drivers, trends, and outliers should be explored
and carefully analyzed for reliability and relevance. Primary data sources,
obtained from the original source and usually traceable to an audited document,
should be used when possible. Backup data should be collected and used to help
identify cost drivers and cross-check results.
Section Revision:
September 20, 2019
4.2.2-7
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
o Recipients should carefully consider data sources and the applicability, potential
limitations, allowances, risks and uncertainty. This is especially true for NSF
major facilities where estimates often include research and development,
prototypes, university work, software and cyber-infrastructure, and unique,
complex, new and/or evolving technologies.
o The best estimating method should be chosen for each WBS element. The
following cost estimating methodologies should be used, in order of preference,
if the data exists: (1) Actual/historical data for the systems or operations being
estimated; (2) Detailed engineering build-up; (3) Parametric data with
adjustments to reflect differences (e.g., technical, size, weight, quantity,
location, schedule); (4) Analogous data with adjustments to reflect differences;
(5) Expert opinion, only if a secondary methodology is used to substantiate.
o Data sources, content, time, units, calculations and results, explanations for
choosing a particular estimating method or reference, and circumstances
affecting the data should be clearly documented in the CEP and Cost Book BOE.
7. Develop point estimate and compare it to an independent cost estimate:
o Recipients are encouraged to obtain independent cost estimates (ICEs) and cost
estimate reviews to help validate and improve the quality of the estimate before
submitting proposals to NSF. Recipients should address this as part of the CEP, as
described in Section 4.2.2.1. Operations proposals do not typically warrant an
ICE since analogous historical costs are readily available or the basis of estimate
will typically not have the breadth and depth of technical and cost detail that is
expected for a construction award.
o As noted in Sections 4.2.1 and 4.2.2.1, NSF utilizes ICEs and independent cost
estimate reviews done by external panels and independent contractors or
agencies. An ICE is required prior to construction awards. An independent cost
estimate review of some type is required of operations proposals prior to initial
operations, renewal, and competition of awards. These ICEs and independent
cost estimate reviews are used to validate the Recipient estimates, negotiate
awards, check for compliance with GAO best practices and Uniform Guidance
Cost Principles, and inform the NSF cost analysis. Far in advance of reviews, the
NSF PO, G/AO or CO, LFO Liaison, and Cost Analyst determine the type, timing,
scope, and team required. Recipients should be prepared to support these
efforts, address any findings, and participate in reconciliations of proposals with
ICEs.
8. Conduct sensitivity analysis:
o Done to test the sensitivity of cost elements to changes in estimating input
values and key assumptions so that key cost drivers and the range of potential
costs can be identified, highlighted for Recipient management and NSF, and a
Section Revision:
September 20, 2019
4.2.2-8
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
strategy can be developed to deal with them. Sensitive elements are those
where small changes in variables can create the greatest changes in cost.
o Can be done rigorously and quantitatively by examining the effect of changing
one assumption, ground rule, or cost driver at a time while holding all other
variables constant to understand which variable most affects the cost estimate.
The changes should not be arbitrary or subjective (e.g., +/- %), but rather
determined by subject matter experts based on available data.
o Sensitivity analysis tries to isolate the effects of changing one variable at a time,
while risk or uncertainty analysis examines the effects of many variables
changing all at once. The results of the sensitivity analysis can therefore be used
to help identify and quantify risks that are then used in a probabilistic risk
assessment to develop the contingency budget and confidence level.
o The results of the sensitivity analysis can also inform decisions when analyzing
alternatives for design, acquisition, construction, operations, and maintenance.
Analyses can also drive actions to avoid, mitigate, transfer, or accept a risk.
o For operations estimates that may consist largely of level of effort work, a more
qualitative sensitivity review could be performed and justification provided that
there are no particularly sensitive elements and therefore little or no potential
impact.
o The major contributing variables within the highest percentage cost elements
are the key cost drivers that should be considered in the analysis. May be a
ground rule and assumption, especially those least understood or most at risk of
changing. For NSF major facilities, sensitive elements may include electricity,
fuel, major commodities, inflation specific to certain cost categories,
requirements changes, location, domestic versus foreign sources/procurements,
acquisition strategy.
9. Conduct risk and uncertainty analysis:
o Described further in Sections 4.2.5 and 6.2 below. The risk register data, basis of
estimate, assumptions, and detailed methodology used to calculate contingency
budgets must be documented and provided if contingency is requested. As
described in Section 3.4 for construction projects, this information is
documented in the Project Execution Plan components 4.6 Budget Contingency,
4.10 Schedule Contingency, and 6. Risk and Opportunity Management.
o For operations, also see Section 4.2.6. These analyses are not typically required
for operations awards unless a separate contingency budget is requested for
facility or instrumentation upgrades or replacement projects. However, a
summary of key operational risks, their potential cost impacts and mitigation
strategies may be beneficial to articulate as part of the proposal. These could
also be handled as part of the sensitivity analysis (Step 8).
10. Document the estimate: Described throughout Section 4.2.
Section Revision:
September 20, 2019
4.2.2-9
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
11. Present the estimate to management for approval: Described in Sections 2.3, 2.5, 3.4,
3.5, and 4.2.1.
12. Update the estimate to reflect actual costs and changes: Described throughout Section
4.2 and for EVM in Sections 3.4.1 and 6.8. Typically, not required for operations awards
though work plans and budgets may be adjusted annually to reflect actual work done
and updates to planned work.
4.2.2.4 Supplementary Guidance for NSF Budget Categories from the PAPPG
This section discusses types of additional detailed information typically needed by Recipients to
justify the estimates by the required NSF Budget Categories. This information is intended to
supplement the standard guidance for the NSF Budget Categories described in Chapter II.C.2.g
of the PAPPG and depicted in Figure 4.2.2-2. This guidance is not all inclusive nor is it required.
It is intended to clarify NSF expectations, assist Recipients, facilitate NSF review with fewer
iterative resubmissions, and prevent recurrent issues. For each NSF Budget Category, Recipients
should provide specific justifications to demonstrate costs are allowable, allocable, and
reasonable per the cost principles of 2 CFR §200, Subpart E, and realistic.
The PAPPG states that budget justifications must be no more than five pages per proposal.
However, most cooperative agreements for major facility projects (both construction and
operations) will require substantially more pages.
The following apply to the salary data listed below. All Personally Identifiable Information
should be removed from the documentation. If not already covered in the CEP, Recipients
should provide a salary escalation rate for multi-year proposals, which can include a component
for annual raises similar to Cost of Living Adjustments (COLAs) and other pay increases for
promotions within the position classification. Recipients should provide the rationale behind
the salary escalation rate. In some cases, NSF may provide a base escalation rate in the
solicitation for guidance, but Recipients should follow Section 4.2.2.6 of this Guide when
proposing rates. Recipients may contact their NSF PO, G/AO or CO, LFO Liaison, and/or Cost
Analyst for a “Master Labor Schedule” template spreadsheet that can be used to compile all
labor data for ease of estimating and justifying labor costs.
A – Senior Personnel
•
Recipients should provide verification of actual salaries paid for named senior
personnel. Salary rates should be based on actual costs per current rate paid by payroll
register, W-2s, or appointment letters. Recipients should note Academic Year (9-10
month) versus Calendar Year (12 month) appointments or time available to conduct
independent research if such appointments so provide. The Recipient should also
provide documentation to support reasonableness of the salary rate paid, such as salary
rate surveys, salary comparators, Human Resource Department analysis, or other
information.
Section Revision:
September 20, 2019
4.2.2-10
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
•
NSF has a policy which typically limits senior personnel to two months effort in any
given year for standard NSF-funded grants. However, most cooperative agreements for
major facility projects (both construction and operations) have senior personnel effort
well in excess of two months. Compensation in excess of two months, if anticipated,
should be disclosed in the proposal budget, and explained in the budget justification.
B.1 – Postdoctoral Scholars
•
Recipients should provide the average salary rate or rate range for postdoctoral
students at the organization in the field of science. Actual payroll data may not be
available as these may be to-be-hired positions.
B.2 – Other Professionals, Technicians, Programmer, Etc.
•
Since the NSF budget format poses this as a total number of individuals for a total
number of months, additional explanation is generally required to disaggregate the total
for cost analysis. The level of effort will likely need to be obtained by individual or by
position for salary calculations. Recipients should also provide a spreadsheet with the
budget justification that includes: name or position number, location, WBS, title, salary
rate and period, level of effort as a percentage or in person-months, and calculation of
amount for each award year.
•
Recipients should provide supporting documentation for the salary rates of the
technicians, programmers, and other professionals proposed. For these types of
positions, NSF recommends the use of Bureau of Labor Statistics (BLS) Standard
Occupation Classification Codes (SOC) by position title and referencing their positions to
BLS salary rates to establish reasonableness of proposed salary rates. The BLS data is
also available by region or city. Other salary rate survey data may be used, and larger
Recipient organizations may already have established salary ranges and qualification
bases established internally by their Human Resources Departments.
B.3 – Graduate Students
•
Recipients should provide the average salary rate or rate range for graduate students at
the organization in the field of science. Actual payroll data may not be available as these
may be to-be-hired positions.
B.4 – Undergraduate Students
•
Recipients should provide the average salary rate or rate range for undergraduate
students at the organization in the field of science. Actual payroll data may not be
available as these may be to-be-hired positions.
Section Revision:
September 20, 2019
4.2.2-11
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
B.5 – Secretarial – Clerical
•
Recipients should provide the average salary rate or rate range for secretarial clerical
personnel at the organization.
B.6 – Other Personnel
•
Generally, the same as B.2 above but special classifications could justify different
treatment.
C – Fringe Benefits
•
Most Recipient organizations utilize a single tier fringe benefit rate or fringe benefit rate
by class of employee. Occasionally these fringe benefit rates are approved in the
Negotiated Indirect Cost Rate Agreement (NICRA). In such cases, the Recipient can verify
the rate and provide a fringe benefit calculation (rates by class) for a sample project
year. These cases should be noted in the CEP.
•
Some organizations use an actual fringe benefit amount by class of employee. These
amounts vary greatly by employee salary levels. While some fringe benefit costs are
based on a percentage of salaries (such as statutory withholding or contributions to
retirement and Paid Time Off (PTO)), other fringe benefits such as medical insurance
may be a lump sum amount and are not directly tied to salary paid. The Recipient should
provide an estimate of each fringe benefit provided as a percentage to salaries paid
along with a description of the fringe benefit provided as a means to gauge the
reasonableness of the fringe benefit package provided.
•
In both cases, Recipients should explain differences in the treatment of PTO. Some
organizations include this as a component of the fringe benefit rate and others include
the full cost of salary (including PTO) in the salaries as budgeted.
D – Equipment
•
There can be equipment expenses or materials and supplies that individually are less
than the threshold but taken together exceed the equipment threshold, particularly
when installed or fabricated by a Recipient.
•
Recipients should list each item of equipment individually and include a description,
estimated cost, and justification of need. Recipients should typically provide vendor or
catalogue quotes for each item of equipment when available. These quotes should be
indexed and numbered to the equipment items proposed. For unique scientific
instrumentation or other equipment components where vendor quotes are not readily
available, a clear basis of estimate should be provided.
Section Revision:
September 20, 2019
4.2.2-12
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
E – Travel
•
Generally, the cost estimate should be detailed by individual destinations, type of
transportation (airfare or mileage), per diem (lodging and meals) and other associated
expenses. The relation of the travel to the proposed activities should also be included.
For renewal projects, historical costs can be considered as a means of assessing the
reasonableness of travel costs. Where there are large numbers of trips and the actual
locations may not be known in advance, then cost estimating relationships (e.g., average
of $1,500 per traveler per trip) may be used.
F – Participant Support
•
Justification should include the number of participants, stipend amount, travel cost
estimate, and subsistence costs per participant. Recipients should also provide the
number of days or weeks of the training activities to provide a basis for determining
reasonableness of the proposed payments.
•
Participant support costs may not be used for personnel at the Recipient institution.
Note: All contracts for procurements or services needed to carry out the project must be listed
in G.1 – Materials and Supplies,2,3,4, to align with the type of budget activity or in G.6 – Other.
All contracts must follow 2 CFR § 200.317-326 including price and cost analysis, competition,
competition, contacting with women’s, small and minority businesses, and contract provisions.
For procurements by micro-purchase, i.e., purchase of supplies or services using simplified
acquisition procedures, the threshold amount for all awards is $10,000 based on the American
Innovation and Competitiveness Act. Contracts must not be listed in G.5 Subawards.
To assist Recipients in determining the difference between a subaward and a contract, please
refer to the “Subrecipient vs. Contractor Checklist,” developed by the Association of
Government Accountants.
G.1 – Materials and Supplies
•
An itemized listing is not necessary unless an item represents a substantial amount of
costs. Vendor or catalogue quotes, historical costs, or other cost estimating
relationships may be used to establish reasonableness of the cost estimate.
G.2 – Publication, Documentation, Dissemination
•
Recipients should provide an estimate of publication and dissemination costs.
G.3 – Consultant Services
•
For each consultant identified, the Recipient should provide justification that the
proposed rate of pay is reasonable.
Section Revision:
September 20, 2019
4.2.2-13
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
G.4 – Computer Services
•
Where it is established institutional policy to direct charge computer services, the
Recipient may justify and include such costs in the budget. Generally, such recharges
should be based on established internal institution usage rates. Recipients should
provide a supporting institutional statement or policy document and rates by units of
actual usage.
G.5 – Subawards 1
•
Recipients of cooperative agreements are expected to conduct a pre-award risk review
of the subawards to include both cost and price analysis and to identify risk as outlined
in the Uniform Guidance, 2 CFR § 200.331.
•
Recipients should provide NSF with their pre-award analysis of each of the proposed
subawards when submitting for approval of each subaward. Such Recipient pre-award
analysis should include a determination of Subaward risk. This should include an
assessment of financial capability and ensuring the Subrecipient is not on any Federal
Government “do not pay” listing. The Recipient should also have performed a price or
cost analysis of the Subrecipient’s proposed work to ensure the reasonableness of costs.
•
NSF reviews the Recipient’s documentation on each Subrecipient to ensure sufficient
rigor and detail was performed.
•
The Recipient must keep copies of the risk assessment performed, which should detail
any key risks identified and how those risks were mitigated and resolved, cost and price
analysis, and results of searches of the System for Award Management (SAM.gov), and
Federal Awardee Performance and Integrity Information System (FAPIIS).
G.6 – Other
•
Itemized Other costs per PAPPG II C.2.g (vi)(f), including the applicable budget
contingency, should be summed here and described separately in the Comments area of
the form.
•
Budget contingency, when applicable, should be presented as a part of the total amount
of Other Direct Costs under section G.6 on the standard NSF budget form. Budget
contingency budget estimates should be developed in accordance with Sections 4.2.5
and 6.2 of this Guide and should include all fully burdened contingency amounts. The
proposal should include adequate documentation on the basis of estimate for the
contingency amounts, indicating that they were developed in accordance with 4.2.5 and
6.2 and are supportable. Budget contingency and allocations of contingency will be
A subaward is for the purpose of carrying out a portion of a Federal award and creates a Federal assistance relationship with
the subrecipient. See 2 CFR § 200.92 Subaward. Characteristics which support the classification of a subrecipient versus
contractor can be found at 2 CFR § 200.330. See also PAPPG II C.2.g (vi)(e).
1
Section Revision:
September 20, 2019
4.2.2-14
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
called out in the Cooperative Support Agreement by the G/AO under the “Contingency”
section, based on information provided in the negotiated budget justification.
H – Total Direct Costs
•
The total amount of direct costs requested in the budget, to include Lines A through G,
must be entered on Line H.
I – Indirect Costs
•
When the Recipient has a Negotiated Indirect Cost Rate Agreement (NICRA) established
with a cognizant federal agency, the rate and base in that agreement should be used to
compute indirect costs. A copy of the NICRA should be included in the CEP.
•
When a Recipient does not have a NICRA, the Recipient should provide a calculation and
an indirect cost rate proposal. The Recipient should ensure that indirect costs are in
accordance with NSF policies in NSF’s Indirect Cost Rate Proposal Submission
Procedures. Recipients should provide a clear description of rates and application bases.
Recipients should also provide spreadsheet calculation of rate or rates by year clearly
showing exclusions such as sub-contracts greater than $25,000, equipment or capital
expenditures, and participant support. If a Recipient has different indirect cost rates
across NSF budget categories in Figure 4.2.2-2, these rates should be clearly identified
and justified. Any deviation to a Recipient’s normal rate should also be justified.
K – Fee
•
When the Recipient is proposing a Fee amount, it should be presented in line K. A Fee
can only be proposed when the solicitation allows for it. Fee will be called out separately
in the award terms and conditions and based on information provided in the negotiated
budget justification.
4.2.2.5 Fee
The payment of fee may be authorized for major facility construction and operations awards,
unless otherwise prohibited in specific circumstances by NSF. Fees will be evaluated for
reasonableness by the G/AO using a structured approach as prescribed by DACS. In part, the
G/AO may use information such as the negotiation objective set forth in the Decision Memo
and/or other cost negotiation memorandum as the basis for selecting the fee type and
determining the fee amount. The amount of fee will not exceed the statutory limitations
pertaining to cost contracts set forth at 41 U.S.C. 3905, notwithstanding that the fee is provided
through a cooperative agreement. NSF will also provide guidelines for Recipients that receive
fee to encourage the utmost discretion and appropriate consideration in the use of fee, to
include examples of inappropriate uses of fee (e.g., including but not limited to not using fee on
alcoholic beverages or lobbying as set forth at 2 CFR § 200.450 and 48 CFR 31.205-22). NSF will
reserve the authority to review a Recipient’s actual use of fee. Accordingly, Recipients must
Section Revision:
September 20, 2019
4.2.2-15
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
separately track and account for uses of fee provided under NSF awards. The terms and
conditions of each award will specify the fee arrangement. NSF will consider reductions in
future fee if a Recipient’s actual use of fee is in contravention with the guidelines on
inappropriate uses.
4.2.2.6 Escalation
Recipients are not limited to using only broad and publicly available economic assumptions
when doing cost estimates. NSF encourages organizations to use escalation information
appropriate for the known situations or a particular industry as long as they can be justified. For
example, specialized data may be available from the Department of Energy, Department of
Defense, BLS, industry metrics, and/or historic experience with similar items. The justification
for all escalation assumptions and inflation factors (including use of standard OMB inflation
factors) should be included in the CEP and used consistently throughout the BOE.
Estimates should preferably be based on current information but may include appropriate
escalation calculations and justifications to support the planned execution timeframe.
Escalation for raw materials and equipment in technological projects often runs higher than
broad measures of inflation (e.g., the consumer price index) due to inelasticity in pricing (i.e.,
there are few or no substitutes available in the marketplace and demand remains constant).
Recipients should consider cost sensitivity and establishing risks and associated contingency for
future price variability and developing mitigating actions. (See Sections 4.2.2.3 and 4.2.6)
4.2.2.7 Work Breakdown Structure (WBS)
As described in the GAO Cost Estimating and Assessment Guide, a WBS is the essential
cornerstone of every project because it defines in detail the work necessary to accomplish a
project’s objectives. For construction, the WBS is a deliverable-based and hierarchical
framework structure that provides specific, manageable, and schedulable tasks and may be
composed of products, material, equipment, services, data, and support facilities that the
project should yield. An operational WBS may be functional, activity, and/or deliverable based,
depending upon the type of work. Level of Effort tasks should be confined to only those tasks
that are not easily definable as deliverables. The WBS provides a consistent framework for
planning, estimating costs, developing schedules, identifying resources, and determining where
risks may occur. The WBS is a valuable communication tool and provides the means for
measuring program status, e.g., via using Earned Value Management for construction. WBSs are
developed at varying levels of detail but should include at least three levels. Generally, the
number of levels employed should be sufficient to identify and measure progress towards
achieving deliverables, assign responsibility, and enable effective management and reporting.
The number of decomposition levels varies depending on the project’s size and complexity,
technical maturity, organizational constraints, acquisition and construction strategies, and
management’s assessment of need.
Section Revision:
September 20, 2019
4.2.2-16
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.2 Elements of Both Construction and Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Guidance and examples of common WBS elements can be adapted from GAO and other
guidance and tailored for NSF projects, as depicted in Figure 4.2.3-1 and Figure 4.2.4-1. The
benefits of developing standardized or similar WBSs across the portfolio of facilities within an
organization include:
•
Consistent, clear, and familiar reporting structures and organizational relationships
•
Improved efficiency and effectiveness of NSF cost analyses
•
Better characterization of project schedule, scope, and cost
•
Ease of judging completeness and reasonableness
•
Better collection and sharing of data and analysis methods across multiple contractors
and projects to support future cost estimates
•
Better cost tracking over time, and identification of major cost drivers and systemic
problems across contractors and projects
Section Revision:
September 20, 2019
4.2.2-17
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.3 Additional Guidance for Construction Estimates
4.2.3.1 Purpose and Process
As discussed in Sections 4.2.1 and 4.2.2 above, NSF utilizes internal staff, outside experts, and
expert panels at the Conceptual Design, Preliminary Design and Final Design Reviews and
during the Construction Stage to assure that proposed construction cost estimates and budgets
meet expectations, incorporate relevant GAO Cost and Schedule Guide best practices, and are
allowable, allocable, reasonable, and realistic. Cost Estimating Plans and Cost Books should be
updated as necessary during each of the Phases in preparation for the Reviews. NSF documents
all the cost analysis work, technical reviews, audits, etc. for cost analysis as part of its oversight
and assurance roles.
The construction PDR estimate and subsequent NSF analysis must be sufficient to give NSF
confidence in the estimated Total Project Cost (TPC) that advances for National Science Board
authorization and potential inclusion in a future budget request. The FDR estimate and analysis
must be sufficient to give NSF confidence in constructing and commissioning the facility within
the TPC.
4.2.3.2 Construction Cost Book – Introduction and Executive Summary
Construction Cost Books are necessary at the CDR, PDR, and FDR, at minimum, to provide a
comprehensive, consolidated estimate of construction costs.
The Project Execution Plan described in Section 3.4 of this Guide includes a Construction Cost
Book (PEP-4.7) as one component of the overall Construction Project Definition. The Cost
Estimating Plan and Construction Cost Book provide assumptions and detailed information
forming the Basis of Estimate. The following additional high-level information should be
provided as an overview and executive summary (PEP-4.5) to assist with the review process
described in Section 2 of this Guide. Recipients should consult with the PO and G/AO or CO as
necessary to identify any other specific cost reports and content required to support the
review.
•
Overall high-level cost summary charts, tables, profiles, and reports; depicting total and
annual costs; reported both by WBS and in NSF budget format; providing Base Year and
Then Year costs.
•
A comparison of the current total project cost to past estimates and an explanation of
any major changes, including impacts to scope or design.
•
Explanation of how project costs by WBS map to the NSF budget format, including
detailed traceability or crosswalk matrix, described further below.
•
Other reports, as needed, e.g., costs by resource types (subcontract, labor, materials,
travel), cost profiles (total, labor, non-labor, by WBS sub-element), personnel profiles
(Full-time-equivalents by WBS sub-element).
Section Revision:
December 14, 2018
4.2.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.3.3 Construction Cost Book – Format
Major facility construction projects must employ a deliverable-based and hierarchical WBS that
provides specific, manageable and schedulable tasks and may be composed of products,
material, equipment, services, data, and support facilities that the project should yield. Level of
effort tasks should be minimized for optimizing tracking of spending against budget and
accomplishments against plan in the project Earned Value Management reports.
Examples of potential components of a WBS, common to many NSF plans for construction of
major facilities, are listed in Figure 4.2.3-1 and further described below. The intent is to provide
a standard format to the extent feasible with the vast array of different facility types while
noting that additions and/or alterations to this list are likely, due to the unique nature of each
specific facility.
A basic description of each WBS is as follows:
1.0 Project Administration and Management Office – Include activities related to the
management and administration of the project. This includes quality assurance and
safety, reliability, document control, cost/schedule reporting and control systems, and
configuration management.
2.0
Facility Infrastructure and Civil Construction – Includes the design, procurement,
construction, and integration, of the supporting infrastructure. One example is a
telescope and site construction, consisting of the facility enclosure, dome, and
telescope mount.
3.0
Scientific Equipment and Instrumentation – Includes unique and specialized scientific
equipment. For example, field sensors and gages.
4.0
Computers and Cyber-Infrastructure – Includes hardware and software needed to
operate the system and collect and analyze data.
5.0
Systems Integration, Testing, and Commissioning – Includes the overall systems
infrastructure and personnel needed to integrate other WBS elements to ensure they
work correctly together for testing, commissioning, training, and operations.
Section Revision:
December 14, 2018
4.2.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.3-1
Construction WBS and Cost Book Sample Format
1.0 Project Administration and Management Office
1.1 Project Management Office
1.2 Site Office
1.3 Science Office
1.4 Education and Public Outreach
1.5 Safety and Environmental Assurance
2.0 Facility Infrastructure and Civil Construction
2.1 Sub-element X
2.2 Sub-element Y
2.3 Sub-element Z
3.0 Scientific Equipment and Instrumentation
3.1 Subcomponent X
3.2 Subcomponent Y
3.3 Subcomponent Z
4.0 Computers and Cyber-Infrastructure
4.1 Data Infrastructure
4.2 Data Products
5.0 Systems Integration, Testing, and Commissioning
5.1 Common Utilities and Support Equipment
5.2 Early System Assembly, Integration, and Testing
5.3 Acceptance Testing
5.4 Training
5.5 Science Verification
4.2.3.4 Construction Cost Book – Detail
This section discusses additional detailed information needed for a high-quality Recipient cost
estimate and NSF cost analysis. This information is intended to supplement the standard GAO
best practices, grant guidance in the PAPPG, and industry standards and good practices 1. The
guidance should improve project execution, clarify NSF expectations, assist Recipients, facilitate
Examples: AACE International Recommended Practice No. 18R-97, COST ESTIMATE CLASSIFICATION SYSTEM – AS APPLIED IN
ENGINEERING, PROCUREMENT, AND CONSTRUCTIONS FOR THE PROCESS INDUSTRIES; and AACE International Recommended
Practice No. 34R-05, BASIS OF ESTIMATE
1
Section Revision:
December 14, 2018
4.2.3-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
NSF review with fewer iterative resubmissions, and prevent recurrent issues. It is understood
that this information will become further refined as the Design Stages advance.
Presentation and Linkages
•
Individual WBS element costs must have a sound, fully justified and documented, and
sufficiently detailed Basis of Estimate. Figure 4.2.3-2 below provides an example
Construction Cost Book Sheet depicting the format and content typically needed to
consolidate the “Cost Model Data Set” and to provide the appropriate level of detail and
BOE. This sheet includes the following information:
o WBS and activity codes and descriptions, per the WBS Dictionary, to index the
cost estimate to a specific deliverable
o Statement of Work describing the scope
o Estimator Name and Date of Estimate
o Resource Descriptions
o Cost Basis Codes describing the estimate methodology (e.g., expert opinion,
analogy, parametric, engineering build-up, historical data)
o Direct Costs with Units and Hours
o Associated Fringe and Indirect Costs
o NSF Budget Category Code corresponding to the budget categories depicted in
Figure 4.2.2-2 and Section 4.2.2.4 above, to allow mapping between WBS subelements in Construction Cost Book and NSF Budget Categories on NSF Budget
Forms
o Basis of Estimate source data, with breakout of sub-elements, typically including
direct input from technical experts in that area with calculations using material
and labor quantities and unit prices, with clear assumptions and sources
referenced
o Associated risk, uncertainty, sensitivity, or contingency information, if any.
•
Estimates must have clear traceability, including the following, as appropriate, for CDR,
PDR, FDR, and Construction:
o The total estimated cost should correlate to current drawings, specifications, and
schedules.
o Lower levels of the WBS must correctly roll-up to the higher levels, and the
application of rates and factors must be consistent with the Cost Estimating Plan,
basis of estimate, supporting rate agreements, and Recipient accounting
practices.
•
WBS sub-element costs should be readily mapped to NSF Budget Categories depicted in
Figure 4.2.2-2 and Section 4.2.2.4 above; for example:
o If each cost element on the Figure 4.2.3-2 Cost Book Sheet is assigned an NSF
Budget Category Code (e.g., “A for Senior Personnel,” “E1 for Domestic Travel,”
Section Revision:
December 14, 2018
4.2.3-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
“G4 for Computer Services,” “I for Indirect Costs”), then the WBS elements can
readily be compiled into standard NSF Budget Forms, which depict total cost
types across all WBS elements (e.g., all personnel, equipment, travel, indirect, or
computer services costs across all WBS elements)
o If databases are sufficiently detailed, documented, and traceable, then
automatic sorting and summarizing of costs will be facilitated for various
purposes and for different reporting formats.
•
Cost estimates may be directly linked to scheduling tools, to allow automatic cost
updates with schedule changes.
BOE Refinement Process
•
Because of the hierarchical nature of the WBS, it is possible, over time, to refine the
level of detail at which the project scope, schedule, and task-based costs are captured.
Throughout the Design Stage the task and cost fidelity will increase, and eventually,
during the construction of the Project, the plans will be fully detailed. As the project
moves through the phases, detailed engineering build-up estimates using current
quotes and prices should be collected, such that the proportion of estimated costs
based on expert opinion, analogy, or parametric estimates is reduced. As the project
finalizes plans for the start of construction the basis of estimate should include more
vendor catalogue, quoted, or proposed contract prices.
•
Direct labor rates, quantities, and skills mix should be justified, including information
from subawards.
•
If using consultants and contractors, Recipients should carefully justify substantial
consulting costs, including the type of work performed, quantity of time proposed, and
its cost, as compared to potentially less expensive current employee labor to accomplish
the proposed work.
•
Cost estimates should include adequate funding for project management, including the
use of appropriate project management tools such as project management control
software and associated staff support.
•
The major facility construction cost estimate may include commissioning (i.e.,
integration, testing, acceptance, and operational readiness), including funding for staff
to perform these activities and train the operations personnel. Roles change as a project
progresses from construction through commissioning and eventually to operations; time
and staffing requirements need to be carefully calculated in advance, with clear
demarcation between construction funded scope and operations scope as discussed in
Section 3.4.1, Components of a Project Execution Plan; Commissioning (Component 15
of the PEP).
•
Where partnerships are involved, monetary contributions to acquisition and eventual
operations and usage should be timely, sufficient, and well documented in the PEP and
IMP.
Section Revision:
December 14, 2018
4.2.3-5
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
•
Cyber-infrastructure technical requirements and costs (both initial cost and continuing
costs of hardware, software, maintenance, upgrades and operations) should be carefully
considered and periodically validated. Rapid advances in computing may require
upgrades as often as every 3 to 5 years.
•
Cost of evolving technologies should be considered as part of budget development and
through acquisition planning. For example, it may be appropriate to include higher
allowances in the BOE, or higher impacts as part of the budget contingency
development, and plan for procurement late in the Construction Stage.
Section Revision:
December 14, 2018
4.2.3-6
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.3 Additional Guidance for Construction Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS)
Figure 4.2.3-2
Section Revision:
December 14, 2018
Construction Cost Book Sheet Sample Format
4.2.3-7
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.4 Additional Guidance for Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.4 Additional Guidance for Operations Estimates
4.2.4.1 Purpose and Process
In addition to the specialized scientific expertise required for operations, award solicitations can
also include expectations for estimating budgets, business systems, and operational and
financial reports. As discussed in the NSF PAPPG, individual solicitations, and Sections 2.5, 4.2.1,
and 4.2.2, these systems and reports help ensure the science mission can be met in a cost
effective way.
NSF utilizes internal staff, outside experts, and panel reviews to ensure cost estimates and
budgets meet expectations, incorporate relevant GAO Cost Guide best practices, and are
allowable, allocable, and reasonable. The NSF Cost Analysis document is used as an award
decision tool that captures all the cost analysis work, technical reviews, audits, etc. for cost
analysis as part of its oversight and assurance roles. It is incumbent on NSF to plan and budget
for effective research and educational use of facilities, as well as the costs to operate and
maintain the facility long term. It is incumbent upon the Recipient to ensure their operations
proposal is well-documented, accurate, comprehensive, and credible.
Operating budgets should include, when appropriate, resources to provide a continuing
program of advanced research and development (R&D) that will enable a facility to evolve its
scientific program and best meet the needs of the research community. Funding for these kinds
of up-grades may also come from separate equipment and/or instrumentation programs within
the Directorate or Division.
4.2.4.2 Operations Awards Proposals – Overview
In addition to the guidance for Annual Work Plans described in Section 2.5.1 of this Guide and
Proposals for Operations and Management described in Section 3.5, additional information may
be requested by the PO or via the operations and management award solicitation. Recipients
should consult with the PO and G/AO or CO as necessary to identify any other specific cost
reports and content required to support the review.
•
Periodic plans that may include an executive summary, narrative overview, strategic and
annual objectives correlated to NSF mission needs, and an annual operating budget
focusing on any significant changes from previous plans. Plans may also include
expected scope, milestones, outcomes and impacts, developments, challenges and
opportunities, as necessary.
•
Explanation of how program costs within functional areas are coded or otherwise
related to the NSF Budget Categories depicted in Figure 4.2.2-2 and Section 4.2.2.4
above.
Other reports, such as annual cost by resource types (subcontract, labor, materials,
travel), cost profiles (total, labor, non-labor, by sub-element), and personnel profiles
(Full-time-equivalents by sub-element).
•
Section Revision:
September 20, 2019
4.2.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.4 Additional Guidance for Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.4.3 Operations Awards Proposals – Format
As discussed in Section 4.2.2.7, there are numerous benefits of standardizing the framework for
accomplishing operational goals with a Work Breakdown Structure (WBS). An operational WBS
may be functional, activity, and/or deliverable based, depending upon the type of work, but the
amount of scope assigned to Level of Effort tasks should be minimized for better tracking of
spending against budget and tracking of accomplishments against plan. An example of a
hierarchical WBS for an operations award is provided in Figure 4.2.4-1 below. The intent is to
provide a standard format to the extent feasible with the vast array of different facility types
while noting that additions and/or alterations to this list are likely, due to the unique nature of
each specific facility. The level of detail contained in the cost reports may vary depending on
Programmatic management requirements and cost analysis effort.
Section Revision:
September 20, 2019
4.2.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.4 Additional Guidance for Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.4-1
Operations WBS and Budget Sample Format
1.0 Project Director, Management, and Administration Office
1.1 Director’s Office
1.2 Project Management Office
1.3 Site Office
1.4 Education and Public Outreach
1.5 Safety and Environmental Assurance
1.6 Administrative Services
2.0 Science Operations
2.1 Research Planning
2.2 Experimental and Operations Support
2.3 Data Analysis
2.4 Calibrations and Data Quality
2.5 Special Projects
3.0 Significant/Important Infrastructure Modernization, Overhaul, Upgrade, Replacement,
Expansion
3.1 Equipment
3.2 Facilities/Infrastructure
3.3 Computer Systems, Instrumentation
4.0 Facility and Equipment Operations, Maintenance, Engineering, and Support Services
4.1 Operations
4.1.1 Scheduling
4.1.2 Operating
4.1.3 Testing
4.2 Maintenance
4.2.1 Corrective Maintenance
4.2.2 Preventive Maintenance
4.3 Utilities
4.3.1 Energy (e.g., electricity, natural gas, central heating, central cooling)
4.3.2 Information Technology, Communications, Cyber-Security
4.3.3 Security
4.3.4 Water
4.4 Other/General Support Services
5.0 Contingency (If justified, and supported by appropriate risk analysis and management)
Section Revision:
September 20, 2019
4.2.4-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.4 Additional Guidance for Operations Estimates
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.4.4 Operations Awards Proposals – Detail
This section discusses additional detailed information, as follows, typically needed for a highquality Recipient estimate and NSF cost analysis. This information is intended to supplement
the standard GAO best practices and guidance in the PAPPG. The guidance should improve
execution, clarify NSF expectations, assist Recipients, facilitate NSF review with fewer iterative
resubmissions, and prevent recurrent issues. For existing awards, the Recipient should consult
with the PO.
•
When power costs are significant and volatile, a strategy for dealing with price
fluctuation should be developed as part of the operations plan. Other examples of items
that may require separate consideration are expendables – such as cryogens, gases and
spare parts – and ancillary equipment such as refrigerators and IT equipment.
•
Separate funding sources and revenue streams (e.g., visitor center fees) should be
clearly delineated.
•
Education and Public Outreach costs should be explicitly identified and explained.
•
Multiyear budgets should take inflation into account, using factors discussed in Section
4.2.2.6 above.
Contingency, if requested, must be in compliance with Section 4.2.6 of this Guide.
Section Revision:
September 20, 2019
4.2.4-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.5 Risk Planning for the Construction Stage
4.2.5.1 Implementation of NSF’s No Cost Overrun Policy
The intent of NSF’s “No Cost Overrun” policy (see Section 1.4 of this Guide) is to instill diligence
and rigor in establishing the TPC and giving NSF a strong oversight position. Mechanisms for
offsetting potential cost increases are described herein and include, in order of precedence and
assuming appropriate use in accordance with NSF policy and practice:
1. Re-planning
2. Use of contingency
3. Use of management reserve (if authorized)
4. De-scoping if science/cost trade study determines that science loss is acceptable
5. Request NSB authorization to increase TPC if science loss due to available de-scope
options is not acceptable
NSF uses the following practices to implement the five mechanisms above:
•
“Management reserve” is not allowable in the Recipient’s portion of the risk-adjusted
Total Project Cost (TPC) estimate; only “contingency.” Management Reserve (if
authorized) is held by NSF.
•
Directorates must be responsible for the first 10% of cost overruns which exceed the
Board authorized TPC, or some portion of the NSF-held management reserve (if needed
and included in the authorized TPC) as determined by the Director.
•
At the Preliminary Design Review (PDR), projects must have a time-phased, prioritized
de-scoping options that equates to at least 10% of the baseline scope budget.
•
In support of NSF’s “No Cost Overrun” policy, projects must use a confidence level for
contingency estimates between 70 and 90 percent (under a probabilistic approach)
based on the particulars of the project and the inherent ability to de-scope.
•
NSF will hold budget contingency through project completion, in an amount up to 100%
of the total NSF-approved contingency budget, until it can be justified for obligation.
•
Although the initial TPC becomes public (i.e. through the budget request) after PDR, the
TPC under the “No Cost Overrun” policy is set at award (post-FDR) to allow for
refinement during the Final Design Phase.
Engagement with the National Science Board (NSB) on any cost overruns is based, in
part, on their delegation order for award supplements.
•
If there is reason to believe that mechanism #5 above will be implemented and the TPC will
increase by 10% or more, the Sponsoring Organization shall immediately notify the Chief Officer
for Research Facilities (CORF). In accordance with statute (see Section 1.4.7 of this Guide), NSF
is required to notify Congress within 30 days in writing when there is reasonable cause to
believe that the TPC will increase by 10% or more.
Section Revision:
December 15, 2020
4.2.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.5.2 Introduction to Budget Contingency
Budget contingency is a critical component of the comprehensive planning and execution of the
construction of major research facilities. This document describes the policies and procedures
concerning the planning, use, and oversight of budget contingency in the construction of
facilities fully funded by NSF and to the NSF-funded component of the scope when NSF partners
with other entities. It also describes the NSF’s process for assessing the sufficiency of
contingency, evaluating the effectiveness of management plans used for administration of
contingency, and NSF’s oversight role in the use of contingency funds.
For all assistance awards with academic institutions and non-profit organizations, contingency
is held by the Recipient in accordance with the Uniform Guidance (§ 200.433).
Federal Acquisition Regulation (FAR) governs the planning, use and oversight of contingency for
contracts with commercial organizations. Regardless of where contingency is held, the
requirement for a well substantiated risk assessment and contingency estimate, as well as a
robust oversight and administration is essential. Estimating contingency and managing risk is an
integral part of the project planning and execution process. NSF positions on contingency,
management reserve and de-scoping must be considered by the Program and the Recipient as
part of that process. Although strategies for other types of contingency are mentioned here,
this document is only intended to address management of the budget contingency.
The definition of contingency varies widely among project management practitioners and
federal agencies. For NSF, 1 budget contingency covers the “known unknowns” and is used to
mitigate identified cost or schedule risks as described in the Project Execution Plan 2 (PEP). The
estimated risk-adjusted TPC, as defined in Section 9 of this Guide, is developed in accordance
with the GAO Cost Estimating and Assessment Guide, 3 as explained elsewhere in this Guide.
OMB’s cost principles in the Uniform Guidance address budget contingency, and define it as:
… that part of a budget estimate of future costs (typically of large construction projects, IT
systems, or other items as approved by the Federal awarding agency) which is associated
with possible events or conditions arising from causes the precise outcome of which is
indeterminable at the time of estimate, and that experience shows will likely result, in
aggregate, in additional costs for the approved activity or project. Amounts for major
project scope changes, unforeseen risks, or extraordinary events may not be included.
NSF terminology aligns with that of AACE International, the Association for the Advancement of Cost Engineering, and of the
Project Management Institute’s Project Management Body of Knowledge (PMBOK Guide). See Section 6.2.3.2 for NSF
definitions of contingency and management reserve.
1
See Section 3.4 for details regarding the PEP. Note that the PMBOK Guide refers to “Project Management Plan” rather than
PEP, but the NSF definition of PEP is equivalent.
2
Note that the NSF definitions and treatment of contingency and management reserves differ from those used in the GAO Cost
Estimating and Assessment Guide.
3
Section Revision:
December 15, 2020
4.2.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
In contrast, “Management Reserve” 1 is often used by industry and other organizations to cover
the unforeseen risks, or the “unknown unknowns.” Except in rare circumstances, NSF does not
normally hold a management reserve for a specific project as part of the TPC. As a result, the
Directorate is responsible for the first 10% of costs which exceed the authorized TPC. To
mitigate this risk, the project’s prioritized and time-phased de-scoping plan should equal at
least 10% of the baseline budget when established at PDR. The ability to de-scope varies widely
by project and the impacts on the eventual scientific capabilities of the facility will also vary.
The scope contingency should be well considered and strive to minimize negative impacts. The
Directorate may also choose to cover the cost overrun from programmatic funding (and
increase the TPC) in lieu of de-scoping if it deems the science-support capabilities of the facility
would be too severely impacted. 2 See Section 2.4.1 of this Guide for required approvals.
The PEP describes a construction project’s scope, budget, schedule, and identified risks. It also
articulates the project’s plans for accomplishing the intended scope while satisfying the
constraints of budget and schedule and managing those risks. An essential component of the
PEP is the Risk Management Plan (RMP), which describes the project’s procedures for risk
identification, analysis, monitoring, and handling (including de-scoping if required) so that the
project has a high likelihood of being accomplished within the total available budget. Budget
contingency is only one tool used to control project risk. The RMP will also include methods and
tools to manage scope contingency, schedule contingency, and provide robust risk handling and
monitoring processes. Refer to Section 6.2, Risk Management Guidelines, for additional
information.
The development of budget contingency entails estimating the future potential impacts of
identified possible adverse events to the project (i.e., risks) if those events are ultimately
realized. In accordance with the Uniform Guidance, NSF requires the use of widely accepted risk
management practices (including parametric and probabilistic methods depending on project
maturity) to estimate a range or distribution of contingency. An appropriate value is then
selected from that range that will enable the project to successfully complete the required
scope within the TPC that is sent forward for National Science Board (NSB) authorization. In
support of NSF’s “No Cost Overrun” policy, confidence levels must be in the 70-90%3 range
following PDR depending on the nature of the project; including the ability to de-scope. This
applies even for higher risk projects. The resulting TPC estimate, including estimated
contingency required, will ultimately factor into NSF’s decision on whether or not to proceed
The GAO Cost Estimating and Assessment Guide uses the term “management reserve” for funds held for mitigation of “known
unknowns” whereas NSF uses the term “contingency.” For GAO, management reserves are included in the budget baseline and
are managed at the contractor level. The value of the contract includes these known unknowns in the budget base, and the
contractor decides how much money to set aside.
2 Directorates are able to do this as a result of NSF’s “transfer authority” which is dependent on continued inclusion in the
appropriation act. The language may require that congressional appropriation committees be notified in advance of any
reprogramming. Directorates should consult with the Budget Office during the decision-making process.
1
The GAO Cost Estimating and Assessment Guide states that confidence levels of 70 to 80 percent are used for high-risk
projects, particularly with projects having higher design complexity and technology uncertainty as with NSF-funded facilities.
3
Section Revision:
December 15, 2020
4.2.5-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
with the project. This policy position is in no way intended to discourage the construction of
cutting-edge, high risk facilities needed to advance scientific understanding. It is intended to
give a high degree of confidence that the project will come in on budget and clearly articulate
the level of risk involved so that sound decisions can be made. Following construction start, if
subsequent analysis shows that confidence is declining and the Board-authorized TPC will be
exceeded, NSF requires that a reduction in scope be considered as a strategy to bring the costs
back in line with the budget.
Since development of contingency is statistically-based, there is a chance that not every risk will
be realized at its maximum impact. Therefore, it is possible that contingency dollars will remain
at the end of the project. Even if contingency is allocated, the Project may come in under
budget for other reasons. Once project objectives are met and the project completed, any
residual funds must be de-obligated and returned to NSF at which time NSF will request
possible re-allocation of those dollars to other agency priorities. Awarded contingency shall be
held by the Recipient until project completion, but no later. Budget, Finance and Award
Management (BFA), the Large Facilities Office (LFO), and the Program Office will conduct a
project close-out with the Recipient in accordance with NSF practice and as described in Section
2.4.2 of this Guide.
Major strategies used by NSF to ensure accountability in the management of contingency
budgets include:
1. Contingency budgets are developed in accordance with widely accepted standards for
risk assessment and planning. Contingency budget, scope, and schedule are similarly
derived from probabilistic, bottom-up assessments of the entire project scope.
2. Contingency budgets are evaluated for reasonableness by NSF through use of expert
review panels convened by the Program that examine the BOE and methodology and
compare the cumulative contingency amounts with historical experience on similar
projects. This happens at each phase of the project (Conceptual, Preliminary, and Final
Design) at increasing levels of refinement. Other divisions within NSF, and potentially
contracted experts, will also evaluate the contingency estimate as part of the total
project cost assessment as it moves through these phases.
3. NSF will obligate and allocate contingency based on need and performance of the
Recipient. The overall status of remaining contingency, future liens on contingency, and
all allocations and returns of contingency funds (as risks are realized or retired) are
reported on a periodic basis as specified in the award instrument. Balances will be
measured against the total NSF-approved contingency budget and the allocated
contingency to date. This is part of the standard project reporting and requires archiving
in the permanent electronic record used by NSF (FastLane/eJacket).
4. Management and use of contingency is documented separately through the
configuration and change control process and must reference the associated Work
Breakdown Structure (WBS) elements and/or the previously identified Risk. The Earned
Section Revision:
December 15, 2020
4.2.5-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Value Management (EVM) framework for financial status reporting will eventually
reflect movement of contingency into the Performance Measurement Baseline (PMB)
budget (increase or decrease in Budget at Completion; BAC). Although traceable as
allocations or returns to the contingency budget, contingency dollars become part of
the PMB and are no longer separately identifiable as contingency once incorporated.
5. All project expenditures must be used only for scope as defined by the elements of the
NSF approved project definition, and all are subject to financial audit.
6. Management of contingency is described in the Configuration and/or Contingency
Management Plan (PEP-6.3). In this plan, thresholds are established (based on the
nature of the project) on who has the authority to approve the use of contingency.
These thresholds are also documented in the award instrument. Below the thresholds,
the Recipient has authority to manage and allocate contingency budget to specific inscope elements of the project WBS following the Configuration Change Control Process.
Above these thresholds, approvals from NSF are required, with the level of approval
corresponding to the magnitude of the proposed change.
7. Financial controls prevent the cumulative Recipient cash draws from exceeding the
obligated spending authority in NSF’s financial system.
4.2.5.3 Contingency Planning and Assessment during Conceptual Design
A budget estimate, like the measurement of a physical quantity, has a value and an uncertainty
dependent on where the project is in the design process. The uncertainty in the budget
estimate is a consequence of identification of foreseen project risks and other “known
unknowns” that are under the control of the project; including scope that is not cost effective
to define in detail during preconstruction planning or the earlier phases of design. The ability to
estimate these risks and uncertainties naturally changes over time as the design is refined and
the understanding of the project matures. Recipients are required to develop methods for
qualitative and quantitative assessment of these risks, and to develop an optimized risk
handling strategy that evolves with the project. 1 Regardless of the phase, the BOE for
contingency development must be sound and well-documented but remain appropriate for
that phase.
For the Conceptual Design Phase, both the baseline estimate, and the uncertainty of that
estimate should be based on expert judgment and parametric models developed by the project
planners based on scaling and extrapolating historical data from projects with similar
characteristics. When NSF conducts the Conceptual Design Review (CDR), it expects that
Recipient will have developed a risk-based, budget contingency estimate at a similarly refined
level of detail; one that is based on estimates for major elements or functional components of
See Section 6.2, Risk Management Guidelines, for more information about formulating and implementing Risk Management
Planning, and standard references on project management, such as the PMBOK Guide, for a detailed explanation of the
individual steps in Risk Management Planning: risk identifications, qualitative and quantitative risk analysis, risk handling, and risk
monitoring.
1
Section Revision:
December 15, 2020
4.2.5-5
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
the proposed facility. NSF will conduct the CDR using a panel of experts able to apply prior
experience to assess the reasonableness of the budget and contingency estimates. The budget
contingency estimate will be evaluated by NSF as part of its first internal cost analysis for the
project based on the CDR deliverables. This initial cost analysis will help inform the cost book
and other deliverables developed during the Preliminary Design Phase.
4.2.5.4 Contingency Planning and Assessment during Preliminary Design
During the Preliminary Design Phase, NSF requires Recipients to develop budget estimates and
associated risk estimates that are “bottom up” assessments 1 that consider every element of the
entire project, using as inputs the definitions of the lowest appropriate WBS elements. For each
lowest level element, the project should estimate its expected cost, excluding unusual risks or
occurrences that are outside the control of the project. The project should also separately
estimate, at the appropriate WBS element for the risk described, the technical, cost and
schedule risks or uncertainties using a widely accepted method that is employed by all
estimators. NSF expects to see the project utilize a probabilistic method of calculating a range
of risk exposures appropriate to the project area in question and the maturity of the risk
assessment. Expert judgment should always be applied to both the inputs (BOE) and outputs of
this process, to the reasonableness of potential cost and schedule impacts, and to the
applicability of the process to specific areas of the project. In some circumstances, such as
where specialized knowledge of a particular technical area or market condition exists, it can be
appropriate to override the outputs based on expert intervention. Supporting documentation
should clearly articulate which risks elements were considered and how they were modified
when making any adjustments to the model outputs.
It is not always realistic or even feasible to mitigate all anticipated risks. It is extremely unlikely
that typical projects will encounter all of the risks and the full extent of possible consequences
that have been identified. The contingency estimate should be appropriate to manage only the
ensemble risk, which is much more likely to occur than the sum of the individual risks. This
approach produces a more likely estimate for the TPC compared to an approach where Control
Account Managers increase individual WBS elements to cover risk. Use of rigorous probabilistic
cost estimating methods that estimate confidence levels for the TPC (such as Monte Carlo
methods based on probability distributions for risk) are preferred and NSF highly encourages
application of these methods where practical. As a result of these estimating activities, the
project should develop the contingency estimate that provides a high degree of confidence that
the project can be completed within budget per NSF’s “No Cost Overrun” policy.
Budget, scope, and schedule risk are usually correlated to some extent. A change in scope, for
instance, may mean more costs and additional schedule. Risk analysis and budget and schedule
See Section 6.2, Risk Management Guidelines, for more information about formulating and implementing Risk Management
Planning, and standard references on project management, such as the PMBOK Guide, for a detailed explanation of the
individual steps in Risk Management Planning: risk identifications, qualitative and quantitative risk analysis, risk handling, and risk
monitoring.
1
Section Revision:
December 15, 2020
4.2.5-6
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
contingency estimation methods should consider the degree of correlation in estimating an
appropriate level of budget contingency.
Budget contingency is developed based on risk assessment of individual WBS elements, but
once defined; it loses its identification with any specific cost element and is fungible throughout
the project to manage the overall project risk. Until then, contingency is held separately from
the project baseline budget estimate 1 that is used for Earned Value Management reporting, but
is included in the total project cost, regardless of the award instrument.
NSF requires the PEP to contain a baseline that defines the project’s intended scope, budget,
schedule, risk, and management plans. The PEP will include provision of schedule and scope
contingency 2 for use by the Project Manager, developed according to the following additional
considerations:
Schedule contingency: The construction schedule should be developed in the same manner as
the budget contingency estimate, following the WBS structure at the
lowest available level of detail. The project should make a technical
estimate for each task’s duration and its dependence on other tasks.
Scope contingency:
NSF requires projects to assess possible use of scope contingency and
develop a plan to make effective use of scope contingency, if necessary,
during construction. This provides the project with an additional tool to
manage the overall project.
NSF requires, at Preliminary Design Review (PDR), that the contingency budget, schedule, and
scope are the outcome of detailed planning by the project for how best to handle the various
risks that have been identified. Some risks are most effectively handled proactively by investing
in additional developmental and design activities or resources intended to prevent the risk from
occurring.
At the PDR, NSF requires a funding profile by fiscal year that includes the commitment and
obligation of funds, plus anticipated contingency needs. The profile should be a consequence of
the project’s formulation of a resource-loaded schedule for EVM reporting. Since PDR informs
the budget request to Congress, this allows NSF to determine the year-by-year construction
funding profile. The annual Congressional appropriation must be sufficient to accomplish the
work proposed and provide the financial resources needed to manage the risk activities
foreseen during that period.
The budget contingency estimate will be further evaluated by NSF as part of its second internal
cost analysis for the project based on the PDR deliverables. This second cost analysis will give
1
That is, contingency is not included within the Budget at Completion (BAC). TPC = BAC + contingency.
2
See Section 6.2.3 for definitions.
Section Revision:
December 15, 2020
4.2.5-7
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
assurance on the TPC brought forward to the NSB as well as help inform the cost book and
other deliverables developed during the Final Design Phase in preparation for award.
4.2.5.5 Development of the Contingency Use Process
NSF examines the RMP at PDR to ensure that the PEP describes a formal process for Change
Control1 that includes the allocation of contingency within the project during construction. NSF
approval of the RMP, including the change control process, must be documented and
maintained in the agency’s permanent record. Under the RMP, the Project Manager (or other
designated individual) should have budget authority to transfer to or from the contingency
category to specific WBS elements, via a process that follows the project’s Configuration
Change Control Plan. A typical change control process, for example, may involve written
application to the Project Manager by the affected Control Account Manager(s) and formal
review and recommendation by a Change Control Board (CCB) consisting of all other system
leads. The Program Officer must have the authority to then approve the requested funds, reject
the request, or request a change in schedule, technical scope or other corrective action. All CCB
change requests are to be logged, documented, and archived by the project, with the logs and
documentation provided on a periodic, pre-determined basis to NSF for review. The defined
CCB process must include a provision for seeking prior written approval from NSF (Program
Officer or higher depending on the magnitude) for all actions that exceed the thresholds
specified in the award instrument or NSF policy.
The CCB change request document, whether forwarded to NSF for approval or not, must have
the minimum content requirements necessary to comply with relevant cost principles as well as
to maintain an audit trail. See the sample change control request form at the end of this
section. This process must be examined by NSF for compliance before approval of the Change
Control Plan. CCB documentation must specify all control accounts that budget is being
allocated to or recovered from, and tie to budgets itemized by cost element (i.e., labor,
materials, supplies, etc.). Contingency allocations must be supported by analysis demonstrating
that the proposed amounts to be allocated are considered reasonable and allowable and
should be linked to the WBS and/or Risk Register ID. Allocations from contingency and returns
to it change the PMB budget. Therefore, it is essential that historical information be logged and
maintained in a manner that allows NSF to systematically track the evolution of the PMB from
its initial release at award through all subsequent changes. In other words, PMB budgets must
be traceable through historical records to the initial PMB release.
4.2.5.6 Contingency Planning and Assessment during Final Design
NSF requires the project to refine its cost estimates following PDR, adding additional definition
and improved confidence with the tasks associated with accomplishing the project deliverables.
At the Final Design Review (FDR) the budget estimate should be substantially based on
externally obtained cost estimates (vendor quotes, bids, historical data, etc.). This added
1
Section 2.4, Construction Stage, contains additional information about NSF expectations for conducting change control.
Section Revision:
December 15, 2020
4.2.5-8
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
definition is expected to result in a change in the project’s estimated Budget at Completion
(BAC) and in the accompanying budget contingency, while keeping the sum of the two at or
below the PDR TPC used to inform the budget request. Also, as part of the FDR, NSF assesses
the methodology employed by the project to further refine its cost and contingency estimates
including schedule and scope adjustments. All of this information would then factor into the
total project cost assessment being refined and evaluated by NSF to make the initial
construction award.
4.2.5.7 Contingency Use and NSF Oversight during Construction
NSF will negotiate the award instrument with the Recipient to fund project construction
activities (Construction Stage). This instrument will specify the contingency amounts and
include thresholds 1 above which prior written NSF approval is required before the Project
Manager may allocate contingency (as described in the approved Change Control Process, PEP8.2). Contingency is allocated to, or from, specific WBS elements increasing or decreasing the
Budgeted Cost of Work Scheduled (BCWS). The thresholds will vary depending upon the
particulars of each project. Working with the Recipient, NSF will employ the following criteria
when establishing the threshold or thresholds. These considerations must be documented in
the award file as well as in the PEP and the IMP.
•
Award and Subaward amounts – A larger award amount may warrant establishment of
higher thresholds to lower administrative burden.
•
Sufficiency of project plans and designs – More detailed project plans, specifications
and designs generally lead to higher confidence and better bids which may allow the
thresholds to be higher.
•
Nature of identified project risks – Higher risk associated with the nature, timing and
severity of certain project work packages may warrant a lower threshold.
•
Review Recommendations – Expert panel findings and recommendations should be
considered in setting thresholds.
•
Recipient or Subrecipient past performance history – Available past performance
information may help to indicate whether a Recipient’s change control process is
adequate or whether the Recipient has been successful in identifying contingencies,
e.g., use and accuracy of contingency logs, and therefore support a corresponding
appropriate threshold. Poor performance would support a lower threshold.
•
Known audit findings and their disposition – Relevant audit findings/dispositions
should be considered in establishing thresholds.
•
Sufficiency of Recipient administrative systems – The adequacy of compliance with
financial and administrative systems including accounting systems, historical cost data,
and financial reports may impact the thresholds.
Thresholds are necessary to allow the project to respond in a timely way to small, immediate needs for use of contingency
during construction. This avoids potential cost increases that could result from delay.
1
Section Revision:
December 15, 2020
4.2.5-9
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
•
Degree of NSF substantial involvement in the project – The complexity and risks
associated with the project may warrant more NSF involvement and hence lower
thresholds.
Once construction begins, the actual cost for some specific WBS elements may exceed the
estimated cost and the Project Manager can choose to allocate contingency in accordance with
the process defined in the PEP for Change Control. In other cases, the actual cost will be less
than the estimates, and the Project Manager may decide to transfer budgeted funds from the
affected WBS elements to contingency. In either case, whether it’s a risk realized or a risk
retired, the Change Control documentation should tie this transfer back to an identified risk
element in the Risk Management Plan to be allowable.
Contingency funds are to be used only to support scope that is part of the NSF-approved
project baseline, as defined in the PEP and successive CCB actions. Depending on the
thresholds, Project Manager, CCB, NSF, and NSB approvals are required to modify the project
scope. Unexpended contingency funds may not be used to support operations or other outof-scope activities.
4.2.5.8 Reporting Requirements
Each project in construction must report monthly to NSF on the status of the project, while
projects in the Design Stage are highly encouraged to submit a monthly report. Details on the
required information for the monthly reports are outlined in Section 4.6.2, Recipient
Performance Reports.
Projects are expected to periodically compute the estimate to complete (ETC) and estimate at
completion (EAC) and compare the EAC to the Budget at Completion (BAC). At least annually,
the project should update the remaining risk exposure to establish a risk-adjusted estimate at
completion (RAEAC) for comparison to the TPC. The updated remaining risk exposure should be
based on the quantitative risk analysis with current risks and uncertainties. NSF will monitor the
financial information provided and compare the available contingency to the estimated
remaining risk exposure. NSF may request a recovery plan if the contingency budget appears
inadequate to manage remaining risk. Recipients should consult with the PO and GA/O or CO as
necessary on the format for the monthly reports. The information contained in the monthly
report is not intended to supersede or replace other reporting requirements as specified in the
Cooperative Agreement or Cooperative Support Agreement.
All CCB actions, irrespective of amount, or whether they increase or decrease the BAC, must be
reported directly to Program Officer at least quarterly. All CCB actions exceeding defined
thresholds for allocation of budget, schedule, or scope contingency must be approved by NSF as
codified in the PEP-8.2 and the CA. NSF-approved CCB actions must be made part of the
award’s permanent record. For all assistance awards, CCB documentation is maintained in
NSF’s electronic record system (eJacket) in accordance with the award terms and conditions.
Section Revision:
December 15, 2020
4.2.5-10
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
NSF’s financial system controls prevent the cumulative Recipient cash draws from exceeding
the obligated spending authority. All funds are retained within NSF’s obligated award amount
to be drawn down by the Recipient for allowable expenses once needed. NSF conducts various
post-award monitoring activities, such as periodic external reviews (whose scope includes
financial as well as technical status), site visits, and single and program-specific audits to
monitor compliance.
4.2.5.9 Partnership Considerations
NSF may partner with other entities to plan and construct a major facility. The guidelines within
this document are applicable when NSF funds a particular scope of work within a larger overall
project. Risk assessment and contingency development processes are to be applied to those
WBS elements funded by NSF. Similarly, the Recipient managing construction must report on
the use of contingency during construction in accordance with the requirements regarding use
of contingency funds.
More complex situations may arise when NSF funds a proportion of the total project cost, or
where NSF contributes along with others to a common fund to build specific WBS elements
within the context of a larger project. Because overall project risk is reduced as more WBS
elements are aggregated into the risk analysis and managed through a centrally held
contingency fund during construction, NSF encourages the development of unified
management for project planning and execution of the entire project scope wherever practical.
However, NSF recognizes other partners may have different processes for planning, funding,
and conducting oversight, making it challenging to form a unified management structure.
Consequently, the award instrument must define the specific procedures for handling
contingency in those circumstances. Program Officers are advised to consult with the Division
of Acquisition and Cooperative Support to determine an effective approach consistent with the
principles of federal laws and regulations. The Large Facilities Office may be able to provide
models of various approaches that have been used successfully in other projects.
Section Revision:
December 15, 2020
4.2.5-11
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.5-1
Section Revision:
December 15, 2020
Sample of a Change Control Request Form, with instructions for filling out the various sections
4.2.5-12
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Section Revision:
December 15, 2020
4.2.5-13
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.5 Risk Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Section Revision:
December 15, 2020
4.2.5-14
Major Facilities Guide: NSF 21-107 (July 2021)
4.2.6 Budget Contingency Planning during the Operations Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.2.6 Budget Contingency Planning during the Operations Stage
The processes and procedures to handle risk are very different between major facility
construction and operations awards. As with construction, there are many inherent risks with
operations. However, the risks are markedly different in nature. Estimates for operations are
usually based on well-understood historical information and experience with routine risk
exposure included in the Basis of Estimate (BOE) as part of the “most likely cost”. The work
itself is based on the day-to-day activities of science support staff and required consumables
rather than the production, assembly and testing of discrete deliverables.
Operations award use, in approximate order, the following strategies:
•
Routine risk impacts are included in the BOE as part of the most likely cost.
•
Re-budgeting authority is used by the Recipient per the award terms and conditions.
•
Recipient reduces the level of science support effort (with NSF approval if significant).
•
Recipient requests supplemental funding; assuming proper justification, availability of
funds and recommendation by Program.
In contrast, risk handling on construction awards uses the strategy per Section 4.2.5 and NSF’s
“No Cost Overrun” policy.
As stated above, it is generally more appropriate for operating budgets to include only explicitly
identified allowances 1 for repairs, replacement, maintenance and other factors such as
“technology refresh” for cyber-infrastructure or other similar up-grades. However, any request
for budget contingency must comply with paragraph § 200.433 of the Uniform Guidance.
Unless a separate contingency budget is justified and fully supported through a formal risk
assessment and a Risk Management Plan, projects should use a systematic program to identify
the potential costs and operational impacts of both recurring and non-recurring events to
develop these allowances and clearly articulate this information as part of the basis of estimate.
A separate contingency budget may be preferable if the operational plan includes significant
upgrade funded through R&RA that should be managed as a separate sub-project. Finally, since
“contingency” has a specific meaning under the Uniform Guidance, and “management reserve”
cannot be held by the Recipient, these terms should not be used in the basis of estimate.
1 Definition in Lexicon is adapted from AACE International Recommended Practice No. 10S‐90, Cost Engineering Terminology,
Rev. March 1, 2016 and AACE International Skills and Knowledge of Cost Engineering, 6th Edition.
Section Revision:
December 14, 2018
4.2.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.1 Schedule - Introduction
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.3
SCHEDULE DEVELOPMENT, ESTIMATING, AND ANALYSIS
4.3.1 Schedule - Introduction
A schedule is a management tool used for planning and executing work during any Stage of a
Facility’s life cycle. Schedules address both how and when the work is to be performed by
identifying the activities needed to accomplish the scope of work and by time-phasing these
activities with durations and schedule logic. Time-phasing involves identifying the key
relationships between activities to determine the proper sequence necessary to accomplish the
work.
A project schedule, also referred to as a schedule model, identifies the necessary activities with
interdependencies along a timeline to complete a specific deliverable or defined scope of work
with a beginning and an end. Project schedules are typically used to manage work during the
Design and Construction Stages of a Facility’s life cycle. While NSF does not have a no schedule
overrun policy similar to the “No Cost Overrun” policy (NCOP), a reliable schedule is critical for
the Construction Stage. Schedules used for the Operations Stage of a Facility’s life cycle are
generally performance goals defined as events or milestones on a timeline and may or may not
have activities with identified interdependencies. An operations program may use separate
schedules to manage upgrades or renewal projects.
The GAO Schedule Assessment Guide 1 is intended for project schedules and identifies ten best
practices associated with creating and maintaining reliable critical path method (CPM)
schedules. Refer to the GAO Schedule Assessment Guide for discussion of concepts associated
with CPM and the specifics of each best practice. Recipients are required to utilize the GAO
Schedule Assessment Guide in the development of Construction Stage schedules for major
facility projects, as defined in Section 1.4 of this Guide, regardless of the award instrument
employed. As noted in Section 1.1 of this Guide, award instruments can take the form of
cooperative agreements or contracts. Unless otherwise noted, construction schedules must
comply with the applicable federal regulations, as implemented by NSF in this Guide, the
Proposal and Award Policies and Procedures Guide (PAPPG) or the Guide to the NSF Contracting
Process. Recipients should also use the GAO Schedule Assessment Guide when developing
schedules to manage design activities. The GAO scheduling best practices have limited
application to the schedules typically used for operations, such as bar charts or milestone
charts, and is not required guidance for Operations Stage schedules.
The guidance in this section applies to the development of construction schedules and provides
NSF expectations associated with the GAO scheduling best practices taking into consideration
NSF’s policies and practices. This guidance also explains NSF’s schedule analysis practices
aligned with the Design stage-gate reviews discussed in Section 2.3 of this Guide and the format
and supporting justification for Recipient schedules. By following this guidance Recipients
U.S. Government Accountability Office (GAO) Schedule Assessment Guide: Best Practices for Project Schedules (GAO-16-89G
December 2015, or subsequent revision)
1
Section Revision:
December 15, 2020
4.3.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.1 Schedule - Introduction
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
should expect to develop a high-quality and reliable schedule, which also enables an efficient
review by NSF.
Development of a construction schedule starts during the Conceptual Design Phase, evolves
during the Design Stage, and is expected to be ready to support construction by the end of the
Final Design Phase. For a major facility project, an activity-based resource-loaded schedule with
network logic is required for advancement to the Construction Stage. This resource-loaded
schedule (RLS) provides the basis for the performance measurement baseline (PMB) to be used
to monitor the project performance and forecast future milestones during the Construction
Stage. The RLS is also used to develop the time-phased construction budget plan during the
Design Stage.
A high-quality and reliable schedule, effectively controlled, is a key element to successful
project execution. A project’s resource-loaded schedule is the foundation that integrates scope,
budget, and time. Therefore, it is used to establish the budget and schedule contingencies, to
develop the time-phased funding needs and to measure and forecast performance. At the
Preliminary Design Review (PDR), NSF requires a funding profile by fiscal year that includes the
commitment and obligation of funds, plus anticipated contingency needs. The profile should be
developed using the Construction Stage resource-loaded schedule and the quantitative
assessment of risks and estimating uncertainties. 1 Following the Final Design Review (FDR), the
resource-loaded schedule establishes the PMB and, with the schedule contingency, informs the
total project duration authorized by the National Science Board (NSB).
Developing a high-quality and reliable schedule requires the knowledge and experience of both
the activity owners and the project scheduler(s). Activity owners responsible for managing the
work and the most experienced team members performing the work should be responsible for
estimating the resources and identifying the interdependencies of the activities to execute the
work. The complexity of a schedule typically drives the experience level of the person(s)
developing and maintaining the schedule and the selection of a scheduling software tool. A
Construction Stage schedule for a major facility project will usually require a scheduler properly
trained and experienced in critical path method scheduling and the scheduling tool. Different
scheduling software packages have different select features that require someone experienced
with that software tool to ensure a reliable schedule. Various scheduling software packages use
different terms to define a component of work performed during the course of a project –
activity and task. The use of the term activity in this guidance is interchangeable with the term
task.
See Section 6.2, Risk Management Guidelines, for more information about formulating and implementing Risk Management
Planning, and standard references on project management, such as the PMBOK Guide, for a detailed explanation of the
individual steps in Risk Management Planning: risk identifications, qualitative and quantitative risk analysis, risk handling, and
risk monitoring.
1
Section Revision:
December 15, 2020
4.3.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.2 Characteristics of a Reliable Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.3.2 Characteristics of a Reliable Schedule
The GAO Schedule Assessment Guide identifies four characteristics of a high-quality, reliable
schedule – (1) comprehensive, (2) well-constructed, (3) credible, and (4) controlled. Each of the
GAO ten scheduling best practices aligns to one of these four characteristics. Various other
industry scheduling good practices can also be generally aligned to one or more of these
characteristics. Refer to the GAO Schedule Assessment Guide for details on each of the best
practices and the mapping of best practices to the characteristics.
As discussed in Section 1.1 of this Guide, NSF does not directly construct or operate the
facilities it supports. NSF’s responsibility is for overseeing the Recipient’s performance. The
Construction Stage schedules are developed and managed by the Recipient and do not include
government activities. The discussion below provides NSF expectations, associated with the
GAO scheduling best practices grouped by characteristic, for Recipient developed construction
schedules.
4.3.2.1 Comprehensive
The schedule must include all the activities to complete the full scope of the project to be
funded by an NSF construction award, if authorized, including all subaward and subcontract
efforts. The schedule must be clearly aligned with the work breakdown structure (WBS).
Section 4.2.2.7 of this Guide provides guidance and examples for development of the WBS
elements. The schedule shall be resource loaded with all the labor, materials, equipment, and
travel assigned to detailed activities and planning package activities. Detailed activities should
be developed to allow discrete progress measurement. A planning package activity contains a
defined scope of work, typically under the responsibility of one organization, without detailed
schedule activities and typically will occur in the distant future.
With the long duration of major facility projects, the use of planning packages in the RLS is an
efficient method to ensure budget is allocated for work scope that doesn’t yet have the level of
information to define the detailed activities to perform the work. For example, at the beginning
of a project, scope associated with commissioning is commonly identified as one or more
planning packages near the end of the schedule. As the project progresses, planning packages
are broken into detailed activities. Incremental conversion of work from planning packages to
detailed activities is commonly known as “rolling wave” planning. Increments for rolling wave
planning may be event-driven (test, review, milestone, procurement) or time-based, such as
every 6 months. If a project is using incremental planning, the process should be defined as part
of managing and controlling the schedule.
The duration assigned to each schedule activity should be the most probable duration factoring
in the planned level of resources. Activities should have relatively short durations and be
consistent with information provided in the Basis of Estimate (see Section 4.2 of this Guide). For
activities that do not lend themselves to a short duration, it may be necessary to document the
activity’s scope in steps or to use another measurement method for evaluating progress.
Section Revision:
December 15, 2020
4.3.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.2 Characteristics of a Reliable Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Planning package activities will normally reflect longer durations until broken into detailed
activities. Planning packages need to be of sufficient detail to establish a credible sequence of
execution for the overall project. Durations of level of effort (LOE) activities, such as
management and other oversight efforts, may be time-based or derived from the span of other
discrete activities. Planning package and LOE activities should be identified as such in the
schedule.
The schedule should include a sufficient number of milestones to manage decision points and
interfaces (internal and external) and to monitor technical progress at different levels of the
project. External milestones may be associated with collaborative partnership efforts, reviews,
funding, facility operations, etc. Typically, external milestones are constrained within the
scheduling tool. The Recipient should consult with the NSF Program Officer (PO) to identify
programmatic milestones and high-level milestones for reporting to NSF. Lower-level
milestones will facilitate more frequent tracking of the project's progress. Milestones should be
coded to reflect their level of significance.
4.3.2.2 Well-Constructed
The attributes of a well-constructed schedule are primarily associated with the logic used to
define the interdependencies of all the schedule activities and establish the critical path. The
critical path is the longest path of activities between a project’s start and its finish and is used
to establish the performance measurement baseline (PMB) duration. Projects with multiple
deliverables or in collaboration with external partners may need to identify additional chains of
activities that are considered critical to achieve project objectives and high-level milestones. All
activities necessary to accomplish the project deliverables must be logically sequenced typically
with predecessor activity finishing before its successor activity starts.
The usage of constraints and lags to fix start or finish dates should be kept to a minimum, as
they obscure the visibility of schedule logic, make it more difficult to manage the schedule, and
hinder the ability to provide reliable forecast dates as the schedule is progressed. Schedule
visibility tasks (SVT) or schedule calendars may be used to help minimize the use of constraints
and lags. SVTs are schedule activities with no resources assigned whose duration is greater than
zero and typically, represent external effort that is not part of the PMB. SVTs may also be used
to increase management visibility to items otherwise represented as lag or constrained
milestones. Based on the project parameters, constraints and/or lags may be necessary to
effectively manage a project. The basis for constraints and lags used in a schedule should be
explained in the Schedule Basis Document as discussed in Section 4.3.3 of this Guide.
During schedule development, Recipients should perform schedule health assessments to
analyze the schedule integrity. Schedule health metrics contain checks designed to indicate
potential activity interdependency issues. At a minimum, a schedule health assessment should
include missing predecessors-successors, relationship types, leads and lags, and hard
constraints. Other potential checks to consider in assessment of the schedule include logic
density, high free float, critical path tests, path convergence, and resource rates. All schedule
Section Revision:
December 15, 2020
4.3.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.2 Characteristics of a Reliable Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
health assessment checks should be used as an assistance for assessment of the schedule
construction quality and to optimize the schedule. They should not be used as a pass or fail test.
The activity durations and the logic sequences should be validated by activity owners and
technical experts. A valid critical path is calculated by the scheduling tool, fully vetted, accepted
by the activity owners and the project management team, and aligned with the project
execution strategy. The critical path represents the sequence of the activities that drive the
earliest possible project completion date and establishes the PMB end date milestone. If the
critical path runs through management activities, the schedule should be carefully examined to
confirm the schedule logic.
Schedule contingency is needed to provide time for uncertainty in the activity durations and
schedule impacts due to risks. Schedule contingency is typically estimated using statistical
analysis or judgement based on past experience. The project end date is based on the PMB
duration plus the established schedule contingency. The award end date is generally the project
end date plus additional time for closeout of the award. The award duration is less than or
equal to the NSB authorized duration. While NSF does not have a no schedule overrun policy
similar to the “No Cost Overrun” policy (NCOP) discussed in Section 1.4.6 of this Guide,
Recipients are expected to exercise discipline to keep projects on schedule. Section 2.4.2.2 of
this Guide discusses NSF’s no-cost schedule extension (NCE) practices.
4.3.2.3 Credible
The schedule must align with the project execution approach and show how the work will be
integrated to achieve project objectives, including activities performed by subawardees and
contractors. The schedule should clearly define the sequence of activities and be horizontally
and vertically traceable through the activity relationship logic. If lower-level, more detailed
schedules are utilized in addition to the project schedule, milestone linkages should be
established to show the vertical traceability between the project schedule and the lower-level
schedule(s). The schedule should utilize milestones with predecessor activities to define
completion of major components and/or deliverables, hand-offs between different
organizations, key events, etc. The NSF Program Officer (PO) may define specific milestones for
the Recipient to include in the project schedule.
For major facility projects, the amount of schedule contingency is determined by a probabilistic
risk analysis and selecting a finish date with a confidence level between 70%-90%. The schedule
risk analysis shall be based on the project risk register with identified schedule impacts and
probabilities and activity duration uncertainty. In addition to the project end date, the total
float or schedule margin for major deliverables should be reviewed and evaluated.
For further discussion on risk registers and schedule risk analyses, refer to Section 6.2 of this
Guide. Prior to conducting a schedule risk simulation, the schedule should be assessed against
GAO’s comprehensive and well-constructed best practices and systematically checked to
confirm the dependability of the risk analysis model. The results from the schedule risk analysis
Section Revision:
December 15, 2020
4.3.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.2 Characteristics of a Reliable Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
including the contingency amounts, method of calculation, project end date, and confidence
level should be documented in the Project Execution Plan 4.10 Schedule Contingency. Schedule
contingency is held separately from the performance measurement baseline (PMB) and
allocations of schedule contingency to and from the PMB are managed through formal change
control.
4.3.2.4 Controlled
The baseline schedule, known as the performance measurement baseline (PMB), is set postFDR with the construction award. The resource-loaded schedule (RLS) is the basis for the PMB.
Every project will have changes to the plan as it is being executed; therefore, effective change
control and disciplined schedule maintenance procedures are necessary. Changes to the
baseline schedule logic due to detailed planning or re-planning should be managed through
formal change control. This includes schedule changes that do not use budget and/or schedule
contingency. The different levels of milestones used to monitor technical progress will typically
correspond to approval thresholds in the change control process. As schedule contingency is
used, the PMB end date is revised.
The schedule must be updated regularly with actual progress, referred to as the progress
schedule, for comparison with the baseline schedule and to forecast dates for milestone
completion. The current projected milestone dates reported in the Construction Stage
performance reports shall be generated using the progress schedule with the same logic as the
baseline schedule. This comparison identifies the specific activities and events that are the
source of current schedule variances or impending problems in meeting milestone dates. If
lower-level schedules are utilized to manage project scope including major subawardees and
contractors, the project needs to establish a process to maintain vertical traceability and ensure
the consistency between the project schedule and the lower-level schedules.
The project management team reviews schedule updates to verify and assess effects and
identify actions as needed. The Recipient’s Project Director reports the project status including
a narrative on accomplishments and challenges to the NSF Program Officer (PO) on a periodic
basis. For major facility projects, the update period is monthly and earned value management
(EVM) is required. Refer to Section 4.6.2 of this Guide for more information on the status
performance reports and Sections 4.6.3.6 and 6.8 of this Guide for information on EVM.
Section Revision:
December 15, 2020
4.3.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.3 Developing and Estimating a Baseline Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.3.3 Developing and Estimating a Baseline Schedule
The total project duration for the Construction Stage is set post-FDR and defined in the
construction award as two components - the performance measurement baseline (PMB)
schedule duration and the schedule contingency. The construction award duration should be
based on the total project duration plus additional time for project closeout as determined by
the Grants and Agreements Officer.
The development of the PMB is an iterative process as the project execution plan matures
through the Design Stage. The process to develop a reliable schedule would generally follow
the steps described below. First, a project would select schedule method, technique(s), and
tool(s). For major facility projects, the critical path method, rolling wave planning, Monte Carlo
simulations and Primavera scheduling software tool are commonly used.
4.3.3.1 Steps
Step 1. Define the total scope of work into deliverables and manageable parts or phases.
This is referred to as the work breakdown structure (WBS) and provides structure to the
schedule. WBSs are developed at varying levels of detail but should be at least to a level of
manageable tasks that can be assigned to one responsible organizational element. For
major facility projects, the WBS used in the schedule is same as the WBS used in the cost
estimate. Refer to Section 4.2.2.7 of this Guide for more on work breakdown structures.
Step 2. Identify project goals, major internal and external interfaces. In discussions with
the various project stakeholders, the project management team identifies major internal
and external interfaces and develops the project goals including high-level milestones and
target dates. In the event that the NSF funded project scope is a part of a larger overall
project, the technical interfaces and the organizations of the overall project may affect how
the NSF part of the scope should be executed. Equipment may be furnished from external
entities or there may be other “hand-offs” with external partners. There could also be
interfaces and “hand-offs” of components between collaborating institutions within the
NSF-funded scope. Operational facilities may have target dates for shutdown periods for
facility modifications or a required sequence of deliverables to minimize impacts to
operations. Establishing such interface milestones will provide clear visibility to the project’s
overall approach and ensure better management of the project schedule in the execution.
Step 3. Develop schedule activities and technical milestones. Schedule activities represent
the specific actions to be performed to produce a specific scope of work. The detail of these
actions becomes more defined as the project proceeds through the Design Stage. The
project management team works with the activity owners to ensure that all work scope has
been identified at the appropriate level of detail. The use of long-duration activities to
reduce schedule complexity needs to be balanced with the ability to manage the project
and measure progress. The schedule should also include lower-level milestones that will
Section Revision:
December 15, 2020
4.3.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.3 Developing and Estimating a Baseline Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
facilitate more frequent tracking of the project's progress. Milestones are also useful to
track progress of externally funded activities that are included in a project schedule.
Planning package activities are commonly used for work in the distant future. Baseline
schedules will have activities for the near-term work defined to a level to execute the work
and measure progress. Projects that use planning package activities should identify these
activities in the schedule software and have a process to ensure they are converted to
detailed plans in a timely manner. For major facility projects, the conversion of planning
package activities during the Construction Stage should be managed through the change
control process.
Step 4. Determine durations for each activity. Activity durations should be the most likely
estimate considering the available or planned level of resources. Activity durations should
not factor in risks or nonwork periods. Calendars in the scheduling software should be used
to account for nonwork days and/or periods. Durations of planning package activities
should be based on analogies to historical projects, experience, or productivity rates.
Step 5. Logically sequence activities. The project management team and activity owners
identify the predecessor-successor logic relationships between activities and milestones
utilizing three types of scheduling relationships (Finish-to-Start, Start-to-Start, Finish-toFinish), along with required lead or lag times. 1 The majority of relationships within the
schedule should be finish-to-start relationships. For reliable forecasting in progress
schedules, planning package activities need to be at a level of detail to maintain a proper
sequence of the work and the use of lags should be minimized.
Step 6. Define and assign resources to activities. Resources are broadly categorized as
labor and materials and supplies (M&S) in the scheduling software. M&S is any cost other
than labor and includes materials, procurements, contracted labor, subcontracts, travel, etc.
A project may use a Resource Breakdown Structure (RBS) to organize a list of the resources
required to complete the scope of work. The resource-loaded schedule defines the
performance measurement baseline (PMB) and reflects the expected (planned) accrual or
actual costs for the activities. An obligation baseline can also be created based on resource
spreads or obligation activities. A fund obligation profile is only used to match time-phased
funding at the time the PMB is established and is not used for earned-value analysis.
The basis of estimates (BOEs) and project cost estimates are the supporting documentation
for the resources loaded into the schedule activities. NSF construction proposals for major
facility projects must be reported both by WBS and NSF budget format per Section 4.2.2.2
of this Guide. The scheduling software may be used to “tag” resources and generate the
cost data for the NSF budget format. Refer to Sections 4.2.2 and 4.2.3 of this Guide for more
information on the NSF budget categories and construction proposal formats.
1
Refer to GAO Schedule Assessment Guide for additional information regarding activity relationships.
Section Revision:
December 15, 2020
4.3.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.3 Developing and Estimating a Baseline Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Step 7. Perform schedule calculations. Schedule calculations are performed using the
scheduling software. Early and late dates, critical path, and activity float are determined.
Calculations can be performed at various times during the preparation of the schedule to
allow for preliminary reviews and resource leveling.
Step 8. Review and analysis. The project management team and the activity owners should
be actively involved in reviewing the results of the schedule calculations. The review should
consider the project objectives, milestone completion dates, critical and near-critical paths,
float values and required resources (compared to resource availability) to determine the
acceptability of the schedule. Where alterations are required, changes are made to the
schedule logic, resource allocations, and/or durations, and then the schedule is reanalyzed.
Step 9. Assign risk-based schedule contingency. Part of the scheduling process includes the
project management team determining the risk-based schedule contingency that is derived
from the estimated duration uncertainty and risks associated with a set of activities and/or
the overall project. Schedule contingency is used (like budget contingency) to accommodate
approved baseline changes and resultant schedule impacts without impacting overall
project schedule objectives. Refer to Sections 4.2.5 and 6.2 of this Guide for more
information on development of schedule contingency.
Step 10. Prepare schedule information. The scheduling software is used to produce various
reports and graphics such as critical path, resource utilization profiles, milestone summary,
time-phased budget, etc. A summary of the baseline schedule and schedule contingency are
part of the project definition and are included in the Project Execution Plan (PEP). For major
facility projects, refer to Section 3.4.1 of this Guide for the specific schedule information to
be provided in the PEP.
4.3.3.2 Schedule Documentation
The baseline schedule is accompanied by a basis document that provides parameters and
underlying assumptions used in the development of the schedule for all project stakeholders’
understanding. A well-written schedule basis document will also help oversight groups in the
assessment of a schedule’s validity and reliability. For major facility projects, the Schedule Basis
Document must include the following components at a minimum:
•
General description of the overall approach to achieve the project goals that gives a
high-level framework of the schedule network logic, the external dependencies, and key
drivers of the critical path.
•
Identify key dates used in the development of the schedule such as life-cycle dates,
decision dates, hand-off dates, etc.
•
List of schedule assumptions such as external constraints, procurement durations,
construction calendar/seasons, operations integration requirements, funding
parameters, any significant resource limitations, items excluded from the schedule, etc.
•
Basis for the constraints, lags, leads, open-ended activities used in the schedule.
Section Revision:
December 15, 2020
4.3.3-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.3 Developing and Estimating a Baseline Schedule
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
•
Project management team’s assessment of the schedule integrity and quality using GAO
Schedule Assessment Guide’s characteristics of a reliable schedule – comprehensive,
well-constructed, credible, and controlled.
o For well-constructed, the assessment should include the results from a software
quality assessment tool such as Acumen Fuse with explanations for elements that
exceed standard metrics.
•
Schedule contingency analysis and results.
A schedule management plan or estimating plan typically describes the policies, procedures,
and tools to be used for development and management of the schedule. It is not the same as
the Schedule Basis Document but may include some similar components. The following are
additional components that could be included in the Schedule Basis Document that may be
useful in an independent review of the Recipient’s schedule.
•
Identification of scheduling software options used, i.e. calendars, activity identifications
(LOE, task dependent, schedule visibility, planning packages, etc.), project-specific
coding used, calculation of critical paths, progress override contrasted with retained
logic, progress updates with duration updates, etc.
•
Method(s) used for resource leveling – an explanation of how the project determined
that the time-phased manpower requirements from the schedule are aligned with the
project staffing plans.
•
Description of the process to convert planning packages to detailed packages or the
rolling wave planning, if used. This may be included in an earned value management
system description.
Section Revision:
December 15, 2020
4.3.3-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.4 Schedule Maintenance During Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.3.4 Schedule Maintenance During Construction Stage
The baseline schedule or PMB maintains the original agreed-upon activities and milestone
dates, unless altered in accordance with the project’s change control procedures. Work
progress is measured at regular intervals by the project management team and maintained in a
progress schedule, also referred to as a working, forecast, or status schedule. A comparison of
the progress and baseline schedules indicates the extent to which the project is ahead of or
behind schedule. This comparison also identifies the specific activities and events that are the
source of current schedule variances or impending problems. If earned value is used, the
schedule status cycles must coincide with the accounting month end to ensure consistency of
earned value calculations and reporting.
4.3.4.1 Baseline Schedule or PMB
A baseline schedule is maintained during project execution for comparison against the progress
schedule. All changes to the baseline schedule, including activity durations, logic, resources, etc.
should be documented and approved by the project management team. For major facility
projects, the change control process and approval thresholds are to be documented in the
Project Execution Plan and/or the EVM procedures.
The performance measurement baseline (PMB) end date is based on a technically driven
schedule within funding limitations and does not include schedule contingency. The total
project duration establishes the project risk-adjusted end date. A project may want to use
schedule buffers to manage or monitor interim milestones or external deliverables to the
project such as subcontract work. These types of schedule buffers should be identified as
schedule margin with SVTs in lieu of lags. If a schedule margin (buffer) activity is used in the
baseline schedule, its duration should be zeroed out prior to running a schedule risk analysis. By
doing so, the schedule analysis can be used to determine the margin durations needed to
achieve specific milestones or deliverable requirements. The schedule margin activity should
not drive the PMB end date. Schedule contingency amounts are not included in the PMB due to
the NSF requirement that contingency is held and managed separately from the baseline.
4.3.4.2 Progress Schedule
The progress schedule records the project progress status and forecasts activity and milestone
dates of the remaining work. The performance measurement baseline (PMB) end date should
be constrained to create float calculations and identify high-level milestones with negative
float. If a delay is deemed significant, the project management team should develop a plan to
examine options for schedule recovery. If the negative float cannot be mitigated, use of
schedule contingency may be necessary to update the baseline milestone date.
At regular intervals, the project management team reviews planned and completed activities to
determine progress. Various methods are used to assess the status for different kinds of
activities to ensure that progress is being determined objectively. Status information from the
activity owners typically includes activity start and finish dates, percentage complete for on-
Section Revision:
December 15, 2020
4.3.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.4 Schedule Maintenance During Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
going activities, forecast completion dates, and milestones achieved. The project management
team should vet the progress schedule results and forecast dates prior to status reporting. It is
important to note that progress information is not used to modify dates in the baseline
schedule. The baseline dates, duration, resources, etc. are only changed utilizing the baseline
change control process.
For work performed under subawards and contracts (referred to as subcontracts), the project
must identify appropriate reporting inputs to ensure objective measurement of progress.
Subcontracts may be based on milestones or require the subcontractor to develop a schedule
that supports the project schedule. The project management team needs to establish
procedures to ensure accurate progress reporting and reliable forecasting from the progress
schedule.
When progress schedule updates forecast significant change in the schedule and cost to
complete, associated revisions should be made to the Estimate to Complete (ETC) to develop a
new Estimate at Completion (EAC). Significant changes to the ETC should be considered for a
baseline change or as a minimum tracked as a lien against budget contingency. Prudent
maintenance of the control account-level estimates at completion ensures that the EAC reflects
a valid projection of project costs. The EAC should be based on performance to date and new
estimates for remaining work but does not include risks and opportunities within the project’s
risk register unless they are realized. Refer to Sections 4.2.5.8 and 6.2.11 of this Guide for more
information regarding EAC and risk-adjusted estimate at completion (RAEAC).
Section Revision:
December 15, 2020
4.3.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.5 NSF Analysis of Construction Stage Resource-Loaded Schedules
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.3.5 NSF Analysis of Construction Stage Resource-Loaded Schedules
NSF uses various oversight tools to assess the reliability of the Recipient’s schedule and inform
NSF stage-gate decisions. The discussion below describes at a high level how these tools are
used to review the Recipient’s schedule against the GAO scheduling best practices and the
documentation needed to conduct these reviews. Questions about these reviews should be
directed to the Program Officer and/or the relevant Grants and Agreements Officer or
Contracting Officer. Appendix II of the GAO Schedule Assessment Guide identifies qualitative
information and key documentation that GAO Auditors use to assess a schedule.
4.3.5.1 Schedule Review Component of Stage-Gate Reviews (CDR, PDR, FDR)
The construction schedule develops as the project moves through the Design Stage to readiness
for construction. Section 2.3 of this Guide describes each of the stage-gate reviews and NSF
expectations for readiness for a project to advance. Figure 2.1.3-2 illustrates the progressive
phases within the Design Stage and NSF Decision Points. At the Conceptual Design Review
(CDR), the schedule is high-level with key milestones and typically based on analogy with similar
projects and/or experience of technical experts. At Preliminary Design Review (PDR), a
resource-loaded schedule is required at a level sufficient to develop a time-phased budget and
estimate contingencies. As the design matures toward the Final Design Review (FDR), the
schedule is refined with more detailed activities to be ready for construction and to be
baselined for EVMS.
Based on internal guidance for PDR and FDR, the schedule will be reviewed for complete work
scope (GAO Best Practice 1), sufficient resources and duration to execute the project, (GAO
Best Practices 3, 4, and 7), credible sequence of work (GAO Best Practices 2, 5, and 6), and
appropriate schedule contingency for risks and estimating uncertainties (GAO Best Practice 8).
The external panel provides the expert experience to review the credibility of the schedule
sequence, logic, and duration, and the resource requirements of activities. Based on the results
of the CDR and PDR, NSF should provide guidance to the Recipient for implementation into the
FDR schedule relating to the GAO scheduling best practices. At the end of the Final Design
Phase, the Recipient needs to have a construction ready schedule that is reliable as defined by
the GAO schedule characteristics to advance to the Construction Stage.
To support the schedule review at PDR and FDR, the Recipient should provide the following
schedule documents with the Project Execution Plan (PEP):
•
Work breakdown structure (WBS) dictionary
•
Full schedule sorted by the WBS - Gantt view
•
Critical path and near critical path schedule(s) – Gantt view
•
List of project milestones by WBS
•
Schedule Basis Document
•
Risk register with schedule impacts identified
Section Revision:
December 15, 2020
4.3.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.5 NSF Analysis of Construction Stage Resource-Loaded Schedules
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
•
Schedule contingency analysis and results
4.3.5.2 Schedule Review Component of Independent Cost Estimate Reviews
As discussed in Section 4.2 of this Guide, NSF may utilize independent cost estimates and cost
estimate reviews, in some cases performed by independent contractors or other government
agencies, to inform the NSF Cost Analysis. In conjunction with an independent cost estimate
review, NSF may include an independent review of the Recipient’s schedule and schedule
contingency analysis using GAO scheduling best practices or the development of an
independent schedule and schedule contingency. NSF’s selection of the type and scope for an
independent cost estimate review should follow internal guidance.
An independent review of a Recipient’s schedule would typically include an assessment of the
GAO schedule characteristics, Comprehensive (GAO Best Practices 1, 3, and 4) and WellConstructed (GAO Best Practices 2, 6, and 7), in accordance with NSF expectations as described
in Section 4.3.2 of this Guide. This review may also assess the methodology used by the
Recipient for the schedule contingency analysis (GAO Best Practice 8). The external panel for a
stage-gate review would usually provide the expert experience to review the schedule risks and
associated impacts used in the schedule contingency analysis.
To support development of an independent schedule and schedule contingency, the Recipient
will need to provide the same detailed technical information that was used to develop the
schedule such as:
•
Technical specifications and requirements
•
System design drawings and technology selections
•
Key assumptions
•
Work breakdown schedule
•
Schedule Basis Document
•
Schedule management plan, if used
To support an independent review of the Recipient’s schedule and schedule contingency
analysis, the Recipient should provide the following documents in addition to the stage-gate
review project documents:
•
Baseline resource-loaded schedule source file
•
Schedule contingency analysis source file(s)
•
Major subcontractor schedule, if applicable, and the associated terms and conditions
•
Schedule management plan, if used
4.3.5.3 Schedule Review Component of NSF EVMS Verification Review
As part of the NSF’s earned value management system (EVMS) verification review, discussed in
Section 4.6.3.6 of this Guide, the Recipient’s processes for maintaining a performance
measurement baseline (PMB) schedule (GAO Best Practice 10) and updating the progress
Section Revision:
December 15, 2020
4.3.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.3.5 NSF Analysis of Construction Stage Resource-Loaded Schedules
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
schedule (GAO Best Practice 9) are assessed per EIA-748 EVM guidelines for implementation of
EVMS. This review specifically addresses statusing the schedule and measuring performance,
change control processes and documentation, and vertical traceability with lower-level
schedules (i.e., subcontractor schedules) as applicable (GAO Best Practice 5). Per internal
guidance, the EVMS verification review is informed by other NSF reviews including the FDR
stage-gate review for assessment of EIA-748 EVM guidelines associated with other scheduling
best practices such as complete work scope in the schedule (GAO Best Practice 1) and resources
assigned to all the activities (GAO Best Practice 3).
To support an EVMS verification review, the Recipient should provide the following documents
in addition to the Final Design Review (FDR) project documents for assessment of the GAO
scheduling best practices:
•
EVM system description
•
Change control process description
•
Project controls’ schedule procedures for schedule progressing and maintenance
•
Baseline resource-loaded schedule source file
•
Major subcontractor schedule, if applicable, and the associated terms and conditions
•
Schedule management plan, if used
4.3.5.4 Schedule Review Component of NSF Cost Analysis
As part of the NSF Cost Analysis, conducted following internal guidelines, the Recipient’s
schedule will be assessed for alignment with GAO scheduling best practices to determine if the
schedule meets the four characteristics of a reliable schedule as discussed in Section 4.3.2 of
this Guide. This schedule analysis will be led by the Large Facilities Office (LFO), will include a
technical evaluation from the stage-gate review, and may include input from an independent
schedule review and/or EVMS verification review. Refer to Section 4.2.1 of this Guide for more
information on the NSF cost analysis process and timeline.
Section Revision:
December 15, 2020
4.3.5-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.4 System Integration, Commissioning, Testing and Acceptance
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
4.4
SYSTEM INTEGRATION, COMMISSIONING, TESTING AND ACCEPTANCE
System integration, commissioning, testing, and acceptance are Recipient functions, and are an
essential part of complex construction/acquisition projects. Failure to perform them, or to
adequately plan for them, can lead to serious cost and schedule overruns. The Recipient is
required to describe its plans for system integration, commissioning, testing and acceptance in
the PEP (Table 3.4.1-1, Component 15). The Program Officer (PO) approves these plans, but is
also required to include periodic review of progress in these areas:
•
System Integration – combining and coordinating the many physical and performance
interfaces in a project;
•
Commissioning – substantiating the capability of the facility to function as designed by
bringing various system components on line first sequentially and then in simultaneous
operations to study and affirm the interaction among subsystems;
•
Testing – assessing the operation of the facility by applying the criteria established in
the PEP to measure acceptable performance; and
•
Conditions for Acceptance – specifying the expected condition of the facility, its
performance attributes, the tests the Recipient will perform, and the data it will
consider prior to accepting the facility or components of the facility and declaring it
ready for Operations and Maintenance. In some cases, a phased approach to
acceptance will be required. For example, for distributed-but-integrated facilities or for
facilities with complex instrumentation and equipment, the PO will want the Recipient
to demonstrate performance and perform acceptance procedures for part of the system
prior to proceeding with construction and/or acquisition of other systems. The PO, in
consultation with the Integrated Project Team (IPT), will determine whether the
Recipient will conduct the tests and accept the facility or whether the PO will participate
in the testing and accept the facility on behalf of the government.
Frequently, some aspects of construction and/or acquisition overlap with initial operation. A
detailed Operational Readiness Plan (PEP-15.2) should be developed by the Recipient at least
one year prior to the anticipated commencement of commissioning activities. Elements of
commissioning and transition from Construction Stage to Operations Stage are first addressed
during Conceptual Design Phase and become progressively more detailed as planning evolves.
During construction, the PO reviews the plan, utilizing internal staff, external experts,
consultants, external review panels and the resources of the Large Facilities Office. The review
of the plans for commissioning and acceptance should consider the following questions:
•
Will the project have parallel periods of construction/acquisition and operations, with
some components coming on line earlier than others?
•
What is the project’s strategy for facility acceptance, operational readiness review, site
safety and security, and training of operational staff and members of the research
community utilizing the facility?
•
What are the project plans for transitioning staff from construction to operational
support activities? Is there a plan to bring in personnel with the requisite technical skills
Section Revision:
May 30, 2019
4.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.4 System Integration, Commissioning, Testing and Acceptance
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
to operate and support the facility at appropriate times? Have training needs been
addressed?
•
What risks to the project might result from contractor interference during periods of
beneficial use or occupancy as construction activities conclude?
•
What risks to the project might result from operations delays?
•
What contracting strategies are employed to ensure that priority tasks are completed in
a timely way and do not delay operational readiness?
•
What are project plans for obtaining use and occupancy permits, or satisfying other local
regulatory criteria?
•
Do the budgets reflect a proper allocation between construction/acquisition and
operations?
Even if limited operations are undertaken, the changeover from construction funding to
operational funding does not have to occur until the facility has been accepted and the PO
ensures that the budget is estimated accordingly. Where operational funding will be used prior
to acceptance, the PO will ensure that the budget justification clearly describes the changeover
and that the earlier changeover is estimated and budgeted accordingly, per the Segregation of
Funding Plan (PEP-15.4).
Section Revision:
May 30, 2019
4.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.5 Documentation Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.5
DOCUMENTATION REQUIREMENTS
The Recipient is responsible for ensuring that a document management system is in place that
provides for retention and retrieval of essential and significant documentation related to the
project. Recipient documentation may take many forms, from informal e-mail communications
to formal letters, bids and contracts. NSF strongly prefers that this system be electronically
accessible via Internet, rather than paper-based, but recognizes that some paper records are
necessary. The documentation system should not only aid in identifying the types of documents
to retain but should also contain appropriate controls over official documents such as drawings
to ensure that only the most recent drawings are being used and that only authorized
personnel are able to access and modify them. A sound document management system will
help prevent miscommunications and misunderstandings and will ensure that the facility
operators have the information required to maintain the facility.
Recipients should retain financial records, supporting documents, statistical records and other
records pertinent to the award instrument (CAs, grants, subawards, and contracts) for a period
of three years after submission of the Final Project Report. In addition, access to any pertinent
books, documents, papers and records should be made available to the NSF Director, Office of
Inspector General, and the Comptroller General of the United States or any of their duly
authorized representatives to make audits, examinations, excerpts and transcripts in
accordance with either the Uniform Guidance or Federal Acquisition Regulation (FAR)
requirements.
The documentation required, and the responsibility for producing and maintaining it, varies
within the facility life cycle. During the Design and Development Stage, the Program Officer
(PO) is responsible for producing and maintaining documentation related to review and
approval of awards. Managing the documentation pertaining to the review and processing of
proposals and awards is the PO’s responsibility throughout the life of the project. Chapter VI of
the Proposal and Award Manual (PAM) requires that proposal decisions be clearly documented.
Chapter XII of the PAM requires that NSF award records be retained and either retired or
disposed of in accordance with Federal law and regulation. NSF documentation should include
all partnership and other agreements, standard eJacket submission in the NSF-required format,
the Internal Management Plan (IMP), the Project Definition (typically defined in the PEP), the
record of oversight (including all reviews and reports), and all significant project
correspondence.
During the Construction Stage, essential and significant documentation includes the record of
any decision affecting the cost, schedule or baseline. At a minimum, the following forms of
documentation should be retained:
•
•
•
•
Memorandum of Understanding (MOU) and any other project agreements or deals;
Architectural, engineering, shop and as-built drawings;
Correspondence identifying problems, the resolution process, and the final decision;
Contingency use log;
Section Revision:
May 2, 2016
4.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.5 Documentation Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
•
Change requests and approvals; and
System integration, commissioning, testing, and acceptance plans and results.
During the Operations & Maintenance Stage, the Recipient documents facility performance in
terms of:
•
The facility itself – e.g., historical record of all costs related to maintenance (preventive,
deferred, repairs and/or emergency), operating time, and scheduled as well as
unscheduled downtime, and
•
Use of the facility for research and education (including a record of users that includes
the name, affiliation, funding agency, award number and annual award amount for each
user).
Section Revision:
May 2, 2016
4.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.1 Introduction to Oversight, Reviews, Audits, and Reporting
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.6
REQUIREMENTS FOR NSF PERFORMANCE OVERSIGHT, REVIEWS, AUDITS, AND
REPORTING
4.6.1 Introduction to Oversight, Reviews, Audits, and Reporting
Oversight, reviews, audits, and reporting requirements change as a facility moves through its
life cycle and differ substantially between the Design, Construction, and Operations Stages. The
Recipient is responsible for complying with the reporting requirements contained in the award
instrument (e.g., technical and financial reporting), this Guide, and in the Proposal and Award
Policy and Procedures Guide (PAPPG) – particularly with respect to property management and
final reporting and requirements for closeout of the award. The Recipient is also responsible for
managing the project and providing internal oversight of its own activities. This may require
internal reporting and reviews by committees established by the Recipient institution. NSF is
responsible for reporting its performance goals for construction projects based on Recipient
EVM reports, per the Government Performance and Results Modernization Act (GPRAMA) of
2010.
Reviews and reporting are an important part of the oversight process that allows the PO to
monitor performance against the project plan and goals. Due to the complex nature of major
facilities, the level of oversight will be considerably greater than for a typical NSF research
grant. The Program Officer (PO) has primary responsibility for oversight of the facility in
accordance with the Internal Management Plan (IMP) and through various reviews and reports,
such as consultation and coordination with the Large Facilities Office, coordination of assurance
through the NSF Integrated Project Team (IPT), and periodic updates to the Facilities
Governance Board (coordinated through the CORF) and the NSB.
Reviews and reporting incur certain costs. Depending on the size of the project and the
distribution of the information, these costs may be significant enough to warrant explicit
inclusion in the project budget. Review and reporting plans and costs should be identified in the
PO’s IMP and in the Recipient’s PEP so that they can be adequately considered in the project
budget and schedule. The PO should clearly define the reporting requirements that are the
responsibility of the Recipient in the award instrument and these requirements should be noted
as milestones on the project schedule for construction. The Recipient’s Project Director adheres
to their internal practices regarding financial and business operations controls, 1 and internal
reporting (e.g., to the Principal Investigator, Dean, etc., as applicable and required).
It is important that consideration be given to Conflict of Interest rules and Privacy Act
restrictions when distributing and sharing reports containing proprietary or confidential
information.
1
See NSF "Business Systems Review (BSR) Guide.”
Section Revision:
November 25, 2016
4.6.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.2 Recipient Performance Reports
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.6.2 Recipient Performance Reports
Reporting requirements vary by facility life-cycle stage (Design, Construction, Operation, and
Divestment) and are specified, either explicitly or by reference, in the Terms and Conditions
associated with the award between NSF and the Recipient. Performance reports are generally
provided on a monthly and/or quarterly basis, with a comprehensive annual report provided by
a predetermined date. Separate reporting activities are required for MREFC funds used for
construction activity if the facility also receives Research and Related Activities (R&RA) funds.
These are specified in the Terms and Conditions of the separate cooperative agreements for
each type of funding.
NSF may occasionally request that Recipients provide additional information on a specific
activity, often as a result of a unique request from within the government. Such requests may
not be specifically included in the cooperative agreements due to their ad hoc and individual
nature, nevertheless a Recipient must be prepared to respond. Examples include responses to
audit requests from the Office of Inspector General and queries from the US Congress and
Executive Branch agencies. Some projects, particularly those with construction activities or
frequent changes in design, will need more frequent reporting intervals. For example, providing
the written minutes from a weekly construction meeting is common practice.
During the Construction Stage, the Project Director, who is responsible for executing and
controlling the project in accordance with the PEP and the award instrument, reports to the
Program Officer (PO) on a periodic basis (monthly for major facility projects and no less than
quarterly in other cases). Those reports should include the following:
•
PROJECT STATUS. A narrative to include the accomplishments and challenges during the
reporting period, including major scientific and/or technical accomplishments and
milestones achieved. Management information such as changes in key personnel,
budget issues, subaward/contractor performance, as well as any other information
about which the PO needs to be aware should also be included;
•
CURRENT PHOTOS. Recent photos with a written description and photographer
acknowledgements;
•
INTEGRATED PROJECT SCHEDULE. Chart or table of performance reporting milestones,
indicating which are on the baseline critical path, the current and forecast PMB
completion date and other key milestones on which the EVM is based;
•
FINANCIAL SUMMARY AND PROJECTIONS. A narrative describing the amount of
construction funding to date and the amount of costs incurred thus far; a discussion of
Earned Value metrics with attention to changes from prior month, an estimate of the
amount of contingency funds needed to complete the project, and a funding summary
and projections indicating actual funding and projected funding by fiscal year;
•
EVM DATA TABLE. Earned Value metrics (BAC, PV, EV, AC, CV, CPI, SV, SPI, EAC, ETC) at
least at the second level of detail in the WBS; Complete, Scheduled and Budget Spent
Section Revision:
December 15, 2020
4.6.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.2 Recipient Performance Reports
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
percentages; PMB and forecast completion dates, remaining budget and schedule
contingencies; and risk exposure;
•
STAGE TIMELINE OF TOTAL CONSTRUCTION SCHEDULE. “S” curve table comparison of
the Actual Cost of Work Performed (ACWP) with the Budgeted Cost of Work Performed
(BCWP) by quarter within each fiscal year up until the present quarter; and the
Budgeted Cost of Work Scheduled (BCWS) for those quarters and extending to the end
of the construction phase;
•
ROLLING TWELVE-MONTH WINDOW. “S” curve table depicting the same data as the
previous table in a twelve-month snapshot centered on the month of the report;
•
SV AND CV TREND GRAPH. Cost and schedule variances and performance indexes (CV &
CPI and SV & SPI) over a twelve-month period;
•
DISCUSSION OF VARIANCE AND CORRECTIVE ACTIONS AT THE WBS LEVEL. Review of
current or anticipated problem areas and corrective actions in a variance report at an
appropriate control account, work package, or WBS level as agreed upon with NSF for all
cost and schedule variances > ±10%, including explanation of causes, impacts at
completion, and management actions 1;
•
BUDGET CONTINGENCY. Available balances of budget and schedule contingency, as a
total amount (dollars or calendar days), and for budget contingency as a percentage of
the estimated cost to complete (ETC) the project. A “liens” list of projected amounts of
possible future calls on contingency and an updated change log indicating all
contingency allocations (“puts and takes”);
RISK MANAGEMENT. Identify top risks including the probability weighted cost exposure
and trigger dates; narrative on risk updates including new risks, revised estimates of
impact, mitigation strategies, etc.; Updated remaining risk analysis results (at least
annually).
•
For major facility projects in the Construction Stage, the PO is responsible for providing to the
LFO Liaison a copy of the monthly project report in a standard format provided by the Head,
Large Facility Office (HLFO). Smaller-scale projects will provide status reports to the PO with a
frequency and level of detail defined in their respective Management Plans (MPs). In every
case, the PO is responsible for keeping the appropriate NSF staff (Grants and Agreements or
Contracting Officer, Division Director (DD), Assistant Directors (ADs), Integrated Project Team
(IPT) members, etc.) informed of the project status.
In executing and controlling the project, the Recipient manages the project to the Performance
Measurement Baseline (PMB) definition for cost, schedule, and scope. As part of routine
reporting process, the Recipient will notify the PO of project cost and schedule variances
exceeding ±10%, including explanations of causes and any correction actions, at the WBS levels
agreed upon between NSF and the Recipient. Per Section 4.6.4 NSF’s Performance Metric for
Variance reports provided by Recipients are used by NSF in its metrics for construction project performance goals, In
accordance with the GPRA Modernization Act of 2010. See Section 4.6.4, NSF Performance metric for Construction.
1
Section Revision:
December 15, 2020
4.6.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.2 Recipient Performance Reports
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Construction, the Recipient will notify the PO of total project cost and schedule variances, at
Level 1 of the WBS, exceeding ±10%, including a recovery plan, with an associated timeline.
Section Revision:
December 15, 2020
4.6.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in the
Budget, Finance, and Award Management Office
4.6.3 Recipient Performance Reviews and Audits
4.6.3.1 Recipient Internal Reviews
The Recipient will demonstrate appropriate internal management by conducting its own
reviews and internal audits in conformance with generally accepted accounting standards,
project or operations management practices, and State and Federal regulations as appropriate.
These reviews include, but are not limited to, technical, design readiness, procurement
readiness, progress and planning, risk, safety, and acceptance reviews as well as self-audits. The
kind and frequency of all reviews and audits should be addressed in the Project Execution Plan
(PEP-14.2). Although internal review team’s members are typically project staff, consideration
should be given to the inclusion of outside subject matter experts, who can provide a valuable
independent perspective. Consideration should also be given to inviting NSF staff as observers.
4.6.3.2 NSF External Reviews
NSF requires periodic external reviews that provide advice on the status and anticipated future
performance of the facility activities. The frequency and content of these reviews are specified
in the terms and conditions of the award instrument. Typically, the periodic reports (monthly,
quarterly, annual) are used to help evaluate and monitor progress and provide information to
review panels. Additional ad hoc reviews may be conducted by the PO under certain
circumstances, such as significant re-planning of construction projects, changes in management
structure, and major changes in research technical design, direction, or scope. These reviews
should determine the extent to which the facility is meeting the goals of their Annual Work Plan
as well as the overall goals for the award, discuss any upcoming challenges and highlight good
practices and lessons learned that could be applied to other NSF major facilities. Whenever
possible, the review should be conducted at the facility itself by an external panel with
expertise in the construction and operations of large scientific facilities. 1 The panel should
produce a formal written report to NSF. In all cases, NSF should develop a review charge
written to elicit advice matched to the specific needs and challenges of the facility at the time.
Invitees to the review must include the PO, the Grants and Agreements or Contracting Officer,
and staff from the Large Facilities Office. These reviews should be coordinated with other
reviews and audits listed in Section 4.6.3, such as the Business Systems Review (BSR).
Careful consideration should be given to the selection of independent reviewers, and in all
cases the skill sets of the reviewers should be matched to the type and kind of review to be
conducted. Broad programmatic review panels charged with reviewing all aspects of a project
will generally have representation from the academic and broader national/international
research community, as well as experts in administrative aspects of facilities/project
management. A review panel focusing on specific administrative or technical aspects of a
project would have a different set of skills.
Consult with the NSF Program Officer for guidance and good practices with respect to planning and executing External
Reviews of NSF’s major facilities.
1
Section Revision:
July 26, 2021
4.6.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in the
Budget, Finance, and Award Management Office
The PO may use a standard review “template” 1 developed by the Division or Directorate. These
provide a broad outline against which the project can be compared and may include checklists
that can be used to assess the status of the project. These formats can be particularly helpful in
the pre-award phase in ensuring that the project is ready to be implemented.
4.6.3.3 Business Systems Reviews (BSR)
The BSR is one of NSF’s advanced monitoring activities that assist with oversight and provide
assurance of the suite of business systems that support the administrative management of a
major facility. These reviews are designed to provide reasonable assurance that the business
systems (people, processes, and technologies) of NSF Recipients are effective in meeting
administrative responsibilities and satisfying federal regulatory requirements including those
requirements listed in NSF’s Proposal & Award Policies & Procedures Guide (PAPPG).
Specifically, a BSR verifies that administrative, including financial, policies and procedures are
written; evaluates the extent to which these policies and procedures conform to OMB
requirements, NSF expectations, and other applicable federal regulations; and validates they
are being used to administratively manage the major facility in each of the core functional
areas. BSRs are also intended to provide an opportunity for cross-fertilization of ideas through
the identification of good practices and serve to refocus Recipients on the importance of
administrative quality.
The LFO has the lead role in coordinating the assessment of these systems by using desk
reviews and site visits to determine if the administrative business systems used in managing the
facility meet NSF expectations and are in compliance with federal regulations.
BSRs are conducted on a regular review cycle which is informed by the internal NSF annual
Major Facility Portfolio Risk Assessment. For major facility projects, the BSR can be used to
strengthen the Institutional capacity in advance of a construction award. Risk factors reviewed
during the annual Major Facility Portfolio Risk Assessment typically include:
•
The timing and associated findings of other related reviews or audits of administrative
business systems;
•
Management structure providing administrative business systems support, and;
•
Significant changes in funding levels or the Recipient’s award administration.
Further information and various details of the BSR process are provided in the BSR Guide, 2
which defines and establishes the procedures for the planning, execution and follow-up
activities associated with conducting BSRs. The Guide also defines the roles and responsibilities
of NSF staff assigned to BSR activities and identifies core functional and targeted review areas.
1
Please contact the cognizant NSF PO for details and a description of good practices and/or preferred templates.
2
See "Business Systems Review (BSR) Guide” at the NSF Large Facilities Office website, https://www.nsf.gov/bfa/lfo/index.jsp.
Section Revision:
July 26, 2021
4.6.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in the
Budget, Finance, and Award Management Office
4.6.3.4 Incurred Cost Audits
For cooperative agreements, NSF is required to conduct an incurred cost audit for major facility
awards at the end of the award and during execution of the award based on an annual risk
assessment and other oversight activities conducted by NSF. These incurred cost audits are
required for construction awards but may be implemented for operations awards based on risk.
Recipients should be prepared for such an audit at any time. For contracts, incurred cost audits
are performed in accordance with the FAR, the cognizant Federal Agency procedures, and
terms and conditions of the contract.
The Incurred Cost Audit provides prudent oversight for those responsible for the effectiveness,
efficiency, and economy of the Recipients' operations and the use of Federal funds. The
Incurred Cost Audit is performed to obtain reasonable assurance about the allowability,
allocability, and reasonableness of costs incurred and to help prevent waste, fraud, and abuse.
In preparation for an incurred cost audit, Recipients are required to submit financial
expenditures (incurred cost) data to NSF on a frequency determined by the Grants and
Agreements Officer and the Cooperative Agreement’s Terms and Conditions. It will be no less
frequently than annually. The NSF Financial Data Collection Tool for Major Multi-User Research
Facilities (Tool) assists Recipients in preparing and recording financial expenditure information
for its cooperative agreements for major facilities, is required for submission of the financial
expenditures data. The Tool is a macro-enabled Excel workbook that provides Recipients a
single, standardized method for submitting direct and indirect expenditure data with minimal
effort and helps to ensure data quality and accuracy.
To complete the data collection, Recipients will need to collect expenditure information for all
active Cooperative Agreements and Cooperative Support Agreements during the awards’
performance period. To submit expenditure data, Recipients can download the Tool file from
NSF at: http://www.nsf.gov/bfa/lfo/lfo_documents.jsp. Also, Recipients can access a video
tutorial on how to complete the Tool at: https://www.nsf.gov/bfa/dcca/csb/index.jsp. Once
downloaded, Recipients should save the file and then begin to enter data following the
Instructions tab of the Tool and the video tutorial. Questions regarding this Tool should be
directed to bfadacsmfaudits@nsf.gov and the responsible NSF Grants and Agreements Officer.
4.6.3.5 Accounting System Review or Audits
Recipients must maintain adequate internal controls, policies and procedures, and reliable
accounting systems. NSF’s efforts to implement additional procedures to enhance oversight
capability include requiring accounting system analysis in certain circumstances. An accounting
system analysis may be completed using NSF resources or with the assistance of contract
support. NSF uses different levels of reviews or audits to comply with this oversight
requirement. Specifically, prior to entering into a major facility award, NSF may review recent
Business Systems Reviews, other business site visit reports, available audit reports, and/or
analysis of the Recipients’ accounting procedures and practices in those cases where such a
Section Revision:
July 26, 2021
4.6.3-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in the
Budget, Finance, and Award Management Office
review has not been performed within the past two years or otherwise deemed necessary by
the Grants and Agreements Officer. NSF may procure accounting system reviews or analysis for
Recipients when a Grants and Agreements Officer cannot conclude whether a Recipient’s
accounting systems are adequate or determined needed during the annual risk assessment.
4.6.3.6 Earned Value Management Verification, Acceptance, and Surveillance
NSF requires major facility project Recipients use EIA-748 compliant Earned Value Management
System (EVMS) as an integrated management tool for successful project planning and
execution. To ensure that the Recipient’s EVM data provide timely, accurate, and reliable
performance information, NSF conducts project EVMS verification, acceptance, and surveillance
based on the processes recommended in the National Defense Industrial Association (NDIA)
Earned Value Management (EVM) Guides 1 as part of the project oversight and monitoring
activities. Section 6.8 of this Guide discusses basic principles and guidelines for implementation
of EVMS.
The project should demonstrate it has a structured management process that follows the
principles of EIA-748 EVMS standards and provides a sound basis for performance
measurement, problem identification, corrective actions, and management re-planning
activities as required. NSF’s EVMS verification and acceptance of a project is intended to ensure
that the implementation of EVMS for the project is appropriately tailored to the project’s
management needs. For the project to utilize the full benefits of EVMS and aid the successful
execution of the project plan, the EVMS should be properly scaled and the 32 guidelines applied
in a way that reflects the size, complexity, risk, and nature of the work. NSF’s acceptance of a
project’s EVMS is not intended to be a certification of a Recipient’s EVMS. As a result, it should
not be used by other government or contracting agencies, nor can it be extended to other NSF
projects managed by the Recipient. If a Recipient has a current EVMS certification from another
Federal Agency, the NSF EVMS verification review may be modified, but NSF acceptance will
still need to be documented and on-going surveillance performed.
The project’s EVMS verification is performed through a Compliance Evaluation Review (CER)
process. NSF strongly encourages projects to utilize EVM to the extent practicable during the
Design Stage to prepare for full implementation during the Construction Stage. NSF will
complete the CER well in advance of the award of construction funds. The NSF acceptance of
the project’s EVMS should occur before actual physical construction or major acquisitions
commence and will be based on acceptable resolution of the findings from the CER.
The LFO has responsibility for NSF’s EVMS verification, acceptance and surveillance process.
Working closely with the Project Management Team, the LFO Liaison and the Program Officer
will determine the best time for conducting the initial CER and any follow-up activities for
acceptance. After acceptance, periodical surveillance reviews should be conducted during the
Construction Stage to ensure that the accepted EVMS is being maintained and followed, and
1
Refer to Section 7 References of this Guide for a list of these NDIA guides.
Section Revision:
July 26, 2021
4.6.3-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in the
Budget, Finance, and Award Management Office
that the EVM data and information are being used to inform project management decision
making. The frequency and focus of surveillance reviews are determined by the Program Officer
in consultation with the Large Facilities Office via the LFO Liaison but are generally conducted as
part of the annual construction review to minimize burden. The scope of the surveillance
reviews can be inclusive of all EIA-748 guidelines or can concentrate on specific areas of
interests. Targeted surveillance reviews may result from corrective actions, new procedures,
and/or demonstration of practice.
Section Revision:
July 26, 2021
4.6.3-5
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.4 NSF Performance Metric for Construction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.6.4 NSF Performance Metric for Construction
In accordance with the Government Performance and Results Modernization Act (GPRAMA) of
2010 (Public Law 111-352); Empowering the Nation Through Discovery and Innovation: NSF
Strategic Plan for Fiscal Years (FY) 2011-2016; and OMB requirements, NSF developed goals to
measure construction/upgrade performance based on EVM systems used to monitor project
cost and schedule. For MREFC-funded projects1 under construction and more than 10 percent
complete, the NSF performance metric goal (when using EVM) is keeping both the total
project cost and schedule variances against the Performance Measurement Baseline at, or
better than, negative 10 percent. Projects that are less than 10 percent complete are not held
to this goal because EVM data is less meaningful statistically in the very early stages of a
project.
Total project variances exceeding 10 percent (positive or negative) should be reported to NSF
by the Recipient and be accompanied by an explanation and a proposed plan and timeline for
recovery or accommodation of the cost and schedule shortfalls (e.g., use of contingency, descope). 2
This includes facilities whose construction is partially supported by funds or in-kind contributions from outside agencies. In
such cases, the variance for the total project and the variance for the NSF-funded portion should be reported separately. For
example, if the total project variance is -7% and the NSF portion is -12%, then the -12% would be reported, accompanied by a
recovery plan. Alternatively, if the total project variance is -15% and the NSF variance is -11%, then both variances would be
reported with appropriate recovery plans with timelines.
1
2
See section 4.6.2 for requirements on variance reporting at lower WBS levels for monthly progress reports.
Section Revision:
November 25, 2016
4.6.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.6.5 Re-Baselining
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.6.5 Re-Baselining
If maintaining the original project definition via re-planning is no longer possible, and the scope,
total project cost, or approved project end date are in jeopardy, the Recipient will consult with
the PO to determine whether re-baselining the project is warranted. When deciding which
course of action to pursue, the PO will need to balance the effect of failing to achieve the
project’s performance goals against the impact on the research and education proposed for the
completed facility.
The PO should consult with the NSF Integrated Project Team and the Directorate/Division
Leadership, prior to authorizing re-baselining a project. Variances may result from many
factors – for example, inadequate project planning or management, or factors not within the
Project Director’s (or manager’s) control. Examples include failure to identify the complexity in
particular tasks (such as integration), failure to budget for adequate labor, materials or time
versus unexpected increases in the cost of labor and/or materials, unavailability of labor and/or
materials, unusually severe weather, etc.
For construction projects, uncertainties are normally managed through re-planning 1 and the
use of contingency, per Section 4.2.5. Re-baselining for construction projects occurs for
variances that result in:
1. Increases above the NSB-authorized Total Project Cost (TPC),
2. A change in the total project duration, and/or
3. Major changes in scope.
NSF approvals are required per Section 2.4.1 of this Guide. If only the schedule is extended
without an increase in TPC, the terms and conditions of the award instrument apply (i.e. NSF
policy on No-Cost Extension for CAs).
Re-baselined projects will generally go through external panel review depending on the timing
and expediency required, all stage complete NSF cost analysis, and Board authorization. Once a
re-baselined Project Definition has been authorized, the re-baselined requirements replace the
Performance Measurement Baseline as the standard against which progress is measured.
Consequently, costs exceeding budgeted amounts in the initial PMB are not referred as
“overruns” once a new project PMB has been implemented by the project management and
accepted by NSF.
1
See Section 2.4.1 for the definitions of “re-baselining” and “re-planning”.
Section Revision:
November 25, 2016
4.6.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.6.6 Project Personnel and Competencies
Successful execution of construction projects and on-going programs of the scale and
complexity typical of NSF major facilities requires skilled people who collectively possess a
broad range of professional competencies. The minimum set of competencies that NSF
considers essential for managing its major facilities is detailed in Section 4.6.6.3 and addresses
all the life cycle stages of a managing facility. It is expected that fulfillment of these
competencies will be achieved differently by each managing organization.
From the NSF perspective, there are two categories of personnel that are involved with project
execution. One category is Key Personnel and the second is the Project Team. Guidance
applicable to Key Personnel is detailed in Section 4.6.6.1. and guidance applicable to the Project
Team is detailed in Section 4.6.6.2. As a group, some combination of individuals identified as
either Key Personnel or Project Team members is responsible for possessing the suite of
competencies listed in Section 4.6.6.3 and the full breadth of necessary knowledge, skills, and
experience.
4.6.6.1 Key Personnel
Key Personnel are individuals considered essential to successful project execution and/or
facility operations and are named specifically in the original proposal and ultimately in the
award documentation. 1 For major facilities funded via contracts, certain individuals are
designated as Key Personnel and are listed by name and title in the contract (see NSF
Contracting Manual, 2020). For Major Facilities funded via cooperative agreements, NSF’s
Proposal and Award Policies and Procedures Guide (PAPPG) requires that all NSF grants and
cooperative agreements identify a Principal Investigator (PI) or Project Director (PD). Major
facilities may have both and they would automatically be considered Key Personnel. In addition
to the PI and PD roles, Key Personnel positions appropriate for a major facility project may
include a Project Manager or Operations Manager, Deputy Project Director, Associate Directors,
or similar senior staff members.
Other than the PI and PD, the managing organization proposes what additional, if any, Key
Personnel are named. For example, in addition to the positions mentioned above, acquisitions
and contract management may be considered so important for success that the organization
assigns a dedicated Procurement Officer and includes this position as Key Personnel. The
competency(ies) fulfilled by Key Personnel should be identified and maintained over time as
discussed in Section 4.6.6.3.
In accordance with the PAPPG and the NSF Contracting Manual, NSF has approval authority
over the PI and any co-PIs and reserves the right to approve other positions that are identified
and named as Key Personnel in the original proposal and ultimately in the award
Major Facilities use the term “Key Personnel” as opposed to “Senior Personnel” on other NSF awards to maintain consistency
with terminology used in major facility award documents.
1
Section Revision:
December 15, 2020
4.6.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
documentation. 1 Following award, any proposed substitutions or replacements to Key
Personnel must be submitted in advance, with all necessary supporting documentation to
assess competencies, to the cognizant NSF Program Officer for review. No changes may be
implemented without prior formal written notification by an NSF Grants and Agreements
Officer or NSF Contracting Officer.
The following position descriptions include guidance from the PAPPG and general expectations
of these roles in executing a major facility project or program.
•
•
•
Principal Investigator (PI) - This position is responsible for the scientific, technical, and
budgetary aspects of the award and is generally the individual responsible for
submitting the project proposal to NSF. The Principal Investigator is ultimately
responsible for all aspects of successfully executing the project and/or facility
operations, including ensuring that it meets its scientific and technical objectives and
interfacing with NSF and the broader science community. For the purposes of this guide,
PI/co-PI is interchangeable with Project Director/co-Project Director if not proposed as
separate positions.
Project Director (PD, may also be the PI) - The Project Director is typically responsible
for the day-to-day management of the project, generally reports to the PI (if proposed
as a separate position), and may be named as a co-PI. This position may transition to the
Operations Stage to help ensure continuity once construction is complete.
Project Manager (PM) or Operations Manager (OM) - This position is responsible for
managing the project’s construction or scientific activities on a day-to-day basis. For
construction projects, this would include major deliverables, the project’s schedule, and
budget, and earned value metrics to monitor project progress against the current plan.
The PM is essential in the Construction Stage of a major facility project but is optional in
the Development, Design, Operations, and Divestment Stages depending on the planned
activities. The PM may also serve in other capacities such as a deputy Project Director.
For facilities in the Operations Stage, the Operations Manager could be considered an
analogous position to Project Manager. NSF would have approval authority if this
position is identified as Key Personnel or otherwise required in the award
documentation.
4.6.6.2 Project Team
The Project Team comprises additional staff who collectively, with the Key Personnel, possess
the competencies detailed in Section 4.6.6.3. The Project Team may comprise any combination
of individuals or organizational units, such as an Office of Sponsored Research. The
competency(ies) attributed to a Project Team individual or organizational unit should be
identified and substantiated as discussed in Section 4.6.6.3. Approval of Project Team members
is not required but NSF should be notified when there are significant changes.
1
The ability to approve other Key Personnel is based on specific requirements detailed in the governing NSF award documents.
Section Revision:
December 15, 2020
4.6.6-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Project Team members should be identified in a project’s proposal or annual report where the
managing organization discusses fulfillment of the competencies detailed in Section 4.6.6.3.
This documentation allows NSF, via proposal and annual reviews, to assess whether
competencies are adequately covered by the Key Personnel and Project Team.
4.6.6.3 Competency Requirements for Major Facility Management
The knowledge areas listed in Table 4.6.6.3-1 are considered necessary for effective project and
program management (P/PM) of a major facility and are based on the Government-wide P/PM
standards developed as part of the Program Management Improvement Accountability Act
(PMIAA), Public Law No. 114-264. Under NSF cooperative agreements and certain applicable
contracts, the Recipient performs many of the management roles normally done by federal
project/program managers at other agencies that use contracts. 1 As given in Section 2.1.6 of
this Guide, NSF’s role is oversight of the activities performed by the Recipient, including the
proper use of federal funds.
Table 4.6.6.3-1
PMIAA Areas of Program Management Standards and Principles
Knowledge Area
Change Management
Communications Planning, Stakeholder Engagement,
and Coalition Building
Performance Management
Portfolio Management
Contracting and Acquisition Management
Customer Service
Evaluation
Process Improvement
Project Management
Requirements Development and Management
Financial Management
Human Capital Management
Information Management
Risk Management
Strategic Planning
The competencies listed in Table 4.6.6.3-2 are derived from these knowledge areas. While there
is not a one-for-one mapping between these knowledge areas and the competencies in Table
4.6.6.3-2, there is a close alignment to increase the likelihood of successfully executing the
project or program. The competencies in Table 4.6.6.3-2 have been tailored to reflect the
characteristics of NSF major facility projects.
The managing organization should be able to identify the Key Personnel and Project Team
(named individual or organizational unit) that collectively fulfill the suite of competencies listed
in Table 4.6.6.3-2. All competencies must have at least one resource assigned; however, the
same resource may be assigned to fulfill more than one competency. Some competencies are
required to be assigned to Key Personnel as indicated in the “Assigned Resource” columns in
Refer to Sections 1.1 and 1.2 of this Guide for discussion on the use of cooperative agreements and contracts for major facility
awards and the order of precedence of NSF policies and statutory requirements.
1
Section Revision:
December 15, 2020
4.6.6-3
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 4.6.6.3-2. Fulfillment of other competencies may be provided by Project Team individuals
or organizational units.
Not all competencies are necessary for each stage of a project’s lifecycle. In some lifecycle
stages there is no requirement for one or more competencies to be fulfilled and the
competency requirement is designated in Table 4.6.6.3-2 as “Optional.” The decision by the
managing organization of whether a competency is considered essential should be based on
the nature of the proposed activities on the award. For example, if an operations award
includes a major upgrade, Project Management and Earned Value Management competencies
may be beneficial if the project has a large budget (e.g., >$20 million) or long duration (2 or
more years).
As requested by NSF, the managing organization should submit documentation (resume, etc.)
substantiating the assigned resource’s expertise and qualifications for each assigned
competency based on the solicitation or as part of a review and when proposing a change in
Key Personnel or Project Team members. As stated in Sections 4.6.6.1 and 4.6.6.2, NSF approval
is only required for Key Personnel. While NSF does not approve Project Team members as it
does Key Personnel, substantiating documentation relating to competencies is still required
when changes to the Project Team are made, if such documentation is requested by NSF; this
allows NSF to document and verify that competencies are adequately covered even though NSF
does not have approval/concurrence authority over individuals.
If a competency is assigned to an individual Key Personnel or Project Team member, then the
substantiating documentation should include a resume, certification, or similar document(s)
describing the individual’s expertise and qualifications relating to the assigned competency. If a
competency is assigned to the Project Team via an organizational unit, the applicable training
or certification requirements for individuals to work within that organization should be
provided, rather than those of the individuals themselves; this allows NSF to verify that the
competency is addressed by the organizational unit even though NSF does not have
approval/concurrence authority over individuals within the unit. Likewise, if an external
contractor is providing a specific competency as an individual, the qualifications should be
specific to that individual whereas if the contractor is fulfilling the competency as an
organizational unit type, the applicable training or certification requirements required for the
individuals within the organization should be provided.
Section Revision:
December 15, 2020
4.6.6-4
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 4.6.6.3-2
Competency
Competency Resource Assignment Requirements
Assigned Resource per Life cycle Stage
Development
Design
Construction
Operations
Divestment
Project Management
Optional
KP
KP
Optional
Optional
Program Management
Optional
Optional
Optional
KP
Optional
Earned Value Management
Optional
Optional
KP or PT
Optional
Optional
Risk Management
Optional
Optional
KP or PT
KP or PT
Optional
Cost Estimating
Optional
KP or PT
KP or PT
KP or PT
Optional
Business Process
Reengineering
Optional
Optional
Optional
KP or PT
Optional
Compliance
KP or PT
KP or PT
KP or PT
KP or PT
KP or PT
Contracting and Acquisition
Optional
KP or PT
KP or PT
KP or PT
Optional
Financial Management
Optional
KP or PT
KP or PT
KP or PT
Optional
Data Management
Optional
Optional
KP or PT
KP or PT
Optional
Information Technology
Optional
KP or PT
KP or PT
KP or PT
Optional
Workforce Management
Optional
Optional
KP or PT
KP or PT
Optional
Stakeholder Management
Optional
KP or PT
KP or PT
KP or PT
Optional
Resources indicated as Key Personnel (KP) or Project Team (PT)
A general description for each of the listed competencies in Table 4.6.6.3-2 is provided in Table
4.6.6.3-3. These descriptions are intended to be general and reasonably in alignment with the
guidance established in PMIAA and are not considered a fully authoritative set of definitions.
Table 4.6.6.3-3
Competency Descriptions
Competency
Description
Project Management
• Demonstrates general and specialized knowledge of the principles,
methods, and tools for project management, with “project” defined as a
temporary endeavor with a defined scope, cost, and completion date. A
project may be part of a larger program or portfolio.
• Demonstrates knowledge of the strategies, techniques, and processes
used to plan, monitor, and control project scope; includes collecting
requirements, defining scope, creating a work breakdown structure,
validating scope, and controlling scope to ensure project deliverables
meet requirements.
• Demonstrates knowledge of the strategies, techniques, and processes
used to plan, develop, and control project schedules and track project
milestones, activities, and deliverables, including timeframes and
assigned resources.
• Demonstrates knowledge of the principles and methods to identify,
solicit, analyze, specify, design, and manage requirements, and able to
systematically assess how well a project is working to achieve intended
outcomes.
Section Revision:
December 15, 2020
4.6.6-5
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Competency
Description
Project Management
(continued)
• Skilled in the use of project management controls to analyze project
budget and schedule information and to generate reports with the
primary focus of answering two fundamental questions: (1) How much is
the project going to cost at completion and will the project finish on
budget? (2) How long is the project going to take and will the project
finish as scheduled?
• Knowledge of the principles, methods, and tools of quality assurance,
quality control, and reliability used to ensure that a project, system, or
product fulfills requirements and standards.
• Skilled at recording and controlling changes to the performance baseline
(cost, scope, and schedule).
• Able to identify and align project needs to the science mission and goals.
• Skilled in satisfying internal and external customers through successful
project execution; able to communicate and report progress to the NSF
Program Officer.
Program Management
• Demonstrates knowledge of the principles, methods, and tools for the
coordinated management of a program, including oversight of a set of
programs, projects, contracts, and other work that supports scientific
goals.
• Able to provide oversight of multiple projects, integrate dependent
schedules and deliverables, and conduct related activities (for example,
benefits management, life cycle management, and program
governance).
• Able to plan for and manage capital assets and develop budgets,
cost/benefit analyses, and investment decision documentation for
evaluation and justification of program costs.
• Demonstrates knowledge of the strategies, techniques, and processes
used to plan, monitor, and control the level of scientific support; includes
collecting requirements, defining scope, creating a work breakdown
structure, validating scope, and controlling scope to ensure program
deliverables meet requirements.
• Demonstrates knowledge of the strategies, techniques, and processes
used to plan, develop, and control program schedules and track major
sub-project milestones, activities, and deliverables, including timeframes
and assigned resources.
• Skilled in implementing Continuous Process Improvement (CPI) initiatives
to leverage organizational strategy and performance management data
to identify and eliminate waste, reduce variation, and satisfy customer
needs.
• Skilled in long-term planning, implementing actions needed to realize
scientific goals, and mitigating likely challenges and barriers to achieving
the desired outcomes.
• Demonstrates knowledge of the principles and methods to identify,
solicit, analyze, specify, design, and manage requirements, and able to
systematically assess how well a program is working to achieve intended
outcomes.
Section Revision:
December 15, 2020
4.6.6-6
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Competency
Description
Program Management
(continued)
• Knowledge of the principles, methods, and tools of quality assurance,
quality control, and reliability used to ensure that a project, system, or
product fulfills requirements and standards.
• Able to identify and align program needs to the science mission and
goals.
• Demonstrates knowledge of change management principles, strategies,
and techniques required for effectively planning, implementing, and
evaluating change in the organization.
• Skilled in satisfying internal and external customers through successful
program execution; able to communicate and report progress to the NSF
Program Officer.
• Demonstrates knowledge of the principles and methods for evaluating
program or organizational performance using financial and nonfinancial
measures, including identification of evaluation factors (for example,
workload, personnel requirements), metrics, and outcomes, addressing
both the science and operations.
Earned Value Management
• Demonstrates knowledge of the Electronic Industries Alliance (EIA)-748
on Earned Value Management Systems (EVMS) and how to use it as an
integrated management tool for successful project planning and
execution.
• Able to apply the 32 guidelines described in EIA-748 when developing
and implementing the project EVMS.
• Skilled at scaling the guidelines based on the size, complexity, and type
of work effort needed to successfully manage the project.
Risk Management
• Demonstrates knowledge of principles, methods, and tools for risk
management.
• Skilled in identification, evaluation, mitigation, management, and
oversight of risks and opportunities within a project or program.
• Able to remedy potential issues and implement improvements to reduce
risk, including through the development of risk mitigation plans.
Cost Estimating
• Demonstrates knowledge of the principles and methods of cost
estimating, including the best practices (twelve steps) identified in the
GAO Cost Estimating & Assessment Guide.
• Able to develop a Cost Estimating Plan and Cost Book that reflects NSF
and GAO guidance.
Business Process Reengineering
• Demonstrates knowledge of methods, metrics, tools, and techniques for
restructuring and improving business processes.
Compliance
• Skilled in ensuring that the award is managed in compliance with
applicable Federal laws, regulations, and guidance.
Contracting and Acquisition
• Demonstrates knowledge of the process and procedures for soliciting,
executing, monitoring, and closing contracts and other award
instruments in compliance with Recipient organization procurement
policies.
Section Revision:
December 15, 2020
4.6.6-7
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Competency
Description
• Able to develop acquisition plans, processes, subawards, and contracting
strategy consistent Federal laws, regulations, and guidance.
Financial Management
• Demonstrates knowledge of procedures for assessing, evaluating, and
monitoring programs or projects for compliance with Federal laws,
regulations, and guidance, including OMB Uniform Guidance (2 CFR §
200), relating to financial management.
• Able to prepare, justify, and/or administer the budget for project or
program areas.
• Able to plan, administer, and monitor expenditures to ensure costeffective support of programs and policies, e.g., through use of financial
controls and audits.
• Skilled in assessing the financial condition of a project or program.
Data Management
• Demonstrates knowledge of the principles, procedures, and tools of data
management, such as modeling techniques, data backup/recovery, data
mining, data standardization processes, etc.
• Able to plan/budget for, manipulate, and control access to
information/scientific data during the project’s or program’s lifecycle.
Information Technology
• Able to manage information technology resources, such as personnel,
equipment, etc. that support the project or program.
• Demonstrates knowledge of the three pillars of information security
programs (Governance, Resources, and Controls) and how to develop
and manage a robust cybersecurity program.
Workforce Management
• Able to manage workforce requirements to meet organizational and
program goals within budget constraints and to ensure employees are
appropriately recruited, selected, appraised, and rewarded.
Stakeholder Management
• Demonstrates knowledge of the concepts, practices, and techniques
used to identify, engage, influence, and monitor relationships with
stakeholders; able to collaborate across organizational boundaries and
engage in partnerships and team building.
Section Revision:
December 15, 2020
4.6.6-8
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Partnerships Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.7
PARTNERSHIPS
4.7.1 Partnerships Overview
For both major facility and smaller projects, partnerships are an essential consideration –
beginning at project development and all the way through divestment. Partnerships may take
many forms, but typically include coordinated funding from states or state institutions, other
federal agencies, 1 non-governmental entities, and foreign funding agencies. International
partnerships are generally the most complex.
Key issues in these partnerships, whether international or the result of interagency or state
collaboration, present several important challenges that the Recipient and PO need to consider
carefully.
The first is “culture shock.” The science or engineering cultures in different countries will
generally exhibit great variations in procedures when it comes to funding, managing and
overseeing, constructing and operating a facility. Differences often include lack of mutual
understanding or considerably different contexts for defining the role and function of project
management. It is typically very challenging for each nation to manage its part of the project
unless there is a means for integrated management and oversight by the central Project
Manager.
The Project Director or Manager should be in place before funds are released and, to be most
effective, should be given budget authority (or authority over in-kind resources) and should not
simply act as a coordinator. In terms of oversight, reviews of project status by U.S. agencies are
not universally accepted. U.S. agencies use reviews heavily, but not all countries do. In some
countries, reviews that uncover problems may be received without a sense of urgency and may
not be acted upon quickly. U.S. partner agencies may be able to insist upon resolution of issues
when playing a majority role in funding; if not, other steps should be taken. Full project
transparency is essential to success.
A second important issue is early negotiation with international partners. There is a need to
start with a clear understanding by all partners as to how the construction project is to be
managed, or, in the case of operations, how the facility is to be operated. It is also important to
know how agencies (ministries) in different countries view the project in terms of shared goals,
the science or engineering case for the project, and its priority. If participating partner countries
all rate the priority of a project at the highest level, then commitments carry more weight.
Funding risks associated with international partnerships should be assessed and fallback plans
developed regarding potential changes in commitment. Finally, early negotiation also provides
See “Best Practices for Federal Research and Development Facility Partnerships,” IDA Science & Technology Policy Institute,
IDA Paper P-5148 Log: H 14-000676, for guidance or models on forming interagency federal partnerships.
1
Section Revision:
December 15, 2020
4.7.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.1 Partnerships Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
a means to establish and maintain regular agency-to-agency contacts, providing an early
understanding of funding pressures and other emerging pressures in each country.
The cognizant NSF Program Officer should be informed of potential international partnerships
early in the process and kept apprised of significant developments. Prior to entering into formal
arrangements with foreign collaborators, the Recipient shall provide written notification to the
cognizant Program Officer according to the terms and conditions of the CA or contract. The
cognizant Program Officer will coordinate with the NSF Chief of Research Security Strategy and
Policy to ensure that potential international partnerships are in compliance with U.S. law, NSF
policy, and current foreign policy and geo-political considerations.
Section Revision:
December 15, 2020
4.7.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.2 Partnership Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.7.2 Partnership Funding
Funding of projects involving partnerships is obviously a central consideration. International
partner agencies need to understand the funding processes in the different countries involved.
The complexity of the NSF process can lead to misunderstandings regarding the schedule of
funding and project approvals. Because of the great variation among countries as to how labor
costs are counted, it is good practice to adopt standard costing techniques for equipment,
labor, commissioning and operations. MOUs need to be developed, detailing the foreign
contributions. In some cases, these contributions may be in cash or in-kind level of effort; but
deliverables should be clearly specified, and the contributions should be valued in U.S.
equivalent terms (including all labor costs) for projects in which NSF is the lead agency. To aid
project management and eventual close-out, it should also be made clear what scope NSF and
the other partner are either paying for or contributing (by WBS element) and proper
segregation of funding rules employed as appropriate.
As with all such projects, contingency funds (or their equivalent) need to be identified by all
partners. There is great variation in practice among countries, again because labor costs may or
may not be included in contributions to the project. This can have a great impact. For example,
in a cost-overrun situation it may become expeditious to simply stretch the project out. This
may work for one country, resulting in less focus on schedule issues; but it generally does not
work for U.S. projects where “standing army” costs are directly allocated to the total cost for
construction of a facility.
In addition, when partner funding is in cash, variations in exchange rate can have a large effect
on the ability of a given country to meet its commitment on deliverables. Therefore, scope
contingencies need to be explored. When international partners do not include adequate
contingency, and the U.S. does, funding “caps” (agreed upon in advance) are an appropriate
practice. Although caps may enforce discipline, they may have other effects. For example, when
there are schedule slips and “standing army” costs rise, caps can limit the deliverables that may
be provided. Strict adherence to caps may therefore compromise the overall performance
goals.
Finally, a facility’s project management and operations plans should be well understood by all
partners. When different countries have responsibilities for separate subsystems, strong system
integration and comprehensive interface documents become very important. The changecontrol process needs to be clearly understood. Change control is made very complex because
performers in one country may be ill equipped to handle or adapt to required changes. It is also
very important to establish a sound schedule baseline and adhere to it.
For partnerships with organizations or agencies in the United States, the following activities are
advised:
•
Evaluate NSF’s role (NSF’s authority and responsibility vary depending on its status as
executive agent or as a majority, equal or minority partner). Assess risks and develop a
Section Revision:
June 17, 2015
4.7.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.2 Partnership Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
plan to address them, e.g., implementation of controls that limit NSF’s exposure to
overruns (see Section 6.2, Risk Management Guidelines For Construction Stage).
•
Ensure that all partners understand the review and approval processes of the other
partners.
•
Prior to entering into a partnership, develop and execute an MOU.
Section Revision:
June 17, 2015
4.7.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
4.7.3 Memorandum of Understanding (MOU) with NSF
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
4.7.3 Memorandum of Understanding (MOU) with NSF
MOUs are broad, general agreements between NSF and other parties to pursue activities of
mutual interest and benefit; cooperate in areas where science and engineering interests
coincide; and provide a framework for cooperation. A typical MOU includes:
•
The purpose of the Understanding; authority of the parties to enter into an
Understanding;
•
Scope of the Understanding, including a project description and the respective
responsibilities of each party for funding, management and oversight (including
procedures for resolving conflicts and dealing with defaults);
Rights of each party with respect to access, ownership and intellectual property
(Chapter VII of the PAM); means for resolving disputes; and
•
•
A termination clause.
MOUs are developed by the PO and cleared according to procedures outlined in Chapter VIII of
the Proposal and Award Manual (PAM).
Section Revision:
December 14, 2018
4.7.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
5 Guidance for Mid-Scale Research Infrastructure Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
5
GUIDANCE FOR MID-SCALE RESEARCH INFRASTRUCTURE PROJECTS
This section addresses guidance for the planning and oversight of mid-scale research
infrastructure projects as defined in Section 1.4 of this Guide. These oversight processes draw
on the well-established good practices within sponsoring directorates and divisions for such
projects, which follow the requirements in the PAPPG. The design and development of a midscale project is used to advance the technical design and develop the project management
processes to establish a Project Execution Plan (PEP) ready for start of the construction stage,
also known as implementation for mid-scale projects.
Mid-scale project oversight requirements are to be tailored based on each project’s unique
characteristics such as the technical scope, the type and mix of work performed (e.g. standard
procurement by the Recipient, software development, or civil construction), and an assessment
of the associated technical and programmatic risks. However, NSF is committed to the principle
that this flexibility does not preclude the requirement for appropriate rigor on the part of NSF
or the Recipient. Appropriate use of NSF major facility oversight practices will be determined on
a case-by-case basis as outlined below.
Similar to major facilities, mid-scale project proposals should include actual costs and budget
estimates for all stages of the project life cycle: development and design, construction or
acquisition, operations, and divestment, although these stages may be less rigorously defined
depending on the project scope and history. For example, actual costs should be included for
investments previously made during development and design when submitting a proposal for
the construction or acquisition. Mid-scale project proposals should also include a Concept of
Operations discussion and the strategy and timeline for divestment so the proposal can include
estimates for eventual operations and divestment. See the Project Execution Plan (PEP)
discussion below. The detailed programmatic requirements for these estimates will be included
in solicitations calling for mid-scale infrastructure.
Budgets should be supported by well-documented Basis of Estimates (BoE) developed in
accordance with the best practices and twelve steps outlined in the GAO Cost Estimating and
Assessment Guide to meet the four characteristics of a high-quality estimate: welldocumented; comprehensive; accurate; and credible (see Section 4.2 of this Guide). Schedules
should be developed following the applicable best practices outlined in the GAO Schedule
Assessment Guide (See Section 4.3 of this Guide).
Selection Criteria: Mid-scale projects are selected based on the merit review criteria detailed in
the program solicitation. However, some typical project characteristics that may be considered
are listed below.
•
•
•
•
Opportunity to enable frontier science and engineering (S&E) research and education;
Compelling research needs;
Priority within the relevant science communities;
Accessibility to an appropriately broad user community;
Section Revision:
December 15, 2020
5-1
Major Facilities Guide: NSF 21-107 (July 2021)
5 Guidance for Mid-Scale Research Infrastructure Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
•
•
•
Level of maturity on collaboration of partnerships;
Budget alignment with solicitation and life cycle cost impacts;
Technical feasibility and consideration of risks; and
As appropriate, a Project Execution Plan (PEP) sufficiently developed for the life cycle
stage of the project.
Unless a risk for award is identified by Office of Budget, Finance, and Award Management (BFA)
or Program during proposal review, there are generally no additional pre-award review
requirements once the NSF review and selection process are complete and a subsequent award
is made.
Review Process: Mid-scale projects are not subject to the formal stage-gate review process
given in Section 2.3 of this Guide. However, the internal proposal review process used by NSF
for construction/acquisition proposals should be sufficiently robust to assess readiness using a
similar philosophy to the Final Design Review but scaled appropriately to the program. This
review process will also include an assessment against the programmatic selection criteria given
in the solicitation.
These reviews, as well as reporting to assess progress against plan during implementation, will
be determined by the Program Officer in consultation with the Grants and Agreements Officer
and LFO Liaison (if applicable, see below). Reporting and progress review requirement will be
codified in the award.
Programmatic Deliverables: Mid-scale projects should be executed using well-established
project management methodology. The specific project management approach used should be
scaled to the needs of the project. For example, project management controls used to manage
project resources and schedules, performance management, financial and progress reporting
requirements, and risk management techniques should be carefully considered such that
burden does not outweigh the benefit.
A Project Execution Plan (PEP) is required for all mid-scale projects in order to document the
foundation for how the project will be managed by the Recipient during the construction stage
(also referred to as implementation). Concurrence on an initial PEP must be reached between
NSF and the proposing organization. It is reasonable to expect the PEP to evolve during the
execution of the award.
The following list provides the minimum required components of the PEP for a mid-scale
project as compared to Section 3.4.1 of this Guide. The contents of each PEP component should
be tailored in both detail and scope to the specifics of the project. Refer to Section 3.4.1 of this
Guide for descriptions of typical elements of each PEP component. Unless otherwise noted in
the solicitation, the sub-topics within each PEP component should be included. Although, some
of the material may also be included in the mid-scale proposal itself, inclusion in the PEP allows
for completeness and reference in the award terms and conditions.
Section Revision:
December 15, 2020
5-2
Major Facilities Guide: NSF 21-107 (July 2021)
5 Guidance for Mid-Scale Research Infrastructure Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1.
2.
4.
6.
8.
9.
10.
12.
13.
Introduction
Organization
Construction Project Definition
Risk and Opportunity Management
Configuration Control
Acquisitions
Project Management Controls. Describe the methods for performance measurement
and management.
Cyber-Infrastructure
Commissioning, including Concept of Operations
The PEP should be included in the Supplementary Information section of the proposal.
If the project will be integrated into a larger facility or instrument, the proposal should include a
section discussing planned system engineering activities. If the site selected has any known or
potential requirements for permitting or environmental impact studies, a discussion of this
should be included in the PEP. Section 6.5 of this Guide discusses environmental regulations
associated with construction or modification of facilities. Inclusion of other PEP components
detailed in Section 3.4.1 of this Guide is optional and should consider the unique attributes of
the project.
The Performance Measurement Baseline (PMB) for the project is set by the proposed scope,
budget, and schedule as defined at the time of the award. All reporting is done against this
project baseline. The baseline budget, budget contingency and fee (if any) comprise the Total
Project Cost (TPC). The estimation and use of budget contingency (if proposed) must follow
Section 4.2.5.7 of this Guide. Although substantial rigor is required in establishing the TPC, midscale research infrastructure projects are not subject to NSF’s “No-Cost Overrun” policy used
for major facility projects as defined in Section 1.4.6.
A mid-scale project should follow project planning and management practices that suit the
project while supporting sound performance measurement and management needs. The
project management controls should identify the methods and quantitative measures to
compare the technical progress and costs during execution to the planned schedule and
budget. The scope and complexity of a project should be assessed to determine if the project
can benefit from the earned value principles for performance management. NSF does not
require earned value management implementation for mid-scale projects. NSF has established
a scaled earned value management approach with reduced administration burden. Refer to
Sections 6.8.4 and 6.8.5 of this Guide for more information on NSF’s scaled earned value
management approach.
Section Revision:
December 15, 2020
5-3
Major Facilities Guide: NSF 21-107 (July 2021)
5 Guidance for Mid-Scale Research Infrastructure Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
PO Oversight: At the earliest practical point, each mid-scale project is assigned a cognizant NSF
Program Officer (PO) 1 with primary responsibility for award management and project oversight.
The PO (or POs) creates a Management Plan (MP) to document key project characteristics, the
planned oversight approach, and any extraordinary exceptions or additions to the guidance
presented in this section as part of the program solicitation development in accordance with
NSF policy. Formal Internal Management Plans (IMP) used for major facilities are not required.
Depending on the funding account, the PO assigned to manage mid-scale projects may be
required to be permanent NSF employees, as required by statute. The technical background
and experience of the cognizant NSF PO for a mid-scale project should be appropriate to latestage design and construction activities as determined by the Sponsoring Organization.
Interaction with the Office of Budget, Finance and Award Management (BFA): Program
Officers recommending mid-scale projects as defined in Section 1.4 of this Guide with a TPC
over $20 million must consult with BFA prior to award. To support oversight of stand-alone
mid-scale projects, the Large Facilities Office (LFO) will generally appoint one LFO liaison to
assist Programs and the Grants and Agreements Officers in the appropriate BFA unit, either
Division of Grants and Agreements (DGA) or Division of Acquisition and Cooperative Support
(DACS), depending on the nature of the project and the funding program. For mid-scale projects
that are upgrades to major facilities, the LFO Liaison assigned to that facility will provide
support to Program and the Grants and Agreements (or Contracting) Officer.
Integrated Project Teams: Mid-scale projects consisting of upgrades to existing NSF major
facilities should be coordinated through the NSF Integrated Project Team (IPT) for that facility.
Formal IPT’s are not required for stand-alone mid-scale projects.
Budget Inclusion and the National Science Board: Inclusion in the NSF budget is either done as
part of a Directorate-level or agency-wide program as described in the solicitation. While major
facilities require formal NSB authorization for future budget inclusion, the level of engagement
for mid-scale projects by the National Science Board is based on the annual award amount, the
current Board award authorization thresholds, and the account from which the project is
funded.
1
Also referred to within NSF as Program Director or Program Manager.
Section Revision:
December 15, 2020
5-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6
6.1
SPECIAL TOPICS AND SUPPLEMENTARY MATERIALS
INTRODUCTION
This section contains extensive supplementary information on special topics having to do with
the National Science Foundation (NSF) role in planning, oversight, and assurance of major
facility projects. The materials are presented in a tutorial format to be of particular benefit to
individuals newly involved with major facility projects. They are based primarily on current
standards and good practices for project management.
Section Revision:
July 17, 2015
6.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2
RISK MANAGEMENT GUIDELINES FOR CONSTRUCTION STAGE
6.2.1 Introduction
Project risk management is a process which increases the probability of a successful project by
identifying threats to the project, assessing the nature of those threats, and identifying actions
that can be taken to either reduce the probability of those threats occurring or reduce the
impact of the threats to the project. Even on a simple project, things seldom go as planned.
With the highly-technical, scientifically ground breaking, and long duration projects undertaken
by the NSF there will be many changes required to the baseline plan as a project matures.
Successful projects anticipate problems, work to avoid those problems, and limit the impact
those problems will have on a project.
Risk management serves two purposes; one is to forecast impacts of possible events on the
project’s cost and schedule, the other is to prioritize and inform project decisions on alternate
strategies to mitigate the cost or schedule impact of a possible event or increase the technical
performance margin of a system or subsystem. The former (quantitative risk analysis) creates a
framework for quantifying the risks to the project goals in terms of cost in dollars, schedule in
days, and performance for the purpose of forecasting the final cost, schedule, and performance
of the complete project. The latter (qualitative risk analysis) helps the team sort through the
hundreds and perhaps thousands of risks to identify and address the ones that are most likely
to have the most significant impact on the project.
Qualitative risk analysis practices have remained relatively unchanged recently while
quantitative risk analyses have been evolving rapidly as the software tools and their integration
with scheduling software packages have evolved. While quantitative risk analysis has become
easier and more sophisticated, it is unlikely to fully replace qualitative risk analysis because the
quantitative analysis requires validated inputs that are more labor intensive to produce. Most
projects utilize the qualitative risk analysis practices for their month-to-month risk management
and implement quantitative risk analysis only when they need to re-forecast the estimate at
complete cost and completion date of the project.
Risk management involves all project personnel. With an effective risk management program
every project team member should be able to state the top project risks as well as the top risks
to their subsystem. Risk management has an inherent Malmquist (completeness) bias – there
will always be more risks to a project then are reflected in the risk register. To minimize this
effect every project team member from every perspective in the program should be
contributing threats, opportunities, and mitigation ideas to the risk board. The team also needs
to be well aware of the risks associated with their subsystems, so they recognize how a mistake
in their area would impact the overall project (an aspect of human error prevention and project
safety).
Some projects refer to risk management as risk and opportunity management, to emphasis to
the team that they should also be thinking about opportunities for changes in the baseline plan
Section Revision:
November 25, 2016
6.2.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
that could save cost, save schedule, or improve performance. This section follows the Hulett
definition of risk that is in the Project Management Body of Knowledge (PMBOK) that includes
opportunities in the definition of risk. Project teams should remind each other to keep thinking
about new opportunities as well as threats to the project.
Risk and opportunity management feeds into the key decisions that make a project successful.
It is a core activity for project managers, systems engineers, subsystem leads, program officers,
and review panels.
NSF requires major facility Recipients to develop and follow formalized Risk Management
during the Design and Construction Stages of major facility projects.
Successful Risk Management entails early recognition, proactive planning, and aggressive
execution of all risk management processes. Ideally Risk Management begins as early as the
initiation stage of the project life cycle. This Guide provides detailed information on the Risk
Management 1 methodologies and strategies commonly applied during project planning and
execution.
There are three key products of Risk Management as applied to NSF construction projects:
•
A Risk Management Plan that sets out how risks will be identified and managed by the
project following standard risk management processes and practices,
•
A Risk Register, or tracking tool, that documents identified risks, and
•
A determination of Risk Exposure and the related amount of Contingency needed to
control risks, based on quantitative risk analysis.
The Risk Management Plan (RMP) is a required element of the Project Execution Plan (PEP)
described in Section 3.4 of this Guide (often as a separate document). A RMP should be
included in the project planning and proposals no later than the start of the Conceptual Design
Phase. The Plan should identify the responsibilities for risk management and describe the Risk
Management process that will be followed— including roles and responsibilities, procedures,
criteria, tools, and techniques to be used to identify, analyze, respond to, and track project
risks. The level of detail in the plan, and the scope, timing, and level of risk analysis should be
commensurate with the maturity and complexity of the project and may evolve and change
over time. An example of an acceptable RMP outline is shown in Table 6.2.5-1.
The Risk Register – typically a spreadsheet or data base – is a tracking tool that includes a
description of all risks that are deemed to be important to achieving project success, along with
an assessment of those risks that allows them to be prioritized for effective management. The
Risk Register also includes the risk handling strategy, the person to whom each risk has been
assigned for accountability purposes, the current status of the risk handling strategy, and
The NSF Program Officer, as part of oversight responsibilities, identifies project-related agency risks to NSF, formulates
mitigation strategies, and documents them in an Internal Management Plan (IMP), accessible only to NSF staff.
1
Section Revision:
November 25, 2016
6.2.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
comments. An example of a commonly used style for a Risk Register is given in Figure 6.2.7-6. It
should be noted that appropriate tracking tools will vary among projects because the types of
information and indicators being monitored vary from project to project. The selection and
definition of a tracking system to be used in a project should be commensurate with the size
and complexity of the project and should be defined in the project’s RMP.
Risk Management strategy involves the estimation of overall risk exposure and the
determination of an adequate amount of contingency – a quantity of money, scheduled time,
or reductions in scope intended to recover project objectives if uncertainties and risks occur
with negative impacts. The values for cost and schedule contingencies are taken from
distributions generated by Monte Carlo simulations with probability and impact ranges for
uncertainty and risks for activities defined in the baseline. The confidence levels for meeting a
chosen project end date and total cost should lie between 70% and 90%. Scope contingency
involves identifying lower priority tasks that can be delayed or dropped from the project
without a crippling impact to project objectives. De-scoping may be used if the project forecast
indicates that cost or schedule overruns are likely. For NSF major facility projects, these
contingencies are held separately from the project Performance Measurement Baseline (PMB) 1
budget, schedule, and scope. Strategies for using contingency are detailed in the project Risk
Management Plan. Contingency is controlled and managed through the project
Configuration/Change Control Process (CCP). The use of contingency is subject to approval by
project leadership, and by NSF, if amounts are above certain thresholds, as defined in the
cooperative agreement (CA).
While the text of this section tends to refer to projects in construction, good risk management
practices can be useful throughout a project’s life cycle, including during operations. “The best
laid schemes of mice and men / Often go awry.” Implementing preventative mitigations and
pre-planning alternative strategies will reduce the likelihood and impact of these events.
The following subsections provide guidelines for planning the Risk Management processes,
developing the RMP, creating a Risk Register, and calculating a quantized measure of risk
exposure that leads to the establishment of contingencies. Examples of accepted or good
practices are included as guidelines.
Performance Measurement Baseline (PMB) is the approved cost and schedule plan established at award for accomplishing the
scope that can be changed only through formal change control process.
1
Section Revision:
November 25, 2016
6.2.1-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.2 Definition of Project Risk and Risk Exposure
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.2 Definition of Project Risk and Risk Exposure
Risks are defined many ways. One of the most inclusive definitions, and the one used in these
guidelines, is; “… an uncertain event or condition that, if it occurs, has a positive or negative
effect on at least one project objective.” 1
Most international standards agree that risk is made up of both threats and opportunities.
Capturing and capitalizing on opportunities to reduce costs, save time, and improve technical
performance may improve the possibility of finishing on time, budget, scope and quality by
offsetting the negative impact of threats to those objectives. The tools and methods employed
in managing threats are also used to identify and take advantage of these opportunities for
reducing project cost or schedule or improving technical performance. NSF requires
Opportunity Management as a necessary component of risk management.
Project Risk Exposure is the quantized result on project objectives of various risks and
uncertainties occurring. Project risk exposure is usually expressed as an amount of budget or
time that is the output of a Quantitative Risk Analysis that combines probability of occurrence
with consequence. Project risk exposure diminishes over time as risks are realized or avoided
and should always be less than or equal to remaining contingency amounts.
This definition is used in the Guide to the Project Management Body of Knowledge, (PMBOK® Guide), Project Management
Institute, 5th Edition, 2013, Chapter 11.
1
Section Revision:
May 2, 2016
6.2.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.3 Definition of Allowable Contingencies
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.3 Definition of Allowable Contingencies
Contingencies are a necessary component of risk management for NSF projects – they provide
the wherewithal and flexibility to control risks and realize opportunities. Contingency
allocations are for in-scope deliverables and are to be used to mitigate identified risks and
uncertainties that may impact a projects ability to achieve approved project objectives.
6.2.3.1 Allowable Contingency
Most risk management guides define two general types of budget and schedule contingency:
Contingency: “a planned amount of money or time which is added to a baseline
estimate to address specific, identified risks.” 1 It is an estimated amount based on
various risk management techniques and managed by the project.
Management reserve: “a planned amount of money or time which is in addition to the
baseline estimate and contingency to address unforeseeable events.” 2 It is not managed
by the project.
NSF does not normally carry management reserves as defined above. For NSF projects, only the
first type, Contingency, is allowed in the Recipient’s portion of the NSB authorized Total Project
Cost (TPC). This means that the estimation of contingency amounts should be tied to risks
identified at the time the total budget and duration are set, and that such contingency can only
be used to mitigate those pre-identified risks. See Section 6.2.7 for using proper quantitative
estimating methodologies for determining risk-based contingencies.
In addition to budget and schedule contingency planning, NSF requires projects to assess
possible use of scope contingency and to develop a plan to make effective use of scope
contingency options, if necessary, during construction. This provides the project with an
additional tool to manage the overall project. Use of all contingency is managed through formal
change control processes, as described in Section 4.2.5.
6.2.3.2 Contingency Definitions
Contingency for NSF projects includes cost, schedule, and scope amounts, as defined below:
Budget Contingency: An amount added to a baseline budget estimate to allow for identified
items, conditions, or events for which the state, occurrence, or effect is uncertain, and that
experience shows will likely result, in aggregate, in additional costs. Typically estimated
using statistical analysis or judgment based on past asset or project experience.
Identified risks are often referred to as “known unknowns” in the literature. In other words, a risk that can be identified during
planning is “known,” but the probability of occurrence and the extent of its impact cannot be determined with accuracy and are
therefore “unknown.”
1
Unforeseeable events are those that are not or cannot be identified during planning and are typically referred to as “unknown
unknowns” in the literature. They may also include low probability, extreme events that are beyond project control, such as the
effects of terrorism and war, natural disasters with impacts beyond expected historical ranges, or global economic crises.
2
Section Revision:
May 30, 2019
6.2.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.3 Definition of Allowable Contingencies
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
For major facility construction projects, the amount of budget contingency is determined by
performing a probabilistic risk analysis on the baseline cost and schedule and selecting a total
project cost with an acceptable confidence level (typically between 70%-90%). See Section 4.2.5
for details on total project cost requirements. Budget contingency is held separately from the
Performance Measurement Baseline (PMB) and allocations of budget contingency to and from
the PMB are managed through formal change control.
Schedule contingency: An amount added to a baseline schedule estimate to allow for
identified delays, conditions, or events for which the state, occurrence, or effect is uncertain,
and that experience shows will likely result, in aggregate. Typically estimated using
statistical analysis or judgment based on past asset or project experience.
For major facility construction projects, the amount of schedule contingency is determined by
performing a probabilistic risk analysis on the baseline schedule of activities and selecting a
commitment finish date with a confidence level between 70%-90%. The project end date is
determined by the sum of the baseline duration and the selected contingency amount.
Schedule contingency is held separately from the PMB and allocations of schedule contingency
to and from the PMB are managed through formal change control.
Scope contingency: Scope included in the project baseline definition that can be removed
without affecting the overall project’s objectives, but that may still have undesirable effects
on facility performance. Identified scope contingency should have a value equal to at least
10% of the baseline budget.
Scope contingency can be retained or deleted, depending on project risk performance and
available contingency, in order to stay within the total project cost. A scope management plan
includes a time-phased estimate of available budget and or time from de-scoping options,
based on key decision points. See Section 4.2.5 for details on requirements. Implementation of
scope contingency options is managed through formal change control.
Section Revision:
May 30, 2019
6.2.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.4 Risk Management Steps and Methodology
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.4 Risk Management Steps and Methodology
The steps involved in the Risk Management process have been defined variously by different
practitioners. For the purposes of these guidelines, the Risk Management process is defined as
comprising the following steps: 1
•
•
•
•
•
•
Risk Management Planning
Risk Identification
Qualitative Risk Analysis
Quantitative Risk Analysis
Risk Response Planning
Risk Monitoring and Control
The relationship between these steps is shown in Figure 6.2.4-1.
Figure 6.2.4-1
Picture of Six Risk Management Processes (According to PMI)
Risk Management
Planning
Risk
Identification
Risk Monitoring
and Control
Qualitative Risk
Analysis
Risk Response
Planning
Quantitative Risk
Analysis
The Risk Management steps outlined above are iterative and continuous and any one step, or
all of the steps, could be active at any given time. Risk analysis is performed continuously
throughout the project life cycle. For example, a conceptual risk analysis may be conducted to
facilitate selection between alternative options, to determine the level of project management
1
The six steps are the same as the processes described in the PMBOK® Guide Chapter 11.
Section Revision:
May 2, 2016
6.2.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.4 Risk Management Steps and Methodology
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
required, to identify where the challenges lie, and to determine the level of technical
information and development activity necessary to achieve project success. That risk analysis is
then updated during each of the life cycle phases of the project. Performing risk analysis is
particularly necessary in preparing for key project decisions. Periodic reviews of the risks at
appropriate intervals should be performed to identify new risks, to evaluate progress in risk
handling strategies, as well as to evaluate changes during the project development and
implementation cycles. Risk Management Planning, and the RMP, may also need to be
re-addressed at times of significant change, such as transitions from one project life cycle to
another or during a re-baselining with significant modifications to the project baseline.
Section Revision:
May 2, 2016
6.2.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.5 Risk Management Planning
Planning begins by developing and documenting a Risk Management strategy. Early efforts
establish the purpose and objectives, assign responsibilities for specific areas, identify
additional technical expertise needed, describe the assessment process and areas to consider,
delineate considerations for mitigation planning, define a rating scheme, dictate the reporting
and documentation needs, and establish report requirements. The strategy to manage root
causes provides the program team with direction and a basis for planning. The output of risk
management planning is a written document – the Risk Management Plan (RMP) – containing
the details of how risk will be managed through application of tools and processes defined in
the plan. See the next subsection for a description of requirements for the RMP.
One key strategic decision that should be made early in the Risk Management planning is the
selection and assignment of personnel with appropriate capability in Risk Management to lead
and/or guide the planning and analyses. As will be seen from the topics presented in the
analysis portion of the section, the art and science of risk management can be extremely
complicated for complex, high risk projects. While project managers, scientists and engineers
may have expert knowledge and judgment for identifying, estimating impacts from, and
defining mitigation for individual risks, they are usually not expert in estimating the overall or
aggregate risk exposure to the project from the combined impact of many individual risks.
Finding qualified resources to meet the risk management requirements of the project,
particularly for establishing the amount of contingency, should be a high priority for early
planning in order to ensure that methodologies and programming tools can be selected and
implemented in a timely manner. Options include sending existing staff for specialized training
in risk management and tool usage, directly hiring risk management experts, contracting with
industry, or some combination of the above.
A second early key decision is the determination of what risk assessment methodologies and
tools will be used, from first estimates through construction. The sophistication of the
appropriate risk assessment tools typically increases with advancing planning detail and
maturity, as well as with increasing project complexity. A project that includes a high number of
procurements and in-house tasks typically requires software applications and methods that use
a fully resource-loaded schedule for risk assessment and contingency estimation, while a
project entailing management of a single large contract may be adequately served by tools and
methods that use cost spreadsheets and summary level schedules. Choosing the appropriate
tools and methods at the outset can avoid the need and the burden of changing to different
systems as the project planning matures.
Risk Management planning is iterative. Normally, the risk management methodology and
procedures are defined as part of the risk management process planning early in the Design
Stage, but they may be extended or modified during design and execution as long as the efforts
remain within approved scope.
Section Revision:
May 2, 2016
6.2.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.5.1 Risk Management Plan (RMP)
The Risk Management Plan (RMP) describes how risks are documented and risk management
will be applied on the project. It is an integral part of the PEP, as outlined in Section 3.4 of this
Guide. The level of detail in the plan and the scope, timing, and level of risk analysis should be
commensurate with the complexity and maturity of the project as it advances through Design
and Construction Stages. The plan is a living document used throughout design and
implementation and should therefore be under configuration management. The Risk
Management Plan should include the following elements:
•
•
•
•
•
•
Risk Management Strategy and Approach
Roles and Responsibilities
Processes used to apply the Risk Management process
Baseline definition for Calculating Risk Exposure and Contingency needs
Contingency Estimating and Management
Resources assigned to and schedule, cost, and timing of risk management activities
The Recipient should periodically review the RMP and revise it, if necessary. Some events may
drive the need to update an existing RMP, such as: (1) the baselining of a project,
(2) preparation for a major decision point, (3) technical audits and reviews, (4) an update of
other project plans, and (5) a change in major project assumptions. A sample format with the
expected content for a Risk Management Plan (RMP) is outlined in Table 6.2.5-1.
Table 6.2.5-1
Section
Sample Format for a Risk Management Plan
Description
1.
Introduction
This section should address the purpose and objective of the plan, and provide a
brief summary of the project, to include the approach being used to manage the
project, and the acquisition strategy.
2.
Definitions
Definitions used by the Recipient should be consistent with NSF definitions for ease
of understanding and consistency. However, the NSF definitions allow program
officers flexibility in constructing their risk management programs. Therefore, each
Recipient’s RMP may include definitions that expand the NSF definitions to fit its
particular needs. For example, each plan should include, among other things,
definitions for the ratings used for technical, schedule, and cost risk in qualitative
risk analysis.
3.
Risk Management
Strategy and
Approach
Provide an overview of the risk management approach, to include the status of the
risk management effort to date, and a description of the project risk management
strategy.
4.
Organization
Describe the risk management organization of the Recipient and list the roles and
responsibilities of each of the risk management participants.
Section Revision:
May 2, 2016
6.2.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
5.
Resources
Implications of
the Plan
The resources to be used in managing risk on the project should include the time of
management and project team members as well as risk specialists and contractors if
appropriate, to effectively manage the risks on the project. These risk management
costs should appear specifically in the project budget.
6.
Schedule
Implications of
the Plan
The time periods in the project schedule when risk management activities are
planned to occur. Activities providing sufficient time to perform the tasks and
milestones to record their completion should be inserted in the project schedule
and statused along with the schedule statusing plan.
7.
Risk Management
Process and
Procedures
Describe the project risk management process to be employed, i.e., risk planning,
qualitative and quantitative risk assessment, handling, monitoring and
documentation, and a basic explanation of these components. Also provide
application guidance for each of the risk management functions in the process. If
possible, the guidance should be as general as possible to allow the project’s risk
management organization flexibility in managing the project risk, yet specific
enough to ensure a common and coordinated approach to risk management. It
should address how the information associated with each element of the risk
management process will be documented and communicated to all participants in
the process, and how risks will be tracked to include the identification of specific
metrics if possible.
8.
Risk Planning
This section describes the relationship between continuous risk planning and this
RMP. Guidance on updates of the RMP and the approval process to be followed
should be included.
9.
Risk Identification
This section of the plan describes the identification process. It includes procedures
to be used for examining the critical risk areas and processes to identify and
document the associated risks.
10. Risk Register
Analysis and
Ranking
This section summarizes the analyses process for developing a qualitative or
quantitative risk rating and populating the Risk Register. This rating is a reflection of
the potential probability of each risk and the impact of each risk on the project
schedule, cost, scope and quality. It also describes how the risk analysis data will be
collected and maintained throughout the project’s life cycle.
11. Probabilistic Risk
Analysis and
Contingency
This section describes the way the project will analyze the implications of identified
and quantified risks on the total project schedule and cost objectives or major
milestones. Typically, a Monte Carlo simulation is used based on the project
resource-loaded schedule or on the cost estimate if a schedule is not available. This
section also describes the use of the risk analysis results for setting contingency
amounts and prioritizing risks for risk mitigation.
12. Risk Handling
This section describes the risk handling options and identifies tools that can assist in
implementing the risk handling process. It also provides guidance on the use of the
various handling options for specific risks.
13. Risk Monitoring
This section describes the process and procedures that will be followed to monitor
the status of the various risk events identified including the frequency and
organizational level of risk review. It provides for identification and calibration of
Section Revision:
May 2, 2016
6.2.5-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
new risks should they arise. It should provide criteria for the selection of risks and
risk mitigations to be reported on, and the frequency of reporting. Guidance on the
selection of metrics should also be included.
14. Risk Management
Information
System,
Documentation
and Reports
This section describes the management information system structure, rules, and
procedures that will be used to document the results of the risk management
process. It also identifies the risk management documentation and reports that will
be prepared; specifies the format and frequency of the reports; and assigns
responsibility for their preparation and dissemination.
15. Risk Exposure and
Contingency
Management
This section describes the specific process and procedures used to determine
construction project risk exposures and the concomitant contingencies for scope,
cost, and schedule. It describes contingency management plans and processes and
ensures that contingency use is linked to both an identified risk and an appropriate
Work Breakdown Structure (WBS) element within project scope.
6.2.5.2 Roles and Responsibilities
Typically, the Project Manager or a designated Risk Manager (RM) is responsible for leading the
identification and analysis of project risks. All stakeholders (e.g., users, designers, and sponsors)
involved in the project are asked to provide input on what they deem to be the risks for the
project, possible risk mitigations, and ways to capture potential opportunities. The RM
consolidates the information collected and creates the list of risks with accompanying
attributes and manages the response to the risks. An example of a Roles and Responsibilities
table for key stakeholders and project staff that meets requirements is shown below Table
6.2.5-2.
Table 6.2.5-2
Roles
Example of a Risk Management Roles and Responsibilities Table
Responsibilities
Organization
Management
• Support the risk management process. Encourage all levels of the project organization to
participate fully and openly in the process.
• Make project decisions based in part on the results of risk analysis.
• Provide the culture that supports risk management and welcomes honest and realistic
results.
Risk Manager
• Oversee the Identification and documentation of new risks (threats and opportunities) in
the risk register
• Oversee the analysis of risks by the project team and work with them to develop risk
response plans (mitigate, avoid, accept, and transfer).
• Oversee reporting and tracking of risk activities during project status meetings
• Document and communicate risk activities frequently with stakeholders
• Review risks as they are concluded, and identify lessons learned
• Recommend and champion mitigation strategies to the Change Control Board (CCB) on
behalf of the risk management team.
Section Revision:
May 2, 2016
6.2.5-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Project Team
Members
• Assist the RM with the risk identification, qualitative and quantitative risk analysis and
development of risk response plans (mitigate, avoid, accept, and transfer). Participate in
risk workshops and interviews to provide risk data.
• Submit new threats, opportunities, and mitigations into the risk system as they arise.
• Assist the RM with the development and execution of risk response plans
• Attend project risk status meetings, as needed, and assist RM with the reporting and
tracking of risk activities
• Assist the RM with documenting and communicating the risk (threats and opportunities)
activities frequently with stakeholders
• Review risks as they are concluded, and identify lessons learned
Risk Owner
• Assist the risk originator (PM, RM, project team member, etc.) with development of the
risk descriptions
• Assist the RM and project team with the analysis development of risk response plans
(mitigate, avoid, accept, and transfer) contingency plans
• Update the risk register with modifications to risks
• Monitor the risk triggers and update the risk register
• Attend project status meetings, as needed, and assist the PM with reporting and tracking
risk activities
• Assist the RM and project team with documenting and communicating risk activities with
stakeholders
• Capture risk closure notes in the risk register and lessons learned
Section Revision:
May 2, 2016
6.2.5-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.6 Risk Identification
6.2.6.1 Risk Identification Process
Risk identification is an organized approach for determining, listing, and describing events that
might impact a project’s objectives (for example, time, cost, scope, and performance). It is an
iterative process that is conducted throughout the entire project life cycle. The risks are
described with a basis as to why this event is considered a risk. Identification relies on the skill,
experience, and insight of project personnel and subject matter experts (SMEs), as well as the
Recipient’s project manager, the NSF Program Officer, and the NSF Grants and Agreements
Officer. The objective of risk identification is to describe all the relevant risks so that the group
can focus on uncovering the probability and impact of the risks on project objectives (used in
qualitative risk analysis) or activities / costs affected (used in quantitative risk analysis). The
process for performing Risk Identification, along with inputs and outputs, is given
in Figure 6.2.6-1.
Figure 6.2.6-1
Risk Identification Process
Process
• Project schedule,
scope, budget
• System and project
artifacts
• Meeting Minutes
• Environmental
factors, PEP
Inputs
• Review key project/ phase deliverables
• Meet with the Risk Manager and project team
on a regular basis to discuss and identify
potential risks
• Interview key stakeholders
• Review existing risks to validate
• Update risk or add new risk to Risk Register
• Brainstorm for risk at status meetings
• WHEN: at the beginning of projects, project
milestones, and at weekly/ monthly status
meetings
• New and updated
risks
• Updated Risk
Register with
description
• Potential risk
responses
Outputs
Techniques used in identifying risks include leveraging existing project artifacts and guidance
documents, as well as proactively searching and gathering information to assist in that identification. The quality and completeness of risk identification is primarily dependent upon the
knowledge and experience of the project team and its commitment to risk management
processes. For example, the following basic methods can be used to assist in the identification
of risks:
Section Revision:
May 2, 2016
6.2.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
•
•
•
•
Brainstorming
Diagramming
Interviewing
Analysis of existing project artifacts
Comparison to historical information
Formal risk identification is performed in the early part of the project life cycle and as part of a
continuous effort during the project life cycle. Any person associated with the project should be
encouraged to continually identify potential project risks. Risk Identification, whether in
workshops or in interviews, should include at least the following participants:
•
•
•
•
•
•
•
Project managers
Project team leaders
Project team members
Business stakeholders
SMEs Contractors
SMEs outside the project team, for unbiased perspective
Project partners (e.g., foreign agencies, organizations with diverse objectives)
One immediate outcome of risk identification is the populating of the Risk Register, or tracking
tool, with the identified risks. The priorities based on impacts to project objectives and plans for
handling and reducing impacts will be added after analysis and risk handling planning, as
described in later sections. The Risk Register provides a means of tracking and reporting status
as risks occur and mitigation strategies are implemented and is an important tool for Risk
Management implementation.
6.2.6.2 Risk Identification and the Risk Register
The Risk Register includes a description of all risks that are deemed to be important to
achieving project success (from the Risk Identification process) along with an assessment of
those risks (using Qualitative Risk Analysis) that allows risks to be prioritized for effective risk
management. The results of identification, qualitative analysis, and risk handling – the major
components in the Risk Register – can lead to further analysis (Quantitative Risk Analysis, for
example).
Each risk should be assigned a unique identification number or code. Once a risk is entered into
the Register, it is never deleted. Its state may be changed to inactive (for example, retired,
closed, or merged with another risk), but it should never be deleted from the register. Risk IDs
are never reused.
The Register should be accessible (read-access) to all project members – the primary objective
of the Register is to keep the project team thinking pro-actively about how to avoid or mitigate
threats and take advantage of opportunities. It can be a spreadsheet, data base, or a specialized
risk management software tool. Changes to the Register should be managed through the risk
Section Revision:
May 2, 2016
6.2.6-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
process, which often may restrict ability to make changes (write-access) to a small team or to
the Risk Manager. The mechanism for all project members to read, comment on, and submit
new risks and mitigations should be established in the RMP.
The examples shown here and in Section 6.2.7, Qualitative Risk Analysis – Risk Register Ranking,
lead to a Risk Register containing a ranked subset or summarized set of risks based on
individual, qualitative impact analysis. Note that numeric impacts determined by the qualitative
method for individual risks may not be summed or combined to give a value for overall project
risk exposure.
Further discussion of Risk Register content is given in Section 6.2.7, Qualitative Risk Analysis –
Risk Register Ranking, with a sample Risk Register shown in Figure 6.2.7-6.
6.2.6.3 Risk Description
The risk description serves as a key point in the Risk Register and will be generated and updated
as needed. If there is a trigger event that causes the risk or foretells the risk’s occurring, it
should be described since it will specify what condition(s) would launch the risk and maybe
activate a contingency plan.
Risks (both threats and opportunities) are typically identified and tracked using the following
sentence structure for the Risk Description:
“Because of (some cause) a (risk) may occur, and (consequences) will happen.”
Example:
“Because foreign exchange rates may change, the cost of components in WBS 3.1 and 2.6
may increase or decrease, causing cost variances which affect contingency use.”
Using this format helps to distinguish the uncertainty or risk from its cause and its
consequence, a distinction which is important for mitigation planning. For instance, a statement
that “we have 5 schedule risks” is focused on an objective (schedule) that is impacted, not the
root cause of the risk or uncertainty. Alternatively, a statement that “the risk is that the
technology is really hard” does not lend itself to mitigation efforts. Difficulty of technology is a
fact or a “cause” in this format, which cannot be changed. The risk may be that “we do not have
the right skills on the project to handle the complexity” or “we may have to rely on third parties
to gain control of this technology.” That is a risk that can perhaps be mitigated. A possible risk
description for this scenario may be:
Because the technology for the major components in WBS 2.7 is very advanced, we may
have to rely on third parties for design, with the consequence that we have less control over
cost and schedule and an added burden of increased communication efforts.
Risks should be identified that are both internal, perhaps under project control, and external,
likely to be beyond project control. Risks for which there are no plans for mitigation should also
Section Revision:
May 2, 2016
6.2.6-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
be included in the register. 1 However, note that NSF does not allow the use of contingency for
risks that are commonly referred to as “unknown unknowns” 2 such as exceptional events or
major changes in scope. These exceptional events may not be included in quantitative risk
analyses used in determining contingency amounts.
6.2.6.4 Risk Identification Concerns
Efforts should be made to identify all risks to the project as early as possible, employing all
stakeholders identified in and using the techniques listed in Section 6.2.6.1, Risk Identification
Process. At the time of the risk analysis there are likely to be risks that are currently unknown
but may be revealed at a later date. When they become apparent, they can be then analyzed
and a “corrective action” can be specified and implemented. The objective of the Risk
Management program is to minimize the number of unanticipated issues and to address them
when possible and prudent.
Some people believe that project risk is often underestimated in both cost and schedule,
leading to well-known, sometimes notorious, overruns. 3, 4 Historical experience suggests that
mega-projects suffer from such problems systematically. 5 Strategic or overarching risks are
often missed in the risk identification process since the participants do not think globally, only
locally. Systemic or overarching risks are often not discussed or even considered during risk
identification. There may be cultural bias that leads to optimistic thinking in which threats are
systematically underestimated, outcomes are assumed to be achievable with less than realistic
effort and the potential for setbacks and rework is ignored. Any tool or technique that will
encourage people to “think outside of the box” when identifying project risk will help identify
the possibility of large overruns – when caught early, these risks may be manageable.
One common issue in identifying and collecting project risks is that people’s response and
participation in the identification process may be “stove-piped.” That means that people will
ordinarily discuss threats and opportunities that have to do with their own area of
concentration. In practice the project teams and other SMEs have experience and knowledge
outside these narrow areas, so the data collection method used should encourage them to
think broadly and strategically when identifying risks. Reminding risk identification participants
of external, organizational and project management source-areas of risks can help elucidate
strategic risks that they know about but that are outside their narrow area of technical
The fact that “we cannot do anything about it” or “we choose to accept the risk” does not disqualify it as a risk to the project.
One can argue that these risks should be in the Risk Register and certainly in the Quantitative Risk Analysis.
1
2 In many risk management guides, a portion of contingency is designated as management reserve for “unknown unknowns.”
NSF does not allow this.
See Bent Flyvbjerg, Nils Bruzelius, and Werner Rothengatter, Megaprojects and Risk: An Anatomy of Ambition, 2003
Cambridge University Press, and Glenn Butts and Kent Linton, “NASA the Joint Confidence Level Paradox – a History of Denial,”
NASA Cost Symposium, April 28, 2009.
3
4
Challenges to Meeting cost, Schedule, and Performance Goals, NASA IG-12-21
5
Edward W. Merrow, Industrial Megaprojects, 2011, Wiley.
Section Revision:
May 2, 2016
6.2.6-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
expertise or their work assignment. Often the use of a standard Risk Breakdown Structure
shown in Figure 6.2.6-2 will encourage risk identification participants to think more broadly
about risks to the project.
Figure 6.2.6-2
Typical Risk Breakdown Structure (RBS) 1
Different approaches to Risk Management may subdivide risks into various categories for
analysis. For illustrative purposes, this guide will use cost, schedule, and technical or
performance risk as the categories used in examples. Other risk categories commonly in use are
programmatic, business or economic, design requirements, software, and technology risks.
Alternatively, the OMB Risk Categories shown in Figure 6.2.6-3 could be used as guidelines for
identifying the various types of risk that apply to the project (refer to “OMB Risk Categories”
document in Critical Infrastructure Protection (CIP) for detailed descriptions and examples of
these categories). See the Government Accountability Office (GAO) Cost Estimating and
Assessment Guide, GAO-09-3SP, Chapter 14, for more examples. Some projects may also decide
to differentiate between internal and external risks. In many cases, it may be advisable to use
different categories for various parts of a project. For instance, categories of risk may be
different for software development than for hardware procurement. Each project should
decide which categories are most appropriate for its use while establishing the Risk
Management Plan and processes.
1
This RBS was the initial model for the RBS in the PMBOK® Guide, Chapter 11 of the Project Management Institute
Section Revision:
May 2, 2016
6.2.6-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.6-3
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
OMB Risk Categories: to be used as a starting point for projects to select their own categories
Schedule
Initial Costs
Life Cycle Costs
Technical Obsolescence
Feasibility
Reliability of Systems
Dependencies and Interoperability
Surety (Asset Protection)
Risk of Creating a Monopoly
Capability of Agency to Manage the
Investment
11)
12)
13)
14)
15)
16)
17)
18)
19)
Overall Risk of Project Failure
Organizational and Change Management
Business
Data/Info
Technology
Strategic
Security
Privacy
Project Resources
Another related social or cultural issue in identifying project risks is that people are often
uneasy about (or even afraid to be) discussing risks that can be embarrassing or harmful to the
project. This unease is often experienced during risk workshops or in other group settings.
Social pressures to conform (“groupthink”) – to suppress dissenting opinions clearly unpopular
to the group, including management, to agree with others against personal opinion just to
move the workshop along, and to defer to people perceived to have greater expertise even
when in disagreement – often make it difficult for some people to speak out. 1 A possible
solution to the impacts of social pressure is to conduct one-on-one, in-depth interviews with
SMEs in which the interviewee is promised confidentiality. Such interviews often yield honest
opinions about what might affect the project’s success. Usually some or most of the risks
revealed and discussed in these sessions are not on the organization’s risk register and would
not be analyzed in qualitative or quantitative risk analysis in the absence of the interviews. For
these reasons it is important to provide a safe environment for project team members and
others to identify and discuss project risks.
Risk identifiers may have concerns about including risks that are 100% likely to happen
(sometimes these are called “uncertainties or issues”) in the Risk Register. If the risks are 100%
likely to happen and their impacts are known, they should be included in the PMB. Often,
however, a risk that is certain to occur will have an impact that is not already included in the
project execution plan and that needs to be handled somewhere else, such as in the risk
analysis. Or the risk may have an uncertain impact on project objectives. These situations call
for the risk to be identified, even if it is already happening or certainly will happen, so the risk
can be included in the Risk Register and the subsequent quantitative risk analysis. The objective
of risk identification is to describe all the relevant risks so that the group can focus on
uncovering the probability and impact of the risks on project objectives (used in qualitative risk
These phenomena are discussed in Understanding and Managing Risk Attitude, David Hillson and Ruth Murray-Webster,
Gower, 2005.
1
Section Revision:
May 2, 2016
6.2.6-6
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
analysis) or activities / costs affected (used in quantitative risk analysis). Once a risk that is
certain to occur has been included in the PMB it can be removed from the Risk Register.
Care should be taken to provide the same thoroughness of identification for events far in the
future as well as for the near term for projects that have an execution period of several years.
Project team members usually find that it is easier to identify and discuss the risks that are
current or on the near horizon than those that may occur much later in the project. Adding to
the difficulty is the fact that future events may not be well defined at the time of risk analysis.
The risk identification exercise should take special care to encourage the participants to look
into the future, maybe with the help of lessons learned documents or their own experiences on
prior projects, to see what risks are far down the project life cycle. Another useful technique is
to “walk through” the activities planned for later execution. Examples of unidentified risks may
include unexpected legal changes, technical performance issues, resource losses, etc. Other
sources of future risks might include the reliance on unproven or even just conceptualized
technology or on doing business with an organization or in a country unknown to the sponsors.
The risk identification should include thinking about risks that have happened on other similar
projects or might occur in later phases of this project. If the team spends some time discussing
these down-stream risks, they can perhaps remember other projects or conceptualize the
existence of risks that would otherwise go unreported.
Section Revision:
May 2, 2016
6.2.6-7
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.7 Qualitative Risk Analysis – Risk Register Ranking
6.2.7.1 Purpose of Qualitative Risk Analysis
Qualitative risk analysis involves determining the probability of the occurrence of a risk,
assessing the consequences of this risk on specific project objectives (time, cost, scope and
quality) if it occurs, and using the two dimensions of a risk (probability and consequence) to
identify a rank or “risk level.” This risk level represents a judgment as to the relative risk to the
project objectives and the project as a whole and is categorized as Low, Moderate, or High.
Based on the risk level, risks can be prioritized for mitigation to responses. The results of Risk
Identification, Qualitative Risk Analysis, and Risk Response Planning comprise the main
elements of the Risk Register. The process for performing Qualitative Risk Analysis, along with
inputs and outputs, is given in Figure 6.2.7-1 below.
Figure 6.2.7-1
• Risk and uncertainty
descriptions and
calibration
• Project artifacts such
as schedule, estimate
• Client and team
interviews for risk
data
• Other data gathering,
lessons learned
Inputs
Qualitative Risk Analysis Process
Process
• Assess the schedule and cost against good
practices
• Conduct in-depth confidential risk interviews for
probability, impact and activities / costs affected
• Calibrate and assign inherent uncertainty and
estimating error, providing for more uncertainty
for work performed in the future
• Run Monte Carlo simulation on the risk-assigned
project schedule or estimate
• Prioritize the risks to the project
• Mitigate the high-priority risks and create a
post-mitigated result
• Likelihood of
finishing on time
and budget
• Needed cost and
time contingency
amounts
• Identification of high
priority risks and
needed risk
mitigations
Outputs
Achieving risk reduction is an integral part of setting priorities, sequencing project work, and
responding to the most serious risks first. Thus, the identified risks should be prioritized.
Note that Qualitative Risk Analysis is applied only to individual risks and is not used in
estimating overall project risk exposure or in determining contingency amounts. The analysis of
impact or consequence, however, may serve as input to Quantitative Analysis used to estimate
overall risk exposure.
6.2.7.2 Considerations When Performing Qualitative Risk Analysis
A number of factors complicate qualitative risk analysis, including:
•
Risk data, like data about the future contained in cost estimates or project schedules,
have a significant content based on subjective, expert judgment. The evaluation of risks
Section Revision:
November 25, 2016
6.2.7-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
to cost and schedule therefore generate approximate rather than precise results. “There
are no facts about the future.” 1
•
The term “risk” includes both “threats” and “opportunities” for NSF purposes. Project
risk team members should look for uncertainties that could help improve the project’s
results or offset threats. Both threats and opportunities should be examined for total
project impact, since opportunities for one project participant may be considered
threats by another, and vice versa.
•
The probability that a risk may occur and the impact if the risk were to occur should be
evaluated separately before combining the two parameters in a risk matrix. The idea
that “the risk is unlikely so its impact must be low” confuses the two parameters of
probability and impact. SMEs should be asked to estimate the impacts as if the risk has
occurred. Probability and impact will be considered together in the combined risk matrix
approach.
•
It is good practice to assess risks’ impact on separate objectives such as time, cost,
scope or quality/performance Impact ranges rather than creating a single, overall risk
level for the risk. Thus, ranking levels are defined for each of these objectives. For
instance, a risk can be judged to have a high impact on time but a moderate impact on
cost and a low impact on scope.
•
The definitions of impact on each project objective (very low, low, moderate, high and
very high) are set by the Risk Manager and documented in the RMP.
•
The definitions of combined risk level for probability and impact taken together (low,
moderate or high; or green, yellow or red) in the Risk Matrix are set by the Risk Manager
and documented in the RMP.
•
The impact of an individual risk may be modest and still be considered a high or very
high priority for mitigation. This is because the combined or aggregate risk of many
moderate risks may be high. The project may want to mitigate some low or moderate
risk in order to reduce the combined threat from many risks.
•
The risk register should include only root cause risks. Risks as defined in the plan may
not be mutually exclusive, that is, they may have the same root cause risk. Put another
way, if two or more risks are not mutually exclusive as written, their common root cause
risk should be identified and used instead.
6.2.7.3 Limitations of Qualitative Analysis
There are some limitations to the practice of qualitative risk analysis. Recognizing these will
help the organization appreciate and use the results correctly:
Lincoln E. Moses, Administrator of the Energy Information Administration, Administrator’s Message to the Annual Report To
Congress, 1977, Volume Three.
1
Section Revision:
November 25, 2016
6.2.7-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Qualitative Risk Analysis addresses the impact of individual risks on project objectives
one at a time. As such it is dedicated to prioritizing individual risks based on subjective
estimates of probability and subjective estimates of the impact to the project objectives.
It is not equipped to forecast or estimate overall project results such as the finish date
or total cost.
•
Qualitative Risk Analysis is unlikely to yield valid quantitative results since it usually does
not include all possible correlations and outcomes for impacted activities from a single
risk. SMEs should consider the risk with its probability of occurring and all the activities
in the schedule it would affect if it occurred, whether or not those activities are on the
risk critical path (the risk equivalent to the critical path in CPM scheduling). They should
also evaluate whether other risks might prevent mitigation of a particular risk from
resulting in much improvement. All of this analysis is being done in the individual SME’s
head. Such complex calculations are best handled by the Quantitative Risk Analysis
simulation method described in Section 6.2.7.
•
The estimation of the impact of a risk on cost should consider the impact of that risk
directly on cost plus the impact indirectly on the cost of time-dependent resources if the
risk also affects time. This is another calculation that individuals are not well-equipped
to make without a computer but is handled well by Quantitative Risk Analysis.
•
Judging whether a risk has a high-priority for the project would involve reviewing the
conclusions on each objective and asking the risk manager to prioritize the objectives.
Some projects are time sensitive and some are budget driven, others have a fixed scope
or could be de-scoped. These factors would need to be considered to determine
whether the risk is low, moderate or high priority for the project as a whole. Some
software packages that perform qualitative risk analysis assume that if a risk is “high”
for any objective it should be judged to be “high” for the entire project. There is no real
basis for doing so, since the risk may be judged to be high for an objective that is not the
most important for the specific project under consideration.
6.2.7.4 Qualitative Risk Analysis – Probability and Impact Assessment
Risk level determination can be done using a variety of techniques. The method given here
begins by assigning qualitative values for event probability and impact/consequence(s) to each
objective separately. These will then be used to determine a qualitative risk level. A key feature
of this method is that it requires independent assessment of the probability and consequence
of a risk.
The probability of a risk occurring is usually given to a range of possible probabilities of
occurring. Similarly, consequences are usually expressed in levels that represent ranges of
impacts judged by the risk manager to be of very low, low, moderate, high or very high impact
as the result of one risk among many. The ranking of a risk as it is applied to a particular
objective (e.g., time) is determined by the combination of probability and impact ranges, where
the project manager or some other stakeholder (e.g., the customer) determines which
Section Revision:
November 25, 2016
6.2.7-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
combinations would indicate that the risk is high, moderate of low priority for further study,
quantitative risk analysis or handling.
The following steps provide the details of this Qualitative Analysis method:
1. Address each risk statement from the Risk Identification process individually.
2. Determine the qualitative probability of occurrence value (P) range that best describes
the probability for each risk with appropriate basis and justification. Discuss the
probability that the risk might occur on the project with some noticeable effect on the
objective being discussed. Estimate the probability for the risk without regard to which
objective(s) the risk affects if it occurs. The probability of occurrence is for the duration
of all project phases. Table 6.2.7-1 provides an example of typical criteria for
establishing probability values.
3. Determine the qualitative consequence or impact of occurrence value I range that best
describes the impact for the objective such as time, cost, scope or performance for each
risk with appropriate basis and justification. In the evaluation, assume that the risk has
occurred and determine the recovery time, the cost of recovery, and the impact on
scope or quality. The consequence of occurrence is for the duration of all project phases
and for the objective being assessed. Table 6.2.7-2 provides typical criteria for
establishing consequence values. This table illustrates the different definitions that are
applied to the implications for time, cost, scope, and quality. Of course, these definitions
should be tailored to the project by the project manager or some other stakeholder
(e.g., the owner or customer).
Section Revision:
November 25, 2016
6.2.7-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 6.2.7-1
Sample Risk Probabilities Table
Each project should determine the appropriate number of levels and their definitions that match that project’s
circumstances.
Probability of
Occurrence Descriptor
Probability of Occurrence
Numerical Ranges
equivalent levels 1
Very Low
<0.1
Low
>0.1 but <0.4
Will not very likely occur in the life cycle of the project
or its facilities; or the probability of occurrence is
greater than 10% but less than or equal to 40%.
Moderate
>0.4 but <0.6
Will likely with middling probability (e.g., a coin flip) to
occur sometime during the life cycle of the project or its
facilities; or the probability of occurrence is greater than
40% but less than 60%.
High
>.6 but <.8
Likely to occur with more than 60 percent chance during
the project, or the probability of occurrence is between
60% and 80%
Very High
>0.8
Will likely occur sometime during the life cycle of the
project; or the probability of occurrence is greater than
or equal to 80%.
Table 6.2.7-2
Criteria in Words
Will not likely occur anytime in life cycle of the facilities;
or the probability of occurrence is less than equal to
10%.
Sample Risk Consequences 2 Table
The descriptors for the objectives of cost and time are explicitly given as numbers while those for scope and quality
are expressed in narrative descriptions.
Defined Conditions for Impact Scales of a Risk on Major Project Objectives, e.g., Time
Definition for Threats Only
Project Objective
1
Very Low
Low
Moderate
High
Very High
Cost
Insignificant Cost
Increase
<5% Cost Increase
<5 - 10% Cost
Increase
<0 - 20% Cost
Increase
>20% Cost
Increase
Time
Insignificant Time
Increase
<5% Time
Increase
<5 - 10 % Time
Increase
<0 - 20 % Time
Increase
>20% Time
Increase
Scope
Scope Decreases are
Barely Noticeable
Minor Areas of
Scope Affected
Major Areas of
Scope Affected
Scope Reduction
Unacceptable to
Sponsor
Project End Item
is Effectively
Useless
Quality
Quality Degradation
Barely Noticeable
Only Very
Demanding
Applications are
Affected
Quality
Reduction
Requires
Sponsor
Approval
Quality
Reduction
Unacceptable to
Sponsor
Project End Item
is Effectively
Useless
The scales still should be calibrated per the discussion and reference in Section 6.2.9.
An earlier version of this table was used in the PMBOK® Guide. The percentage ranges should be adjusted by the project
manager for the project and translated into days and dollars for ease of use.
2
Section Revision:
November 25, 2016
6.2.7-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Notice that the definitions for time and cost can be quantitative but that those for scope and
quality are generally descriptive. Sometimes a project’s scopes can be quantified, though it may
have several dimensions. Quality or performance might be measurable in terms of failure rates
or number of “fixes” that would be needed. The more the impact levels can be quantified the
more the responses by different people will be comparable. The project manager can calibrate
the numerical ranges for the specific project. The consequence definitions of very low, low,
moderate, high or very high (or Negligible, Marginal, Significant, Critical, and Crisis) may vary
considerably from a small to a large project. These tables should be provided as part of the
RMP.
It is preferable to refer to the numerical levels when gathering qualitative risk data since
definitions in words are often misleading. For instance, two people may use the term “Likely”
but mean different values. Or, one may say the risk is “likely” to occur and another may say
“unlikely” but mean the same numerical value, or at least in the same “bucket” or range of
values. Research has shown that the lack of overlap in assigning probability values with
common word definitions is severe. 1 (See Figure 6.2.7-2.)
Figure 6.2.7-2
Overlap in Risk Probability of Occurring When Descriptors Are Used
Expert judgment is required in risk analysis, just as it is for project scheduling and project cost
estimating. That is why several or many people need to be involved in providing their opinions
and experiences when assessing project risks. With multiple people assessing the probability
and impact of each risk against each objective, such as in the recommended in-depth
confidential interviews, there will be differences of opinion between them. The risk analyst has
1
Private research conducted by Dr. David Hillson in 2004 and presented at a PMI EMEA conference.
Section Revision:
November 25, 2016
6.2.7-6
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
to consolidate the data from different sources into one set of parameters for each risk and
objective from the dissimilar responses. This process uses expert judgment.
6.2.7.5 Alternative Approach to Qualitative Risk Impact Analysis ‒ Maxwell
An alternative approach is that proposed by F. D. Maxwell for projects such as those supported
by the Aerospace Corporation or the Space & Missile Systems Center of the US Air Force in Los
Angeles. See Table 6.2.7-3. The “risk driver category” is not the same as the project objectives,
but rather describes where the risks might be originating. This was not used by Maxwell in
conjunction with the probability before 1990, but it does illustrate definitions of impact that
were used on many aerospace and scientific projects. Maxwell stated that:
Special attention should be given to first-of-a-kind risks because they are often associated
with project failure. First-of-a-kind risks should receive a critical or crisis consequence
estimate unless there is a compelling argument for a lesser consequence value
determination.
Table 6.2.7-3
Risk Driver Category
Maxwell Risk Driver Assessment Framework 1
Very Low
Risk Level
Low Risk Level
Medium
Risk Level
High Risk
Level
Very High
Risk Level
1
Required Technical
Advancement
Nothing New
Minor
Modifications
Only
Major
Modifications
State of the
Art
Beyond State
of the Air
2
Technology Status
Currently in
Use
Prototype Exists
Under
Development
In Design
Concept State
3
Complexity
Simple
Somewhat
Complex
Moderately
Complex
Highly
Complex
Highly
Complex with
Uncertainties
4
Interaction/
Dependencies
Independent
of Other Risk
Drivers
Dependent on
One Additional
Risk Driver
Dependent on
Two Additional
Risk Drivers
Dependent on
Three
Additional Risk
Drivers
Dependent on
more than
Three
Additional
Risk Drivers
5
Producibility
Established
Demonstrated
Feasible
Known
Difficulties
Infeasible
6
Process Controls
Statistical
Process
Controls (SPC)
Documented
Controls
(No SPC)
Limited
Controls
Adequate
Controls
No Known
Controls
7
Manufacturing
Precision
High
Adequate
Limited
Margins
Known but
Inadequate
Unknown
8
Reliability
Historically
High
Average
Known Limited
Problems
Serious
Problems of
Unknown
Scope
Infeasible
Developed by F. D. Maxwell at the Aerospace Corporation. Included in “Estimating Cost Uncertainty when only Baseline Cost is
Available,” quoting R.L. Abramson and S. A. Book, “A Quantification Structure for Assessing Risk-Impact Drivers,” Laserlight
Networks, briefing presented to the 24th Annual DOD Cost Symposium (Leesburg, VA, September 5-7, 1990).
1
Section Revision:
November 25, 2016
6.2.7-7
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Risk Driver Category
Very Low
Risk Level
Medium
Risk Level
High Risk
Level
Very High
Risk Level
9
Criticality to Mission
Nonessential
Minimum
Impact
Known
Alternatives
Available
Possible
Alternatives
Exist
“Show
Stopper”
10
Cost
Established
Known History
or Close
Analogies
Predicted by
Calibrated
Model
Out of Range
of Experience
Unknown or
Unsupported
Estimate
11
Schedule
Demonstrated
Historical
Similarity
Validated
Analysis
Inadequate
Analysis
Unknown or
Unsupported
Estimate
Low Risk Level
6.2.7.6 Qualitative Risk Analysis – Risk Level Matrix
Once the probability and impact level of each identified risk is agreed to, the risk’s position is
determined on the probability and impact matrix shown in the following figures. The vertical
matrix axis labels in the figures correspond to the definitions for probability levels given in Table
6.2.7-1, and the horizontal axis labels correspond to the values for impact defined in Table
6.2.7-3. Combinations of probability and impact for a risk are shaded as red, yellow and green
for high, medium, and low risk level. The risk manager, project manager, or other stakeholder
should set these regions for each risk level, based on an understanding of the definitions of the
axes, which would cause the risk to rise to the appropriate level of attention.
Figure 6.2.7-3 shows a risk probability and impact matrix for one objective that is symmetrical.
Figure 6.2.7-4 shows a risk probability and impact matrix for an objective that emphasizes the
impact of the risk on its red-yellow-green status. This asymmetrical risk matrix indicates that
any risk that has a very high impact will achieve “high risk” or “red risk” status without regard to
the probability that the risk will occur on the project. Of course, the definitions of risk impact
and probability buckets defined in the RMP will determine the relative score that the risk
achieves.
Section Revision:
November 25, 2016
6.2.7-8
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.7-3
Symmetrical Risk Level Matrix
Figure 6.2.7-4
Asymmetrical Risk Level Matrix
The Risk Level for each objective for a defined risk depends upon where it falls in the Risk Level
Matrix according to the axes definitions. For example, a cost risk with an estimated probability
of 70% of occurrence and an estimated impact of $280K, or cost increase of 9% for the item at
risk, would fall into the High probability range and the Moderate cost impact range, according
to Table 6.2.7-1 and Table 6.2.7-2. Thus, the Risk Level for cost for this particular risk occurring
falls into the High, or “red” range in Figure 6.2.7-3.
It is important to scale probability and impact so that the risks can be distinguished. On the one
hand, if the lower bound for an impact score of very high is easy to reach there will be many
Section Revision:
November 25, 2016
6.2.7-9
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
risks with the same “red” assessment and the method will not distinguish risks for priority Risk
Handling. On the other hand, definitions of high or very high impacts that are very difficult to
reach will lead to very few or no “red” risks. While that may be true for some projects it would
be unusual for an NSF project with a high scientific impact.
The objective of the matrix is to communicate the choice of priorities for monitoring and
response, which may best be done with the 2-D diagrams (5x5) shown herein. Depending upon
the activity and the ability to differentiate the risk levels, other matrices may be chosen by the
risk analysis team. For example, a 5x5x4 risk level (the fourth level represents the 4 objectives)
matrix would then have five values for probability, five for consequence and four for objectives.
Recall that Risk Management entails the identification and ranking of opportunities as well as
threats. 1 Opportunities that are assessed to be in the “High” category are viewed as “lowhanging fruit” that can be easily claimed for the project if sought. For instance, if people are
coming off another similar project where they have had a good result, our project will benefit if
we can encourage or otherwise get them to join our project team. However, if such an
opportunity is not recognized in a timely manner, those productive people will go to other
projects. Another example is a potential cost saving if older but acceptable technologies can be
used in place of cutting-edge solutions without impacting performance or quality. This type of
cost savings is common in data acquisition and storage systems, for instance.
The butterfly or mirror risk probability and impact matrix below shows scoring threats and
opportunities in similar ways. The red-yellow-green ranges for threats have been discussed. The
red risks for opportunities are those that have a high likelihood of occurring and if they occur,
they help the project achieve its objectives, if only by offsetting threats. Risk Response of
opportunities needs to be proactive in order to secure these opportunities for the project.
Figure 6.2.7-5
Probability and Impact Matrix including Threats and Opportunities
An early discussion of the use of opportunities in project management can be found in: Effective Opportunity Management for
Projects: Exploiting Positive Risk, David Hillson, Marcel Dekker, 2004.
1
Section Revision:
November 25, 2016
6.2.7-10
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.7.7 Risk Level Input to the Risk Register
The Risk Levels per each objective for all identified risks are entered into the Risk Register. See
the sample Risk Register in Figure 6.2.7-6. It is common practice to also include a column in the
matrix for the probability descriptor for ease of reference. As mentioned before, it is good
practice to list the Risk Level for each project objective separately and not combine them into a
single risk level for the stated risk. Projects may choose, however, to designate a flag to identify
some risks as “Major” or “Top” risks. These Top Risks are judged by the project management to
call for more aggressive management and more frequent monitoring than other risks.
Communicating and tracking the status of the top project risks is a key element of project
management. The Risk Management Plan should address the frequency with which these
significant risks are tracked. Top risks should be reviewed and evaluated during standard subsystem team meetings and reviews as well as at project status meetings.
Projects should also include a status report for the top risks in the various required reports to
NSF, including the monthly report, as well as for reviews. One simple method for
communicating the summary status of top risks to various stakeholders is shown in the sample
Top Risk Matrix shown in Figure 6.2.7-7, which shows risk level and trend data for selected
risks.
Section Revision:
November 25, 2016
6.2.7-11
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.7-6
Sample Risk Register with Risk ID Number, Associated WBS Identification, Qualitative Probability and Impact for Initial and Post-mitigation States, and
Mitigation Actions
Pre-Mitigated Scores
Probability and Impacts
Post-Mitigated Scores
Resulting Risk Score
Risk
ID
Risk
Description
Associated
WBS
Prob.
Risk
Occurs
On
Sched.
On
Cost
On
Scope/
Quality/
Performance
Risk
on
Sched.
Risk
on
Cost
Risk on
Scope/
Quality/
Performance
PM1
If …., then …
2.2
H
VH
H
M
H
H
TECH1
If …., then …
3.2.5
H
VH
H
H
H
EXT8
If …., then …
3.1.3
M
VH
H
M
ORG4
If …., then …
1.2
M
H
M
PM4
If …., then …
2.4.1
M
H
TECH5
If …., then …
3.3.1
M
TECH6
Risk Action
Risk
Mitigation
Actions
Trigger
or
Watch
date
Major
Risk
Flag
Resulting Risk Score
Prob.
Risk
Occurs
On
Schedule
On
Cost
On
Scope/
Quality/
Performance
Risk
on
Sched.
Risk
on
Cost
Risk on
Scope/
Quality/
Performance
M
M
M
M
L
M
M
M
H
H
M
H
M
L
H
M
M
H
H
M
L
H
M
L
M
M
L
M
H
M
M
L
M
M
M
M
M
M
H
M
H
H
M
L
L
L
L
L
L
L
VH
H
M
H
M
M
VL
M
M
M
L
M
L
L
H
M
M
M
M
M
VL
L
L
M
L
L
M
EXT6
L
H
H
L
M
M
L
VL
M
L
L
M
L
L
PM2
M
L
H
L
M
H
M
L
L
H
VL
L
M
L
TECH9
VL
VH
VH
L
M
H
L
VL
H
VH
L
M
H
L
TECH10
VL
VH
M
VL
M
L
L
VL
M
L
VL
L
L
L
Section Revision:
November 25, 2016
Risk
Owner
Probability and Impacts
6.2.7-12
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.7-7 Sample Top Risk Matrix and Status Report, showing list of project risks selected as most significant to monitor on a frequent basis, with ranking and trend data
Note that top risks include some with Low and Medium criticality (or ranking), as well as those evaluated as High criticality.
Project Top Risk Matrix
Trend Rank
L
LI
KI
K
E
E
L
HI
O
H
O
O
D
O
D
5
RSK-025
RSK-025
RSK-105
4
SOF-RSK-005 SOF-RSK-001
RSK-098
RSK-065
RSK-101
RSK-076
RSK-103
3
SOF-RSK-007
RSK-066
2
RSK-053
SOF-RSK-003,
RSK-102
SOF-RSK-010
RSK-106
RSK-100
SOF-RSK-009
SOF-RSK-002,
RSK-059
SOF-RSK-006
RSK-051
1
1
2
3
4
High
Med
Low
Section Revision:
November 25, 2016
Risk ID
Approach
L x C Trend
Approach
Decreasing (Improving)
M – Mitigate
Increasing (Worsening)
W – Watch
Unchanged
A – Accept
New since last period
R – Research
Title
Inadequate Observatory
Performance for initial
operations
1
(4X4) RSK-025
M
1
(4x4) RSK-098
M
Operations Staff Fatigue
2
(3X4) RSK-102
M
Lack of Adequate
Planning for data analysis
pipeline
4
(3x3) RSK-103
M
Lack of Data Extraction
and Analysis Tools
5
(4x2) RSK-105
M
Scientist support staff
resources
6
(2x5) RSK-059
M
Loss of single-point-failure
component
7
(3x3) RSK-076
M
Maintaining Science
Instruments
8
(2X4) RSK-100
M
Instrument support
structural Integrity
9
(3x3) RSK-101
M
Undocumented Hardware
Requirements
3
(3X4) RSK-106
M
Science Instrument
Hardware Change
Control
10
(2x3) RSK-066
W
Operations Staff
Retention
11
(2x3) RSK-053
W
Changes in Safety and
Reliability Requirements
5
CONSEQUENCE
CONSEQUENCE
Criticality
LxC
6.2.7-13
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.7 Qualitative Risk Analysis – Risk Regist
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.7.8 Other Qualitative Risk Analysis Methods
Expected monetary value, simulation, Bayesian probability theory, reliability, and the use of
decision trees or its inverse, failure mode effects and criticality analysis (FMECA) are other risk
analysis methods that are used in project management. They are not described in detail here
but may be researched using the given references.
Expected monetary value is the product of the risk event probability multiplied by the value of
the gain or loss that will be incurred. Schedule impacts and intangibles (i.e., a loss may put the
organization out of business) should be considered when using this approach. This method for
scaling contingency amounts does not take advantage of information about the range of
possible impacts or probabilities. It can only provide a mean value of the contingency, not some
other target level of confidence. It is not good for time risks or cost risks that have time risk
components. 1
Any schedule of a real project can easily be handled using Monte Carlo simulation techniques, 2
discussed in the next section on Quantitative Risk Analysis. Simulation uses a model of a system
such as the project schedule to simulate a project using Monte Carlo analysis. Monte Carlo
“performs” the project many times so as to provide a statistical distribution of calculated
results under many different scenarios, since in each scenario different risks may occur with
different combinations of impact. The use of Monte Carlo analysis to estimate the risk schedule
or cost distribution by statistically combining risk costs is illustrated in the next section.
A decision tree is a diagram depicting key interactions between decisions and associated events
and uncertainties as understood by the decision-maker. 3 A FMECA is a bottoms-up version of a
decision tree, building up from the elements to the decisions. Either approach helps the analyst
to divide a problem into a series of smaller, simpler, and more manageable events that more
accurately represent reality to simplify decision-making.
Bayesian probability theory treats probability as a degree of belief or uncertainty in a given
statement. More information may be found in Foundations of Risk Analysis. 4
1
Integrated cost-schedule risk analysis is presented in Section 6.2.8, Quantitative Risk Analysis.
For schedule impact the organization should not use the Program Evaluation and Review Technique (PERT or the Method of
Moments) to represent project risk in schedules. This method underestimates risk for the type of projects addressed herein.
Refer to “Project Schedule Risk Analysis: Monte Carlo Simulation or PERT?” David T. Hulett, PM Network published by the
Project Management Institute, February 2000, pp. 43 ff
2
See Recommended Practice 85R-14, Use of Decision Trees in Decision Making, Association for the Advancement of Cost
Engineering International (AACEI), 2014, David T. Hulett principal author. Also, “Use Decision Trees to Make Important Project
Decisions,” David T. Hulett, Cost Engineering (published by AACEI, July / August 2014.
3
4
Pages 62 and 64 of Foundations of Risk Analysis by Aven.
Section Revision:
November 25, 2016
6.2.7-14
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.8 Quantitative Risk Analysis – Estimating Contingency
6.2.8.1 The Purpose of Quantitative Risk Analysis
Quantitative risk analysis can analyze the impact of all of the risks and uncertainties on the
project objectives of overall time and cost. Hence quantitative risk analysis can derive results
that qualitative risk analysis cannot provide, i.e., the likely finish date and project cost when all
risks are considered within a model of the entire project.
Quantitative risk analysis allows the risk analyst to estimate:
•
•
•
How likely is the project to meet its schedule and cost goals?
How much schedule and cost contingency is needed to achieve the project’s desired
level of certainty?
Which risks are causing any potential overrun and are thus high priority for risk
mitigation?
Quantitative risk analysis allows the analyst to estimate the finish date and cost of the project
based on a probability distribution created by applying Monte Carlo simulation to a project plan
such as the schedule, cost estimate or cost-loaded schedule. The inputs are uncertainty and
discrete risk events, although there may also be probabilistic branches, weather / calendar
effects and even conditional branches. The process for performing Quantitative Risk Analysis is
shown in Figure 6.2.8-1 below. Outputs are the estimated total cost and finish date and
associated contingency amounts above the baseline input cost and finish date.
Figure 6.2.8-1
• Risk and uncertainty
descriptions and
calibration
• Project artifacts such
as schedule,
estimate
• Client and team
interviews for risk
data
• Other data
gathering, lessons
learned
Inputs
Quantitative Risk Analysis Process
Process
• Assess the schedule and cost against good
practices
• Conduct in-depth confidential risk interviews for
probability, impact and activities / costs affected
• Calibrate and assign inherent uncertainty and
estimating error, providing for more uncertainty
for work performed in the future
• Run Monte Carlo simulation on the risk-assigned
project schedule or estimate
• Prioritize the risks to the project
• Mitigate the high-priority risks and create a
post-mitigated result
• Likelihood of
finishing on time
and budget
• Needed cost and
time contingency
amounts
• Identification of
high priority risks
and needed risk
mitigations
Outputs
A quantitative risk analysis requires an accurate, up-to-date schedule as well as up-to-date risk
data to be useful. The schedule used for analysis is often not the detailed resource-loaded
schedule (RLS) but is a summary schedule that can be resource loaded.
Section Revision:
December 15, 2020
6.2.8-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
While software tools have made it relatively simple to run a Quantitative Analysis, the
preparation work for a simulation run can take significant time and effort. Often, projects use
Qualitative Analysis for month-to-month risk management and use Quantitative Analysis for
establishing a new baseline or calculating an updated risk-adjusted estimate at completion
(RAEAC).
There are several commercial packages available that provide tools and programs for
performing Quantitative Analysis using cost estimates and/or resource-loaded schedules.
While NSF strongly recommends probabilistic analysis methods for estimating total project risks
and contingency amounts, it does not endorse or recommend any particular program or
product.
A typical result of a quantitative schedule risk analysis using one such commercial tool, in this
case a schedule risk analysis histogram of possible end dates, is shown in Figure 6.2.8-2. The
estimated ranges of impact of risks and uncertainties on the duration of scheduled activities
were fed into a Monte Carlo simulation program that generated a distribution of possible end
dates based on a resource-loaded schedule. For the histogram below, the horizontal axis shows
the possible end dates. The right vertical axis shows the end dates for the confidence level
curve. 1 The dotted lines on the plot represent the end dates for which the confidence level for
completion by that date is 50% and 80% respectively. For this example, the PMB end date is
11/20/2015. If the project elects to use the 80% confidence level, then the chosen project finish
date is 7/28/2016, indicating that the project needs to mitigate or provide contingency for an
additional 8.3 months beyond the baseline date.
NSF sets a required range for the confidence level unless an exception is requested and approved by NSF. See Section 4.2.5 for
details.
1
Section Revision:
December 15, 2020
6.2.8-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-2
Typical Result of a Quantitative Schedule Risk Analysis
Another typical output, from quantitative analysis of a resource-loaded schedule, is a time-cost
scatter diagram. Figure 6.2.8-3 plots cost on the y-axis against end date on the x-axis. A line is
drawn through the slope of the distribution. The plot illustrates the important fact that time
and cost are related. In this case, longer schedule activities with labor-type resources generate
higher cost.
Section Revision:
December 15, 2020
6.2.8-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-3
Time-Cost Scatter Diagram: Each data point represents one realization of the simulation
6.2.8.2 Key Elements in Quantitative Risk Analysis
The platform for quantitative risk analysis is the project cost estimate or project schedule. Since
most cost estimates are developed in a spreadsheet, a risk analysis of the project’s cost
estimate alone is often conducted using a software package that simulates a spreadsheet
model. 1 Schedule risk analyses simulate a project schedule, so software that is able to simulate
schedules developed in the organization’s preferred scheduling package should be used. 2
Integrated cost-schedule risk analyses involve a good-quality PMB schedule (i.e., without cost
Two commonly used packages are @RISK from Palisade Corporation and Crystal Ball from Oracle. (NSF does not endorse or
recommend any particular package.)
1
There are several schedule simulation packages available. Two of the schedule simulation packages with the most capabilities
are Polaris from Booz Allen Hamilton and Primavera Risk Analysis from Oracle. Others include Acumen RISK from Deltek, Risky
Project from Intaver Institute, @RISK for Project from Palisades, JACS from Tecolote and Full Monte from Barbecana. (NSF does
not endorse or recommend any particular package.)
2
Section Revision:
December 15, 2020
6.2.8-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
or schedule contingency) with loaded resources representing the cost estimate attached to the
activities they support.
The elements of risk that may affect the cost, duration, or both cost and duration of a project
include uncertainty, identified discrete risk events, and possible discontinuous events.
•
Uncertainty represents inherent variability in predicting the outcome of future events.
The uncertainty may be from people and organizations’ inability to do the same thing
the same way reliably or from the fact that future events cannot be predicted with
complete accuracy. Uncertainty has a probability of 100% (since it is always present) and
an estimated range for duration or cost. The range often has a positive tail (opportunity)
and a negative tail (threat) such as -5% and +10%. These ranges represent the
confidence in the estimates of activity duration or cost element actually occurring as
estimated. The uncertainty ranges are often specified as a 3-point estimate with low,
most likely and high values for a specified distribution shape, often a triangular
distribution. For every iteration in a simulation, the software pulls a random impact
multiplier for each duration and/or each cost item from within the chosen distributions.
That value, say 1.07, is then multiplied by the activity duration or element cost in the
model to get the value to be used for that iteration.
o There may be asymmetry in the range of uncertainty since it is often easier to
overrun than underrun an estimated value. Hence the optimistic tail of the
distribution may not have as much probability as the pessimistic tail has. Also,
the most likely value may not be the assigned value in the schedule or estimate.
Hence a fairly typical uncertainty range could be .95, 1.05, and 1.15 – the middle
value implies that the duration or cost is most likely 5% higher than in the
baseline model.
o The range of uncertainty can also be used to cover potential, but as yet
unidentified, discrete risks that may surface later in the project than at the time
of analysis. The inability to identify discrete risks is common for events that occur
significantly later in the project or for activities that cannot yet be well defined.
Most often these uncertain risks can be addressed by allocating a wider range of
uncertainty to these durations or costs than to those assigned to better
understood activities occurring in the early years of the project. In this way the
generally higher level of uncertainty for durations and costs in the later years of
the project can be included in the risk analysis leading to the size of the
contingency reserve.
o Some types of activities have more inherent uncertainty than others. It may be
more difficult to make estimates of duration and cost for testing than for design,
whereas fabrication may be somewhere in between. Therefore, some categories
of activities may have wider uncertainty ranges than others. These activity-type
specific uncertainty bands are sometimes termed reference ranges.
•
Discrete Risk Events include those already identified and quantified in the Risk Register
as well as any that may be discovered when interviewing for risk data to use in the
Section Revision:
December 15, 2020
6.2.8-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
quantitative risk analysis. Discrete risks are specified by their probability and range of
impact if they happen to activity durations or cost elements.
o The probability determines the fraction of the Monte Carlo iterations that they
appear in.
o The impact range is related to the duration of the individual activities or size of
cost line items that they are assigned to. Hence the concept of impact range for
quantitative analysis is not the same as that used for qualitative risk analysis,
which is impact on the final date or total cost for the entire project.
o A risk can affect many activities or cost elements. Activities or cost elements can
be affected by more than one, sometimes many, discrete risks.
o Discrete risks can be represented by adding a risk to a cost element or schedule
activity or by specifying a multiplicative factor to apply to the estimated cost (risk
register method) or activity duration (risk driver method).
•
Discontinuous Risk Events are discrete events that can have consequences beyond
adding duration to existing activities or cost to an existing budget element. Technically
challenging projects such as NSF facilities typically have numerous discontinuous risks.
Capturing a complete list of these risks is critical to effective RM and project success. For
example, failing a qualifying test (or other discontinuous event) may require adding new
activities and cost to the schedule in order to recover from the event. These activities
and cost elements are almost certain not to be in the baseline schedule or cost estimate
since those artifacts are usually based upon success of the baseline plan.
6.2.8.3 Platforms for a Project Quantitative Risk Analysis
A project schedule risk analysis starts with a good-quality Critical Path Method (CPM) schedule:
•
The schedule can be a summary or roll-up of the detailed schedule of the project and
should not have any padding or contingency for risk. Estimated project end date and
schedule contingency duration are outputs of the risk analysis. The detailed project
schedule is not always a good candidate for risk analysis input since it usually has several
thousand activities and may be difficult to debug. That is, the detailed project schedule,
perhaps a contractor’s schedule, may not conform to scheduling best practices. 1 Hence,
and in recognition that a schedule risk analysis is a strategic analysis of the project,
summary or “analytical” schedules may be used instead of the detailed schedule. This
analytical schedule needs to represent all the work of the project (including contractor
and other participants such as the customer) and be validated against CPM good
practices. It is recommended that the summary or analytical schedule format adheres to
the project WBS to facilitate reporting of contingency usage.
Characteristics of a schedule used for Quantitative Risk Analysis are:
One source of complete scheduling best practices is the Government Accountability Office (GAO) Schedule Assessment Guide,
2015.
1
Section Revision:
December 15, 2020
6.2.8-6
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
1.
2.
3.
4.
It represents all the work of the project,
All logic links are established,
All constraints are appropriate, and
It is resource loaded.
Since the schedule validation process can require significant effort by project leaders, some
references 1 recommend creating a 300-1000 line summary schedule from the project IMS and
resource loading it with a minimal number of summary resources. (Some multi-billion-dollar
projects have been known to use as few as eight summary resources.) That methodology is
followed in the exercise demonstrated in the following case study.
A cost risk analysis starts with a complete, PMB cost estimate:
•
The PMB cost estimate is complete for all in-scope work but does not include any builtin “padding” or contingency for risk. The estimated cost contingency amount is an
output of the risk analysis. The cost estimate is usually specified in spreadsheet format
and may be simple or detailed. It is recommended that the summary or analytical
schedule format adheres to the project WBS to facilitate reporting of contingency usage.
An integrated cost-schedule risk analysis starts with a resource-loaded schedule for a PMB with
cost and schedule estimates:
•
A schedule, either analysis or detailed level, that is loaded with resources. For the
purpose of a risk analysis the resources do not have to be detailed at the same level as
the Cost Book, but they do have to distinguish between time-dependent (e.g., labor,
rented equipment) resources that will cost more if their activities are longer and timeindependent (e.g., materials, purchased equipment) resources that may have variable
cost but not because of uncertainty in duration. Again, it is recommended that the
summary or analytical schedule format adheres to the project WBS to facilitate
reporting of contingency usage.
All quantitative risk analyses require:
•
Good quality risk data collected in the Risk Register but usually enhanced using good
interview techniques. Note that SMEs are often more willing to talk freely about
extreme good and bad possible risk results in confidential interviews.
•
A professional schedule risk simulation package 2 that can perform a Monte Carlo risk
analysis simulation on a risk-loaded schedule.
1 David T. Hulett, principal author, Recommended Practice 57R-09, Integrated Cost and Schedule Risk Analysis Using Monte
Carlo Simulation of a CPM Model, AACEI, 2011.
There are several different software packages that can do this analysis. The package used for these charts and tables is
Polaris® from Booz Allen Hamilton.
2
Section Revision:
December 15, 2020
6.2.8-7
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
An organizational culture that is committed to conducting an unbiased and realistic risk
analysis and to use its output, such as total risk to objectives or prioritized risk events to
be mitigated in order to improve the prospects of the project.
6.2.8.4 Case Study: Quantitative Risk Analysis 1 Exercise
These steps will be illustrated with a simple case study of an integrated cost-schedule risk
analysis of design and fabrication of a space vehicle, as shown in the resource-loaded Gantt
chart schedule shown in Figure 6.2.8-4.
Figure 6.2.8-4
Resource-Loaded Schedule Used for a Simple Case Study of an Integrated Cost-Schedule Risk
Analysis for Design, Fabrication, Testing, and Delivery of a Space Vehicle
This is a project starting June 1, 2008, with a ship to launch site end date of November 20, 2015.
The project cost is estimated at $651.6 million. 2 Resources are shown on the bar chart and
include mostly labor, with some equipment in the First Stage and Upper Stage Fabrication
activities.
1 David T. Hulett, principal author, Recommended Practice 57R-09, Integrated Cost and Schedule Risk Analysis Using Monte
Carlo Simulation of a CPM Model, AACEI, 2011.
This schedule has been developed in Microsoft Project. Another popular scheduling package is Primavera P6 from Oracle.
Most schedule simulation packages can import projects from these two scheduling packages.
2
Section Revision:
December 15, 2020
6.2.8-8
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
In this case study the resources, as shown in Table 6.2.8-1, are few and summary.
Table 6.2.8-1 Resources for Quantitative Risk Analysis Example
Resource Name
Type
Preliminary Designers
Work
Detail Engineers
Work
Fabrication
Material
Integrators
Work
Integration Testers
Work
Specification Writers
Work
Unit Testers
Work
Fabricators
Work
6.2.8.5 Schedule Risk Analysis ‒ Uncertainty
The schedule risk analysis starts with uncertainty reference ranges, estimated by the project
SMEs. Recall that the probability for uncertainty occurring is 100%, and thus occurs for all
simulation iterations for all assigned durations. The ranges shown in Table 6.2.8-2 are the SMEs’
estimates of uncertainty in the task durations. Note that three of these imply that the SME
interviewees assess the “Most Likely” value to be greater than the durations in the schedule.
This may be because they view the schedule as being built with optimistic durations or that
more has been learned about activity durations, leading to a higher estimate of the “most
likely” durations. Although not shown here, their evaluation could also have resulted in lower,
mostly durations. The use of Risk Drivers allows these distributions to have both threat and
opportunity tails.
Table 6.2.8-2
Schedule Duration Risk Reference Ranges
Activity Category
Low
Most Likely
High
Designers
0.90
1.00
1.20
Fabricators
0.95
1.05
1.20
Integrators
0.95
1.05
1.20
Requirement Writers
0.90
1.00
1.15
Testers
0.85
1.10
1.25
Notice that these are fairly narrow ranges that represent inherent variability, for instance, but
do not represent the impact of discrete risks on the activity durations. These ranges are applied
to the activities in the named categories by a triangular distribution, in this case, from which
the computer pulls at random a multiplicative factor that is applied to the schedule duration.
The example exercise demonstrated here uses 5,000 iterations because the software is fast, but
3,000 iterations would generally be enough.
Section Revision:
December 15, 2020
6.2.8-9
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Uncertainty ranges should be wider the further out into the future the work is being planned
and estimated. This is because it is harder to estimate durations or costs several years into the
future, since the work has not been contracted yet and may not actually be detailed with any
specificity. Also, there will be risks in the future that cannot be identified today as discrete risks
but should be provided for with wider uncertainty ranges.
The analysis is performed using the reference ranges. If the analysis stopped at this point with
just uncertainties, the schedule results would look like the histogram shown in Figure 6.2.8-5
below. The 80th percentile has been chosen as the target level of confidence for this example.
The target confidence level for actual projects is chosen by the project or the customer. 1 The
related cost risk histogram shown in Figure 6.2.8-6 represents the effect of duration uncertainty
alone on the costs for time-dependent resource.
Figure 6.2.8-5
End Date Results for Schedule Duration Uncertainties
To show these results one software package, Polaris, was chosen. However, these results can be achieved using Primavera
Risk Analysis, JACS, Risky Project and others.
1
Section Revision:
December 15, 2020
6.2.8-10
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-6
Cost Result for Schedule Duration Uncertainties
The results can also be shown in tabular form, with the 5% and 95% values included to
determine if the total range is believable. For uncertainties alone, the results in Table 6.2.8-3
are believable.
Table 6.2.8-3
Results with Schedule Uncertainties Assigned
Schedule
Baseline
5%
50%
80%
95%
Dates
20-Nov-15
8-Feb-16
19-May-16
14-Jul-16
7-Sep-16
2.6
6.0
7.8
9.6
Months from Base
Cost
Baseline
5%
50%
80%
95%
Dollars (millions)
651.6
660.6
684.4
697.1
709.5
1%
5%
7%
9%
% above Base
Because this is an integrated cost-schedule risk analysis there is interest in the relationship
between time and cost. This is shown in a finish date – total cost scatter diagram shown in
Figure 6.2.8-7. The scatter plot has a dot for each of all 5,000 iterations. The scatter slope
indicates the positive relationship between time on the horizontal axis and cost on the vertical
axis. The curved line toward the top right of the scatter represents those combinations of cost
and schedule results that exhibit a 70% probability of meeting both objectives, given the
uncertainties applied to the cost-loaded schedule. The target of 70% confidence level for
Section Revision:
December 15, 2020
6.2.8-11
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
budgeting and scheduling was chosen in this case since it is often used by several government
funded agencies, such as NASA. 1
Figure 6.2.8-7
Total Cost and End Date Scatterplot for Schedule Uncertainties
Note that the schedule uncertainty data in this example are assumed to be not correlated. If
they were correlated (i.e. if one is high in its range then the others would also be high in their
ranges), the extremes in cost and time would be greater and the correlation between time and
cost would be tighter than shown above. The analyst should explore whether the uncertainty
distributions should exhibit correlation or not. If so, then the analyst will want to exploit the
capabilities of the chosen analysis package to handle correlations.
6.2.8.6 Schedule Risk Analysis ‒ Discrete Risks Added as Drivers
The second step is to identify, calibrate and assign discrete risks to the project schedule. The
risks used in this example are applied to the categories of activities, including design,
fabrication, integration, testing and requirements. For this exercise the risks are given generic
names, but in an actual analysis the risks would be taken from the Risk Register and augmented
by risks discussed in the confidential risk interviews. The generic risks for this exercise, with
their probabilities are shown in the top section in Figure 6.2.8-8. One risk, “Organizational risk
affecting all,” has been selected to show its assigned impact range next to the triangle symbol
1
See: “Understanding the Joint Confidence Level (JCL) at NASA,” NASA Office of Evaluation at 9/4/14.
Section Revision:
December 15, 2020
6.2.8-12
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
on the right: Min 0.85, mode 1.05, Max 1.3. The Organizational risk has a probability of 70% and
is assigned to all tasks since its impact is felt on everything. Although the description has not
been filled in for this exercise, the organizational risk could stem from “lack of ready access to
key decision makers that can increase durations” or to “organizational red tape that could slow
decision making,” for example.
Figure 6.2.8-8
Schedule Risk Drivers – Organizational Risk
When these risk drivers are assigned to multiple tasks or activities, those activities’ durations
become correlated since (1) if the risk occurs it occurs for all activities to which it is assigned,
and (2) the multiplicative factor chosen for that iteration is applied to all of those activities. If
only one risk is involved the activities become 100% correlated. If other risks are also assigned
the correlation between activity durations is reduced. In this way the risk driver method models
how correlation occurs so SMEs do not have to guess at the correlation matrix. With the
addition of discrete risks to the analysis, the schedule impacts are more pronounced, and the
results show a later start (by 15.4 months) and higher cost (by $100 million) than with just the
uncertainties for the 80% confidence level. See Table 6.2.8-4 below. Note that the cost increase
is due to schedule duration risk drivers alone, and not to any cost uncertainty or risk.
The scatterplot in Figure 6.2.8-9 shows greater correlation of time and cost risk than the
previous plot showing uncertainties only, since the Organizational risk driver was assigned to all
activities.
Section Revision:
December 15, 2020
6.2.8-13
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 6.2.8-4
Results with Schedule Uncertainties and Discrete Risks Assigned
Schedule
Baseline
5%
50%
80%
95%
Dates
20-Nov-15
5-Jan-16
28-Dec-16
26-Oct-17
16-Aug-18
1.5
13.3
23.2
32.9
Months from Base
Cost
Baseline
5%
50%
80%
95%
Dollars (millions)
651.6
650.7
730.6
797.6
865
0%
12%
22%
33%
% above Base
Figure 6.2.8-9
Total Cost and End Date Scatterplot Showing Greater Correlation of Time and Cost Risk
6.2.8.7 Cost Risk Analysis ‒ Uncertainty and Discrete Risk Drivers
The last consideration in this simple example is whether there are uncertainties and discrete
risks for cost which would cause cost variations that are independent of schedule.
Examples of uncertainty could be errors in the time independent cost of fabrication, variances
in the time-dependent activities’ daily “burn rate” due to uncertainty in the number of
Section Revision:
December 15, 2020
6.2.8-14
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
hours/workers needed per day, and/or uncertainty in the estimated salaries. These risks, if they
occur, are in addition to the cost impact from schedule duration risks already discussed in the
previous material. The cost estimating error on the burn rate of labor or total cost of
equipment can be entered by resource as uncertainties, with probability of 100% and a range of
impact. Example uncertainty reference ranges for cost uncertainty as applied to different
resources for this exercise are shown in Figure 6.2.8-10.
Figure 6.2.8-10
Uncertainty in the Burn Rate and Total Cost
Resources and Their Utilization Uncertainty
UID
Resource
Type
Planned Units per Unit
or Day
Min - Most Likely - Max
Rate Per Unit or Day
1
Preliminary Engineers
Time Dependent
640
600 - 650 - 700
2
Detail Engineers
Time Dependent
960
900 - 960 - 1,020
3
Fabrication
Time Independent
1
0.9 - 1.05 - 1.15
4
Integrators
Time Dependent
1,200
1,100 - 1,250 - 1,500
5
Integration Testers
Time Dependent
1,200
1,150 - 1,250 - 1,550
6
Specification Writers
Time Dependent
800
750 - 800 - 850
7
Unit Testers
Time Dependent
800
700 - 825 - 950
8
Fabricators
Time Dependent
720
680 - 720 - 760
Discrete Risk drivers affecting cost can also be included to the analysis, in addition to the
uncertainty factors. These cost factors can be entered as the implication of identified risk
drivers, just as in the previous exercise for schedule drivers. If both cost and schedule risks
occur, the burn rate, cost estimate, and duration will vary, and each driver will cost to vary.
While new risks may be entered that just affect the burn rate or total cost of equipment, the
existing risks with schedule drivers already included can have those impacts as well. For
example, a cost factor has been added to the Risk Driver Editor for the previously identified
Organizational risk affecting all tasks, as shown in the Figure 6.2.8-11.
Section Revision:
December 15, 2020
6.2.8-15
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-11
Screenshot of Risk Driver Editor
After running the program with the addition of cost uncertainties to resources and allowing risk
drivers to affect costs directly rather than only through schedule risk, there is a direct impact on
cost, as can be seen in Table 6.2.8-5 below. The schedule table is not shown since the cost
drivers included in the exercise do not by themselves impact duration. Note that some risks will
have just schedule duration uncertainties and risk drivers, some will have just cost uncertainties
and drivers, and some will have both. Cost will be affected in all cases, but schedule is affected
only for those risks with duration uncertainties and drivers.
Table 6.2.8-5
Results with Uncertainties and Cost Risks Assigned
Cost
Baseline
5%
50%
80%
95%
Dollars (millions)
651.6
679
838.6
975
1100
4%
29%
50%
69%
% above Base
Adding the uncertainty and risks affecting the costs independently of time to the simulations
results in a time-cost scatterplot shows less connection between time and cost, as shown in
Figure 6.2.8-12.
Section Revision:
December 15, 2020
6.2.8-16
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-12
Scatterplot Showing Less Connection between Time and Cost
6.2.8.8 Handling Inflation
Inflation is a part of the NSF budgeting and project planning. The program should select an
acceptable source for the future inflation rate and use it in the baseline and the risk analysis of
that baseline. For the case study in this exercise, the baseline cost is projected at $651.6 million
in base year dollars, that is, without inflation. With risks but no inflation the risk analysis
simulation shows a cost in base year dollars of $975 million at the 80th percentile of certainty.
The analysis program can be used to factor in inflation if the cost estimating has been
performed in base year dollars. Adding the factor of cost inflation and setting it at the rate of
3% causes the risked cost at the P-80 level to increase to $993 million in then-year dollars as
shown below in Figure 6.2.8-13:
Section Revision:
December 15, 2020
6.2.8-17
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 6.2.8-13
Scatterplot with Addition of Cost Inflation Factor
The value of 3% inflation may be a most likely number, but the software used in this exercise
does not support an uncertain inflation level in simulation. A suggestion is to perform two
scenarios where the inflation rate is either lower or higher than 3%.
•
At 2% inflation the cost is estimated at $987 million
•
At 4% inflation the cost is estimated at $1 billion.
These scenarios can help understand the total “then dollar” cost of the project that is risk
adjusted, and the impact of the inflation assumption on that number.
6.2.8.9 Prioritizing the Discrete Risks ‒ Risk Mitigation Workshop
The organization is encouraged to use these results to help improve the prospects of the
project by mitigating the important risks. To do this the risks are prioritized. See Table 6.2.5-1
for sample prioritization from this exercise. This prioritization method uses the Monte Carlo
simulation, a 60-year-old method, and the schedule which the project team is using to manage
or at least summarize the project. It is thought that this prioritization of risks is more realistic
than that using qualitative methods resulting in the risk register, in part because it recognizes
the structure of the schedule and handles correlations.
Section Revision:
December 15, 2020
6.2.8-18
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.8 Quantitative Risk Analysis – Estimating
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 6.2.8-6
Savings and Days Saved
ID
Name
Cost Savings
Days Saved
7
Organizational risk affecting all
$196.15M
207
7
Uncertainty
$48.03M
152
6
External risk affecting all
$34.97M
41
2
Risk affecting fabrication
$23.8M
31
4
Risk affecting testing
$4.28M
34
3
Risk affecting Integration
$10.86M
32
1
Risk affecting design
$5.35M
10
5
Risk affecting requirements
$0
0
The risk mitigation exercise should be done in a workshop setting since many people have to
contribute and commit to the mitigations.
•
This workshop includes the PM, DPM, team leads and others involved in mitigation of
risk.
•
Given the prioritized list of risks for a project that may overrun cost and schedule
targets, the project team can develop risk mitigation actions. The mitigation workshop
estimates the improvement in the probability and impact parameters is expected to
result from the various mitigations planned for each identified risk (uncertainty cannot
be mitigated in concept).
•
For the mitigation actions to “count” against the project risk management should
commit to them as evidenced by their post-mitigation budget, schedule and assignment
of people to monitor the risks and their mitigations. These risks should be added to the
risk register as well so they are reviewed frequently.
•
Each risk mitigation action accepted is modeled and the post-mitigation amount of risk
to time and cost is recorded, along with the cost of the risk mitigation. A post-mitigation
simulation will determine how much benefit is expected from the mitigations.
•
The final report includes post-risk mitigation results and the overall project cost and
schedule risk if those risk mitigation actions are taken and mitigate the risks. Note that
the original cost and schedule target will generally not be met since that would require
complete mitigation of the risks that caused the estimate of overrun in the risk analysis
itself.
Section Revision:
December 15, 2020
6.2.8-19
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.9 Risk Response Planning
Figure 6.2.9-1
• Risk Register
• Meeting and
stakeholder
interviews
with
dependent
organizations
Inputs
Risk Response Planning Process
Process
• Confirm whether mitigation is required for risks
based on the Probability/Impact Ranking Chart
• Confirm root causes
• Develop a plan to mitigate the risk and provide a
Description and an Owner in the Risk Register
• Chart the risk mitigation events in time
throughout execution and map the improvement
in risk status to mitigation events
• WHEN: at the time risks are identified, at weekly/
monthly status meetings, and when changes to
the risks occur
• Updated Risk
Register
• A separate action
item or schedule for
implementing risk
mitigation
Outputs
A known risk (often referred to as a “known unknown”) is a risk that has been identified and
can be calibrated (probability and impact) and analyzed. Examples of known risks may include
strategic or overriding aspects of the project environment such as poor project management
practices, lack of resources, multiple projects, external dependencies, relationships between
project participants, technical complexity etc. Identified risks need to be proactively managed
throughout the project life cycle by identifying who owns the management of that risk and by
outlining risk symptoms, triggers, and contingency plans that would prevent the risk from
occurring or that would lessen the project impact should it occur.
The Risk Response Planning step includes considerations related to risk mitigation and response
planning. This includes the assignment of one or more persons to take responsibility for each
identified risk and the development of measures and action plans to respond to the risk should
it become an issue. PMI PMBOK® Guide defines Risk Response Planning as the process of
developing options and actions to enhance opportunities and to reduce threats to project
objectives.
Risk response actions for threats are generally categorized as: 1
1
•
Avoid – This strategy involves changing the project to eliminate the threat from
identified risk
•
Mitigate – This strategy involves taking early action to reduce the likelihood and/or
impact of risk
•
Transfer – This strategy involves shifting the responsibility and ownership of the risk to
another party. Although this strategy is seldom used for NSF projects, it typically
This listing and these descriptions are described in the PMBOK® Guide, 5th Edition, 2013 PMI
Section Revision:
May 2, 2016
6.2.9-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
involves purchasing insurance against the type of risk or requiring vendors to assume
more risk.
•
Accept – This strategy involves acknowledging the threat as part of the project and
accepting the consequences of its occurrence. An example of this is political or
legislative risk that is out of the control of the project team. The consequence of
acceptance may mean that contingency resources may need to be applied if the risk is
realized.
Risk response actions for opportunities are generally categorized as:
•
Exploit – This strategy seeks to eliminate the uncertainty associated with this
opportunity to ensure it happens. This is similar to Avoid threats.
•
Enhance – This strategy seeks to increase the probability and / or the positive impacts of
the opportunity. This is similar to Mitigate threats.
•
Share – This strategy seeks to share the benefits of the opportunity with another
organization that is in the best position to secure the opportunity for the project. This is
similar to Transfer for threats.
•
Accept – This strategy accepts an opportunity if it arises but does not envision pursuing
it, similar to Accept for threats.
For the most part, project risk response planning will consist of defining risk thresholds for
action, confirming risk triggers, and then planning a mitigation strategy and/or developing
backup plans if risks occur. A risk trigger is an event or events that activate the execution of a
backup plan, should the risk become an issue. Triggers should be specified in the Risk Definition
in the Risk Register, as well as the date that risk resolution is required for each risk. Mitigation
strategies identify actions that may minimize or eliminate project risks before the risk occurs or
document decisions to accept the consequences of risks without action. A risk may have several
mitigation activities that attempt to balance the reduction in the probability and/or the severity
of the risk occurrence with the cost-effectiveness of the mitigation strategy. Mitigation planning
requires that the root cause(s) of the risk be identified and that the mitigation strategy and
plans be aligned accordingly. Backup plans define actions to be taken in response to identified
risk triggers in hopes of reducing potential project impact as a result of a realized risk (often
defined in the literature as an “issue”).
A tabulated example of the impact of Risk Response evaluation is given below in Table 6.2.9-1.
Section Revision:
May 2, 2016
6.2.9-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 6.2.9-1
Impact of Risk Handling on Project Cost
Before Handling
After Handling
Residual Risk Cost Estimates ($K)
Risk Item or Basis
Risk Level
Worst Case Handling Cost Implement
Cost ($K)
Strategy Handling
Risk
Level
Best
Case
Most
Likely
Worst
Case
Redesign to solve problems identified during reviews
Moderate
3,360
Mitigate
75
Low
0
150
500
Do analyses or design per external comments
Moderate
390
Avoid
0
--
N/A
N/A
N/A
Rework design documents during concept evolution
Moderate
5,720
Mitigate
0
Moderate 0
750
2,500
Redesign for add’l equipment for ops or pretreat interface
Moderate
160
Mitigate
0
Low
40
100
Design for sintering equipment
High
500
Mitigate
308
Moderate 0
0
200
Redo design for SNF re-sizing
Moderate
200
Accept
0
Moderate 0
50
200
Redesign; contamination control in process room
Moderate
5,000
Mitigate
361
Moderate 0
300
3,000
Change design basis, due to scale-up impact
Low
50
Accept
0
Low
0
15
50
Redesign, for SC furnace
Low
800
Mitigate
0
Low
0
0
50
Redesign to add gas-trapping system
Low
1,550
Accept
0
Low
0
0
1,550
Rework to add waste streams to design
High
3,000
Mitigate
0
Moderate 0
250
2,300
Rework robotic features design
High
7,440
Mitigate
53
Moderate 0
500
2,000
Redesign for characterization
High
5,000
Mitigate
176
Moderate 0
600
3,000
Redesign to meet canister requirements
Moderate
3,000
Accept
0
Moderate 0
100
3,000
Design for new cables
Moderate
400
Mitigate
0
Low
0
0
50
Redesign for additional MC&A equipment
Moderate
400
Mitigate
0
Low
0
0
50
Section Revision:
May 2, 2016
0
6.2.9-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Risk Item or Basis
Risk Level
Worst Case Handling Cost Implement
Cost ($K)
Strategy Handling
Risk
Level
Best
Case
Most
Likely
Worst
Case
Redesign, to apply new structural criteria to 105L
Moderate
1,500
Mitigate
300
Low
0
0
700
Redesign, per SGS inputs
Low
500
Accept
0
Low
0
0
500
Redesign for changes, per NRC interface
Moderate
200
Mitigate
0
Low
0
0
150
Additional utility design features
Moderate
500
Accept
0
Moderate 0
300
500
Delays initiating design, awaiting R&D completion
High
5,360
Mitigate
0
Moderate 0
240
720
Delays redesigning for classified process control system
Low
60
Avoid
0
--
N/A
N/A
N/A
Add features to meet IAEA
Moderate
500
Mitigate
0
Low
0
0
50
Uncertainty in obtaining contingency funds
Moderate
2,000
Avoid
0
--
N/A
N/A
N/A
Disposal of bundling tubes
Moderate
100
Avoid
75
--
N/A
N/A
N/A
Decontamination of final-product canister
Moderate
500
Avoid
341
--
N/A
N/A
N/A
Storage location for depleted uranium
Moderate
100
Avoid
75
--
N/A
N/A
N/A
Availability of emergency generator and fuel tank
Moderate
40
Avoid
0
--
N/A
N/A
N/A
Redesign for necessary structural supports
Moderate
300
Avoid
225
--
N/A
N/A
N/A
0
3,295
21,170
Arithmetic Sums:
Section Revision:
May 2, 2016
48,630
1,989
6.2.9-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The risks with mitigation plans and risk triggers are all listed in the Risk Register with their
Qualitative Risk Assessment status. After Risk Response Planning has been performed the entire
entry for the risk includes:
•
Its statement or definition
•
Analysis and ranking of initial risk
•
Assignment to a risk owner
•
Risk mitigation actions and backup plans
o Costs
o Timing and risk triggers
o Expected results
•
Status of mitigation efforts
•
Analysis and ranking of residual risk after mitigation
The last item listed above is the expected residual risk and ranking after mitigation has been
applied. This is accomplished by repeating the analysis of probability and ranked impact on
project objectives with successful mitigation assumed. Thus, the Risk Register shows “before”
and “after” views of the analysis, with risks migrating down from red to yellow to green with
the mitigation steps that cause the improvement in risk status and timing of those steps. The
sample risk register shown in Figure 6.2.7-6 shows columns with headings for “Pre-Mitigated”
and Post-Mitigated” analysis results.
Section Revision:
May 2, 2016
6.2.9-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.10 Risk Monitoring and Control
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.10 Risk Monitoring and Control
Risk Management requires continuous monitoring of project risk and iterative application of the
risk identification, analysis, and response processes. Existing risks need to be monitored,
controlled, and ultimately retired, while new risks should be identified and added to the Risk
Management process. Risk Monitoring and Control is the process of identifying, analyzing, and
planning for new risk, keeping track of and re-analyzing identified risks, monitoring risk
symptoms and triggers, reviewing the execution of risk responses strategies while evaluating
their effectiveness, and reporting status to stakeholders. The Risk Register, as the tool that
supports Risk Management and provides a means of communication, should be kept up to date
with status and changes. The frequency and process for reviewing project risk is set out in the
project Risk Management Plan.
Risk Control includes the process of regularly updating the Risk Register and communicating to
stakeholders the latest risk status, with resulting impacts on the project and mitigation plans.
Reporting of project and program level risks should be included as part of regularly scheduled
status meetings with, and in formal status reports to, internal project members as well as
eternal stakeholders and the NSF. The NSF has emphasized the need to communicate the risks
at regularly scheduled status meetings to ensure that continued focus and awareness is placed
on risk management.
Figure 6.2.10-1
• Risk Register
• Meeting and
stakeholder
interviews with
dependent
organizations
• Supporting
documentation
from project
artifacts
Inputs
Risk Monitoring and Control Process
Process
• Repeat risk identification to determine if new risks
should be put on the Risk Register and handled
• Regularly review the project for the risks reported on
the Risk Register to see if they have changed
probability or impact or should be retired
• Review the mitigation plans and validate that they still
represent an acceptable approach
• Disseminate mitigation strategies to stakeholders and
responsible resources. Follow on action items for
mitigation plans and report on status
• Use Risk Register to create and communicate the risk
disposition
• Ensure a continuous review of the risks and
monitoring for the mitigation and contingency efforts
• Updated Risk
Register
• Corrective actions
(not originally
identified as
mitigations)
• Report and
communicate on
new risks and
updated items
Outputs
When risks are resolved, they should be retired from the list of active risks. When the project
ends, the risk register may be closed. If some risks pose other future threats to the program or
future projects, consideration should be given to re-opening a risk with the appropriate
operations management, or at the program level as an “ongoing risk.”
Section Revision:
May 2, 2016
6.2.10-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.11 Contingency Management for Risk Mitiga
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.11 Contingency Management for Risk Mitigation
6.2.11.1 Contingency Budget Timeline
NSF expects the project to refine its WBS budget estimates following the Preliminary Design
Review (PDR), adding additional definition to the tasks associated with accomplishing the
project’s deliverable activities. At Final Design Review (FDR) the PEP budget estimate should be
substantially based on externally obtained cost estimates (vendor quotes, bids, historical data,
etc.). This added definition is expected to result in an increase to the project’s estimated
Budget at Completion (BAC) and project schedule, and a concomitant reduction in its budget
and schedule contingencies, while TPC and the risk-adjusted, committed schedule finish date
remain constant. The quantitative risk analysis should have a component to anticipate this
increase in cost and time so that the original contingency amounts are sufficient to provide for
this increase.
As a project progresses, the baseline cost estimate and schedule will typically be exceeded and
contingency amounts of dollars and time will be used. Periodically the project cost estimate
must be revised to reflect all new information, including actual costs and use of contingency
funds, adjustments to the risk profile, learning curves for manufactured items, etc. This new
estimate of the cost of the remaining work is called the Estimate to Complete (ETC), and the
Actual Cost of Work Performed (ACWP) + ETC is equal to the latest revision of the Estimate at
Completion (EAC). The EAC should be compared to the BAC to identify potential liens on the
remaining contingency. For NSF projects, risks are not included in the ETC, EAC, BAC, or PMB
due to the NSF requirement that contingency be held and managed separately from the
baseline. The risk-adjusted estimate at completion (RAEAC) is the ACWP + ETC + remaining risk
exposure. If the RAEAC is greater than the TPC, de-scoping may be necessary. See Section 4.2.5
for details on requirements for budget contingency use.
The project should create and maintain an expected contingency allocation profile. Contingency
allocation profiles usually do not track the commitment or spending profiles. For many projects,
the highest use of both schedule and budget contingency occurs during procurement and
during final commissioning/integration phases. A contingency allocation curve for such a
project would be bi-modal, with one peak for procurements activities and another for
significant contingency amounts held back until the end of the project, even though the
spending curve may be low near the end of the project. Although risk does burn down over
time, there may be significant reworking of hardware, for example, needed as a result of
knowledge gained during integration and commissioning activities.
6.2.11.2 Change Control for Contingency Adjustments
Adjustments to cost, schedule, and scope are documented and approved under the project
Change/Configuration Control Process (CCP, PEP-8.2). The Risk Management Plan describes
how the project uses the Change/Configuration Control Process (CCP) to assign contingency to
specific WBS elements when risks materialize, and how budget contingency is de-allocated
from WBS elements and returned to the contingency category when budget underruns occur.
Section Revision:
December 15, 2020
6.2.11-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.11 Contingency Management for Risk Mitiga
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The Change Control Process should be initiated when the Total Project Cost target is
established at the Preliminary Design Review and followed for the duration of the project. All
change control actions that affect the use of contingency – cost, schedule, or technical
performance and scope – should include a link to an identified and documented risk and
indicate the affected WBS elements at the first meaningful level of technical differentiation
within the project. The CCP must make provision for seeking prior approval from the NSF
Program Officer for all actions exceeding thresholds as defined in the CA. All change requests
are to be archived by the project and made available for review by NSF. The Project must keep
a log of all change actions such that contingency actions, including puts and takes, can be
reported and summarized. See Section 4.2.5 for further details and a sample Change Request
form.
Note that use of contingency does not automatically require a change to the baseline. For
instance, a change control action can authorize contingency to cover a cost overrun which is
tracked as a variance on the baseline Budget at Completion (BAC). In such a case the
contingency can be incorporated into either the BAC or the EAC. In the first instance, the BAC is
changed. In the second, the variance from the BAC remains and can be used for trending and
other information. See Section 4.2.5 for further details on approval levels for use of
contingency.
Adjustments to contingency should include taking advantage of opportunities to assign savings
and underruns to contingency. Savings should not be left in associated WBS elements if they
are above thresholds set out in the Risk Management Plan, nor should they be shifted to other
tasks without going through the Change Control Process for return to contingency and
subsequent allocation to a different WBS element. Budget and cost underruns should be moved
to contingency as risks are retired and WBS elements are closed out and reconciled. Savings
realized through the implementation of planned de-scoping options should also be placed into
contingency. Returning the savings allows the best use of contingency for overall project
priorities.
6.2.11.3 Liens List: Forecasting and Opportunity Management
The Project should maintain a Liens List of planned future adjustments to contingency as a
forecasting tool that tracks actions that have not yet been incorporated into the BAC or EAC.
The list may document items such as very high probability risks with trigger points for action,
deferred scope held as contingency until a decision date, realized risks needing draws on
contingency that require more definition for a change control action to be implemented, and
anticipated opportunities for returns to contingency. It can also be used to record the need for
contingency to cover budget and schedule variances that will not/cannot be mitigated. It does
not serve the same purpose as a watch list or major threats list from the Risk Register. It acts as
an escrow or staging account for planned or near certain contingency allocations.
The List should include a description of the identified risk and the anticipated action, with
estimates of budget and schedule impacts, and anticipated decision date for any CCB action.
Section Revision:
December 15, 2020
6.2.11-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.11 Contingency Management for Risk Mitiga
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
The affected WBS elements should be identified, at the second level (or the first meaningfully
specific level of scope description), where known.
Projected amounts of future adjustments to contingency in the Liens List are to be periodically
reported to NSF. NSF recommends including this information within the monthly status report
as well.
6.2.11.4 Updates of the Estimate at Completion and Risk Exposure
The project should maintain an estimate of total costs with risk exposure by periodically
updating the schedule, the Estimate at Completion (EAC), and the analysis of overall project
risk. This is referred to as the Risk-Adjusted Estimate at Completion (RAEAC). Budget and
schedule contingency amounts should be appropriate for the risk exposure throughout the
project life cycle. During concept and early design development, a qualitative risk analysis and
risk register may provide an adequate estimate of risk exposure for both the design and
construction planning estimates. As project planning reaches the Preliminary Design Phase, the
drawbacks of qualitative analysis – limited subset of risks, ignored correlations, and arithmetic
sums of averages – do not allow that method to adequately portray total project risk. 1 Project
planners should transition to quantitative risk analysis in order to establish a substantiated total
project cost at the time of the PDR.
For the Construction Stage, the initial budget contingency is a part of the total project cost as
defined by the award instrument. As time goes by, risk exposure changes with risk mitigation,
new knowledge, and new circumstances. The amount of remaining budget contingency
fluctuates over time with assignments to risk mitigation and return of any savings; either from a
risk being retired or work packages coming in below the estimated budget. The total remaining
available budget contingency should be compared to the remaining risk exposure to determine
whether the project has adequate funds to cover anticipated risks. Remaining available
contingency should always equate to the difference between the total project cost minus the
Estimate at Completion (EAC) and any liens.
The sum of the EAC and liens should include variances (backward looking actuals) and updated
estimates (forward looking forecasting) in the current plan, not the target baseline BAC. The
EAC should equal the BAC only at project start and after major changes to the baseline from replanning or re-baselining. For NSF projects, risk and uncertainties are not included in the ETC,
EAC, BAC, or PMB due to the NSF requirement that contingency is held and managed separately
from the baseline.
It is good practice to re-estimate EAC and Risk Exposure yearly, unless stated otherwise in the
CA. Specific dates may also be appropriate times for re-evaluation, such as at major milestones
dates. The Project Manager periodically re-assesses the current risk assessment to identify and
1
Projects usually adopt a more conservative certainty target such as the 80th percentile.
Section Revision:
December 15, 2020
6.2.11-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.11 Contingency Management for Risk Mitiga
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
address any new risks that arise as the project progresses. This assessment should result in a
determination of whether cost and schedule contingency remain sufficient for project risks.
6.2.11.5 Contingency Use and NSF Oversight during Construction
The NSF Program Officer must concur on all CCB actions exceeding CA- or contract-defined
thresholds for allocation of budget, schedule, or scope contingency. NSF will negotiate a CA
with the Recipient institution to fund project construction activity which will specify thresholds
above which prior NSF approval 1 is required before allocation of contingency (following formal
CCB review) to specific WBS elements. Contingency may only be used to support in-scope work
for the approved project baseline. See Section 4.2.5.7 for additional details.
6.2.11.6 Documentation and Reporting of Contingency Use
Risk management actions involving Change Control actions fall under the following
documentation and reporting requirements, as stated in more detail in Section 4.2.5:
•
All Change Control Requests, irrespective of amount or whether they increase or
decrease the BAC, are to be reported directly to NSF Program Officer
•
The Recipient will keep an archive of all Change Control Requests
•
The Recipient will keep a summary log of all Change Control Requests
•
Projected amounts of future adjustments to contingency (“liens”) are to be periodically
reported to NSF.
NSF recommends including this information within the monthly status report. Note that
National Science Board (NSB) approvals 2 are required when Change Control actions exceed the
even higher thresholds defined by NSB policy.
The required summary log of all Change Control actions should include the following:
•
Change control action title,
•
Change control document reference number,
•
Change control approval date,
•
Amounts of change in budget, scope, and/or schedule, for each affected and identified
WBS element,
•
Any adjustments to contingency amounts,
•
WBS elements affected by the changes (at WBS Level II or at the first meaningful level
of technical differentiation within the project)
•
Risk Register ID number and description for the risk being addressed, and
Thresholds are necessary to allow the project to respond in a timely way to small, immediate needs for use of contingency,
such as field changes during construction. This avoids potential cost escalation that could result from delay.
1
2
See Section 2.4, Construction Stage, for details on NSF policy on how and when NSB authorization is required.
Section Revision:
December 15, 2020
6.2.11-4
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.11 Contingency Management for Risk Mitiga
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
NSF approval date if required.
Monthly reports must also include the status of contingency as part of the Earned Value
Management (EVM) reports. See Sections 4.2.5.8 and 4.6.2 for details on reporting.
Section Revision:
December 15, 2020
6.2.11-5
Major Facilities Guide: NSF 21-107 (July 2021)
6.2.12 Partnership Considerations for Conting
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.2.12 Partnership Considerations for Contingency Management
NSF may partner with other entities to plan and construct a major facility. The guidelines within
Section 4.2.5.9 of this document are applicable when NSF funds a particular scope of work
within a larger overall project. Risk assessment, contingency development processes and
contingency status reporting are to be applied to those WBS elements to be funded by NSF.
NSF encourages the development of unified management for project planning and execution of
the entire project scope wherever practical.
Section Revision:
May 2, 2016
6.2.12-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3 Guidelines for CyberSecurity of NSF’s Major Facilities
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3
GUIDELINES FOR CYBERSECURITY OF NSF’S MAJOR FACILITIES
6.3.1 Introduction
Data creation, sharing, and analysis are central to the progress of science. As information has
becoming increasing digital and increasingly accessible from anywhere, the confidentiality,
availability, and integrity of information and information systems has raised the importance of
cybersecurity considerations. Cybersecurity protects the availability of instruments and
systems; promotes trust in, and availability of, data; and provides confidence in the integrity of
the research resulting from use of facility information and information resources. At the same
time, inappropriate, inefficient, and ineffective cybersecurity compliance regimes can be costly
in time, human capital, and funding. Cybersecurity programs for a facility must therefore be
well-aligned with the mission and appropriately balance risk with cost and research innovation.
Thus, a cybersecurity program is based on a structured approach to planning, developing, and
maintaining levels of information security and risk appropriate to a facility’s mission and phase.
A cybersecurity program should be scoped to the key assets, resources, and the full lifespan of
the facility. It is necessarily a living program that adapts, adjusts, and advances. As such,
cybersecurity programs require reporting, evaluation, and updating as appropriate.
Section Revision:
July 26, 2021
6.3.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.2 Major Facility Cybersecurity Program
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3.2 Major Facility Cybersecurity Program
Uniform Guidance §200.303 states that the Recipient’s internal controls, including technology
infrastructure and security management, should be compliant with guidance published by the
Comptroller General or Committee of Sponsoring Organizations of the Treadway Commission
(COSO) 1. Further, the Cooperative Agreement Supplemental Financial & Administrative Terms
and Conditions (CA-FATC) for Recipients of major facilities or Federally Funded Research and
Development Centers (FFRDC) 2 requires an information security program 3 and identifies a
modest set of required components for the program. Additionally, an information security plan
is a required element of the Construction Stage Project Execution Plan (PEP) per Section 3.4 of
this Guide.
The foundation for developing and maintaining a project’s cybersecurity program lies in the
research mission and goals of the facility itself. Related foundational considerations are project
phase, size, complexity, project budget, and the project’s required data management plan
which identifies key information assets. In addition, geographic and institutional distribution
can be an important overall factor. The four pillars of a cybersecurity program which rest on
this foundation are mission alignment; governance; resources; and controls. Like other facility
project components, the cybersecurity program should be appropriately represented in
standard project documents and NSF oversight activities such as the project execution plan,
project risk management plan, project budget, project reports, and project reviews.
The following sections define and describe a suggested framework for the facility cybersecurity
plan. This framework is based on the previously mentioned four pillars of information security
programs: Mission Alignment, Governance, Resources, and Controls. 4 Major facilities may use
these pillars as a framework for founding, operating, evaluating, and improving their
information security programs, and meeting the award terms and conditions. Since there are
interdependencies among the pillars, an integrative approach is required. The exact content
and emphasis of the information security program should be tailored to the mission, phase, size
and scope of an individual facility.
The four pillars of a cybersecurity program rely on a project-specific inventory of “information
assets” to be protected. Risk-based approaches to protection of information assets are further
determined by a project-tailored “information classification” 5 which recognizes varying degrees
1
Standards for Internal Control in the Federal Government and Internal Control – Integrated Framework,
These documents are updated every 6 to 12 months. Check at https://www.nsf.gov for the most recent version that applies to
major facilities or FFRDCs.
2
3For
the purposes of this section, there is no distinction among the terms “information security,” “cybersecurity,” and “IT
security” as referenced in the award terms and conditions. However, this section specifically addresses digital information.
4
See, NSF Cybersecurity Center of Excellence program guidance, e.g., ., https://www.trustedci.org/framework
5
https://www.scribd.com/document/203236714/CISO-Perspectives-Data-Classification
Section Revision:
July 26, 2021
6.3.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.2 Major Facility Cybersecurity Program
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
of value, priority, and/or sensitivity of the information assets. The information asset inventory
and information classification are described in the Mission Alignment section.
Section Revision:
July 26, 2021
6.3.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.3 Mission Alignment
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3.3 Mission Alignment
A cybersecurity program exists to support its organization’s mission and should maximize benefit to the
mission. Effectively tailoring the cybersecurity program requires an understanding of the organization’s
mission. A properly tailored cybersecurity program accounts for the positive and negative impacts
security can have on the organization’s mission. Overinvesting in cybersecurity results in increasingly
diminishing returns, wasting resources that could have been used to directly advance the mission.
Similarly, underinvesting in cybersecurity is also a bad investment, as an investment in preventative
cybersecurity can be significantly smaller than the cost of a cyber incident.
6.3.3.1 Stakeholders & Obligations
Organizations must identify and account for cybersecurity stakeholders and obligations. Cybersecurity
stakeholders are people or entities with interest in or affected by an organization’s cybersecurity and
involves both internal and external stakeholders. Internal stakeholders include the cybersecurity
operator and IT leadership, application developers, system administrators, and information system
users. External stakeholders may include research projects, suppliers, parent organizations, and others.
Researchers supported by the cybersecurity operator may include internal researchers on staff and
external researchers collaborating on projects using the operator’s cyberinfrastructure.
6.3.3.2 Information Asset Inventory
Organizational identification and location of information assets is a prerequisite to competently
securing those assets. See, CIS Critical Security Controls 1 and 2, 1 and NIST 800-53 control CM8. 2 The inventory might include many details, but at a minimum it should identify the asset and
indicate the value or sensitivity of the system and/or classification of the information. The
facility data management plan which is a required document for proposal submission is a key
source for both asset identification and classification.
The asset inventory can be built by manually using publicly or commercially available templates
or worksheets or by constructing a custom database. The Open Science Cyber Risk Profile
(OSCRP) provides guidance on assets to consider for science projects. 3 Trusted CI 4 has
developed substantial guidance 5 for information security programs in NSF projects, including a
template for recording data about information assets. To reduce the manual effort, security
products or scanners 6 can aid with asset discovery and inventory.
1
https://www.cisecurity.org/controls/
2
http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r4.pdf
3
https://trustedci.github.io/OSCRP/OSCRP.html
Trusted CI, the NSF Cybersecurity Center of Excellence, is an NSF-funded center dedicated to assisting the NSF community,
including Major Facilities, with information security needs. More information is available at https://trustedci.org
4
5
https://www.trustedci.org/framework/
6
https://www.cisecurity.org/white-papers/cis-controls-sme-guide/
Section Revision:
July 26, 2021
6.3.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.3 Mission Alignment
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
Information is any communication or representation of knowledge such as facts, data, or
opinions in any medium or form, including textual, numerical, graphic, cartographic, narrative,
or audiovisual. 1 Organizational information may be stored and used within the organization’s
information systems, as well as flow out to third party systems.
An information system is a discrete set of information and related resources (such as people,
equipment, and information technology) organized for the collection, processing, maintenance,
use, sharing, dissemination, and/or disposition of information. 2
6.3.3.3 Information Classification
Information has varying degrees of organizational value, sensitivity, and protection
requirements. 3 These qualities are key factors to consider in analyzing the anticipated impact of
security incidents. In addition, some information assets may be subject to additional external
control (e.g. federal or state privacy laws, international regulations, contractual obligations). In
most cases, information can be classified into two to four categories (e.g., public, internal, and
controlled).
1
See, National Information Assurance (IA) Glossary, CNSS Instruction No. 4009, Apr. 2010.
2
See, 44 U.S.C. 3502
3
https://www.scribd.com/document/203236714/CISO-Perspectives-Data-Classification
Section Revision:
July 26, 2021
6.3.3-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.4 Governance
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3.4 Governance
Recommended governance elements for cybersecurity programs include: roles and
responsibilities; policies and practices, risk acceptance and management.
6.3.4.1 Roles and Responsibilities
Successful cybersecurity programs require an active role for facility leadership in policy
development and implementation, assigning information asset ownership, and accepting
residual risk.
A second essential role is that of information asset owner. This is a person, position, or entity
given formal responsibility for an information asset (or set of assets) within an organization.
There are typically multiple information asset owners. The asset owner understands the risks to
the asset and ensure adequate controls are in place over the life of the project.
In addition, cybersecurity programs should have an identified senior security role, such as a
Chief Information Security Officers (CISO) or Information Security Officer (ISO) as owner of the
cybersecurity program and lead decisionmaker for operational aspects of the cybersecurity
program. This individual also facilitates the formation of informed cybersecurity policies,
practices and risk management decisions by facility leadership and information asset owners.
6.3.4.2 Policies
Every facility project with information assets will require the development, approval and
implementation of some information security policies within its cybersecurity program. Policies
are driven not only by the facility’s information assets and classifications, but also by relevant
regulations. Regulations may be international, national or local. Regulations may be specific to
an information asset (e.g. HIPAA, FERPA, etc.) or to specific needs of a facility collaborator or
user (e.g. ITAR, FISMA). Examples of common policies include: Acceptable Use Policy; Access
Control Policy; Incident Response Policy. Sample templates may be found at the Trusted CI
website 1. Trusted CI’s guidance recommends developing a “Master Information Security Policy
& Procedures” (MISPP) document as an initial policy-making step. A master policy provides an
overview of the project’s information security program including a summary of roles and
responsibilities, as well as an organized list of specific information security policy documents.
Additional sources of policy templates and forms include the Higher Education Information
Security Council (HEISC) Resources Center, 2 and SANS Institute’s Information Security
Templates page. 3 Using example and template policies can streamline the policy production
process, even if substantial customization is warranted. The policies themselves only reduce
1
See, NSF Cybersecurity Center of Excellence program guidance, e.g., https://www.trustedci.org/framework/templates
2
https://www.educause.edu/focus-areas-and-initiatives/policy-and-security/cybersecurity-initiative/resources
3
https://www.sans.org/security-resources/policies
Section Revision:
July 26, 2021
6.3.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.4 Governance
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
risk if attached to other elements of the cybersecurity program (e.g. roles and responsibilities,
controls) and integrated into overall project governance and management.
6.3.4.3 Risk Management and Acceptance
Cybersecurity programs employ a risk-based approach to information security and as such,
there is inherent acknowledgement by NSF that any valuable activity requires acceptance of
residual risk. Residual risk is that risk that remains in the presence of controls. An important
output of a risk-based information security program is a documented set of well-informed risk
management and project leadership acceptance decisions. Due to the rapidly changing
technology landscape, a flexible, informal risk assessment process is often more valuable than a
formal, detailed risk assessment that is out of date long before it is completed. In addition to
the Trusted CI guide which is tailored to the scientific community, the Open Science Cyber Risk
Working Group (OSCRP), has developed and released a “best practices” document 1 to assist
NSF, NIH and DOE projects in assessing cybersecurity risks related to Open Science projects.
Finally, the Armed Forces Communications and Electronics Association International (AFCEA)’s I
Cyber Committee has produced a useful and relevant publication: The Economics of
Cybersecurity, A Practical Framework for Cybersecurity Investment 2.
6.3.4.4 Evaluation
Given the dynamic technology and cybersecurity landscape, organizations should plan for
periodic evaluations of the cybersecurity program, including policies, practices, and controls.
While project management and NSF oversight will involve regular reporting and review of
program milestones, outcome metrics, and incidents, the project is encouraged to consider
periodic self-assessments, external or stakeholder reviews, and evaluation of incident response.
Tools are available from variety of sources to aid in assessment. 3 4
1
http://trustedci.github.io/OSCRP/
2
https://www.afcea.org/committees/cyber/documents/cybereconfinal.pdf
3
https://library.educause.edu/resources/2015/11/information-security-program-assessment-tool
4
https://cset.inl.gov/SitePages/Home.aspx
Section Revision:
July 26, 2021
6.3.4-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.5 Resources
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3.5 Resources
6.3.5.1 Budget
Overall, worldwide cybersecurity spending is on the rise 1 2 driven in part by response to
increased cyber crime and new data protection regulations. There is wide variability in the cost
of cybersecurity as a percent of overall IT budgets (3-12%) depending on the mission of the
institution (e.g. Defense and Aerospace industries have more stringent requirements and
proportionally higher costs.)) and size of the institution (Small institutions may not achieve
economies of scale.). A secondary contribution to variability is the considerable discrepancy in
what is within the cybersecurity budget as well as what is in the IT budget. A 2016 NSF
Cybersecurity Summit examination of public data on DOE open science laboratory cybersecurity
spending 3 indicated that approximately 0.5% of the overall lab budget and 8-12% of the IT
budget (excluding scientific IT) were spent on cybersecurity.
6.3.5.2 Personnel
In addition to the CISO/ISO, ongoing access to skilled cybersecurity professionals is key to a
successful cybersecurity program. The CISO may manage a team of dedicated information
security professionals or oversee staff/activities in various departments within the facility.
Considering the demand for experienced cybersecurity professionals, 4 5 organizations may
need to consider outsourcing security services. In any scenario, information security resources
outside the facility such as a parent institution/campus, peer organizations, commercial security
consultants, intrusion detection and log monitoring services, and incident response services can
be an important source of security expertise, training, evaluation, and recommendations.
While technical skills are important, teaching skills, communication skills and negotiating skills
are endemic to cybersecurity programs and are, therefore, necessary personnel considerations.
1
https://www.gartner.com/newsroom/id/3836563
2
https://securityintelligence.com/news/cybersecurity-spending-poised-to-rise-in-2018-gartner-reports/
3https://static1.squarespace.com/static/5047a5a6e4b0dcecada15549/t/57b4b32dd2b857a1b6827a7f/1471460142220/Cybers
ecurity+Budgets+NSF+Summit+2016.pdf
4
https://www.monster.com/career-advice/article/future-of-cybersecurity-jobs
5
https://www.csoonline.com/article/3201974/it-careers/cybersecurity-job-market-statistics.html
Section Revision:
May 30, 2019
6.3.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.3.6 Controls
Prepared by the Directorate for Computer & Information Science & Engineering (CISE)
6.3.6 Controls
Controls are tailored to the facility’s portfolio of information assets and aligned to protect
confidentiality, integrity, and availability based on the corresponding information classification
for those information assets.
6.3.6.1 Control Set
Controls or mitigations are the administrative, technical, and physical safeguards and
countermeasures implemented to support the facility’s mission and ensure the appropriate
protection of information assets. Control selection, implementation and evaluation are ongoing
processes in any information security program.
While efficiency and effectiveness will require customization of controls, there are several
authoritative sources to assist in selecting baseline controls. 1,2,3 Scientific facilities may merit
special security considerations (e.g. diverse research data flows; identity management for
distributed science communities 4; non-facility device connectivity to facility networks and data;
unique Industry Control (ICS) and/or Supervisory Control and Data Acquisition (SCADA)
systems 5; application software development 6, 7, 8).
1
https://www.cisecurity.org/controls/ ; for more detail including history, see https://www.sans.org/critical-security-controls
2
https://www.asd.gov.au/publications/protect/essential-eight-explained.htm
3
https://www.nist.gov/cybersecurity-framework
4
https://refeds.org/sirtfi
5
https://ics-cert.us-cert.gov/ See, also, NIST SP 800-82r2. Available at http://csrc.nist.gov/publications/PubsSPs.html
6
https://social.technet.microsoft.com/wiki/contents/articles/7100.the-security-development-lifecycle.aspx
7
https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
8
https://software-security.sans.org/resources/swat
Section Revision:
July 26, 2021
6.3.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.4 Guidelines for Planning and Executing External Reviews of NSF's Major Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.4
GUIDELINES FOR PLANNING AND EXECUTING EXTERNAL REVIEWS OF NSF'S MAJOR
FACILITIES
This document, which is in preparation, will describe the process for evaluation and review of
all NSF major facility projects proposed for construction, under construction or currently in
operation. It will provide assistance to the Program Officer (PO) in preparing and planning a
review of the non-research related aspect of the project’s management, budgets, schedule and
related activities. The information contained will offer guidance for three situations: reviews of
facilities in planning; reviews of construction activity; and operational reviews of ongoing
facilities. A description of the overall process of planning and carrying out an external review of
a major facility project will be provided as an aid to the PO or associated staff who may be
unfamiliar with these processes or need a reference source on good practices.
The evaluation and reviews covered in the document include assessment of management,
schedules and budgets, as well as other matters relevant to a major facility project, such as
changes to technical aspects or scope. It does not address the intellectual merit or the broader
impact criteria used to select the project for support, but rather focuses on evaluation of the
Recipient’s planning and implementation activities.
The reports and recommendations from these external reviews are made directly to NSF. NSF
evaluates the review panel input, determines the appropriate response, and issues written
guidance to award Recipients for any subsequent response and action.
Section Revision:
November 25, 2016
6.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.5 Environmental Considerations in Major Facility Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.5
ENVIRONMENTAL CONSIDERATIONS IN MAJOR FACILITY PLANNING
NSF’s funding for the construction or modification of facilities constitutes a Federal Action that
triggers compliance with several statutes designed to protect the Nation’s environmental,
cultural and historic resources. Awareness of, and strict adherence to, all relevant
environmental regulations are extremely important considerations in the planning,
construction and operation of facilities.
These statutes include, but are not limited to, the National Environmental Policy Act (NEPA),
the National Historic Preservation Act (NHPA) and the Endangered Species Act. While NEPA and
its implementing regulations focus on activities that take place within the United States,
proposed activities that take place outside of United States may also be subject to NEPA.
Furthermore, there are international agreements and treaties that deal with environmental
impacts. For further information, see the Proposal and Award Policies Procedures Guide
Chapter II.C.2.j and consult with the PO.
NSF regulations governing compliance with the National Environmental Policy Act (NEPA) are
found at 45 CFR §640. NSF regulations supplement the Council on Environmental Quality’s
(CEQ) regulations, published at 40 CFR §§1500-1508. Program Officers, as required by NSF
regulations, are responsible for evaluating the environmental impacts that may result from the
implementation of a Foundation award and determining into which category incoming
proposals fall (i.e., CATEX, EA, or EIS). Compliance with NEPA also includes providing
opportunities for public input on issues such as potential environmental impacts and ways to
avoid, minimize, and/or mitigate adverse impacts. Determining the required level of
compliance activities – including what documentation, consultation and/or permits may be
required – is a complex task. The Program Officer (PO) should not attempt to determine the
extent of compliance requirements without consulting the Environmental Compliance Team
within NSF's Office of the General Counsel. NEPA compliance may require the preparation of an
Environmental Assessment (EA) in cases where no significant environmental impacts are
expected or the more extensive documentation of an Environmental Impact Statement (EIS)
where adverse effects are anticipated. Failure to take necessary steps can cause undue delays
in a project’s schedule, significant cost escalation and potential federal litigation.
Additionally, in conjunction with or independent of its NEPA compliance, NSF may be required
to initiate consultations with Native Americans and other interested parties pursuant to the
NHPA and/or initiate informal or formal consultation with the U.S. Fish and Wildlife Service
under the Endangered Species Act. These compliance requirements can introduce significant
schedule and cost risk into the project which should be considered and addressed.
Furthermore, there is no special source of funding within NSF to pay for the environmental
compliance process; the cost is normally borne by the program using Research and Related
Activities (R&RA) funds.
Section Revision:
November 25, 2016
6.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.5 Environmental Considerations in Major Facility Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Given the above considerations, the following guidance is offered:
1. It is imperative that the PO contact the Environmental Compliance Team within NSF's
Office of the General Counsel early in the Conceptual Design Phase to seek guidance on
specific requirements for compliance. The time required to complete environmental
compliance can take a year or more depending upon the level of impacts associated
with a proposed project.
2. It is extremely important that the PO and the project get cost estimates for the
compliance process and factor these into the project’s scope, schedule and budget early
in the design process.
The cost drivers associated with these activities (their impact on the project construction cost)
need to be well understood by PDR since the PDR budget and risk assessment provide the basis
for the construction funding request.
Section Revision:
November 25, 2016
6.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.6 Guidelines for Property Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.6
GUIDELINES FOR PROPERTY MANAGEMENT
NSF retains ownership to property it funds only if specified in the terms and conditions of the
award, otherwise title resides with the Recipient. In some limited cases, NSF will retain title to
property (federally-owned property) based on consideration of award-specific details and
circumstances. These special circumstances are identified at the time of award or periodically
during the award when Property is acquired. At the end of the award, NSF may choose to
invoke its conditional interest in NSF-funded property (federally-funded property) to take title
or transfer to another organization. This stewardship responsibility is necessary to protect the
U.S. Government’s and the public’s investment in these unique research facilities.
The requirements for management of property procured using NSF funds are detailed in key
NSF documents: Cooperative Agreement and Cooperative Support Agreement specific to the
award; Chapter IX Grantee Standards, Section D; Property Management Standards of the NSF
Proposal and Award Policies and Procedures Guide (PAPPG); the NSF Cooperative Agreement
FATC, Articles 6, Equipment and 55 Liability; Articles 5, 62, 63, 64, 65, 66 of the Cooperative
Agreement Modifications and Supplemental Financial & Administrative Terms and Conditions
for Major Multi-User Research Facility Projects and Federally Funded Research and
Development Centers; or other supplements. In addition, 2 CFR § 200.310-316 prescribes
standards for managing and disposing of property furnished by the Federal government or
whose cost was charged to a project supported by a Federal award. It is incumbent on the
Recipient to understand these policies and maintain a property management system.
Each NSF major research facility is unique in its mission and its circumstances and thus the
approach to property management will likely be unique but must comply with federal
regulations and NSF policy. The policies and procedures governing the management of
Federally-funded property should cover the following general topic areas:
•
The process for acquisition and procurement;
•
The process for retention of financial records (physical and electronic) necessary for
property audits or closeout;
•
The process for inventory management including custody, location, use, and disposition
(note that awardees should seek instruction from NSF in most disposition cases);
•
The process for marking or identification of property, as appropriate;
•
The process to establish and perform routine and preventative maintenance, and
•
The process to secure property while in operation, storage, or transit.
Section Revision:
December 15, 2020
6.6-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.7 Guidelines for Financial Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.7
GUIDELINES FOR FINANCIAL MANAGEMENT
[Reserved for future content]
Section Revision:
May 2, 2016
6.7-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.1 EVMS Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.8
GUIDELINES FOR EARNED VALUE MANAGEMENT SYSTEMS
Earned Value Management (EVM) is a recognized project management methodology that
provides insight into a project’s technical, cost, and schedule progress. NSF recognizes that a
properly implemented Earned Value Management System (EVMS) can provide accurate and
reliable performance measurement metrics, forecast cost and schedule information, and
uncover potential problems to support sound and timely management decisions. A properly
implemented EVMS is also essential to inform NSF’s oversight of the project.
6.8.1 EVMS Requirements
NSF requires major facility project Recipients to use EVMS as the management tool for project
planning and execution. Major Facility projects should obtain NSF’s acceptance of the project’s
EVMS before actual physical construction or major acquisitions commence. Refer to Section
4.6.3.6 of this Guide for the review requirements and procedures for NSF’s EVMS Verification,
Acceptance, and Surveillance.
6.8.1.1 Seven Basic Principles of EVMS
A project’s EVMS should adhere to the seven basic principles outlined in the EIA-748 Standard
for EVMS:
1. Plan all the project’s work scope to completion using discrete work packages and
planning packages.
2. Break down the project work scope into finite pieces that are assigned to a
responsible person or organization for control of technical, schedule and cost
objectives.
3. Integrate project work scope, schedule, and cost objectives into a performance
measurement baseline plan against which accomplishments are measured. Control
changes to the baseline.
4. Use actual costs incurred and recorded in accomplishing the work performed.
5. Objectively assess accomplishments at the work performance level.
6. Analyze significant variances from the plan, forecast impacts, develop corrective
actions, and prepare an estimate at completion based on performance to date and
the remaining work to be performed.
7. Use the EVMS information in the project’s management processes.
6.8.1.2 Guidelines for NSF’s Verification and Acceptance of EVMS
NSF uses the 32 guidelines of EIA-748 Standard to assess a major facility project’s EVMS. These
EVMS guidelines are high-level and goal-oriented. They state the qualities and operational
considerations of an integrated management system using EVM methods without mandating
detailed system characteristics. They give a project sufficient flexibility within the 32 guidelines
to develop an integrated management process that is tailored to the project’s specific needs.
Therefore, the project’s management team should implement an EVMS in a manner that
Section Revision:
December 15, 2020
6.8.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.1 EVMS Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
employs the most effective and efficient performance management methods and techniques.
The project’s EVMS process should be documented in the Project Execution Plan (PEP-10.2) or
reference an EVM system description document. The document should address how the
32 guidelines of the EIA-748 are implemented and describe how the processes are integrated
into an effective approach for project management.
To realize the full benefit of EVM for effective project management, EVM data should not be
viewed simply as static metrics or as a compliance report. Instead, the project’s EVMS should
be implemented in a manner that can provide the Recipient management team with a reliable
basis for objectively assessing performance against plan, identifying potential issues,
forecasting future trends, and initiating corrective action.
Section Revision:
December 15, 2020
6.8.1-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.2 Tailored Implementation of EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.8.2 Tailored Implementation of EVMS
In order to meet the unique challenges of being on the forefront of enabling scientific research
and functioning within scientific collaborations, tailoring of EVMS is often needed for large
science projects.
“Tailoring” means applying the requirements of the EIA-748 guidelines to fit the project’s
characteristics while still addressing all the guidelines. In other words, tailored EVMS
implements the 32 EVMS guidelines in a manner that fits the specific needs of the project for
effective project management control and reliable performance reporting to NSF, based on the
project’s characteristics, including the type of work, complexity, external dependencies, unique
constraints, etc.
A properly implemented EVMS should be no more complex than is necessary to inform sound
project management decisions while reflecting the business practices and other related
documents (such as the WBS) as outlined in the PEP.
Section Revision:
December 15, 2020
6.8.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.3 Guidelines for Establishing an EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.8.3 Guidelines for Establishing an EVMS
An Earned Value Management System (EVMS) is an integrated set of processes, people and
tools for managing projects using earned value management. The primary purpose for
establishing an EVMS is to support successful management of the project by the Recipient. As
stated above, each project should consider the nature of the project’s work, unique challenges
and constraints in establishing the process to ensure that it supports the project’s management
needs. There is no single correct process for establishing an EVMS. Below is an outline of what
is typically involved:
•
•
•
•
•
•
Obtaining institutional support for the project’s implementation of EVMS
Selecting and implementing the project management tools for EVMS
Assessing the project’s needs and existing processes to establish the project’s EVMS
process following EIA-748 EVMS guidelines
Documenting the EVMS processes and procedures in the PEP
Training the project management team to facilitate implementation of the project’s
EVMS and instill a culture that accepts the use of EVM as a credible management tool
Developing structured surveillance and training programs
•
•
Conducting an NSF EVMS verification review
Receiving NSF’s acceptance that the EVMS meets the intent of 32 guidelines (see
Section 4.6.3.6)
•
Conducting NSF surveillance reviews throughout the Construction Stage
Section Revision:
December 15, 2020
6.8.3-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.4 NSF Scaled EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.8.4 NSF Scaled EVMS
NSF recognizes that the full implementation of 32 EVMS guidelines may add unnecessary
administrative burden. To allow benefit from earned value management (EVM) methodology
without adding extra burden, NSF established the framework of scaled EVM for smaller scale
and less complex projects. Under NSF Scaled EVMS framework, the wide range of scalability
allows extending the practices of EVM basic principles to a project without adding extra burden.
The scaled EVMS means focusing on the seven basic EVMS principles with implementation of
EVMS guidelines that are essential for either reliable project management and/or agency
oversight. In other words, a scaled EVMS recognizes that smaller or less complex projects do
not require the same level of data detail and/or the same level of control rigor that are needed
for large, complex projects. In the NSF Scaled EVMS, not all of the EIA-748’s 32 guidelines are
required. NSF has identified 18 of the 32 EVMS guidelines as appropriate for meeting the
requirements of the seven basic principles. In addition, the degree of implementation of these
EVMS guidelines and the required data detail can vary based on the size and the complexity of
the project. For example, a small and less complex project may find it sufficient to use
milestones with assigned value for performance measurement, while a more complex project
will need more detailed resource-loaded schedule.
Figure 6.8.4-1
Relationship between Scaled EVMS and Full Implementation
Figure 6.8.4-1 illustrates NSF’s definition of EVM scaling and is similar to the illustration in NDIA
EVMS Scalability Guide 1. It shows the relation between the scaled EVM and the EVMS with full
implementation and formal acceptance. The Scaled EVMS Practice Guide in Section 6.8.5 of this
Guide provides more guidance on using the NSF Scaled EVM for project planning, monitoring,
and reporting. Projects using scaled EVMS do not need to have NSF’s formal EVMS acceptance.
National Defense Industrial Association (NDIA) Integrated Program Management Division’s Earned Value Management System
Guideline Scalability Guide, Revision 1, dated November 22, 2016.
1
Section Revision:
December 15, 2020
6.8.4-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.5 Practice Guide to Establish Scaled EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
6.8.5 Practice Guide to Establish Scaled EVMS
This Practice Guide organizes the scaled EVM implementation into four processes and identifies
the minimally required components from the 32 Guidelines needed to realize the benefit of
EVM and to meet the seven basic principles of EVMS as discussed in Section 6.8.1.1 of this
Guide. The guidance below refers to the EIA-748 standard EVMS 32 Guidelines and the EVMS
principles to provide a crosswalk to each process. Refer to NDIA EVMS Intent Guide 1 for
explanations and management value of each of the EIA-748 Standard Guidelines.
6.8.5.1 Process 1: Define and organize the project (Principle 1 and 2)
The goal of this process is to ensure the project scope is well defined with clearly assigned
responsibility for each of its components. This will allow the organization of the project to meet
EVMS Basic Principles 1 and 2. EVMS Guidelines 1, 2, and 5 are the primary reference guidance
for this process, which is broken down into 3 key steps:
1 Define project scope in terms of Work Breakdown Structure (WBS)
Refer to Sections 3.4.1 and 4.2.2.7 of this Guide for the explanation and guidance for
WBS and WBS dictionary. The more detailed levels of WBS, the more data details are
required and need to be managed. The key to having a properly scaled EVM is to set
the WBS level details at a reasonable high level but detailed enough to provide
sufficient visibility of the project’s work scope for management control.
2 Define project organization chart (Organization Breakdown Structure, OBS)
Refer to Section 3.4.1 of this Guide for guidance on the project organizations. To
ensure the project will benefit from EVM, the project’s internal organization
breakdown should link to the WBS and the responsibility for each WBS element
should be clearly identified.
3 Identify organizational responsibility for work, including significant subcontractors,
for sufficient level of management/control
For each of the WBS elements, the project should identify and assign the responsible
person or organization unit for the WBS element’s scope, cost, and schedule
management. For efficient control, there is typically one group responsible for the
full scope at the lowest level of the WBS.
6.8.5.2 Process 2: Establish project cost, schedule, and contingencies (Principle 3)
The goal of this process is to establish the project’s cost and schedule baseline against which
the project’s progress will be measured during execution. This process will ensure the project
meets the expectations of the EVMS Basic Principle 3. In addition to setting the project’s cost
and schedule baseline, the cost and schedule contingencies will also be estimated. Project level
milestones should also be defined and identified in the project baseline schedule. EVMS
National Defense Industrial Association (NDIA) Integrated Program Management Division’s Earned Value Management System
EIA-748-C Intent Guide dated April 29, 2014.
1
Section Revision:
December 15, 2020
6.8.5-1
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.5 Practice Guide to Establish Scaled EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Guidelines 6, 7, 8, 9, 13, and 14 are the primary reference guidelines for this process, which is
broken down into 4 key steps:
1 Schedule the work with logical sequence and task dependencies
Refer to Section 4.3.3 of this Guide for guidance in development of a resourceloaded schedule as part of the project baseline. The more detailed level of activity
breakdown, the more data detail there is to manage. The level should be suitable for
the management control needed. For a less complex project using a scaled EVM, the
activity breakdown for scheduling can be less detailed, and summary level activities/
tasks could be used when the measurement for progress is clear.
2 Identify technical milestones and/or other methods for progress measurement
Refer to Section 4.3.3 of this Guide for guidance on identifying milestones in
development of the baseline schedule. Technical milestones are important
indicators for progress measurement. Milestones with assigned value can be used in
conjunction with summary level tasks for calculating earned value. For some
projects, appropriately time-spaced milestones could be sufficient for the sole
method of progress measurement.
3 Establish time-phased budget by WBS and incorporate indirect cost
Based on the resource assignment for activities in the baseline schedule, a timephased budget can be established for each WBS. Refer to Section 3.4.1 of this Guide
for more explanation on time-phased funding profile. Refer to Sections 4.2 and 4.3
for more guidance on the development of time-phased budgets.
4 Assess project risks and estimate uncertainties to establish cost and schedule
contingency
The project needs to identify technical, cost, and schedule risks and develop a risk
register for management of identified risks. The cost and schedule contingency
estimates should be based on the estimated cost and schedule impacts associated
with the identified risks. Impacts are typically estimated using judgment based on
past experience. Refer to Sections 3.4.1, 4.2.5.4, and 6.2 for guidance on developing
a risk register and establishing the cost, schedule, and scope contingencies. A
probabilistic risk analysis is typically not used on smaller scale and less complex
projects.
6.8.5.3 Process 3: Progress and performance monitoring (Principles 4, 5, 7)
The goal of this process is to ensure the project uses the EVM concept for quantitative
measurement of progress and that the project’s progress data is reliable and used by
management to achieve project goals. EVMS Guidelines 17, 18, 22, 23, and 26 are the primary
reference guidelines for this process, which is broken down into 5 key steps:
1 Define control accounts based on project’s WBS and OBS
The project should set up control accounts at the appropriate level of WBS. The
higher the WBS level for control accounts, the less EV data detail. Properly scaled
EVMS sets the control accounts at the WBS level that suits the management control
Section Revision:
December 15, 2020
6.8.5-2
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.5 Practice Guide to Establish Scaled EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
2
3
4
5
needs. Each control account should have a clearly assigned responsible person as
the control account manager (CAM) and the responsible organization unit for
delivering the scope under this control account. This should be consistent with the
OBS established in Process 1 in Section 6.8.5.1 of this Guide. The CAM is responsible
for completing the corresponding WBS element’s work scope within the control
account’s planned budget and time duration according to the baseline.
Record and summarize actual cost by control accounts
After the control accounts are established, the project should record monthly actual
cost by control accounts. The actual cost should be reconciled with the financial
system’s accounting statements periodically. The project should have a process to
ensure the actual cost report includes accrued costs that are consistent with
completed work to facilitate accurate comparison of actual costs to planned values.
Record task progress and summarize EV for completed work by WBS
The project should assess each work activity’s progress as the basis for calculating
earned value (EV) for all activities, and then summarize the EV for each control
account. The CAM is primarily responsible for providing input on the progress
assessment for all activities.
Summarize schedule and cost performance at select levels of the WBS and perform
variance analysis
Periodically, the project should summarize EV and compare with the baseline plan
and actual costs, typically in a cost performance report (CPR). The project should
establish a variance threshold for the CAMs to perform variance analysis for
understanding the cause(s) of schedule and cost performance variances.
Management actions using information from variance analysis
Based on the information from the variance analyses, the project should take
corrective actions and mitigate risks to ensure the project execution meets the cost
and schedule goals.
6.8.5.4 Process 4: Management analysis and control (Principles 6 and 7)
The goal of this process is to ensure the project uses EV data and forward-looking metrics to
forecast the project’s cost and schedule performance and to allow for early detection of
potential issues. The forward-looking metrics are valuable input that EVM can provide, in
addition to reporting on the past performance. The project should use the forward-looking
metrics to inform management decisions and make timely adjustments to the project plan that
are necessary for the project’s success. The changes to the project’s performance measurement
baseline should be controlled to ensure the integrity of the baseline and the reliability of the EV
data. EVMS Guidelines 25, 27, 28, and 32 are the primary reference guidelines for this process,
which is broken down into 4 key steps:
1 Incorporate major changes to the project plan with change control
When the project makes major changes or adjustments to the project’s plan, such
plan should be incorporated into the performance measurement baseline through
Section Revision:
December 15, 2020
6.8.5-3
Major Facilities Guide: NSF 21-107 (July 2021)
6.8.5 Practice Guide to Establish Scaled EVMS
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
an established change control process. The project should identify approval
authorities and define thresholds for different levels of changes, and timely
incorporate such changes into the baseline upon approval. The changes should be
forward-looking and should not be used to change performance to date.
2 Periodically, update estimates of remaining work
The project should perform cost and schedule estimate updates for the remaining
work periodically, especially when new information is available. Depending on the
task complexity and total project duration, the project can decide the frequency of
such updates to suit the management need. The process for updating the estimates
for each WBS may also help identify potential issues. The forecast and identified
potential issues can be used to inform the management decision process. The
project should also forecast “Estimate at Completion (EAC)” based on the updated
estimates for remaining work and compare the EAC to the total project budget. If
the EAC exceeds the total project budget, management may need to consider
descope options.
3 Update risk assessment and assess the remaining contingencies
The project should update the risk register, assess the risk management plan, and
evaluate the remaining contingencies against remaining risks.
4 Summarize project status and forecast milestones for NSF reporting
The project should summarize the project performance in narrative form and
provide EV data, forecast EAC and forecast milestones in a status report to NSF.
Section Revision:
December 15, 2020
6.8.5-4
Major Facilities Guide: NSF 21-107 (July 2021)
7 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
7
REFERENCES
NSF Reference Documents
NSF Business Systems Review (BSR) Guide
https://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf13100
NSF Proposal and Award Policies and Procedures Guide (PAPPG)
https://www.nsf.gov/pubs/policydocs/pappg18_1/index.jsp
Suite of NSF Terms and Conditions: The NSF website on How to Manage Your Award
http://www.nsf.gov/awards/managing/
Joint National Science Board —National Science Foundation Management Report: Setting
Priorities for Large Facility Projects Supported by the National Science Foundation
(NSB-05-77); September 2005
Setting Priorities for Large Research Facility Projects Supported by the National Science
Foundation, a 2004 National Academies report
US Government Acts and Laws References
42 U.S. Code 1873(b), which states “The Foundation shall not, itself, operate any laboratories or
pilot plants.”
Endangered Species Act
Government Performance and Results Modernization Act (GPRAMA) of 2010 (Public Law 111352)
National Environmental Policy Act (NEPA)
National Historic Preservation Act (NHPA)
NSF’s Authorization Act of 2002, 42 U.S.C.1862n-4(c), signed into law on December 19, 2002,
Public Law 107-368, Section 14(c)
“Uniform Guidance”, 2 CFR § 200, Uniform Administrative Requirements, Cost Principles, and
Audit Requirements for Federal Awards, Office of Management and Budget (OMB),
December 2014
General Facility & Project Management Guides
“Best Practices for Federal Research and Development Facility Partnerships,” IDA Science &
Technology Policy Institute, IDA Paper P-5148 Log: H 14-000676
Guide to the Project Management Body of Knowledge, (PMBOK® Guide), Project Management
Institute, 5th Edition, 2013
National Defense Industrial Association (NDIA) Integrated Program Management Division
(IPMD) Earned Value Management Systems EIA-748-C Intent Guide, April 29, 2014*
Section Revision:
December 14, 2018
7-1
Major Facilities Guide: NSF 21-107 (July 2021)
7 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
National Defense Industrial Association (NDIA) Integrated Program Management Division
(IPMD) Earned Value Management System Acceptance Guide, March 24, 2013, Revision
1*
National Defense Industrial Association (NDIA) Integrated Program Management Division
(IPMD) Surveillance Guide, Revision 2, July 15, 2015
National Defense Industrial Association (NDIA) Integrated Program Management Division
(IPMD) Earned Value Management System Guideline Scalability Guide, Revision 1, dated
November 22, 2016
Cost and Schedule Estimating and Risk Analysis References
Aven, Foundations of Risk Analysis
Butts, Glenn and Kent Linton, “NASA the Joint Confidence Level Paradox – a History of Denial,”
NASA Cost Symposium, April 28, 2009
“Estimating Cost Uncertainty when only Baseline Cost is Available,” quoting R.L. Abramson and
S. A. Book, “A Quantification Structure for Assessing Risk-Impact Drivers,” Laserlight
Networks, briefing presented to the 24th Annual DOD Cost Symposium (Leesburg, VA,
September 5-7, 1990)
Federal Acquisition Regulations (FAR Part 35.017-4, Reviewing FFRDCs)
Flyvbjerg, Bent, Nils Bruzelius, and Werner Rothengatter, Megaprojects and Risk: An Anatomy
of Ambition, 2003 Cambridge University Press
Government Accountability Office (GAO) Cost Estimating and Assessment Guide,
(GAO-09-3SP 2009)
Government Accountability Office (GAO) Cost Estimating and Assessment Guide, (GAO-20195G)
Government Accountability Office (GAO) Schedule Assessment Guide (GAO-16-89G 2015)
Hillson, David, Effective Opportunity Management for Projects: Exploiting Positive Risk, Marcel
Dekker, 2004
Hillson, David and Ruth Murray-Webster, Understanding and Managing Risk Attitude, Gower,
2005
Hillson, David Private research conducted in 2004 and presented at a PMI EMEA conference
Hulett, David T., “Project Schedule Risk Analysis: Monte Carlo Simulation or PERT?” PM
Network published by the Project Management Institute, February 2000, pp. 43 ff
Hulett, David T., “Use Decision Trees to Make Important Project Decisions", Cost Engineering
(published by AACEI, July / August 2014)
Hulett, David T., Integrated Cost-Schedule Risk Analysis, Cost Engineering (AACE International),
November/December 2012, pp. 5-15, with Michael Nosbisch
Section Revision:
December 14, 2018
7-2
Major Facilities Guide: NSF 21-107 (July 2021)
7 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Hulett, David T., principal author, Recommended Practice 57R-09, Integrated Cost and Schedule
Risk Analysis using Monte Carlo Simulation of a CPM Model, AACEI, 2011
Hulett, David T., principal author, Recommended Practice 85R-14, "Use of Decision Trees in
Decision Making", AACEI, 2014
Hulett, David T., “Use Decision Trees to Make Important Project Decisions,” Cost Engineering
(published by AACEI, July / August 2014
Hulett, David T., Integrated Cost-Schedule Risk Analysis, Gower Publishers, 2011
Hulett, David T., Practical Schedule Risk Analysis, Gower Publishers, 2009
Merrow, Edward W., Industrial Megaprojects, 2011, Wiley
NASA's Challenges to Meeting cost, Schedule, and Performance Goals, NASA IG-12-21
Pariseau and Oswalt, Using Data Types and Scales for Analysis and Decision Making
Planning, Budgeting, Acquisition, and Management of Capital Assets, OMB Circular No. A–11
(2014).
“Understanding the Joint Confidence Level (JCL) at NASA,” NASA Office of Evaluation at 9/4/14.
Section Revision:
December 14, 2018
7-3
Major Facilities Guide: NSF 21-107 (July 2021)
8 List of Acro
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
8
LIST OF ACRONYMS
AC
AD
ADR
AICA
AMBAP
AWP
BAC
BFA
BLS
BOE
BSR
CA
CAP
CATEX
CCB
CCP
CDR
CEP
CER
CFR
CISO
CO
CORF
CPI
CPM
CSB
CV
DACS
DD
DGA
DIAS
DRB
EA
EAC
EHR
EIS
ES&H
ETC
EV
EVM
EVMS
Section Revision:
December 15, 2020
Actual Cost
Assistant Director
Accountable Directorate Representative
American Innovation and Competitiveness Act
Award Monitoring and Business Assistance Program
Annual Work Plan
Budget at Completion
Office of Budget, Finance, and Award Management
Bureau of Labor Statistics
Basis of Estimate
Business Systems Review
Cooperative Agreement
Cost Analysis and Pre-Award
Categorical Exclusion (NEPA)
Change Control Board
Change/Configuration Control Process
Conceptual Design Review
Cost Estimating Plan
Compliance Evaluation Review
Code of Federal Regulations
Chief Information Security Officer
Contracting Officer
Chief Officer for Research Facilities
Cost Performance Index
Critical Path Method
Cooperative Support Branch
Cost Variance
Division of Acquisition and Cooperative Support
Division Director
Division of Grants and Agreements
Division of Institution and Award Support
Director’s Review Board
Environmental Assessment
Estimate at Completion
Education and Human Resources
Environmental Impact Statement
Environmental Safety and Health
Estimate to Complete (for Cost)
Earned Value
Earned Value Management
Earned Value Management System
8-1
Major Facilities Guide: NSF 21-107 (July 2021)
8 List of Acro
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
FAR
FDR
FFRDC
FGB
FRP
FY
GAO
G/AO
GPRA
GR&As
HLFO
ICE
IMP
IPT
ISO
IT
LFO
LOE
M&S
MFG
MFWG
MOU
MP
MREFC
NCE
NCOP
NEPA
NHPA
NICRA
NRC
NSB
NSF
O&M
OD
OGC
OMB
PAM
PAPPG
PDR
PEP
PI
PMB
PMBOK
Section Revision:
December 15, 2020
Federal Acquisition Regulations
Final Design Review
Federally Funded Research and Development Center
Facilities Governance Board
Facilities Readiness Panel
Fiscal Year
Government Accountability Office
Grants and Agreements Officer
Government Performance and Results Act
Ground Rules and Assumptions
Head, Large Facilities Office
Independent Cost Estimate
Internal Management Plan
Integrated Project Team
Information Security Officer
Information Technology
Large Facilities Office
Level of Effort
Materials and Supplies
Major Facilities Guide
Major Facilities Working Group
Memorandum of Understanding
Management Plan (for mid-scale projects)
Major Research Equipment and Facilities Construction
No-Cost Extension
No Cost Overrun Policy
National Environmental Policy Act
National Historic Preservation Act
Negotiated Indirect Cost Rate Agreement
National Research Council
National Science Board
National Science Foundation
Operations and Maintenance
Office of the Director
Office of the General Counsel
Office of Management and Budget
Proposal and Award Manual
Proposal and Award Policies and Procedures Guide
Preliminary Design Review
Project Execution Plan
Principal Investigator
Performance Measurement Baseline
Project Management Body of Knowledge
8-2
Major Facilities Guide: NSF 21-107 (July 2021)
8 List of Acro
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
PMCS
PMI
PO
PTO
PV
R&D
R&RA
RAEAC
RBS
RBS
RI
RLS
RM
RMP
S&E
SOG
SME
SPC
SPI
SV
SVT
TPC
WBS
Section Revision:
December 15, 2020
Project Management Control System
Project Management Institute
Program Officer
Paid Time Off
Planned Value
Research and Development
Research and Related Activities
Risk-Adjusted Estimate at Completion
Resource Breakdown Structure (schedule development)
Risk Breakdown Structure
Research Infrastructure
Resource-Loaded Schedule
Risk Manager
Risk Management Plan
Science and Engineering
Standard Operating Guidance
Subject Matter Expert
Statistical Process Control
Schedule Performance Index
Schedule Variance
Schedule Visibility Task
Total Project Cost
Work Breakdown Structure
8-3
Major Facilities Guide: NSF 21-107 (July 2021)
9.1 Lexicon Preface
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
9
9.1
LEXICON
LEXICON PREFACE
This Lexicon contains definitions of project and program management terms used in this Guide,
as applied to NSF major facilities. It is a combination of specialized terms defined by NSF and
used in the management of its major facilities, and terms and definitions commonly used in
professional project and program management. A subset of common project management
terms compatible with NSF usage were selected from a standard source, the PMI Lexicon, 1 for
inclusion in this lexicon.
The Lexicon provides a common set of standard terms and definitions that should facilitate
communication and understanding between stakeholders when used in documents and
correspondence related to major facility management.
The terms and definitions included in this lexicon are in development and are subject to
modifications in future versions.
Entries in italics in this lexicon have been reproduced with permission from Project Management Institute, Inc., [PMI Lexicon],
(2012) Copyright and all rights reserved.
1
Section Revision:
December 14, 2018
9.1-1
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
9.2
TERMS AND DEFINITIONS
Acceptance Criteria. A set of conditions that is required to be met before deliverables are
accepted.
Activity. A distinct, scheduled portion of work performed during the course of a project.
Actual Cost. The realized cost incurred for the work performed on an activity during a specific
time period.
Allowance. Resources included in the basis of estimate for baseline cost estimates to cover the
cost of known but as-of-yet undefined details or requirements for an individual WBS element.
May be used when the level of project definition may not enable certain costs to be estimated
definitively or times when it is simply not cost effective to quantify and cost every small item
included with the WBS element, but reliable correlations are available.
Analogous Estimating. A technique for estimating the duration or cost of an activity or a
project, using historical data from a similar activity or project.
Apportioned Effort. An activity where effort is allotted proportionately across certain discrete
efforts and not divisible into discrete efforts. (Note: Apportioned effort is one of three earned
value management [EVM] types of activities used to measure work performance.)
Approval. The act of officially accepting an idea, action, or plan.
Assistance. The act of giving support or help; making it easier for someone to do something or
for something to happen.
Assumption. A factor in the planning process that is considered to be true, real, or certain,
without proof or demonstration.
Assurance. To give a strong and/or definite statement that something will happen or that
something is true; to give confidence to.
Award Instrument. An agreement between NSF and a Recipient with the terms and conditions
set forth in (cooperative agreements, contracts, etc.).
Backward Pass. A critical path method technique for calculating the late start and late finish
dates by working backward through the schedule model from the project end date.
Baseline. The cost and schedule plan for a scope of work, used during planning. For NSF,
contingency is not included in the baseline but is held and managed separately. A planning
baseline may or may not be under change control. Once a baseline has been approved, is under
change control, and is used as the basis for Earned Value Measurement comparison, it is
referred to as the Performance Measurement Baseline.
Baseline Definition. The description of the approved scope of work and resources for a
construction project, including a hierarchical, product-oriented Work Breakdown Structure
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-1
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
(WBS) and associated WBS dictionary; the cost and schedule Performance Measurement
Baselines (PMB). Any contingency amounts, cost and time, are added to the baseline to
establish the Recipient managed TPC.
Basis of Estimate. Supporting documentation outlining the details used in establishing project
estimates such as assumptions, constraints, level of detail, ranges, and confidence levels.
Bottom-up Estimating. A method of estimating project duration or cost by aggregating the
estimates of the lower-level components of the work breakdown structure (WBS).
*Budget at Completion. The sum of all budgets established for the work to be performed. (For
NSF projects, contingency amounts are not included in the ETC, EAC, BAC, or PMB due to the
NSF requirement that contingency is held and managed separately from the baseline.)
Budget Contingency. See Contingency.
Change Control. A process whereby modifications to documents, deliverables, or baselines
associated with the project are identified, documented, approved, or rejected.
Change Control Board. A formally chartered group responsible for reviewing, evaluating,
approving, delaying, or rejecting changes to the project, and for recording and communicating
such decisions.
Change Control System. A set of procedures that describes how modifications to the project
deliverables and documentation are managed and controlled.
Change Request. A formal proposal to modify any document, deliverable, or baseline.
Closeout. The process by which the Federal awarding agency or pass-through entity determines
that all applicable administrative actions and all required work of the Federal award have been
completed.
Code of Accounts. A numbering system used to uniquely identify each component of the work
breakdown structure.
Conceptual Design Phase. The first phase of the Design Stage, after passing the gate from the
Development Stage, that advances the definition of the scope and requirements, determines
feasibility, and produces updated drafts of most elements of the Project Execution Plan,
including parametric cost and schedule range estimates and a preliminary risk analysis.
Conditional Interest. The government’s right to invoke a transfer of Recipient-titled property;
including to the government or to another Recipient.
Contingency. A planned amount of budget and time added to the baseline estimate to allow for
items, conditions, or events for which the state, occurrence, or effect is uncertain and that
experience shows will likely result, in aggregate, in additional costs. These events are often
referred to as “known-unknowns” and are considered manageable by the Recipient. Budget
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-2
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
and schedule contingency are typically estimated using statistical analysis and professional
judgment based on experience. Budget and schedule contingency are called out separately as
part of the Total Project Cost and Total Project Duration, respectively, and obligated to the
project for the Recipient to manage based on need per NSF policy.
Contingency Report Table. A table containing a list of change control actions and allocations,
with ties to associated WBS elements and identified risk events, for all Performance
Measurement Baseline (PMB) changes that impact the use of contingency.
Constraint. A limiting factor that affects the execution of a project, program, portfolio, or
process.
Construction Stage. The period of time in which funds are obligated for acquisition and/or
construction of a facility that fulfills the terms and conditions set forth in an award instrument
between NSF and the Recipient(s). This Stage ends with the start of the Operations Stage.
Contract. A contract is for the purpose of obtaining goods and services for the non-Federal
entity’s own use and creates a procurement relationship with the contractor. All contracts over
$250,000 require written prior NSF authorization.
Control Account. A management control point where scope, budget, actual cost, and schedule
are integrated and compared to earned value for performance measurement.
Corrective Action. An intentional activity that realigns the performance of the project work
with the project plan.
Cost Book. A compilation of Cost Book Sheets, typically used to present baseline or total project
cost, but may be used to present rolled-up costs for smaller elements or sub-elements.
Cost Book Sheet. A compilation of related information from the Cost Model Data Set, used to
define and present the cost estimate for a particular element or sub-element of a deliverablebased work breakdown structure for construction or a functional, activity, and/or deliverable
based work breakdown structure for operations.
Cost Estimating Plan. A plan to establish and communicate how the preparation, development,
review and approval of the estimate will be completed.
Cost Model Data Set. The cost data used as input to software tools and/or project reports to
organize, correlate, and calculate different project management information.
Cost Performance Index. A measure of the cost efficiency of budgeted resources expressed as
the ratio of earned value to actual cost.
Cost Variance. The amount of budget deficit or surplus at a given point in time, expressed as the
difference between the earned value and the actual cost.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-3
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Crashing. A technique used to shorten the schedule duration for the least incremental cost by
adding resources.
Critical Path. The sequence of activities that represents the longest path through a project,
which determines the shortest possible duration.
Critical Path Activity. Any activity on the critical path in a project schedule.
Critical Path Method. A method used to estimate the minimum project duration and determine
the amount of scheduling flexibility on the logical network paths within the schedule model.
Current Plan. The project cost and schedule plan reflecting the status of progress to date and
updated estimates for completing remaining work that is compared to the approved
Performance Measurement Baseline (PMB), as part of Earned Value Management.
Custody. Protective care or guardianship responsibilities of the Recipient over Federally-funded
property.
Data Date. A point in time when the status of the project is recorded.
Decision Tree Analysis. A diagramming and calculation technique for evaluating the
implications of a chain of multiple options in the presence of uncertainty.
Decomposition. A technique used for dividing and subdividing the project scope and project
deliverables into smaller, more manageable parts.
Defect Repair. An intentional activity to modify a nonconforming product or product
component.
Deliverable. Any unique and verifiable product, result, or capability to perform a service that is
required to be produced to complete a process, phase, or project.
De-Scoping Options (Plan). See Scope Management Plan.
Design Stage. The life cycle stage for detailed planning for projects approved by the NSF
Director at the end of the Development Stage and funded under the formal major facility
planning process. It is divided into the Conceptual, Preliminary, and Final Design Phases; with a
formal and rigorous review gate at the end of each phase to show readiness for advancement
to a higher level of refinement with regard to scope, cost, and schedule.
Development Stage. The facility life cycle stage in which initial high-level ideas are developed
and a consensus built for the potential long-term need, priorities, and general requirements for
a large research facility of interest to NSF and the broader research community.
Discrete Effort. An activity that can be planned and measured and that yields a specific output.
(Note. Discrete effort is one of three earned value management [EVM] types of activities used to
measure work performance.)
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-4
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Divestment Stage. The stage in the facility life cycle encompasses divestment of the facility
starting after the NSF Operations Stage ends and funding for divestment begins. Divestment
options may include partial or complete transfer of a facility to another entity’s operational and
financial control (with or without reduction in project scope), “moth-balling” the facility so that
operations can be restarted at a later date, or decommissioning. Decommissioning may include
complete removal of the infrastructure and site restoration.
Early Finish Date. In the critical path method, the earliest possible point in time when the
uncompleted portions of a schedule activity can finish based on the schedule network logic, the
data date, and any schedule constraints.
Early Start Date. In the critical path method, the earliest possible point in time when the
uncompleted portions of a schedule activity can start based on the schedule network logic, the
data date, and any schedule constraints.
Earned Value. The measure of work performed expressed in terms of the budget authorized for
that work.
Earned Value Management. A methodology that combines scope, schedule, and resource
measurements to assess project performance and progress.
Effort. The number of labor units required to complete a schedule activity or work breakdown
structure component, often expressed in hours, days, or weeks.
eJacket. An electronic Web portal for NSF staff to perform essential business functions related
to proposal and award processing and to access associated documents.
Enterprise Environmental Factors. Conditions, not under the immediate control of the team,
that influence, constrain, or direct the project, program, or portfolio.
*Estimate at Completion. The expected total cost of completing all work expressed as the sum
of the actual cost to date and the estimate to complete. (For NSF projects, contingency amounts
are not included in the ETC, EAC, BAC, or PMB due to the NSF requirement that contingency is
held and managed separately from the baseline.)
*Estimate to Complete. The expected cost to finish all the remaining project work. (For NSF
projects, contingency amounts are not included in the ETC, EAC, BAC, or PMB due to the NSF
requirement that contingency is held and managed separately from the baseline.)
Facility. Shared-use infrastructure, equipment, or instrument - or an integrated network
and/or collection of the same – that is either acquired or constructed to collect, analyze, and
provide necessary data and information in support of research having a major impact on a
broad segment of a scientific or engineering discipline.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-5
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Facility Life Cycle. The sequence of steps or stages that characterize the lifetime of a facility
from beginning to end. For NSF, the stages are Development, Design, Construction, Operations,
and Divestment.
FastLane. NSF online website through which we conduct our relationship to researchers and
potential researchers, reviewers, and research administrators and their organizations. Other
web portals used by Recipients to submit proposal and reporting actions include Grants.gov and
Research.gov.
Fast Tracking. A schedule compression technique in which activities or phases normally done in
sequence are performed in parallel for at least a portion of their duration.
Federally-Funded Property. Any property acquired, fabricated, or improved in whole or in part
with federal funds, whether funded by NSF or any other federal agency.
Federally-Owned Property. Any federally-funded property in the custody of the Recipient
where the agency has retained ownership. The Recipient is subject to use and disposition
requirements in accordance with the award and must submit to NSF annually an inventory
listing of Federally-owned property in its custody.
Final Design Phase. The third and last phase of the Design Stage, after a successful Preliminary
Design Phase, that further refines the project definition and the Project Execution Plan and
demonstrates that project planning and management meet requirements for readiness to
receive funding. The Final Design Phase ends in a potential NSF approval to obligate
construction funds.
Finish-to-Finish. A logical relationship in which a successor activity cannot finish until a
predecessor activity has finished.
Finish-to-Start. A logical relationship in which a successor activity cannot start until a
predecessor activity has finished.
Forward Pass. A critical path method technique for calculating the early start and early finish
dates by working forward through the schedule model from the project start date or a given
point in time.
Free Float. The amount of time that a schedule activity can be delayed without delaying the
early start date of any successor or violating a schedule constraint.
Gantt Chart. A bar chart of schedule information where activities are listed on the vertical axis,
dates are shown on the horizontal axis, and activity durations are shown as horizontal bars
placed according to start and finish dates.
Independent Cost Estimate Review. As defined by the 2009 GAO Cost Estimating and
Assessment Guide, there are eight different types of reviews that may be used by NSF to help
validate the Recipient’s estimate. An Independent Cost Estimate (ICE) is one of the eight types.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-6
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Internal Management Plan. The internal document that defines NSF strategy for conducting
project oversight and assurance, managing NSF risk, and providing project funding.
Lag. The amount of time whereby a successor activity is required to be delayed with respect to a
predecessor activity.
Late Finish Date. In the critical path method, the latest possible point in time when the
uncompleted portions of a schedule activity can finish based on the schedule network logic, the
project completion date, and any schedule constraints.
Late Start Date. In the critical path method, the latest possible point in time when the
uncompleted portions of a schedule activity can start based on the schedule network logic, the
project completion date, and any schedule constraints.
Lead. The amount of time whereby a successor activity can be advanced with respect to a
predecessor activity.
Lessons Learned. The knowledge gained during a project which shows how project events were
addressed or should be addressed in the future for the purpose of improving future
performance.
Level of Effort. An activity that does not produce definitive end products and is measured by the
passage of time. (Note. Level of effort is one of three earned value management [EVM] types of
activities used to measure work performance.)
Liens List. A list of expected adjustments to project scope, budget, and schedule contingency
amounts that are waiting for implementation, including formal change control actions for
planned baseline modifications, scope contingency options held for decision, realized risks, and
coverage of variances.
Logical Relationship. A dependency between two activities or between an activity and a
milestone.
Major Facility. A science and engineering facility project that exceeds $100,000,000 in
construction, acquisition, or upgrade costs to the NSF Foundation.
Management. The act of controlling and making decisions about an operation, organization or
project; the act or process of deciding how to use something; the judicious use of means to
accomplish an end.
Management Reserve. An amount of money or time included as part of the Total Project Cost
estimate to address unforeseen events or uncertainties that are beyond the control of the
Recipient or the agency. These events are often referred to as “unknown unknowns”. The
amount of management reserve (if any) is determined based on agency risk tolerance and
managed exclusively by the agency. Similar “reserves” are not allowable in Recipient estimates
per the Uniform Guidance.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-7
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Milestone. A significant point or event in a project, program, or portfolio.
Most Likely Duration. An estimate of the most probable activity duration that takes into
account all of the known variables that could affect performance.
“No Cost Overrun” Policy. NSF policy requiring that a Total Project Cost estimate established at
the Preliminary Design Phase have adequate contingency to cover all foreseeable risks.
However, NSF conducts its oversight of projects against the Total Project Cost authorized by the
NSB following Final Design Review (FDR).
Operations Stage. The life cycle stage that succeeds Construction and includes the day-to-day
work to operate and maintain the facility and to perform research. Operations may also include
activities to transition from construction to operations, replacement or upgrade activities,
technology research and development, and activities that support planning and staging for the
Divestment Stage.
Opportunity. A risk that would have a positive effect on one or more project objectives.
Optimistic Duration. An estimate of the shortest activity duration that takes into account all of
the known variables that could affect performance.
Organizational Breakdown Structure. A hierarchical representation of the project organization,
which illustrates the relationship between project activities and the organizational units that will
perform those activities.
Oversight. Watchful and responsible care of something or some activity; regulatory supervision.
Ownership [Owned]. The ultimate and exclusive rights and control over property.
Parametric Estimating. An estimating technique in which an algorithm is used to calculate cost
or duration based on historical data and project parameters.
Path Convergence. A relationship in which a schedule activity has more than one predecessor.
Path Divergence. A relationship in which a schedule activity has more than one successor.
Percent Complete. An estimate expressed as a percent of the amount of work that has been
completed on an activity or a work breakdown structure component.
Performance Measurement Baseline. (PMB) The approved cost and schedule baseline for
accomplishing project work scope used as a basis of comparison for Earned Value Management.
The PMB is typically approved and established at the time of the construction award, in the
terms and conditions of the award instrument, and is under formal change control for the life of
the project. (For NSF projects, contingency amounts are not included in the PMB due to the NSF
requirement that contingency is held and managed separately from the baseline.)
Pessimistic Duration. An estimate of the longest activity duration, which takes into account all
of the known variables that could affect performance.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-8
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Phase Gate. A review at the end of a phase in which a decision is made to continue to the next
phase, to continue with modification, or to end a project or program.
Planned Value. The authorized budget assigned to scheduled work.
Planning Package. A component of work within the WBS with budget and duration but without
detailed schedule activities (work package). A planning package should be converted to work
package(s) when the lower-level details of the work are defined and prior to start of the work.
Portfolio. Projects, programs, subportfolios, and operations managed as a group to achieve
strategic objectives.
Portfolio Management. The centralized management of one or more portfolios to achieve
strategic objectives.
Precedence Diagramming Method. A technique used for constructing a schedule model in
which activities are represented by nodes and are graphically linked by one or more logical
relationships to show the sequence in which the activities are to be performed.
Predecessor Activity. An activity that logically comes before a dependent activity in a schedule.
Preliminary Design Phase. The second phase of the Design Stage, after the Conceptual Design
Phase, that further advances the project definition and the Project Execution Plan. It produces a
bottom-up scope, cost, schedule, and risk analysis of sufficient maturity to allow determination
of the Project Total Cost and Duration for a stated future start date and to establish the
construction budget request.
Preventive Action. An intentional activity that ensures the future performance of the project
work is aligned with the project management plan.
Probabilistic Risk Assessment. A quantitative risk analysis that uses probability distributions to
represent the uncertainty usually present in the cost of a deliverable or the duration of a
scheduled activity and discrete risks, in order to obtain a range of outcomes for overall project
cost and finish dates that support selection of contingency amounts as part of risk
management. Many commercial probabilistic risk analysis applications employ Monte Carlo
simulations of project cost and schedule.
Probability and Impact Matrix. A grid for mapping the probability of each risk occurrence and
its impact on project objectives if that risk occurs.
Procurement Management Plan. A component of the project or program management plan
that describes how a team will acquire goods and services from outside of the performing
organization.
Program. A group of related projects, subprograms, and program activities that are managed in
a coordinated way to obtain benefits not available from managing them individually.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-9
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Program Management. The application of knowledge, skills, tools, and techniques to a program
to meet the program requirements and to obtain benefits and control not available by
managing projects individually.
Progressive Elaboration. The iterative process of increasing the level of detail in a project
management plan as greater amounts of information and more accurate estimates become
available.
Project Calendar. A calendar that identifies working days and shifts that are available for
scheduled activities.
Project End Date. The projected date for the completion of all the project baseline schedule
activities plus use of all schedule contingency. (Note that this date may be earlier than, but no
later than, the end date of the award instrument.)
*Project Execution Plan. The document that describes how the project will be executed,
monitored, and controlled.
Project Life Cycle. The series of phases that a project passes through from its initiation to its
closure.
Project Management. The application of knowledge, skills, tools, and techniques to project
activities to meet the project requirements.
Project Management Office. A management structure that standardizes the project-related
governance processes and facilitates the sharing of resources, methodologies, tools, and
techniques.
Project Manager. The person assigned by the performing organization to lead the team that is
responsible for achieving the project objectives.
Project Phase. A collection of logically related project activities that culminates in the
completion of one or more deliverables.
Project Schedule. An output of a schedule model that presents linked activities with planned
dates, durations, milestones, and resources.
Project Scope. The work performed to deliver a product, service, or result with the specified
features and functions.
Project Scope Statement. The description of the project scope, major deliverables, assumptions,
and constraints.
Property. Consists of both real property and personal property. Generally, real property
includes land and things built on land that are not typically moveable, such as buildings.
Personal property is all other property whether it is tangible (having a physical existence) or
intangible (i.e., intellectual property and other financial instruments). Personal property
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-10
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
includes “equipment” which is any tangible property with a useful life greater than 1 year and
typically a per-unit purchase cost of $5,000 or more unless the Recipient sets a lower value for
financial statement purposes. Equipment can range from the very small to the very large as
long as it is moveable, in principle.
Property Trust Relationship. The arrangement where the Recipient has custody of Federallyfunded property for the beneficiaries of the project or program subject to established
obligations and conditions.
Quality Management Plan. A component of the project or program management plan that
describes how an organization's quality policies will be implemented.
Re-Baselining. A modification to the Construction Project Definition that results in a change
that is outside the terms set forth in the award instrument for any of the following: 1) Total
Project Cost (TPC); 2) total project duration; or 3) project scope, except for approved options in
the scope management plan. The initial TPC and award duration are part of the NSB
authorization for the Construction Stage and inform the terms of the award. Re-baselining
actions require special review and approval by NSF beyond those of the typical change control
approval process for re-planning actions.
Re-Planning. A normal project management process to modify or re-organize the Performance
Measurement Baseline cost and/or schedule plans for future work without impacting total
project cost, total project duration, or overall scope objectives; or the implementation of
approved scope management options. Formal change control processes are followed for all
baseline changes. Retroactive changes to past performance should not be included in replanning.
Recipient-Titled Property. Any Federally-funded property in the custody of the Recipient where
the government has not retained ownership, but the property is still subject to established
obligations and conditions. Recipient-titled property is held in trust for the beneficiaries of the
project or program (generally the science community) under which the property was acquired
or improved. This arrangement is otherwise known as the “property trust relationship.”
Generally, the Recipient may not encumber (i.e., place a lien on) the property and must follow
the award terms and conditions on use, management, and disposition of the property. Only
following disposition decisions at the end of the award, would ownership potentially transfer to
the Recipient.
Recovery Plan. A formalized plan of corrective actions to address negative cost and/or schedule
trends for return of the project to within the project definition. The plan should be based on a
comprehensive analysis of the variances and establish a timeline for actions and recovery.
Requirement. A condition or capability that is required to be present in a product, service, or
result to satisfy a contract or other formally imposed specification.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-11
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Research Infrastructure. Any combination of facilities, equipment, instrumentation,
computational hardware and software, and the necessary supporting human capital.
Resource Breakdown Structure. A hierarchical representation of resources by category and
type.
Resource Calendar. A calendar that identifies the working days and shifts upon which each
specific resource is available.
Resource Leveling. A technique in which start and finish dates are adjusted based on resource
constraints with the goal of balancing demand for resources with the available supply.
Responsibility Assignment Matrix. A grid that shows the project resources assigned to each
work package.
Review and Recommend. The act of carefully looking at or examining the quality or condition
of something AND then suggesting that someone taken action or do something.
Risk. An uncertain event or condition that, if it occurs, has a positive or negative effect on one or
more project objectives.
Risk Acceptance. A risk response strategy whereby the project team decides to acknowledge the
risk and not take any action unless the risk occurs.
Risk-Adjusted Estimate at Completion. The expected total cost of completing all work
expressed as the sum of the actual cost to date, the estimate to complete, and the project’s
remaining risk exposure.
Risk Avoidance. A risk response strategy whereby the project team acts to eliminate the threat
or protect the project from its impact.
Risk Breakdown Structure. A hierarchical representation of risks that is organized according to
risk categories.
Risk Category. A group of potential causes of risk.
Risk Exposure. Quantitative impact of risk for a single event, quoted in currency or time, and
typically estimated from probability of occurrence and a likely impact or consequence. Overall
project risk exposure results from an accumulation of individual risk impacts for the work to be
completed, typically determined by applying probabilistic analysis to the set of individual risks.
Risk Management Plan. A component of the project, program, or portfolio management plan
that describes how risk management activities will be structured and performed.
Risk Mitigation. A risk response strategy whereby the project team acts to reduce the
probability of occurrence or impact of a risk.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-12
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Risk Register. A document in which the results of risk analysis and risk response planning are
recorded.
Risk Transference. A risk response strategy whereby the project team shifts the impact of a
threat to a third party, together with ownership of the response.
Rolling Wave Planning. An iterative planning technique in which the work to be accomplished in
the near term is planned in detail, while the work in the future is planned at a higher level.
Schedule Basis Document. A written document to describe the schedule at a high-level
including dependencies, key dates, assumptions, and the project team’s assessment of the
schedule integrity and quality using GAO schedule characteristics.
Schedule Compression. A technique used to shorten the schedule duration without reducing the
project scope.
Schedule Contingency. See contingency.
Schedule Management Plan. A component of the project or program management plan that
establishes the criteria and the activities for developing, monitoring, and controlling the
schedule.
Schedule Margin. An activity with duration and no resources used in the performance
measurement baseline (PMB) schedule to manage or monitor interim milestones or external
deliverable requirements. A schedule margin activity should not be on the critical path that
established the PMB duration. Schedule contingency amounts are not included in the PMB due
to the NSF requirement that contingency is held and managed separately from the baseline.
Schedule Model. A representation of the plan for executing the project’s activities, including
durations, dependencies, and other planning information, used to produce a project schedule
along with other scheduling artifacts.
Schedule Performance Index. A measure of schedule efficiency expressed as the ratio of earned
value to planned value.
Schedule Variance. A measure of schedule performance expressed as the difference between
the earned value and the planned value.
Schedule Visibility Task. (SVT) Schedule activities with no resources assigned whose duration is
greater than zero. SVTs may be wait periods such as concrete curing timing or equipment
delivery within the PMB or may be used to represent external effort that is not part of the PMB.
SVTs may also be used to increase management visibility to items otherwise represented as lag
or constrained milestones.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-13
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Scope Baseline. The approved version of a scope statement, work breakdown structure (WBS)
and its associated WBS dictionary, which can be changed only through formal change control
procedures and is used as a basis for comparison.
Scope Contingency. Scope included in the project baseline definition that can be removed
without affecting the overall project’s objectives, but that may still have undesirable effects on
facility performance. Identified scope contingency should have a value equal to at least 10% of
the baseline budget.
Scope Management Plan. A component document of the Project Execution Plan that describes
how scope contingency is determined, monitored, and controlled over the project lifetime.
Scope Creep. The uncontrolled expansion to product or project scope without adjustments to
time, cost, and resources.
Scope Management Plan. A component of the project or program management plan that
describes how the scope will be defined, developed, monitored, controlled, and validated.
S-Curve Analysis. An earned value management technique used to indicate performance trends
by using a graph that displays cumulative costs over a specific time period.
Secondary Risk. A risk that arises as a direct result of implementing a risk response.
Sponsor. A person or group that provides resources and support for the project, program, or
portfolio, and is accountable for enabling success.
Staffing Management Plan. A component of the human resource plan that describes when and
how team members will be acquired and how long they will be needed.
Stakeholder. An individual, group, or organization that may affect, be affected by, or perceive
itself to be affected by a decision, activity, or outcome of a project, program, or portfolio.
Start-to-Finish. A logical relationship in which a successor activity cannot finish until a
predecessor activity has started.
Start-to-Start. A logical relationship in which a successor activity cannot start until a
predecessor activity has started.
Subaward: Award made by the prime Recipient of an NSF for the purpose of carrying out a
portion of a Federal award and creates a Federal assistance relationship with the Subrecipient.
It does not include payments to a contractor or payments to an individual that is a beneficiary
of a Federal program. A subaward may be provided through any form of legal agreement,
including an agreement that the prime Recipient considers a contract. All subawards require
written prior NSF authorization.
Successor Activity. A dependent activity that logically comes after another activity in a schedule.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-14
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Summary Activity. A group of related schedule activities aggregated and displayed as a single
activity.
Termination. The ending of a Federal award, in whole or in part at any time prior to the
planned end of period of performance.
Title [Titled]. A right to something (for example, property), but the actual rights conferred may
be limited; for example, recipient-titled property routinely carries limitations on use,
management, and disposition under the “property trust relationship” between the government
and the Recipient.
Threat. A risk that would have a negative effect on one or more project objectives.
Three-Point Estimate. A technique used to estimate cost or duration by applying an average or
weighted average of optimistic, pessimistic, and most likely estimates when there is uncertainty
with the individual activity estimates.
To-Complete Performance Index. A measure of the cost performance that is required to be
achieved with the remaining resources in order to meet a specified management goal,
expressed as the ratio of the cost to finish the outstanding work to the remaining budget.
Total Float. The amount of time that a schedule activity can be delayed or extended from its
early start date without delaying the project finish date or violating a schedule constraint.
Total Project Cost. The sum of the baseline budget (including indirect costs), the budget
contingency, fee/profit (as applicable), and management reserve (if authorized) for the
Construction Stage.
The TPC authorized by the NSB following FDR is a “not-to-exceed” figure against which NSF
manages the No Cost Overrun Policy. The initial award may be at or below this figure.
Throughout the Design and Construction Stages, the TPC is an estimate and only at the end of
the project will the final TPC be known.
Total Project Duration. The sum of the amount of time (in months) for the Performance
Measurement Baseline schedule duration and the schedule contingency. The NSB authorized
duration is the estimated project duration plus approximately 6 months.
Trigger Condition. An event or situation that indicates that a risk is about to occur.
Variance Analysis. A technique for determining the cause and degree of difference between the
Performance Measurement Baseline and actual performance.
Variance at Completion. A projection of the amount of budget deficit or surplus, expressed as
the difference between the budget at completion and the estimate at completion.
WBS Dictionary. A document that provides detailed deliverable, activity, and scheduling
information about each component in the work breakdown structure.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-15
Major Facilities Guide: NSF 21-107 (July 2021)
9.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
What-If Scenario Analysis. The process of evaluating scenarios in order to predict their effect on
project objectives.
Work Breakdown Structure. A hierarchical decomposition of the total scope of work to be
carried out by the project team to accomplish the project objectives and create the required
deliverables.
Work Package. The work defined at the lowest level of the work breakdown structure for which
cost and duration can be estimated and managed.
Workaround. A response to a threat that has occurred, for which a prior response had not been
planned or was not effective.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.
Section Revision:
December 15, 2020
9.2-16
Major Facilities Guide: NSF 21-107 (July 2021)
Appendix A: Ranking Criteria for Prioritizing Major Facility Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
10 APPENDICES
Appendix A. Ranking Criteria for Prioritizing Major Facility Projects
Excerpted from the National Academies’ Report: Setting Priorities for Large Facility Projects
Supported by the National Science Foundation
(http://www.nap.edu/books/0309090849/html/R1.html) 1.
First Ranking: Scientific and Technical Criteria Assessed by Researchers in a Field or
Interdisciplinary Area
•
Which projects have the most scientific merit, potential and opportunities within a field
or interdisciplinary area?
•
Which projects are the most technologically ready?
•
Are the scientific credentials of the proposers of the highest rank?
•
Are the project-management capabilities of the proposal team of the highest quality?
Second Ranking: Agency Strategic Criteria Assessed across Related Fields
•
Which projects will have the greatest impact on scientific advances in this set of related
fields taking into account the importance of balance among fields for NSF's portfolio
management in the nation's interest?
•
Which projects include opportunities to serve the needs of researchers from multiple
disciplines or the ability to facilitate interdisciplinary research?
•
Which projects have major commitments from other agencies or countries that should
be considered?
•
Which projects have the greatest potential for education and workforce development?
•
Which projects have the most readiness for further development and construction?
Third Ranking: National Criteria Assessed across All Fields
•
Which projects are in new and emerging fields that have the most potential to be
transformative? Which projects have the most potential to change how research is
conducted or to expand fundamental science and engineering frontiers?
•
Which projects have the greatest potential for maintaining US leadership in key science
and engineering fields?
As referenced in Joint National Science Board —National Science Foundation Management Report: Setting Priorities for Large
Facility Projects Supported by the National Science Foundation (NSB-05-77); September 2005
1
Section Revision:
December 14, 2018
A-1
Major Facilities Guide: NSF 21-107 (July 2021)
Appendix A: Ranking Criteria for Prioritizing Major Facility Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
•
Which projects produce the greatest benefits in numbers of researchers, educators and
students enabled?
•
Which projects most need to be undertaken in the near term? Which ones have the
most current windows of opportunity, pressing needs and international or interagency
commitments that should be met?
•
Which projects have the greatest degree of community support?
•
Which projects will have the greatest impact on scientific advances across fields taking
into account the importance of balance among fields for NSF's portfolio management in
the nation's interest?
Section Revision:
December 14, 2018
A-2
| File Type | application/pdf |
| File Title | Major Facilities Guide: NSF 21-107 (July 2021) |
| Author | Stockhoff, David A |
| File Modified | 2021-08-30 |
| File Created | 2021-08-26 |