Hseep

Homeland Security
Exercise and Evaluation
Program (HSEEP)
April 2013

PRE-DECISIONAL DRAFT

Intro-1

Contents
INTRODUCTION AND OVERVIEW ....................................................................................... INTRO-1
Purpose .......................................................................................................................................... Intro-1
Role of Exercises ........................................................................................................................... Intro-1
Applicability and Scope ................................................................................................................ Intro-2
Supersession .................................................................................................................................. Intro-2
How to Use This Document .......................................................................................................... Intro-2
Revision Process............................................................................................................................ Intro-3
1. HSEEP FUNDAMENTALS ............................................................................................................. 1-1
Overview ............................................................................................................................................. 1-1
Fundamental Principles ....................................................................................................................... 1-1
Exercise Program Management ........................................................................................................... 1-1
Exercise Methodology ......................................................................................................................... 1-2
Exercise Design and Development ............................................................................................... 1-3
Exercise Conduct .......................................................................................................................... 1-3
Exercise Evaluation ...................................................................................................................... 1-3
Improvement Planning .................................................................................................................. 1-3
2. EXERCISE PROGRAM MANAGEMENT ................................................................................... 2-1
Overview ............................................................................................................................................. 2-1
Engage Elected and Appointed Officials............................................................................................. 2-1
Multi-year Exercise Program Priorities ............................................................................................... 2-1
Training and Exercise Planning Workshop .................................................................................. 2-2
Multi-year Training and Exercise Plan................................................................................................ 2-3
Progressive Approach ................................................................................................................... 2-3
Discussion-Based Exercises.......................................................................................................... 2-4
Operations-Based Exercises.......................................................................................................... 2-5
Rolling Summary of Outcomes ........................................................................................................... 2-6
Manage Exercise Program Resources ................................................................................................. 2-7
Exercise Budget Management ...................................................................................................... 2-7
Program Staffing ........................................................................................................................... 2-7
Other Resources ............................................................................................................................ 2-7
3. EXERCISE DESIGN AND DEVELOPMENT .............................................................................. 3-1
Overview ............................................................................................................................................. 3-1
Exercise Foundation ............................................................................................................................ 3-1
Exercise Planning Team and Events ................................................................................................... 3-2
Exercise Planning Team Considerations....................................................................................... 3-2
Exercise Planning Team Positions ................................................................................................ 3-3
Planning Activities ........................................................................................................................ 3-4
Exercise Design ................................................................................................................................... 3-9
Scope........................................................................................................................................... 3-10
Exercise Objectives..................................................................................................................... 3-11
Evaluation Requirements ............................................................................................................ 3-12
Scenario ...................................................................................................................................... 3-12
Exercise Documentation ............................................................................................................. 3-13
Media or Public Affairs Guidance .............................................................................................. 3-19
Exercise Development ....................................................................................................................... 3-20
Planning for Exercise Logistics .................................................................................................. 3-20
Planning for Exercise Control ..................................................................................................... 3-22

Homeland Security Exercise and Evaluation Program

i

Planning for Exercise Evaluation................................................................................................ 3-25
4. EXERCISE CONDUCT ................................................................................................................... 4-1
Overview ............................................................................................................................................. 4-1
Exercise Play Preparation .................................................................................................................... 4-1
Setup for Discussion-Based Exercises .......................................................................................... 4-1
Setup for Operations-Based Exercises .......................................................................................... 4-1
Briefings........................................................................................................................................ 4-2
Exercise Play ....................................................................................................................................... 4-2
Participant Roles and Responsibilities .......................................................................................... 4-3
Conduct for Discussion-Based Exercises ..................................................................................... 4-4
Conduct for Operations-Based Exercises ..................................................................................... 4-5
Contingency Process ..................................................................................................................... 4-7
Wrap-Up Activities ............................................................................................................................. 4-7
Debriefings.................................................................................................................................... 4-7
Player Hot Wash ........................................................................................................................... 4-7
Controller/Evaluator Debriefing ................................................................................................... 4-8
5. EVALUATION .................................................................................................................................. 5-1
Overview ............................................................................................................................................. 5-1
Evaluation Planning ............................................................................................................................ 5-1
Evaluation Team ........................................................................................................................... 5-1
Exercise Evaluation Guide Development ..................................................................................... 5-2
Recruit, Assign, and Train Evaluators .......................................................................................... 5-3
Evaluation Documentation............................................................................................................ 5-3
Pre-Exercise Evaluator Briefing ................................................................................................... 5-4
Exercise Observation and Data Collection .......................................................................................... 5-4
Observation ................................................................................................................................... 5-4
Data Collection ............................................................................................................................. 5-4
Data Analysis ...................................................................................................................................... 5-5
After-Action Report Draft ................................................................................................................... 5-5
6. IMPROVEMENT PLANNING ....................................................................................................... 6-1
Overview ............................................................................................................................................. 6-1
Corrective Actions ............................................................................................................................... 6-1
After-Action Meeting .......................................................................................................................... 6-1
After-Action Report/Improvement Plan Finalization .......................................................................... 6-2
Corrective Action Tracking and Implementation ................................................................................ 6-2
Using Improvement Planning to Support Continuous Improvement .................................................. 6-2
GLOSSARY OF TERMS .................................................................................................... GLOSSARY-1
ACRONYMS AND ABBREVIATIONS .............................................................................ACRONYM-1

Homeland Security Exercise and Evaluation Program

ii

Introduction and Overview
Purpose
The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of guiding
principles for exercise programs, as well as a common approach to exercise program
management, design and development, conduct, evaluation, and improvement planning.
Exercises are a key component of national preparedness—they provide elected and appointed
officials and stakeholders from across the whole community with the opportunity to shape
planning, assess and validate capabilities, and address areas for improvement.
Through the use of HSEEP, exercise program managers can develop, execute, and evaluate
exercises that address the priorities established by an organization’s leaders. These priorities are
based on the National Preparedness Goal, strategy documents, threat and hazard
identification/risk assessment processes, capability assessments, and the results from previous
exercises and real-world events. These priorities guide the overall direction of a progressive
exercise program, where individual exercises are anchored to a common set of priorities or
objectives and build toward an increasing level of complexity over time. Accordingly, these
priorities guide the design and development of individual exercises, as planners identify exercise
objectives and align them to core capabilities 1 for evaluation during the exercise. Exercise
evaluation assesses the ability to meet exercise objectives and capabilities by documenting
strengths, areas for improvement, core capability performance, and corrective actions in an
After-Action Report/Improvement Plan (AAR/IP). Through improvement planning,
organizations take the corrective actions needed to improve plans, build and sustain capabilities,
and maintain readiness.
In this way, the use of HSEEP—in line with the National Preparedness Goal and the National
Preparedness System—supports efforts across the whole community that improve our national
capacity to build, sustain, and deliver core capabilities.

Role of Exercises
Exercises play a vital role in national preparedness by enabling whole community stakeholders
to test and validate plans and capabilities, and identify both capability gaps and areas for
improvement. A well-designed exercise provides a low-risk environment to test capabilities,
familiarize personnel with roles and responsibilities, and foster meaningful interaction and
communication across organizations. Exercises bring together and strengthen the whole
community in its efforts to prevent, protect against, mitigate, respond to, and recover from all
hazards. Overall, exercises are cost-effective and useful tools that help the nation practice and
refine our collective capacity to achieve the core capabilities in the National Preparedness Goal.

1

Core Capabilities are distinct critical elements necessary to achieve the specific mission areas of prevention, protection, mitigation, response,
and recovery. Capabilities provide a common vocabulary describing the significant functions required to deal with threats and hazards that must
be developed and executed across the whole community to ensure national preparedness.

Homeland Security Exercise and Evaluation Program

Intro-1

Applicability and Scope
HSEEP exercise and evaluation doctrine is flexible, scalable, adaptable, and is for use by
stakeholders across the whole community. 2 HSEEP doctrine is applicable for exercises across
all mission areas—prevention, protection, mitigation, response, and recovery. Using HSEEP
supports the National Preparedness System 3 by providing a consistent approach to exercises and
measuring progress toward building, sustaining, and delivering core capabilities.
HSEEP doctrine is based on national best practices and is supported by training, technology
systems, tools, and technical assistance. The National Exercise Program (NEP) is consistent
with the HSEEP methodology. Exercise practitioners are encouraged to apply and adapt HSEEP
doctrine to meet their specific needs.

Supersession
This 2013 iteration of HSEEP supersedes the 2007 HSEEP Volumes. The current version
reflects the feedback, lessons learned, and best practices of the exercise community, as well as
current policies and plans.

How to Use This Document
This document serves as a description of HSEEP doctrine. It includes an overview of HSEEP
fundamentals that describes core HSEEP principles and overall methodology. This overview is
followed by several chapters that provide exercise practitioners with more detailed guidance on
putting the program’s principles and methodology into practice.
The doctrine is organized as follows:
•

Chapter 1: HSEEP Fundamentals describes the basic principles and methodology of
HSEEP.

•

Chapter 2: Exercise Program Management provides guidance for conducting a Training
and Exercise Planning Workshop (TEPW) and developing a Multi-year Training and
Exercise Plan (TEP).

•

Chapter 3: Exercise Design and Development describes the methodology for
developing exercise objectives, conducting planning meetings, developing exercise
documentation, and planning for exercise logistics, control, and evaluation.

•

Chapter 4: Exercise Conduct provides guidance on setup, exercise play, and wrap-up
activities.

•

Chapter 5: Evaluation provides the approach to exercise evaluation planning and
conduct through data collection, analysis, and development of an AAR.

•

Chapter 6: Improvement Planning addresses corrective actions identified in the exercise
IP and the process of tracking corrective actions to resolution.

2

The whole community includes individuals, families, communities, the private and nonprofit sectors, faith-based organizations, and Federal,
State, local, tribal, and territorial governments.
3
The National Preparedness System includes identifying and assessing risks; estimating the level of capabilities needed to address those risks;
building or sustaining the required levels of capability; developing and implementing plans to deliver those capabilities; validating and
monitoring progress; and reviewing and updating efforts to promote continuous improvement.

Homeland Security Exercise and Evaluation Program

Intro-2

Revision Process
The Department of Homeland Security (DHS) Federal Emergency Management Agency
(FEMA) will review HSEEP doctrine and methodology on a biennial basis, or as otherwise
needed, to make necessary modifications and incorporate lessons learned.

Homeland Security Exercise and Evaluation Program

Intro-3

1. HSEEP Fundamentals
Overview
HSEEP doctrine consists of fundamental principles that frame a common approach to exercises.
This doctrine is supported by training, technology systems, tools, and technical assistance, and is
based on national best practices. It is intended to enhance consistency in exercise conduct and
evaluation while ensuring exercises remain a flexible, accessible way to improve our
preparedness across the nation.

Fundamental Principles
Applying the following principles to both the management of an exercise program and the
execution of individual exercises is critical to the effective examination of capabilities:
•

Guided by Elected and Appointed Officials. The early and frequent engagement of
elected and appointed officials is the key to the success of any exercise program. They
provide the overarching guidance and direction for the exercise and evaluation program
as well as specific intent for individual exercises.

•

Capability-based, Objective Driven. The National Preparedness Goal identifies a series
of core capabilities and associated capability targets across the prevention, protection,
mitigation, response, and recovery mission areas. Through HSEEP, organizations can
use exercises to examine current and required core capability levels and identify gaps.
Exercises focus on assessing performance against capability-based objectives.

•

Progressive Planning Approach. A progressive approach includes the use of various
exercises aligned to a common set of exercise program priorities and objectives with an
increasing level of complexity over time. Progressive exercise planning does not imply a
linear progression of exercise types.

•

Whole Community Integration. The use of HSEEP encourages exercise planners,
where appropriate, to engage the whole community throughout exercise program
management, design and development, conduct, evaluation, and improvement planning.

•

Informed by Risk. Identifying and assessing risks and associated impacts helps
organizations identify priorities, objectives, and core capabilities to be evaluated through
exercises.

•

Common Methodology. HSEEP includes a common methodology for exercises that is
applicable to all mission areas—prevention, protection, mitigation, response, and
recovery. This methodology enables organizations of divergent sizes, geographies, and
capabilities to have a shared understanding of exercise program management, design and
development, conduct, evaluation, and improvement planning; and fosters exerciserelated interoperability and collaboration.

Exercise Program Management
Exercise program management involves a collaborative approach that integrates resources,
organizations, and individuals in order to identify and achieve program priorities. Through the

Homeland Security Exercise and Evaluation Program

1-1

management of an exercise program, stakeholders provide oversight to specific training and
exercise activities sustained over time. An effective exercise program maximizes efficiency,
resources, time, and funding by ensuring that exercises are part of a coordinated and integrated
approach to building, sustaining, and delivering core capabilities.
Key elements of HSEEP’s approach to exercise program management include:
•

•

•

•
•

•

Engaging Elected and Appointed Officials to Provide Intent and Direction. Elected
and appointed officials must be engaged early and often in an exercise program. They
provide both the strategic direction for the program as well as specific guidance for
individual exercises. Routine engagement with elected and appointed officials ensures
that exercises have the support necessary for success.
Establishing Multi-year Exercise Program Priorities. These overarching priorities
inform the development of exercise objectives, ensuring that individual exercises
evaluate and assess core capabilities in a coordinated and integrated fashion.
Using a Progressive Approach. A progressive exercise program management approach
includes exercises anchored to a common set of objectives, built toward an increasing
level of complexity over time, and involves the participation of multiple entities.
Developing a Multi-year Training and Exercise Plan. A TEP, developed through a
TEPW, aligns exercise activities and supporting training to exercise program priorities.
Maintaining a Rolling Summary of Exercise Outcomes. A rolling summary report
provides elected and appointed officials and other stakeholders with an analysis of issues,
trends, and key outcomes from all exercises conducted as part of the exercise program.
Managing Exercise Program Resources. An effective exercise program utilizes the full
range of available resources for exercise budgets, program staffing, and other resources.

Exercise Methodology
HSEEP uses a common methodology for planning and conducting individual exercises. This
methodology applies to exercises in support of all national preparedness mission areas. A
common methodology ensures a consistent and interoperable approach to exercise design and
development, conduct, evaluation, and improvement planning, as depicted in Figure 1.1. The
following chapters contain more detailed descriptions of each phase.

Figure 1.1: HSEEP Exercise Cycle

Homeland Security Exercise and Evaluation Program

1-2

Exercise Design and Development
In designing and developing individual exercises, exercise planning team members are identified
to schedule planning meetings, identify and develop exercise objectives, design the scenario,
create documentation, plan exercise conduct and evaluation, and coordinate logistics. At key
points in this process, the exercise planning team engages elected and appointed officials to
ensure their intent is captured and that the officials are prepared to support the exercise as
necessary.
Exercise Conduct
After design and development activities are complete, the exercise is ready to occur. Activities
essential to conducting individual exercises include preparing for exercise play, managing
exercise play, and conducting immediate exercise wrap-up activities.
Exercise Evaluation
Evaluation is the cornerstone of an exercise and must be considered throughout all phases of the
exercise planning cycle, beginning when the exercise planning team meets to establish objectives
and initiate exercise design. Effective evaluation assesses performance against exercise
objectives, and identifies and documents strengths and areas for improvement relative to core
capabilities.
Improvement Planning
During improvement planning, the corrective actions identified during individual exercises are
tracked to completion, ensuring that exercises yield tangible preparedness improvements. An
effective corrective action program develops IPs that are dynamic documents, which are
continually monitored and implemented as part of the larger system of improving preparedness.

Homeland Security Exercise and Evaluation Program

1-3

2. Exercise Program Management
Overview
Exercise program management is the process of overseeing and integrating a variety of exercises
over time. An effective exercise program helps organizations maximize efficiency, resources,
time, and funding by ensuring that exercises are part of a coordinated, integrated approach to
building, sustaining, and delivering core capabilities. This approach—called multi-year
planning—begins when elected and appointed officials, working with whole community
stakeholders, identify and develop a set of multi-year exercise priorities informed by existing
assessments, strategies, and plans. These long-term priorities help exercise planners design and
develop a progressive program of individual exercises to build, sustain, and deliver core
capabilities.
Effective exercise program management promotes a multi-year approach to:
•

Engaging elected and appointed officials

•

Establishing multi-year exercise program priorities

•

Developing a multi-year TEP

•

Maintaining a rolling summary of exercise outcomes

•

Managing exercise program resources

Through effective exercise program management, each exercise becomes a supporting
component of a larger exercise program with overarching priorities. Exercise practitioners are
encouraged to apply and adapt HSEEP doctrine on exercise program management to meet their
specific needs.

Engage Elected and Appointed Officials
Engaging elected and appointed officials in the exercise process is critical because they provide
both the strategic direction for the exercise program, as well as specific guidance for individual
exercises. As representatives of the public, elected and appointed officials ensure that exercise
program priorities are supported at the highest level and align to whole community needs and
priorities. Elected and appointed officials should be engaged early and often in an exercise
program, starting with the development of exercise program priorities at the TEPW. In
developing individual exercises, the exercise planning team should continue to engage their
appropriate elected and appointed officials throughout the exercise planning cycle in order to
ensure the leaders’ vision for the exercise is achieved.

Multi-year Exercise Program Priorities
An exercise program should be based on a set of strategic, high-level priorities selected by an
organization’s elected and appointed officials. These priorities guide the development of
exercise objectives, ensuring that individual exercises build and sustain preparedness in a
progressive and coordinated fashion. Exercise program priorities are developed at the TEPW, as
described in the following sections.

Homeland Security Exercise and Evaluation Program

2-1

Training and Exercise Planning Workshop
Purpose

The TEPW establishes the strategy and structure for an exercise program. In addition, it sets the
foundation for the planning, conduct, and evaluation of individual exercises. The purpose of the
TEPW is to use the guidance provided by elected and appointed officials to identify and set
exercise program priorities and develop a multi-year schedule of exercise events and supporting
training activities to meet those priorities. This process ensures whole community exercise
initiatives are coordinated, prevents duplication of effort, promotes the efficient use of resources,
avoids overextending key agencies and personnel, and maximizes the efficacy of training and
exercise appropriations. TEPWs are held on a periodic basis (e.g., annual or biennial) depending
on the needs of the program and any grant or cooperative agreement requirements.
Participation

When identifying stakeholders, exercise program managers should consider individuals from
organizations throughout the whole community, including but not limited to:
•

Elected and appointed officials responsible for providing direction and guidance for
exercise program priorities and those responsible for providing resources to support
exercises;

•

Representatives from relevant disciplines that would be part of the exercises or any realworld events, including appropriate regional or local Federal department/agency
representatives;

•

Individuals with administrative responsibility relevant to exercise conduct; and

•

Representatives from volunteer, nongovernmental, nonprofit, or social support
organizations, including advocates for children, seniors, individuals with disabilities,
those with access and functional needs, racially and ethnically diverse communities,
people with limited English proficiency, and animals.

Once a comprehensive set of stakeholders has been identified, exercise program managers can
include them in the exercise program by having them regularly participate in TEPWs.
Conduct of the TEPW

When developing exercise program priorities and the multi-year schedule at the TEPW,
stakeholders should engage organizational elected and appointed officials early in the process to
obtain their intent and guidance. TEPW participants also review and consider various factors
such as:
•

Jurisdiction-specific threats and hazards (e.g., Threat and Hazard Identification and Risk
Assessment [THIRA], local risk assessments);

•

Areas for improvement identified from real-world events and exercises;

•

External requirements such as State or national preparedness reports, homeland security
policy (e.g., the National Preparedness Goal), and industry reports; and

•

Accreditation standards (e.g., hospital accreditation requirements), regulations, or
legislative requirements.

Homeland Security Exercise and Evaluation Program

2-2

Figure 2.1 illustrates some of the specific factors for consideration in developing exercise
program priorities.

Figure 2.1: Factors for Consideration in Developing Exercise Program Priorities

Drawing on the above factors and core capabilities, the workshop facilitator leads a group
stakeholder discussion to review exercise program priorities and outline training and exercise
priorities shared across multiple organizations. The group should also develop a multi-year
schedule of training and exercise activities designed to meet those priorities.
At the conclusion of the TEPW, program managers will have a clear understanding of specific
multi-year training and exercise program priorities, and any available information on previously
planned training and exercises that align to those priorities. This combined set of information is
used to develop a multi-year TEP.

Multi-year Training and Exercise Plan
Once the training and exercise program priorities have been outlined, stakeholders develop the
multi-year TEP. The TEP identifies a combination of exercises—along with associated training
requirements—that address the priorities identified in the TEPW.
Progressive Approach
A progressive, multi-year exercise program enables organizations to participate in a series of
increasingly complex exercises, with each successive exercise building upon the previous one
until mastery is achieved. Regardless of exercise type, each exercise within the progressive
series is linked to a set of common program priorities and designed to test associated capabilities.

Homeland Security Exercise and Evaluation Program

2-3

Further, by defining training requirements in the planning process, organizations can address
known shortfalls prior to exercising capabilities.
This progressive approach, with exercises that build upon
each other and are supported at each step with training
resources, will ensure that organizations do not rush into a
full-scale exercise too quickly. Effective planning of
exercises and integration of the necessary training will
reduce the waste of limited exercise resources and serve to
address known shortfalls prior to the conduct of the
exercise. The different types of exercises that may be
included in the multi-year plan are described in the
following sections.

A progressive exercise
program is a series of
exercises tied to a set of
common program priorities.
Each exercise builds on
previous exercises using more
sophisticated simulation
techniques or requiring more
preparation time, personnel,
and planning.

Discussion-Based Exercises
Discussion-based exercises include seminars, workshops, tabletop exercises (TTXs), and games.
These types of exercises can be used to familiarize players with, or develop new, plans, policies,
agreements, and procedures. Discussion-based exercises focus on strategic, policy-oriented
issues. Facilitators and/or presenters usually lead the discussion, keeping participants on track
towards meeting exercise objectives.
Seminars

Seminars generally orient participants to, or provide an overview of, authorities, strategies, plans,
policies, procedures, protocols, resources, concepts, and ideas. As a discussion-based exercise,
seminars can be valuable for entities that are developing or making major changes to existing
plans or procedures. Seminars can be similarly helpful when attempting to assess or gain
awareness of the capabilities of interagency or inter-jurisdictional operations.
Workshops

Although similar to seminars, workshops differ in two important aspects: participant interaction
is increased, and the focus is placed on achieving or building a product. Effective workshops
entail the broadest attendance by relevant stakeholders.
Products produced from a workshop can include new standard operating procedures (SOPs),
emergency operations plans, continuity of operations plans, or mutual aid agreements. To be
effective, workshops should have clearly defined objectives, products, or goals, and should focus
on a specific issue.
Tabletop Exercises

A TTX is intended to generate discussion of various issues regarding a hypothetical, simulated
emergency. TTXs can be used to enhance general awareness, validate plans and procedures,
rehearse concepts, and/or assess the types of systems needed to guide the prevention of,
protection from, mitigation of, response to, and recovery from a defined incident. Generally,
TTXs are aimed at facilitating conceptual understanding, identifying strengths and areas for
improvement, and/or achieving changes in perceptions.
During a TTX, players are encouraged to discuss issues in depth, collaboratively examining
areas of concern and solving problems. The effectiveness of a TTX is derived from the energetic

Homeland Security Exercise and Evaluation Program

2-4

involvement of participants and their assessment of recommended revisions to current policies,
procedures, and plans.
TTXs can range from basic to complex. In a basic TTX (such as a Facilitated Discussion), the
scenario is presented and remains constant—it describes an emergency and brings discussion
participants up to the simulated present time. Players apply their knowledge and skills to a list of
problems presented by the facilitator; problems are discussed as a group; and resolution is
reached and documented for later analysis.
In a more advanced TTX, play advances as players receive pre-scripted messages that alter the
original scenario. A facilitator usually introduces problems one at a time in the form of a written
message, simulated telephone call, videotape, or other means. Players discuss the issues raised
by each problem, referencing established authorities, plans, and procedures for guidance. Player
decisions are incorporated as the scenario continues to unfold.
During a TTX, all participants should be encouraged to contribute to the discussion and be
reminded that they are making decisions in a no-fault environment. Effective TTX facilitation is
critical to keeping participants focused on exercise objectives and associated capability targets.
Games

A game is a simulation of operations that often involves two or more teams, usually in a
competitive environment, using rules, data, and procedures designed to depict an actual or
hypothetical situation. Games explore the consequences of player decisions and actions. They
are useful tools for validating plans and procedures or evaluating resource requirements.
During game play, decision-making may be either slow and deliberate or rapid and more
stressful, depending on the exercise design and objectives. The open, decision-based format of a
game can incorporate “what if” questions that expand exercise benefits. Depending on the
game’s design, the consequences of player actions can be either pre-scripted or decided
dynamically. Identifying critical decision-making points is a major factor in the success of
evaluating a game.
Operations-Based Exercises
Operations-based exercises include drills, functional exercises (FEs), and full-scale exercises
(FSEs). These exercises can be used to validate plans, policies, agreements, and procedures;
clarify roles and responsibilities; and identify resource gaps. Operations-based exercises are
characterized by actual reaction to an exercise scenario, such as initiating communications or
mobilizing personnel and resources.
Drills

A drill is a coordinated, supervised activity usually employed to validate a specific function or
capability in a single agency or organization. Drills are commonly used to provide training on
new equipment, validate procedures, or practice and maintain current skills. For example, drills
may be appropriate for establishing a community-designated disaster receiving center or shelter.
Drills can also be used to determine if plans can be executed as designed, to assess whether more
training is required, or to reinforce best practices. A drill is useful as a stand-alone tool, but a
series of drills can be used to prepare several organizations to collaborate in an FSE.
For every drill, clearly defined plans, procedures, and protocols need to be in place. Personnel
need to be familiar with those plans and trained in the processes and procedures to be drilled.

Homeland Security Exercise and Evaluation Program

2-5

Functional Exercises

FEs are designed to validate and evaluate capabilities, multiple functions and/or sub-functions, or
interdependent groups of functions. FEs are typically focused on exercising plans, policies,
procedures, and staff members involved in management, direction, command, and control
functions. In FEs, events are projected through an exercise scenario with event updates that
drive activity typically at the management level. An FE is conducted in a realistic, real-time
environment; however, movement of personnel and equipment is usually simulated.
FE controllers typically use a Master Scenario Events List (MSEL) to ensure participant activity
remains within predefined boundaries and ensure exercise objectives are accomplished.
Simulators in a Simulation Cell (SimCell) can inject scenario elements to simulate real events.
Full-Scale Exercises

FSEs are typically the most complex and resource-intensive type of exercise. They involve
multiple agencies, organizations, and jurisdictions and validate many facets of preparedness.
FSEs often include many players operating under cooperative systems such as the Incident
Command System (ICS) or Unified Command.
In an FSE, events are projected through an exercise scenario with event updates that drive
activity at the operational level. FSEs are usually conducted in a real-time, stressful environment
that is intended to mirror a real incident. Personnel and resources may be mobilized and
deployed to the scene, where actions are performed as if a real incident had occurred. The FSE
simulates reality by presenting complex and realistic problems that require critical thinking,
rapid problem solving, and effective responses by trained personnel.
The level of support needed to conduct an FSE is greater than that needed for other types of
exercises. The exercise site for an FSE is usually large, and site logistics require close
monitoring. Safety issues, particularly regarding the use of props and special effects, must be
monitored. Throughout the duration of the exercise, many activities occur simultaneously.

Rolling Summary of Outcomes
To help ensure that exercise program priorities are adequately
addressed, exercise program managers should periodically develop
and distribute a rolling summary of exercise outcomes, or rolling
summary report. A rolling summary report provides stakeholders
with an analysis of issues, trends, and key outcomes from all
exercises conducted as part of the exercise program. This report is
designed to:

The rolling summary
report is an analysis of
exercise trends, which
guides the development
of future exercises.

•

Inform elected and appointed officials on the progress of the exercise program;

•

Provide data to support preparedness assessments and reporting requirements; and

•

Enable exercise planners to modify objectives and the exercise schedule to reflect
knowledge gathered from the exercises.

The rolling summary report is not a collection of AARs, but rather an analysis of trends across
exercises. It is developed periodically throughout the series of exercises covered in a multi-year
TEP (e.g., quarterly or biennially, depending how many exercises are conducted). This report is

Homeland Security Exercise and Evaluation Program

2-6

intended to serve as an exercise program management and communications tool, which informs
stakeholders and guides the development of future exercises.

Manage Exercise Program Resources
An effective exercise program should utilize the full range of available resources. Program
managers should ensure that they have planned for an exercise budget, program staffing, and
other resources.
Exercise Budget Management
Effective budget management is essential to the success of an exercise program, and it is
important for exercise managers to maintain awareness of their available resources and expected
expenditures. In developing and maintaining an exercise program budget, program managers
should work with the full range of stakeholders to identify financial resources and define
monitoring and reporting requirements as required by individual exercises.
Program Staffing
Program managers should identify the administrative and operational staff needed to oversee the
exercise program. The TEP can be one basis for determining exercise program staffing needs in
addition to grant funds or other programmatic considerations. Program managers should also
identify gaps between staffing availability and staffing needs. Exercise program managers can
consider alternative means of procuring staff members, such as adding volunteers, students from
universities (e.g., student nurses or emergency management students), or interns.
Other Resources
Exercise program managers should also consider other resources that can support exercises.
Such resources can include:
•

Information technology (e.g., modeling and simulation capabilities)

•

Exercise tools and resources (e.g., document templates)

•

Materials from previous exercises

•

Training courses

•

Mutual aid agreements, memoranda of understanding, and memoranda of agreement

•

Technical assistance

•

Equipment or props (e.g., smoke machines)

Homeland Security Exercise and Evaluation Program

2-7

3. Exercise Design and Development
Overview
In the design and development phase, exercise practitioners use the intent and guidance of their
elected and appointed officials and the exercise program priorities developed in Program
Management to plan individual exercises. Exercise planning teams apply this guidance to shape
the key concepts and planning considerations for an individual exercise or series of exercises.
The eight key steps of exercise design and development include:
•

Setting the exercise foundation by reviewing elected and appointed officials’ guidance,
the TEP, and other factors;

•

Selecting participants for an exercise planning team and developing an exercise planning
timeline with milestones;

•

Developing exercise-specific objectives and identifying core capabilities based on the
guidance of elected and appointed officials;

•

Identifying evaluation requirements;

•

Developing the exercise scenario;

•

Creating documentation;

•

Coordinating logistics; and

•

Planning for exercise control and evaluation.

Exercise practitioners are encouraged to apply and adapt HSEEP doctrine on exercise design and
development to meet their specific needs.

Exercise Foundation
The exercise foundation is a set of key factors that drive the exercise design and development
process. Prior to the beginning of its design, exercise program managers should review and
consider the following items:
•

Elected and appointed officials’ intent and guidance

•

Multi-year TEP

•

Relevant AAR/IPs from real-world events and exercises

•

THIRA or other risk, threat, and hazard assessments

•

Organizational plans and procedures

•

Grant or cooperative agreement requirements.

By reviewing these elements, exercise program managers adhere to the progressive approach to
exercises, and ensure the exercise builds and sustains a jurisdiction’s capabilities while taking
prior lessons learned into account during the exercise design process.

Homeland Security Exercise and Evaluation Program

3-1

Exercise Planning Team and Events
Exercise Planning Team Considerations
The exercise planning team manages, and is
Whole community stakeholders include:
ultimately responsible for, exercise design,
 All levels of government
development, conduct, and evaluation. Using the
 Volunteer organizations
exercise program priorities and guidance from
 Community groups
elected and appointed officials, the team determines
 Private entities
exercise objectives and core capabilities to be
 Nonprofit organizations
assessed; creates a realistic scenario to assess them;
 Faith-based groups
and develops supporting documentation, processes,
 Groups working with individuals
and systems that are used in evaluation, control, and
with disabilities or access and
simulation. Planning team members also help with
functional needs
developing and distributing pre-exercise materials,
and conducting exercise planning meetings, briefings, and training sessions. An Exercise
Director with authority to make decisions for the sponsoring organization provides direction to,
and oversight of, the exercise planning team.
The exercise planning team should be of manageable size yet represent the full range of
participating organizations as well as other relevant stakeholders. For multi-jurisdictional
exercises, planning team members should include representatives from each jurisdiction and
participating functional areas or relevant disciplines. The membership of an exercise planning
team should be modified to fit the type or scope of an exercise, which varies depending on
exercise type and complexity. Usually the exercise planning team is managed by a designated
team leader. To design and develop exercises most effectively, exercise planning teams should:
•

Adhere to a clear organizational structure, with a distinct chain of command, roles and
responsibilities, and accountability to the exercise planning team leader;

•

Use proven management practices, processes, and tools, such as project plans and
timelines, status reports, and other communications;

•

Identify and understand the desired objectives and associated core capabilities for the
exercise, and design and develop the exercise accordingly;

•

Incorporate evaluation planning from the start of exercise design and development; and

•

Use subject-matter experts (SMEs) to develop a realistic and challenging scenario.

Support agencies/organizations including advocates for children, seniors, individuals with
disabilities, those with access and functional needs, diverse communities, and people with
limited English proficiency should also be included throughout the planning process. In doing
so, exercise planners can better understand their perspectives and promote early understanding of
roles, responsibilities, and planning assumptions.
Generally, planning team members are not exercise players. When resources are limited,
exercise planning team members who act as both planners and players should be especially
careful not to divulge sensitive exercise information to other players.

Homeland Security Exercise and Evaluation Program

3-2

Exercise Planning Team Positions
Regardless of the scale and complexity of an exercise, the exercise planning team can be most
effective if it adheres to a coherent organizational structure that clearly delineates roles and
responsibilities. In developing a structure for the planning team, exercise planners may use ICS
principles, as established in the National Incident Management System (NIMS). This structure
can expand or contract to reflect the scope of the exercise and the available resources and
personnel of the participating organizations; depending on available resources, the same
personnel can be used to execute multiple functions. This structure may include the following,
which is illustrated in Figure 3.1:

Figure 3.1: Sample Exercise Planning Team

•

Command Section. The Command Section coordinates all exercise planning activities.
The Command Section includes the exercise planning team leader, who assigns exercise
activities and responsibilities, provides guidance, establishes timelines, and monitors the
development process.

•

Operations Section. The Operations Section provides most of the technical or
functional expertise for scenario development and evaluation. This includes development
of the Master Scenario Events List (MSEL).

•

Planning Section. The Planning Section is responsible for compiling and developing all
exercise documentation. The Planning Section collects and reviews policies, plans, and
procedures that will be assessed in the exercise. This group is also responsible for
planning exercise evaluation. During the exercise, the Planning Section may be
responsible for developing simulated actions by agencies not participating in the exercise
and for setting up a SimCell as required.

•

Logistics Section. The Logistics Section provides the supplies, materials, facilities, and
services that enable the exercise to function smoothly without outside interference or
disruption. This section consists of two subsections: service and support. The service
subsection provides transportation, barricading, signage, food and drinks, real-life
medical capability, and exercise security. The support subsection provides

Homeland Security Exercise and Evaluation Program

3-3

communications, purchasing, general supplies, management of very important persons
(VIPs) and observer processing, and recruitment and management of actors.
•

Administration/Finance Section. The Administration/Finance Section provides
financial management and administrative support throughout exercise development,
including exercise registration support and scheduling.

Planning Activities
This section describes the types of planning activities—often in the form of planning
meetings 4—most useful in exercise design and development. The exercise planning team
members decide the type and number of planning activities needed to successfully plan a given
exercise, based on its scope and complexity. When arranging meeting and exercise site
locations, the planning team should take into consideration those individuals who require
assistance or accommodations during attendance.
Concept and Objectives Meeting

Primary Focus
A Concept and Objectives (C&O) Meeting is the formal beginning of the planning process. It is
held to identify the scope and objectives of the exercise. For less complex exercises and for
organizations with limited resources, the C&O Meeting can be conducted in conjunction with the
Initial Planning Meeting (IPM).
Elected and appointed officials, representatives from the sponsoring organization, participating
organizations, and the exercise planning team leader typically attend the C&O Meeting. The
C&O Meeting helps planners determine the exercise program priorities to be addressed based on
elected and appointed officials’ guidance, design objectives based on those priorities, align
exercise objectives to core capabilities, and identify exercise planning team members.
Discussion Points
Topics or issues generally covered during a C&O Meeting include the following:
•

Exercise scope

•

Proposed exercise objectives and their aligned core capabilities

•

Proposed exercise location, date, and duration

•

Participants and anticipated extent of play for exercise participants

•

Exercise planning team

•

Exercise assumptions and artificialities

•

Exercise control and evaluation concepts

•

Exercise security organization and structure

•

Available exercise resources

•

Exercise logistics

4

HSEEP uses the term “meetings” to indicate smaller events focused on a specific topic (exercise planning), rather than “conferences,” which are
generally larger gatherings with broader agendas.

Homeland Security Exercise and Evaluation Program

3-4

•

Exercise planning timeline and milestones

•

Local issues, concerns, and sensitivities

Tools
The primary tools for the C&O Meeting are an agenda and the background and rationale for
conducting the exercise. A briefing is useful for presenting the exercise background and
rationale, as well as exercise methodology for persons unfamiliar with HSEEP.
Outcomes

The following outcomes are expected from the C&O Meeting:
•

Agreement regarding exercise concept (scope, type, mission area[s], exercise program
priorities to be addressed), exercise objectives, and aligned core capabilities;

•

Consensus on the target exercise timeframe;

•

Anticipated extent of participation;

•

Identification of exercise planning team members; and

•

Exercise planning timeline with milestones, including the date of the next planning
meeting.

Initial Planning Meeting

Primary Focus
The IPM marks the beginning of the exercise development phase. Regardless of whether a C&O
Meeting is held, an IPM should be conducted for all exercises. Its purpose is to determine
exercise scope by getting intent and direction from elected and appointed officials, and gathering
input from the exercise planning team; and to identify exercise design requirements and
conditions (e.g., assumptions and artificialities), exercise objectives, participant extent of play,
and scenario variables (e.g., time, location, hazard selection). The IPM is also used to develop
exercise documentation by obtaining the planning team’s input on exercise location, schedule,
duration, and other relevant details.
During the IPM, exercise planning team members are assigned responsibility for activities
associated with designing and developing exercise documents, such as the Exercise Plan
(ExPlan) and the Situation Manual (SitMan), and coordinating exercise logistics.
Discussion Points
Topics or issues generally covered during an IPM include the following:
•

Clearly defined exercise objectives and aligned core capabilities;

•

Evaluation requirements, including EEG capability targets and critical tasks;

•

Relevant plans, policies, and procedures to be tested in the exercise;

•

Exercise scenario;

•

Modeling and simulation planning;

•

Extent of play for each participating organization;

Homeland Security Exercise and Evaluation Program

3-5

•

Optimum duration of the exercise;

•

Exercise planners’ roles and responsibilities;

•

Decision to record exercise proceedings (audio or video);

•

Local issues, concerns, or sensitivities;

•

Any discussion points typically covered during a C&O Meeting if a C&O Meeting was
not conducted; and

•

Consensus regarding the date, time, and location for the next meeting.

Tools
The primary tools for the IPM are the read-ahead packet, agenda, core capabilities, threat and
hazard information (if applicable), a proposed room layout (if applicable), and the exercise
planning timeline with milestones. A briefing is useful for presenting an overview of the
exercise and meeting discussion points.
Outcomes
The IPM results in desired outcomes, such as:
•

Any outcomes listed in the C&O Meeting section above if a C&O Meeting was not
conducted;

•

Clearly defined exercise objectives and aligned core capabilities;

•

Initial capability targets and critical tasks, which will be reviewed and confirmed prior to
the next planning meeting;

•

Identified exercise scenario variables (e.g., threat scenario, scope of hazard, venue,
conditions);

•

A list of participating exercise organizations and anticipated organizational extent of
play;

•

Draft SitMan or ExPlan;

•

Identification and availability of all source documents (e.g., policies, plans, procedures)
needed to draft exercise documents and presentations;

•

A refined exercise planning timeline with milestones;

•

Identification and availability of SMEs, as necessary, for scenario vetting and/or expert
evaluation;

•

Determination of preferred communication methods among the exercise planning team;

•

Clearly identified and assigned responsibility for exercise logistical issues;

•

A list of tasks to be accomplished by the next planning meeting with established dates for
completion and responsible planning team members identified; and

•

An agreed-upon date, time, and location for the next planning meeting and the actual
exercise.

Homeland Security Exercise and Evaluation Program

3-6

Midterm Planning Meeting

Midterm Planning Meetings (MPMs) provide additional opportunities to engage elected and
appointed officials and to settle logistical and organizational issues that may arise during
exercise planning.
Primary Focus
The MPM is a meeting to discuss exercise organization and staffing concepts, scenario and
timeline development, scheduling, logistics, and administrative requirements. It is also held to
review draft documentation. If only three planning meetings are scheduled (i.e., IPM, MPM, and
Final Planning Meeting [FPM]), a portion of the MPM should be devoted to developing the
MSEL, as needed. See the next section, MSEL Meeting, for more information.
Prior to the MPM, the exercise team leader should engage elected and appointed officials to
provide awareness of the planning process, address any questions, and ensure alignment with
guidance and intent.
Discussion Points
Possible topics or issues for an MPM include the following:
•

Comments on draft exercise documentation

•

Construction of the scenario timeline—usually the MSEL—if an additional MSEL
Planning Meeting will not be held

•

Identification of exercise venue artificialities and/or limitations

•

Agreement on final logistical items

•

Assignment of additional responsibilities

Tools
MPM tools include, but are not limited to, an agenda, IPM minutes, draft scenario timeline, draft
documentation (e.g., ExPlan, Controller/Evaluator [C/E] Handbook), and other selected
documentation needed to illustrate exercise concepts and provide planning guidance.
Outcomes
The following outcomes are expected from the MPM:
•

Fully reviewed SitMan or ExPlan;

•

Draft Facilitator Guide or C/E Handbook, including EEGs;

•

A fully reviewed exercise scenario timeline, which is typically the MSEL (if an
additional MSEL Meeting will not be held);

•

Well-developed scenario injects (imperative if an additional MSEL Planning Meeting is
not scheduled);

•

Agreement on the exercise site; and

•

Finalization of date, time, and location of the MSEL Planning Meeting and/or FPM.

Homeland Security Exercise and Evaluation Program

3-7

Master Scenario Events List Meeting

For more complex exercises, one or more additional planning meetings—or MSEL Meetings—
may be held to review the scenario timeline. If not held separately, topics typically covered in a
separate MSEL Meeting can be incorporated into the MPM and FPM.
Primary Focus
The MSEL Meeting focuses on developing the MSEL, which is a chronological list that
supplements the exercise scenario with event synopses, expected participant responses,
objectives and core capability targets to be addressed, and responsible personnel. It includes
specific scenario events (or injects) that prompt players to implement the plans, policies,
procedures, and protocols that require testing during the exercise, as identified in the capabilitiesbased planning process. It also records the methods that will be used to provide injects (e.g.,
phone call, radio call, e-mail).
Discussion Points
In developing a MSEL, the exercise planning team should first
consider the critical tasks, conditions, and standards set forth by
each exercise objective. A condition is the environment in which
a task is performed; it can be provided by the scenario or through
the MSEL.
If scenario conditions do not trigger performance of the
appropriate critical task, the exercise planning team should
develop a MSEL entry to simulate the desired situation. A wellwritten entry considers the following questions:

MSEL entries are tied to
the Exercise Evaluation
Guide critical tasks to
ensure the critical tasks
and core capabilities can
be demonstrated during
the exercise.

•

Is the event key (i.e., is it directly related to meeting an exercise objective)?

•

What is the desired critical task? Who will demonstrate the critical task?

•

What will stimulate the behavior (e.g., course of play, phone call, actor, video)?

•

Who originates the stimulant? Who receives it and how?

•

What action is the player expected to complete?

•

Should a contingency entry be developed for injection into the exercise in case the
players fail to demonstrate the critical task?

Tools
MSEL Meeting tools include, but are not limited to, previous planning meeting minutes, draft
exercise documentation, and an agreed-upon MSEL template.
Outcomes
Following a MSEL Meeting, the level of MSEL completion may vary. At a minimum, key
events and the time of their delivery are identified, and responsibility for constructing the
remaining events is assigned.

Homeland Security Exercise and Evaluation Program

3-8

Final Planning Meeting

The FPM is the final forum for reviewing exercise processes and procedures. Both before and
after the FPM, the exercise team leader should engage elected and appointed officials to ensure
that the exercise is aligning with their intent, address any questions, and receive any last-minute
guidance.
Primary Focus
An FPM should be conducted for all exercises to ensure that all elements of the exercise are
ready for conduct. Prior to the FPM, the exercise planning team receives final drafts of all
exercise materials. No major changes to the exercise’s design, scope, or supporting
documentation should take place at or following the FPM. The FPM ensures that all logistical
requirements have been met, outstanding issues have been identified and resolved, and exercise
products are ready for printing.
Discussion Points
The following items are addressed during the FPM:
•

Conduct a comprehensive, final review and approve all remaining draft exercise
documents (e.g., SitMan, MSEL, C/E Handbook, EEGs) and presentation materials;

•

Resolve any open exercise planning issues and identify last-minute concerns; and

•

Review all exercise logistical activities (e.g., schedule, registration, attire, special needs).

Tools
The primary tools for the FPM include IPM and/or MPM minutes, an agenda, and previously
finalized and/or drafted exercise documents.
Outcomes
The FPM should not generate any significant changes. The following outcomes are expected:
•

Exercise documents and materials for production are approved;

•

Attendees understand and approve exercise processes and procedures;

•

Last-minute issues are identified and resolved; and

•

Logistical elements, including equipment, facilities, and schedule, are confirmed.

Follow-Up
The exercise planning team finalizes all publications, prepares all supporting materials, rehearses
presentations and briefings, and prepares to conduct the exercise. Prior to the exercise,
documentation and any additional instructions should be disseminated to the appropriate
personnel (e.g., presenters, facilitators, controllers, evaluators, simulators).

Exercise Design
The exercise planning meetings serve as the principal mechanism for executing the major steps
of exercise design. The core components of design include establishing the scope of the
exercise, setting exercise objectives, creating an exercise scenario, developing exercise
documentation, and determining media and public relations guidance.

Homeland Security Exercise and Evaluation Program

3-9

Scope
Determining exercise scope enables planners to “right-size” an exercise to meet the objectives
while staying within the resource and personnel constraints of the exercising organizations. Key
elements in defining exercise scope include exercise type, participation level, exercise duration,
exercise location, and exercise parameters. Some of these elements are determined, or initially
discussed, through program management activities or grant requirements. However, the exercise
planning team finalizes the scope based on the exercise objectives. Alterations to the scope are
reviewed with the exercise objectives in mind; planners must consider whether a change in the
scope will improve or impede the ability of players to meet the objectives.
Exercise Type

A first step in defining exercise scope is determining what exercise type to conduct. The
exercise type is selected based on the purpose of the exercise. If the intent is to review and
discuss a new policy, plan, or set of procedures, a discussion-based exercise may be appropriate.
If the intent is to assess the responders’ knowledge of a plan, policy, or set of procedures, an
operations-based exercise may be appropriate.
Participation Level

Active participation by appropriate entities and key leaders is paramount to meeting the exercise
objectives successfully. Participation level refers to the organizations and level of personnel
(e.g., tactical operators, line supervisors, agency directors) participating in the exercise, as well
as the general number of personnel who will participate in the exercise.
At times, scheduling conflicts, real-world events, or other competing requirements will limit an
organization’s or key players’ ability to participate in an exercise. In this case, exercise
designers will need to simulate the decisions and actions of those participants through an
exercise SimCell. An Extent of Play Agreement (XPA) defines the level of participation.
Exercise Duration

When selecting the exercise duration, the planning team should determine how long it will take
to address the exercise objectives effectively. Discussion-based exercises and some drills are
generally shorter, ranging from a couple of hours to a full day. FEs and FSEs may take longer.
Prevention-focused FEs that exercise the intelligence and information sharing core capability
may last up to 30 days with limited duration of play each day. Resource constraints, including
the opportunity cost of having employees away from their primary roles, should be factored into
determining duration.
Exercise Parameters

Exercise parameters clearly outline what should be included in an exercise scenario based on the
objectives and scope, and what should not be exercised. Often there is a desire to add exercise
activities that fall outside of the scope of the exercise in order to meet diverse planning and
training requirements. While these activities may be useful to an organization, they may impact
the ability of players to meet exercise objectives or may reduce the benefit of the exercise by
diluting its focus. Clearly defining the exercise scope early in the design process will help
exercise planners keep the exercise to a manageable and realistic level.

Homeland Security Exercise and Evaluation Program

3-10

Exercise Objectives
Based on direction from elected and appointed officials, the exercise planning team selects one
or more exercise program priorities on which to focus an individual exercise. These priorities
drive the development of exercise objectives, which are distinct outcomes that an organization
wishes to achieve during an exercise. Exercise objectives should incorporate elected and
appointed officials’ intent and guidance, and exercise participants’ plans and procedures,
operating environment, and desired outcomes. Generally, planners should select a reasonable
number of specific, measurable, achievable, relevant, and time-bound (SMART) exercise
objectives to facilitate effective scenario design, exercise conduct, and evaluation. Table 3.1
depicts guidelines for developing SMART objectives.
SMART Guidelines for Exercise Objectives
Specific

Objectives should address the five Ws- who, what, when, where, and why. The
objective specifies what needs to be done with a timeline for completion.

Measurable

Objectives should include numeric or descriptive measures that define quantity,
quality, cost, etc. Their focus should be on observable actions and outcomes.

Achievable

Objectives should be within the control, influence, and resources of exercise play
and participant actions.

Relevant

Objectives should be instrumental to the mission of the organization and link to its
goals or strategic intent.

Time-bound

A specified and reasonable timeframe should be incorporated into all objectives.
Table 3.1: SMART Guidelines for Exercise Objectives

The exercise planning team aligns each exercise objective to one or more core capabilities.
Figure 3.2 shows the relationship between exercise program priorities, exercise objectives, and
core capabilities.

Figure 3.2: Priorities, Objectives, and Core Capabilities

Aligning objectives to a common set of capabilities enables:
•

Systematic tracking of progress over the course of exercise programs and/or cycles;

•

Standardized exercise data collection to inform preparedness assessments; and

•

Fulfillment of grant or funding-specific reporting requirements.

Homeland Security Exercise and Evaluation Program

3-11

Evaluation Requirements
It is important to develop exercise evaluation requirements early in the design process, as they
will guide development of the exercise scenario, discussion questions, and/or MSEL. Evaluation
requirements clearly articulate what will be evaluated during the exercise and how exercise play
will be assessed. This information is documented in the EEGs.
Once the exercise planning team aligns objectives to core capabilities, it identifies which
capability targets and critical tasks for each core capability are being addressed by the exercise.
Capability targets are the performance thresholds for each core capability; they state the exact
amount of capability that players aim to achieve. Generally, these targets are based on targets
identified as part of an organization’s or jurisdiction’s THIRA or other threat and hazard
identification or risk assessment process. Critical tasks are the distinct elements required to
perform a core capability. Critical tasks may be derived from Mission Area Frameworks,
organizational operations plans or SOPs, or discipline-specific standards.
Scenario
A scenario is an outline or model of the simulated sequence of events for the exercise. It can be
written as a narrative or depicted by an event timeline. For discussion-based exercises, a
scenario provides the backdrop that drives participant discussion, and is contained in a SitMan.
For operations-based exercises, a scenario provides background information about the incident
catalyst(s) of the exercise. The overall scenario is provided in the C/E Handbook, and specific
scenario events are contained in the MSEL.
Exercise planners should select and develop scenarios that enable an exercise to assess objectives
and core capabilities. All scenarios should be realistic, plausible, and challenging; however,
designers must ensure the scenario is not so complicated that it overwhelms players.
A scenario consists of three basic elements: (1) the general context or comprehensive story; (2)
the required conditions that will allow players to demonstrate proficiency and competency in
conducting critical tasks, demonstrating core capabilities, and meeting objectives; and (3) the
technical details necessary to accurately depict scenario conditions and events. The exercise
planning team ensures that the design effort is not characterized by a fixation on scenario
development; rather, the scenario facilitates assessment of exercise objectives and core
capabilities. Because of this, exercise planners should refrain from developing the scenario until
after the scope and objectives of the exercise have been clearly defined. Furthermore, scenarios
should avoid any sensitivity that may arise, such as the use of real names of terrorist groups or
sensitive venues.
Threat or Hazard

The first step in designing a scenario is determining the type of threat or hazard on which the
exercise will focus. Each type of emergency has its own strengths and weaknesses when it
comes to evaluating different aspects of prevention, protection, mitigation, response, and
recovery. The exercise planning team should choose a threat or hazard that best assesses the
objectives and core capabilities on which the exercise will focus. The identification of this threat
or hazard scenario should also be based on the organization’s threat/hazard identification and
risk assessment. 5 Developing and maintaining these risk analyses is an essential component of
5

For further guidance on identifying and assessing risks and associated impacts, please refer to the DHS Comprehensive Preparedness Guide
201: Threat and Hazard Identification and Risk Assessment Guide, First Edition, April 2012.

Homeland Security Exercise and Evaluation Program

3-12

the National Preparedness System, as they enable organizations to identify potential events that
would stress their core preparedness capabilities.
Modeling and Simulation

When incorporated into the development of the scenario
Benefits of using modeling and
and overall exercise design, modeling and simulation can
simulation in exercises include:
bring versatility, cost savings, and fidelity to exercises.
 Enhanced realism
A model is a representation of a system at a point in time
 Efficiency
or space intended to expand an understanding of the real
 Ability to exercise situations
system. Simulation is a method of implementing the
that cannot be safely or
performance of a model, or combination of models, over
realistically replicated
time. Modeling and simulation supports decisionmaking processes by providing human and/or computer
feedback to players during exercise play, thus dynamically representing the impact of their
decisions. For example, human-based simulation during exercises is often manifested through
the SimCell, which represents nonparticipating entities. An example of a computer-based
simulation could include wind damage and storm surge forecasting models developed by the
National Oceanic and Atmospheric Administration, which enable simulation of a hurricane’s
effects on coastal communities.
Modeling and simulation can also be applied in situations where reality cannot be achieved. For
example, for safety reasons a bioterrorism exercise cannot be conducted by releasing a deadly
virus into the environment. However, it is still important to exercise the capabilities necessary to
respond to this type of scenario. The use of modeling and simulation can realistically replicate
variables such as disease propagation, radiation, and chemical attacks.
Exercise Documentation
Comprehensive, organized exercise documentation is critical to ensure an accurate account of the
exercise is preserved. This in turn allows organizations to leverage past documentation to
support future exercises and, more importantly, ensures that all critical issues, lessons learned,
and corrective actions are appropriately captured to support improvement efforts.
While most exercise materials are not sensitive or classified, some materials (e.g., scenario
details) may necessitate restrictions on distribution. It is important for the exercise sponsor(s) to
understand the specific requirements for security marking rules and requirements, access and
dissemination, storage, disposal, and incident reporting of sensitive documents.
Consideration should also be given to the accessibility of presentations and documents, such as
making information available in alternative formats (e.g., large print, compact disc, Braille),
closed captioning or another form of text display, or the provision of sign language interpreters.

Homeland Security Exercise and Evaluation Program

3-13

Table 3.2 lists the key exercise design and development documents identified by the exercise
type and relevant audience.
Document Title

Exercise Type

Distribution Audience

Situation Manual (SitMan)

Seminar (Optional), Workshop
(Optional), TTX, Game

All Participants

Facilitator Guide

Seminar (Optional), Workshop
(Optional), TTX, Game

Facilitators

Multimedia Presentation

Seminar (Optional), Workshop
(Optional), TTX, Game

All Participants

Exercise Plan (ExPlan)

Drill, FE, FSE

Players and Observers

Controller and Evaluator (C/E)
Handbook

Drill, FE, FSE

Controllers and Evaluators

Master Scenario Events List (MSEL)

Drill, FE, FSE, Complex TTX
(Optional), Game (Optional)

Controllers, Evaluators,
and Simulators

Extent of Play Agreement (XPA)

FE, FSE

Exercise Planning Team

Exercise Evaluation Guides (EEGs)

TTX, Game, Drill, FE, FSE

Evaluators

Participant Feedback Form

All Exercises

All Participants

Table 3.2: Exercise Design and Development Documents

Situation Manual

SitMans are provided for discussion-based exercises as the core documentation that provides the
textual background for a facilitated exercise. The SitMan supports the scenario narrative and
serves as the primary reference material for all participants during conduct.
The introduction provides an overview of the exercise—including scope, objectives and core
capabilities, structure, rules, and conduct—as well as an exercise agenda. The next section of the
SitMan is the scenario, which may be divided up into distinct, chronologically sequenced
modules. Each module represents a specific time segment of the overall scenario, based on
exercise objectives and scenario requirements.
Each module is followed by discussion questions, usually divided by organization or discipline.
Responses to the modules’ discussion questions are the focus of the exercise, and reviewing
them provides the basis for evaluating exercise results. These discussion questions should be
derived from the exercise objectives and associated core capabilities, capability targets, and
critical tasks documented in each EEG.
The SitMan generally includes the following information:
•

Exercise scope, objectives, and core capabilities

•

Exercise assumptions and artificialities

•

Instructions for exercise participants

•

Exercise structure (i.e., order of the modules)

•

Exercise scenario background (including scenario location information)

•

Discussion questions and key issues

Homeland Security Exercise and Evaluation Program

3-14

•

Schedule of events

SitMan reference appendices may include, but are not limited to:
•

Relevant documents regarding plans, SOPs, etc.

•

Jurisdiction- or organization-specific threat information

•

Material Safety Data Sheet 6 or agent fact sheet, when applicable

•

A list of reference terms

Facilitator Guide

A Facilitator Guide is designed to help facilitators manage a discussion-based exercise. It
usually outlines instructions and key issues for discussion during the event and provides
background information to help the facilitator answer questions from participants or players.
This guide may also include an evaluation section that provides evaluation staff members with
guidance and instructions on evaluation or observation methodology to be used as well as
essential materials required to execute their specific functions.
Multimedia Presentation

Multimedia presentations are often used to illustrate the general scenario for participants. They
are given at the Start of Exercise (StartEx) and support the SitMan. The presentation should
concisely summarize information contained in the written documentation. Like the SitMan, the
multimedia presentation is also divided into distinct, chronologically segmented modules that,
when combined, create the entire scenario.
This presentation typically contains, at a minimum, the following information:
•

Introduction

•

Exercise scope, objectives, and core capabilities

•

Exercise play rules and administrative information

•

Modules that describe the scenario

The presentations are intended to help focus and drive the exercise as well as add realism. A/V
enhancements to a presentation include video or sounds that convey information to participants.
Exercise Plan

ExPlans are general information documents that help operations-based exercises run smoothly by
providing participants with a synopsis of the exercise. They are published and distributed to the
participating organizations following development of most of the critical elements of the
exercise. In addition to addressing exercise objectives and scope, ExPlans assign activities and
responsibilities for exercise planning, conduct, and evaluation. The ExPlan is intended to be
seen by the exercise players and observers—therefore, it does not contain detailed scenario
information that may reduce the realism of the exercise. Players and observers should review all
elements of the ExPlan prior to exercise participation.

6
Material Safety Data Sheet (MSDS) or Product Safety Data Sheet (PSDS) is intended to provide emergency personnel with procedures for
handling or working with a substance in a safe manner and includes information such as toxicity, health effects, first aid, storage, disposal,
protective equipment, and handling procedures.

Homeland Security Exercise and Evaluation Program

3-15

An ExPlan typically contains the following sections:
•

Exercise scope, objectives, and core capabilities

•

Participant roles and responsibilities

•

Rules of conduct

•

Safety issues, notably real emergency codes and phrases, safety controller
responsibilities, prohibited activities, and weapons policies

•

Logistics

•

Security of and access to the exercise site

•

Communications (e.g., radio frequencies or channels)

•

Duration, date, and time of exercise and schedule of events

•

Maps and directions

Player Handout
The Player Handout provides key information to exercise players. A Player Handout can
supplement the SitMan or ExPlan by providing a quick-reference guide to logistics, agenda or
schedule, and key contact data for players.
Controller and Evaluator Handbook

The C/E Handbook describes the roles and responsibilities of exercise controllers and evaluators
and the procedures they should follow. Because the C/E Handbook contains information about
the scenario and about exercise administration, it is distributed to only those individuals
designated as controllers or evaluators. The C/E Handbook may supplement the ExPlan or be a
standalone document. When used as a supplement, it points readers to the ExPlan for more
general exercise information, such as participant lists, activity schedules, required briefings, and
the roles and responsibilities of specific participants. Used as a standalone document, it should
include the basic information contained in the ExPlan, and detailed scenario information.
The C/E Handbook usually contains the following sections:
•

Assignments, roles, and responsibilities of group or individual controllers and evaluators

•

Detailed scenario information

•

Exercise safety plan

•

Controller communications plan (e.g., a phone list, a call-down tree, instructions for the
use of radio channels)

•

Evaluation instructions

The Controller portion of the C/E Handbook, sometimes known as Control Staff Instructions
(COSIN), provides guidelines for control and simulation support and establishes a management
structure for these activities. This section provides guidance for controllers, simulators, and
evaluators on procedures and responsibilities for exercise control, simulation, and support. The
Evaluation portion of the C/E Handbook, sometimes known as the EvalPlan, provides evaluation

Homeland Security Exercise and Evaluation Program

3-16

staff members with guidance and instructions on evaluation or observation methodology to be
used, as well as essential materials required to execute their specific functions.
Controller and Evaluator Packets
While C/E Handbooks contain detailed information that should be read and understood well in
advance of the exercise, Controller Packets and Evaluator Packets are provided immediately
prior to an exercise to controllers and evaluators respectively. The packets contain key
information from the C/E Handbook and additional information specific to the functional area in
which the given controller or evaluator will be working. This information is needed during
exercise play in order to carry out control and evaluation responsibilities.
Both Controller Packets and Evaluator Packets should contain the following:
•

Essential C/E Handbook information

•

Ground truth document, detailing key elements of the exercise scenario (primarily used
for prevention-focused exercises)

•

MSEL, including injects and events for each responsible controller and evaluator

•

Appropriate EEGs

•

Maps and directions

Master Scenario Events List
A MSEL is typically used during operations-based or complex discussion-based exercises and
contains a chronological listing of the events that drive exercise play. Each MSEL entry should
contain the following at a minimum:
•

Designated scenario time

•

Event synopsis

•

Controller responsible for delivering the inject, with controller or evaluator special
instructions (if applicable)

•

Intended player (i.e., agency or individual player for whom the MSEL event is intended)

•

Expected participant response (i.e., player response expected upon inject delivery)

•

Objective, core capability, capability target, and/or critical task to be addressed (if
applicable)

•

Notes section (for controllers and evaluators to track actual events against those listed in
the MSEL, with special instructions for individual controllers and evaluators)

Scenario timelines listed in a MSEL should be as realistic as possible and based on input from
SMEs. If the activity occurs sooner than the MSEL writers anticipated, then controllers and
evaluators should note the time it occurred, but play should not be interrupted.
Controllers delivering MSEL injects will either be co-located with players in the venue of play,
or they will reside in a SimCell. A SimCell is a location from which controllers deliver
messages representing actions, activities, and conversations of an individual, agency, or
organization that is not participating in the exercise but would likely be actively involved during

Homeland Security Exercise and Evaluation Program

3-17

a real incident. Prior to StartEx, the mechanisms for introducing injects into exercise play should
be tested to ensure that controllers are aware of the procedures for delivering MSEL injects and
that any systems that will be used to deliver them are functioning properly.
The three types of descriptive MSEL events that support exercise play include:
1. Contextual injects introduced to a player by a controller help build the exercise operating
environment and/or keep exercise play moving. For example, if the exercise is designed
to test information-sharing capabilities, a MSEL inject can be developed to direct an actor
to portray a suspect by behaving suspiciously in front of a law enforcement player.
2. Expected action events reserve a place in the MSEL timeline and notify controllers when
a response action would typically take place. For example, during an FSE involving a
chemical agent, establishing decontamination is an expected action that the players will
take without the prompting of an inject.
3. Contingency injects are provided by a controller or simulator to players to ensure play
moves forward to adequately evaluate performance of activities. For example, if a
simulated secondary device is placed at an incident scene during a terrorism response
exercise, but is not discovered, a controller may want to prompt an actor to approach a
player and state that he or she witnessed suspicious activity close to the device location.
This should prompt the responder to discover the device, resulting in subsequent
execution of the desired notification procedures.
MSELs are typically produced in long formats, short formats, or both. Short-form MSELs
usually list injects in a single row in a spreadsheet format. These can be used as a quickreference guide during exercise play or projected onto a large screen in a control cell or SimCell.
Long-form MSELs are used when greater detail is necessary; they include more detailed
descriptions, exact scripting language for actors and simulators, and more detailed descriptions
of expected actions.
Extent of Play Agreements

XPAs can be used to define the organizations participating in the exercise as well as their extent
of play (e.g., one fire station for 8 hours, county Emergency Operations Center [EOC] activated
at level A for 24/7 exercise operations). These agreements are formed between exercise
participants and the exercise sponsor, and can be vital to the planning of an exercise, recruitment
of evaluators, and development of support requirements.
Exercise Evaluation Guides

EEGs are intended to help evaluators collect relevant exercise observations. These documents
are aligned to objectives, and document the related core capability, capability target(s), and
critical tasks. Each EEG provides evaluators with information on what they should expect to see
demonstrated or hear discussed. For more information on EEGs, see Chapter 5: Evaluation.
Participant Feedback Form

At the end of an exercise, participants may receive a Participant Feedback Form that asks for
input regarding observed strengths and areas for improvement that players identified during the
exercise. Providing Participant Feedback Forms to players during the exercise wrap up activities
allows them to provide their insights into decisions made and actions taken. A Participant
Feedback Form also provides players the opportunity to provide constructive criticism about the

Homeland Security Exercise and Evaluation Program

3-18

design, control, or logistics of the exercise to help enhance the planning of future exercises. At a
minimum, the questions on the Participant Feedback Form solicit the following:
•

Strengths and areas for improvement pertaining to the implementation of participating
agencies and organizations’ policies, plans, and SOPs; and

•

Impressions about exercise conduct and logistics.

Information collected from feedback forms contributes to the issues, observations,
recommendations, and corrective actions in the AAR/IP. Feedback forms can be supplemented
by the conduct of a Hot Wash immediately following the exercise, during which facilitators,
controllers, and evaluators capture participant perspectives on the key strengths and areas for
improvement identified during the exercise.
Waiver Forms

Each actor should receive a waiver form prior to the exercise. Signing this form waives liability
for all exercise planners and participants. Exercising entities should use discretion when
recruiting actors under the age of 18 because of additional challenges and concerns related to
liability. If the exercise requires volunteers younger than 18-years-old, parents or legal
guardians must sign their waiver forms.
Weapons and Safety Policy

All exercises, where applicable, should employ a written weapon and safety policy that is in
accordance with applicable State or local laws and regulations. Exercise sponsors should
coordinate the application of this policy with the appropriate safety and/or legal departments as
necessary.
Media or Public Affairs Guidance
Members of the media have the unique ability to fulfill an important function before, during, and
after an exercise. Prior to an exercise, they inform the public that an exercise will take place, and
raise public awareness that the community is preparing for disasters. During an exercise, they
can facilitate the validation of public information plans and procedures. Following an exercise,
the media may release details to the host community on the state of its preparedness, if the
exercise planning team leader provides such information. Therefore, exercise sponsors should
work to incorporate media-related issues into exercise planning.
Press Release

Prior to an exercise, the exercise planning team should develop a written press release to
disseminate to media outlets, including web-based and/or social media outlets, as appropriate.
This release informs the media and the public about general exercise information. Additionally,
this information can be distributed to observers, elected and appointed officials, and other VIPs.
This release should not contain detailed scenario information, such as the type of threat or
hazard, nor should it contain information that might hinder meeting exercise objectives if a
participant were to see it.
Typically, the contents of a media or public information release include the following:
•

Introduction, including sponsor and exercise program information

•

Exercise scope and objectives

Homeland Security Exercise and Evaluation Program

3-19

•

General scenario information

•

Participating agencies or disciplines

Public Announcement

Public announcements should be made prior to any exercise involving public space or space that
will be viewable by the public. This precaution helps avoid confusion on the part of the public.
It will also help the public avoid congestion near the exercise site by providing suggestions for
alternate routes. Announcements can be made through local media, through mass mailings or
pamphlets, and/or on signs near the exercise site.
Media Policy

The agency or organization sponsoring the exercise should decide whether to invite media
representatives to the exercise. If invited, media representatives should have an opportunity
prior to the exercise to conduct interviews with key planners and participants.
At discussion-based exercises, media representatives should not be present during the discussion
of any potentially sensitive information, and filming exercise conduct should be avoided so as
not to inhibit or hinder discussion or the flow of play.
During operations-based exercises, media representatives may be allowed to film certain
activities but should be cautioned not to interfere with exercise play or film any sensitive
operations. Unless media representatives are invited to participate in the exercise, a guide—
typically a public information officer or designee—should escort media representatives at all
times. If mock media or exercise controllers simulating the real-world media are employed
during an exercise to test public affairs training, they should be kept completely separate from
any real-world media representatives who may be observing the exercise.

Exercise Development
Exercise development involves planning for the critical elements of exercise conduct: logistics,
control, and evaluation.
Planning for Exercise Logistics
Logistical details are important, but often overlooked, aspects of an exercise. They can make the
difference between a smooth, seamless exercise and one that is confusing or even unsafe.
Venue

Facility and Room
Meetings, briefings, and exercises should be conducted in facilities that are appropriate for the
exercise scope and attendance. Planners should also ensure that all environmental and historical
preservation documentation is completed if required. Facilities should be reserved solely for
exercise purposes and should be accessible to all participants and free from distractions.
When selecting a facility and room for exercise planning or conduct, planners should account for
the following considerations:
•

Ensure there are enough tables and chairs for every relevant participant.

•

Arrange tables to best suit the meeting or exercise (e.g., U-shaped layout for exercises
requiring facilitation and participant interaction).

Homeland Security Exercise and Evaluation Program

3-20

•

Select a facility with room acoustics that facilitate ease of discussion.

•

Select a facility with accessibility of parking and restrooms for all participants.

Audio/Visual Requirements
A/V requirements are identified during the design phase including individuals assigned to ensure
equipment is properly functioning.
Supplies, Food, and Refreshments
Exercise planners should not assume participants will bring necessary supplies with them.
Writing utensils, notepads, easels, copies of plans and procedures, name badges, and any other
equipment deemed necessary should be procured prior to exercise conduct and provided to
participants.
The exercise planning team should also consider whether food and refreshments can be provided
for participants and observers, in accordance with applicable funding guidance or venue policies.
For discussion-based exercises, it is often beneficial to have a working lunch provided to
minimize disruption to play. For operations-based exercises, hydration of participants is an
important consideration.
Badging and Identification

For security purposes, all exercise participants should wear some form of identification.
Although some players may wear their uniforms, badges are typically used to identify each
exercise participant by name and organization. Where appropriate, name tents should be placed
on tables prior to StartEx to ensure proper seating arrangements. Additionally, each table should
have a table tent identifying the organization or functional area seated at that table.
Registration and Table/Breakout Identification
Participants register upon arrival, for both identification and security reasons. Each participant
should, at minimum, provide their name, organization, telephone number, and e-mail address.
The exercise planning team retains copies of the sign-in sheets, so that participants can receive
follow-up correspondence such as thank-you notes, certificates of completion, copies of the
AAR/IP, and invitations to future planning meetings and exercises.
Actors

Volunteer actors provide added realism and prompt players to provide simulated victim care.
Exercise planning team members can recruit them from local colleges and universities, medical
and nursing schools, drama clubs, theaters, civic groups, emergency response academies, and
Federal and State military units. Consideration should be given to soliciting volunteer actors
from within the access and/or functional needs population to provide an opportunity to practice
meeting the needs of these individuals in a variety of operational environments.
Prior to the exercise, actors should receive the following:
•

Waiver forms for signature, clearing liability for exercise planners and participants;

•

Actor instructions including information on when to arrive, where to report, and other
logistical details; and

Homeland Security Exercise and Evaluation Program

3-21

•

Symptomatology cards containing the signs and symptoms the actor will portray, as well
as information for medical providers.

Parking, Transportation, and Designated Areas

Established parking areas should be clearly labeled for use by participants arriving in personal
owned vehicles. If required, law enforcement personnel should be available to help direct
vehicles to proper parking areas.
Operations-based exercises may also have several key areas for exercise conduct. Designated
exercise areas should be clearly marked, and can include:
•

Exercise Assembly Area. This is a gathering place for all deployable resources that will
be playing in an exercise. The purpose of the exercise assembly area is to gather all
resources and personnel near the exercise site prior to StartEx for safety briefings,
weapons checks, and to ensure that resources and personnel are transported in a safe and
unhurried manner.

•

Operations Area. This is a large space where tactical operations—such as
decontamination, triage, or render-safe procedures—take place.

•

Response Route. This is the path traveled by responding emergency units from the
Assembly Area to the exercise site during a response-focused exercise.

•

Observer/Media Area. This is a designated area that provides observers and real-world
media representatives with a view of the exercise but prevents them from interfering with
exercise play.

Planning for Exercise Control
Exercise control maintains exercise scope, pace, and integrity during conduct under safe and
secure conditions. Key elements of exercise control include controller staffing, structure,
training, communications, and safety and security.
Staffing

The planning team identifies the number of controllers needed during the exercise to deliver and
track information. As a guiding principle, at least one controller should be present at every
venue whenever possible. In addition to controlling the flow of information and release of
MSEL events, positioning a controller at every site helps ensure the exercise is conducted safely
with proper security controls.
During discussion-based exercises, the control staff provides the facilitation. If participants
divide into groups for part of conduct, a table facilitator is assigned to each group. A complex
multijurisdictional FSE, on the other hand, may require hundreds of controllers at field and
headquarters play sites, as well as additional controllers in control cells, to coordinate among the
various play sites or serve in a SimCell. Resource constraints may make placing a controller at
every site challenging. Multitasking personnel to serve as both a controller and an evaluator can
help. While not desirable, exercise planners may also assign selected players to serve as
controllers. Such players/controllers would need to understand clearly how to separate the roles
to avoid feeding advance information into play or otherwise harming exercise integrity.

Homeland Security Exercise and Evaluation Program

3-22

Control Structure and Simulation Cell

The control structure is the framework that allows controllers to communicate and coordinate
with other controllers at other play sites or at a control cell to deliver and track exercise
information. For discussion-based exercises, the structure is usually minimal. For operationsbased exercises, however, the control structure may need to be fairly substantial to allow for
proper coordination.
In an exercise involving field and headquarters play among multiple organizations in one
location, a control cell serves as a central node for sharing information among controllers at the
various sites and for putting all of the information together to form a common exercise picture.
If an exercise contains multiple jurisdictions, particularly multiple levels of government in
different geographic locations, it may be beneficial to establish multiple venue control cells that
communicate and coordinate with each other through a master control cell. When an exercise
does require establishment of multiple control cells, it is important to define their roles and
relationships, including their decision-making hierarchy. Figure 3.3 shows a sample control
structure with multiple control cells.

Figure 3.3: Sample Exercise Control Structure

A SimCell is used to generate injects, receive player responses, and provide information in place
of nonparticipating organizations that would likely participate actively if exercise events were
real. Physically, the SimCell is a working location for a number of qualified professionals who
portray these non-participating organizations. These professionals are knowledgeable of the
organizations they are portraying, and they deliver injects in a realistic fashion. Depending on
the type of exercise, the SimCell may require a telephone, computer, e-mail account, radio, or
other means of communication.
When developing the control structure, exercise planners should consider their resource
environment. Ideally, a control cell will contain a point of contact (POC) or a liaison
representing each participating organization. In exercises involving a mix of classified and
unclassified information, it may be required to separate control cells, with appropriate security

Homeland Security Exercise and Evaluation Program

3-23

firewalls set up to handle classified and unclassified information. Moreover, if an exercise uses a
SimCell to drive exercise play, a determination needs to be made how to staff and integrate it
into the broader control structure.
Controller Training

If all exercise controllers can be recruited from the exercise planning team, there is little need to
develop and provide special training for controllers. If, however, controllers are recruited from
participating entities or other sources outside the planning team, it is very beneficial to provide
some level of advance training to ensure that controllers understand the exercise, their role in it,
and what they need to do.
The training generally includes a basic primer on the exercise design and all of the developed
aspects of exercise control, including the scenario, information delivery methods, control staff,
structure, and communications plan. Controllers are also trained to use the documents (e.g.,
MSEL) and the facilities (e.g., SimCell) that will help them control the exercise.
Communications Plan

The best-designed exercise control structure staffed by the most experienced exercise
practitioners will fail if controllers cannot communicate effectively and efficiently. A
communications section in the C/E Handbook or COSIN serves as a communications plan by
telling controllers who to communicate with, what they need to communicate, and how they will
communicate. This communications section may include:
•

Controller Communications. Controllers at field or headquarters play sites may need to
communicate with controllers at other sites or only with a control cell. Control cells will
need to be able to communicate with all controllers at field or headquarters play sites,
internally, and with other control cells if appropriate. Controllers and control cells may
also need to communicate with players through means other than face-to-face interaction.

•

Timing and Content of Communications. While controllers should communicate
exercise events as they occur, establishing a regular communications schedule with
defined information requirements will help to ensure effective information flow.

•

Communications Methodology. Communications may occur by phone, radio, e-mail,
over a networked system, or a mix. Controllers and control cells will need to be equipped
to use the designated method(s) of communication.

Safety and Security

Controllers also play an important role in ensuring that the exercise is conducted safely in a
secure environment. In exercises involving potentially dangerous field play or the use of
classified materials, the control team designates a safety and/or security controller(s) to focus on
those areas of control.
Safety
Safety is the most important consideration in planning any exercise. For operations-based
exercises, consideration should be given to the following to help ensure a safe environment:
•

Appoint a safety controller(s).

Homeland Security Exercise and Evaluation Program

3-24

•

Dedicate non-exercise participating advanced life support or basic life support ambulance
unit(s) for real-world emergencies that may occur during the exercise.

•

Identify real-world emergency procedures with a code word or phrase.

•

Outline safety requirements and policies.

•

Consider other safety issues outside the scope of exercise control (e.g., weather, heat
stress, hypothermia, etc.).

Security
Because of the sensitive nature of many exercises, it is important for the exercise site to be
secure. Local law enforcement should provide site security where appropriate. Exercises often
also involve sensitive or classified information or procedures. For all exercises involving
sensitive or classified information, exercise planners should identify and adhere to appropriate
security standards to ensure that this information is not compromised. Such measures can
include conducting registration prior to a discussion-based exercise, ensuring that uninvited or
unregistered individuals do not participate, or having law enforcement or security guards monitor
and control access to a play site for the duration of the exercise.
Planning for Exercise Evaluation
Thorough planning and organization prior to an exercise is imperative to effective and successful
exercise evaluation. As described above, the exercise planning team identifies evaluation
elements early in the exercise design process. Additionally, during exercise development, an
evaluation team organizes itself appropriately and develops a comprehensive plan to address how
the exercise will be evaluated. Additional details regarding evaluation planning can be found in
Chapter 5: Evaluation.

Homeland Security Exercise and Evaluation Program

3-25

4. Exercise Conduct
Overview
Exercise conduct involves activities such as preparing for exercise play, managing exercise play,
and conducting immediate exercise wrap-up activities. For discussion-based exercises, conduct
also entails presentation, facilitation, and discussion. For operations-based exercises, conduct
encompasses all operations occurring between the designated StartEx and End of Exercise
(EndEx). Exercise practitioners are encouraged to apply and adapt HSEEP doctrine on exercise
conduct to meet their specific needs. Throughout these efforts, the engagement of elected and
appointed officials by practitioners will ensure that the exercise is addressing the guidance and
intent of officials.

Exercise Play Preparation
Setup for Discussion-Based Exercises
Members of the exercise planning team assigned to support exercise setup should visit the
exercise site at least one day prior to the event to arrange the room, test A/V equipment, and
discuss administrative and logistical issues. On the day of the exercise, planning team members
should arrive several hours before StartEx to handle setup activities and arrange for registration.
Prior to exercise conduct, the exercise planning team must deliver the necessary exercise
materials and equipment, which may include the following:
•

SitMans or other written materials for exercise participants

•

Multimedia presentation

•

Appropriate A/V equipment including televisions, projectors, projection screens,
microphones, and speakers

•

Table tents for each table

•

Name tents for each participant

•

Badges identifying the role of each exercise participant

•

Sign-in sheets

•

Participant Feedback Forms

Setup for Operations-Based Exercises
The appropriate exercise planning team members should begin event setup as many days prior to
the event as necessary, depending on the scope of the simulated environment. Setup entails
arranging briefing rooms and testing A/V equipment, placing props and effects, marking the
appropriate exercise areas and their perimeters, and checking for potential safety issues. On the
day of the exercise, all exercise planning team members should arrive several hours before
StartEx to handle any remaining logistical or administrative items pertaining to setup, and
arrange for registration. A communications check should also be conducted prior to the start of
an operations-based exercise.

Homeland Security Exercise and Evaluation Program

4-1

Briefings
Held before an exercise, briefings educate participants about their roles and responsibilities. By
scheduling separate briefings for elected and appointed officials, controllers and evaluators,
actors, players, and observers, exercise planning team members can avoid giving extraneous
material to different groups and ensure the design, development, and conduct of an exercise
aligns to their elected and appointed officials’ guidance.
Elected and Appointed Official Briefing

Elected and Appointed Official Briefings should occur during design and development and prior
to the conduct of an exercise. The exercise planning team leader should periodically consult
with the elected and appointed officials within the exercise planning team to ensure the exercise
aligns with leadership intent.
Controller/Evaluator Briefing

The C/E Briefing is generally conducted before operations-based exercises. It begins with an
exercise overview and then reviews the exercise location and area, schedule of events, scenario,
control concept, controller and evaluator responsibilities, instructions on completing EEGs, and
any miscellaneous information. Additional training for evaluators may be conducted.
Actor Briefing

The Actor Briefing should be conducted before the exercise, prior to the actors taking their
positions. The actor controller leads this briefing and includes the following information:
exercise overview, safety, real emergency procedures, symptomatology, acting instructions, and
schedule. Identification badges and symptomatology cards are distributed at this briefing.
Player Briefing

Shortly before StartEx, a controller conducts a briefing for all players to address individual roles
and responsibilities, exercise parameters, safety, security badges, and any remaining logistical
exercise concerns or questions. Participant Handouts and ExPlans or SitMans, depending on the
type of exercise being conducted, are often distributed during this briefing. Following the
exercise, controllers ensure that appropriate players attend the post-exercise Hot Wash in their
respective functional area.
Observer Briefing

An Observer Briefing is generally conducted the day of an exercise and informs observers and
VIPs about the exercise background, scenario, schedule of events, observer limitations, and any
other miscellaneous information. Often, observers will be unfamiliar with public safety
procedures and will have questions about the activities they see. Designating someone, such as a
public information officer, to answer questions prevents observers from asking questions of
players, controllers, or evaluators.

Exercise Play
During exercise play, participants accomplish various roles and responsibilities aimed at achieving
exercise objectives and demonstrating core capabilities.

Homeland Security Exercise and Evaluation Program

4-2

Participant Roles and Responsibilities
Table 4.1 describes the exercise participant role and responsibilities associated with exercise
conduct and the applicable exercise types.
Role

Responsibilities

Exercise
Type

Exercise
Director

The Exercise Director oversees all exercise functions during exercise
conduct, oversees and remains in contact with controllers and evaluators,
debriefs controllers and evaluators following the exercise, and oversees
setup and cleanup of the exercise as well as positioning of controllers and
evaluators.

All

Evaluator

Evaluators are chosen based on their expertise in the specific functional
areas they will observe. Evaluators use evaluation documents to
document observations, capture unresolved issues, and analyze exercise
results. Evaluators do not interfere with exercise flow.

All

Lead
Evaluator

The lead evaluator should participate as a member of the exercise
planning team and be familiar with all relevant issues associated with the
exercise, including plans, policies, and procedures; incident command and
decision-making processes; and interagency and/or inter-jurisdictional
coordination issues. The lead evaluator should have the management
skills needed to oversee a team of evaluators over an extended process
as well as the knowledge and analytical skills to undertake a thorough and
accurate analysis of all capabilities.

All

Facilitator

During a discussion-based exercise, the facilitator(s) is responsible for
keeping participant discussions on track with exercise objectives and
ensuring all issues and objectives are explored as thoroughly as possible
within time constraints. If an exercise uses breakout groups, more than
one facilitator may be needed.

Seminar,
Workshop,
TTX, Game

Controller

In operations-based exercises and some games, controllers plan and
manage exercise play, set up and operate the exercise incident site, and
possibly take the roles of individuals and agencies not actually
participating in the exercise. Controllers direct the pace of exercise play,
provide key data to players, and may prompt or initiate certain player
actions and injects to the players as described in the MSEL to ensure
exercise continuity. Controllers issue exercise materials to players as
required, monitor the exercise timeline, and supervise the safety of all
exercise participants. Controllers are the only participants who should
provide information or direction to players. All controllers should be
accountable to one senior controller.

Game,
Drill, FE,
FSE

Senior
Controller

The senior controller (sometimes known as a lead controller) is
responsible for the overall organization of the exercise. The senior
controller monitors actions by controllers and exercise progress, and
coordinates decisions regarding deviations or significant changes to the
scenario caused by unexpected developments during play. The senior
controller debriefs controllers and evaluators after the exercise and
oversees the setup and takedown of the exercise.

Game,
Drill, FE,
FSE

Safety
Controller

The safety controller is responsible for monitoring exercise safety during
exercise setup, conduct, and cleanup. All exercise controllers assist the
safety controller by reporting any safety concerns. The safety controller
should not be confused with the safety officer, who is identified by the
incident commander during exercise play.

Drill, FE,
FSE

Homeland Security Exercise and Evaluation Program

4-3

Role

Responsibilities

Exercise
Type

Exercise
Assembly
Area
Controller

The exercise assembly area controller is responsible for the logistical
organization of the exercise assembly area, including placement locations
for units entering the exercise assembly area, the release of dispatched
units into the field, and coordination of pathways and overall safety within
the exercise assembly area.

FSE

Simulator

Simulators are control staff personnel who role play as nonparticipating
organizations or individuals. They most often operate out of SimCell, but
they may occasionally have face-to-face contact with players. Simulators
function semi-independently under the supervision of SimCell controllers,
enacting roles in accordance with instructions provided in the MSEL. All
simulators are ultimately accountable to the Exercise Director and senior
controller.

Drill, FE,
FSE

Observer

Observers do not directly participate in the exercise; rather, they observe
selected segments of the exercise as it unfolds, while remaining separated
from player activities. Observers view the exercise from a designated
observation area and are asked to remain within the observation area
during the exercise. A dedicated controller or public information officer
should be assigned to manage these groups. In a discussion-based
exercise, observers may support the development of player responses to
the situation by asking relevant questions, delivering messages, or citing
references; however, they generally do not participate in moderated
discussion.

All

Player

Players have an active role in preventing, responding to, or recovering
from the risks and hazards presented in the scenario, by either discussing
or performing their regular roles and responsibilities. Players initiate
actions that will respond to and/or mitigate the simulated emergency.

All

Actor

Actors are typically volunteer personnel responsible for simulating a
specific role in an exercise. Actors are vital to creating a realistic scenario
and can play a variety of roles.

Drill, FSE

Table 4.1: Exercise Participant Roles

Conduct for Discussion-Based Exercises
Multimedia Presentation

The multimedia presentation is a crucial vehicle for conveying information to the players. The
presentation typically starts with brief remarks by representatives from the exercise planning
team or sponsoring organization, and/or elected and appointed officials from the governing
jurisdiction. After the opening remarks, the presentation moves into a brief introductory and
explanatory phase led by a facilitator. During this phase, attendees will be introduced to any
other facilitators, controllers (games only), or evaluators; given background on the exercise
process; and advised about their individual roles and responsibilities.
The facilitator generally presents the multimedia briefing, which describes the scenario and any
relevant background information. The facilitator also leads the discussion, introduces
spokespersons, poses questions to the audience, and ensures that the schedule remains on track.

Homeland Security Exercise and Evaluation Program

4-4

Facilitated Discussion

Facilitated group discussions can occur in a plenary session or in breakout groups, which are
typically organized by discipline or agency/organization. In both formats, a facilitator is
responsible for keeping the discussion focused on the exercise objectives and making sure all
issues are explored within the time allotted. A good facilitator should possess:
•

The ability to keep side conversations to a minimum, keep discussions on track and
within established time limits, control group dynamics and strong personalities, and
speak competently and confidently about the subject without dominating conversation;

•

Functional area expertise or experience;

•

Awareness of appropriate plans and procedures; and

•

The ability to listen well and summarize player discussions.

If feasible and/or appropriate, co-facilitators who are knowledgeable about local issues, plans,
and procedures may assist the lead facilitator. Also, designating a recorder to take notes allows
the facilitator to focus on key discussion issues.
Moderated Discussion

Moderated discussions generally follow breakout discussions. In moderated discussions, a
representative from each group presents all participants with summarized results from a group’s
facilitated discussion. This spokesperson is selected before the facilitated discussion so that he
or she can prepare to speak on behalf of the group. During moderated discussions,
spokespersons summarize the facilitated discussion, present key findings and issues, and discuss
any unresolved issues or questions. At the end of the moderated discussion period, the facilitator
opens the floor for questions.
Time for moderated discussion is generally scheduled at the end of each module, with another
longer period for each at the conclusion of the exercise. During the moderated discussion,
groups should focus only on the material presented in a given module.
Exercise Data Collection

During discussion-based exercises, facilitators help evaluators collect useful data by keeping
discussions focused on exercise objectives, core capabilities, capability targets, and critical tasks.
Additional information about data collection can be found in Chapter 5: Evaluation.
Conduct for Operations-Based Exercises
During conduct of operations-based exercises, the exercise planning team leader normally serves
as the senior controller or Exercise Director. Controllers and evaluators report key activities to
the senior controller. The senior controller is responsible for both commencing exercise play by
announcing StartEx, and announcing EndEx at the conclusion of the scenario, after a certain
period of time has passed, or when all exercise objectives have been met.
Prior to StartEx, rules for exercise play should be disseminated to all participants to establish the
parameters that they must follow during the exercise. These rules help players understand their
roles in the exercise environment, describe appropriate behavior, establish guidelines for
physical contact, and aim to prevent physical harm to individuals or damage to property. Written
rules should be reviewed and approved by appropriate authorities.

Homeland Security Exercise and Evaluation Program

4-5

Exercise areas for operations-based exercises should be clearly defined, and all exercise
operations should take place within these designated areas. The exercise area for an FE is
usually limited to the control or command centers and their onsite staff members. All other
activity and deployment of resources outside of these locations are notional and is simulated by
the SimCell staff. The exercise area for an FSE or drill might include one or more simulated
incident sites, as well as control or command centers. It is important that these areas be clearly
marked to ensure player safety and avoid confusion with real-world operations.
To prevent confusion with real-world communications or accidental deployment of resources, all
communications must be clearly identified as exercise-related. This can be accomplished by
displaying the phrase “exercise material only” prominently on all typed or printed
communications, and by beginning each verbal communication by stating, “This is an exercise,”
or a similar statement as agreed upon by the exercise planning team. Additionally, players
should be supplied with an exercise directory that provides contact information for each of the
simulated organizations portrayed by simulators in the SimCell.
Control

As detailed in Chapter 3: Exercise Design and Development, the control structure for an
operations-based exercise describes how controllers communicate and coordinate with one
another and how they track exercise information. These procedures, as well as clearly defined
roles and responsibilities for each controller, should be detailed in the C/E Handbook. During
exercise play, controllers carry out these responsibilities and closely monitor exercise play to
ensure a safe and effective exercise.
During FE play, SimCell control is particularly important. Because of the great deal of
simulated activity that occurs during FEs, these exercises require a robust and detailed MSEL
and close communication between the site controller(s) and the SimCell. Site controllers should
advise the SimCell on the pace of exercise play, and request more or fewer injects as necessary
to maintain an appropriate pace.
During FSEs and drills, the exercise assembly area controller plays a key role. The exercise
assembly area controller remains in close communication with other controllers throughout the
exercise to ensure safe and realistic deployment of personnel. When a unit arrives at the
assembly area, the exercise assembly area controller takes attendance to ensure all players are
present. Units are positioned according to their deployment times, and qualified individuals
perform a weapons check to guarantee the tagging of all inspected weapons to indicate they are
safe for exercise play. This controller is also responsible for the exercise assembly area’s
logistical organization, including placement locations for units and coordination of exiting
patterns for dispatched units. It is imperative for the exercise planning team to create a
deployment timetable based on realistic response times; failure to do so will result in a
compromised and disorganized exercise. The exercise assembly area controller must be
informed about any updates to the exercise that may require changes to the deployment
timetable, and he or she should update the deployment timetable accordingly.
In all operations-based exercises, it is critical that all exercise controllers take appropriate actions
to ensure a safe and secure exercise environment. These actions may involve monitoring
conditions that impact player and/or actor safety, such as heat stress and other health issues.

Homeland Security Exercise and Evaluation Program

4-6

Exercise Data Collection

During the exercise, each evaluator should use the EEGs to record both quantitative and
qualitative data for capabilities, capability targets, and critical tasks, as assigned by the lead
evaluator. During operations-based exercises, evaluators should be strategically pre-positioned
in locations at which they can gather useful data, and they should track and record participant
actions carefully.
Contingency Process
In order to prevent jeopardizing mission performance in response
to real-world events, the exercise planning team should maintain
An effective contingency
a contingency process to halt, postpone, or cancel an exercise as
process is critical to
necessary. Should the conduct of the exercise put at risk any
ensure the exercise can
be halted, postponed, or
efforts to respond to real-world events or should real-world
canceled if a real-world
events hinder conduct of the exercise, the Exercise Director and
event takes place.
exercise planning team should convene, in coordination with
elected and appointed officials from participating organizations,
to determine the appropriate course of action. Following decision on a final course of action, the
Exercise Director should communicate that course of action to all exercise planners, participants,
and other key stakeholders through all relevant communications mechanisms.

Wrap-Up Activities
Performing thorough exercise wrap-up will ensure that all relevant data is collected to support
effective evaluation and improvement planning.
Debriefings
Immediately following the exercise, a short debriefing should be conducted with exercise
planning team members to ascertain their level of satisfaction with the exercise, discuss any
issues or concerns, and propose improvements. Planners should collect exercise attendance lists,
provide copies to the exercise planning team leader, collect Participant Feedback Forms, and
develop debriefing notes.
Player Hot Wash
A Hot Wash provides an opportunity for exercise participants to discuss exercise strengths and
areas for improvement immediately following the conduct of an exercise. The Hot Wash should
be led by an experienced facilitator who can ensure that the discussion remains brief and
constructive. The information gathered during a Hot Wash can be used during the AAR/IP
process, and exercise suggestions can be used to improve future exercises. Hot Washes also
provide opportunities to distribute Participant Feedback Forms, which, when completed by
players, can be used to help generate the AAR/IP.
For operations-based exercises, a Hot Wash should be conducted for each functional area by that
functional area’s controller or evaluator immediately following an exercise. It can also provide
an opportunity for players to gain clarification on exercise play at other exercise sites or in other
functional areas.

Homeland Security Exercise and Evaluation Program

4-7

Controller/Evaluator Debriefing
The C/E Debriefing provides a forum for functional area controllers and evaluators to review the
exercise. The exercise planning team leader facilitates this debriefing, which provides each
controller and evaluator with an opportunity to provide an overview of the functional area they
observed and to discuss both strengths and areas for improvement. During the debriefing,
controllers and evaluators complete and submit their Participant Feedback Forms. Debriefing
results are captured and may be included in the AAR/IP. Similarly, for discussion-based
exercises, a Facilitator/Evaluator Debriefing is held to review exercise conduct. This debriefing
can be facilitated by the exercise planning team leader and provides a forum for facilitators and
evaluators to discuss strengths, areas for improvement, and progress in completing exercise
objectives.

Homeland Security Exercise and Evaluation Program

4-8

5. Evaluation
Overview
Exercise evaluation maintains the fundamental link between the exercise and improvement
planning. Through exercise evaluation, organizations assess the capabilities needed to
accomplish a mission, function, or objective. This assessment is based on the performance of
critical tasks to capability target levels. Effective exercise evaluation involves:
•

Planning for exercise evaluation;

•

Observing the exercise and collecting exercise data during exercise conduct;

•

Analyzing collected data to identify strengths and areas for improvement; and

•

Reporting exercise outcomes in a draft AAR.

Using a common approach to evaluation supports consistent and meaningful reporting of
exercise results.

Evaluation Planning
It is important that evaluation planning begins during the initial planning phases of the exercise,
as described in Chapter 3: Exercise Design and Development. Identifying clear evaluation
requirements early in the planning process will ensure that the design, development, and conduct
of the exercise best support an effective evaluation. Exercise planners should collaborate to
ensure a consistent approach for evaluating capabilities during an exercise. Additionally, elected
and appointed officials should be engaged early in evaluation planning in order to identify any
specific evaluation requirements. Planning an exercise evaluation typically includes:
•

Selecting lead evaluator and define evaluation team requirements;

•

Developing EEGs, which include objectives, core capabilities, capability targets, and
critical tasks;

•

Recruiting, training, and assigning evaluators;

•

Developing and finalizing evaluation documentation; and

•

Conducting a pre-exercise C/E Briefing.

Through this process, an evaluation team can develop a thorough plan to address how the
exercise will be evaluated.
Evaluation Team
Early in the exercise planning process, the exercise planning team leader should appoint a lead
evaluator to oversee all facets of the evaluation process. The lead evaluator participates fully as
a member of the exercise planning team and should be familiar with the exercise’s objectives. A
lead evaluator should have the management skills needed to oversee a team of evaluators as well
as the knowledge and analytical skills to undertake a thorough and accurate analysis of all
objectives and core capabilities of an exercise. The lead evaluator must have the skills to
effectively communicate and coordinate with the exercise controllers. In addition, the lead

Homeland Security Exercise and Evaluation Program

5-1

evaluator should be familiar with the mission areas and core capabilities associated with the
exercise; plans, policies, and procedures of the participating organizations; incident command
and decision-making processes; and key preparedness doctrine and policy.
The exercise planning team and lead evaluator should determine the structure of the exercise
evaluation team based on the scope of the exercise, the exercise objectives, associated core
capabilities, and critical tasks that will be evaluated during the exercise. Specific security
clearance levels may be required for some exercise play or locations. Exercises that involve
multiple jurisdictions and/or multiple venues should consider assigning site leads, as illustrated
by the example provided in Figure 5.1. A site could be a jurisdiction, a specific emergency
operations center, or another exercise location. These individuals support the lead evaluator and
manage the activities of other evaluators assigned to that location.

Figure 5.1: Sample Exercise Evaluation Team Organization

The exercise planning team and lead evaluator should determine the tools and documentation
needed to support the evaluation team, such as the need for a separate Evaluation Plan instead of
relying on the evaluation section in the C/E Handbook. The lead evaluator also identifies data
collection methods to ensure that information specific to that exercise is recorded.
Consideration should be given to an exercise’s scope and objectives when selecting the number
of individuals for evaluation support. For exercises of limited scope and having fewer objectives
and capabilities, the lead evaluator and one additional person may be all that is needed. For
more complex or larger exercises with a greater number of objectives and capabilities, more
individuals may be required for evaluation and AAR development.
Exercise Evaluation Guide Development
EEGs provide a consistent tool to guide exercise observation and data collection. EEGs are
aligned to exercise objectives and core capabilities, and list the relevant capability targets and
critical tasks. These targets and critical tasks may be drawn from the National Preparedness
Goal and the five national planning frameworks, a threat/hazard identification and risk
assessment product, or from an organization’s own plans and assessments, as described in
Chapter 3: Exercise Design and Development.
EEGs are designed to accomplish several goals:
•

Streamline data collection

Homeland Security Exercise and Evaluation Program

5-2

•

Enable thorough assessments of the participant organizations’ capability targets

•

Support development of the AAR

•

Provide a consistent process for assessing preparedness through exercises

•

Help organizations map exercise results to exercise objectives, core capabilities,
capability targets, and critical tasks for further analysis and assessment

Recruit, Assign, and Train Evaluators
Once evaluation requirements have been defined by the planning team, the lead evaluator
oversees the recruiting, assigning, and training of evaluators. The evaluation requirements play a
critical role in determining how many evaluators are needed, the type of subject matter expertise
they should possess, their assignment during the exercise, and the type of training or instruction
required prior to the exercise. Whenever possible, evaluators should have experience and subject
matter expertise in their assigned functional area. Evaluator assignments should be
communicated to evaluators prior to exercise conduct.
Effective evaluator training ensures that exercise evaluators have a shared understanding of the
key data to be collected and how that data will contribute to the evaluation of the exercise.
Evaluator training typically includes reviewing the following:
•

General information about the exercise, including scope, objectives and aligned core
capabilities, scenario, and schedule;

•

Relevant evaluator documentation (e.g., SitMan, C/E Handbook, evaluation tools); and

•

Appropriate plans, policies, procedures, agreements, or other information that are the
focus of the exercise.

Evaluator training should also include guidance on observing exercise discussion or operations,
and criteria for inclusion of data in the final exercise analysis.
Evaluation Documentation
Once exercise requirements have been defined and evaluation planning completed, the lead
evaluator should finalize the evaluation section of the C/E Handbook or develop the Evaluation
Plan. The C/E Handbook or Evaluation Plan typically contains the following information:
•

Exercise-Specific Details: Exercise scenario, schedule of events, and evaluation
schedule;

•

Evaluator Team Organization, Assignments, and Locations: A list of evaluator
locations, shift assignments, a map of the exercise site(s), evaluation team organizational
chart, and evaluation team contact information;

•

Evaluator Instructions: Step-by-step instructions for evaluators for activities before,
during, and following the exercise; and

•

Evaluation Tools: EEGs, the MSEL or a list of venue-specific injects, electronic or
manual evaluation logs or data collection forms, relevant plans and procedures,
Participant Feedback Forms, and Hot Wash templates.

Homeland Security Exercise and Evaluation Program

5-3

In less complex exercises, the C/E Handbook may be a brief, simple document. For more
complex exercises, the C/E Handbook will be a longer document, containing all the information
and tools that evaluators require.
Pre-Exercise Evaluator Briefing
Before exercise play begins, the lead evaluator should meet with all evaluators to verify roles,
responsibilities, and assignments, and to provide any significant updates (e.g., last-minute
changes to the scenario, new assignments). The Evaluator Briefing provides an opportunity for
evaluators to ask questions and to ensure complete understanding of their roles and
responsibilities. Depending on a variety of factors, including exercise scope, objectives, and
scenario, this briefing may be done in conjunction with exercise controllers, as a
Controller/Evaluator Briefing. Depending on the exercise organization, it may be necessary to
conduct briefings at more than one exercise site.

Exercise Observation and Data Collection
Exercise observations and data collection can differ between discussion-based exercises and
operations-based exercises. Discussion-based exercises often focus on issues involving plans,
policies, and procedures; consequently, observations of these exercises may consist of an
evaluator or a note-taker recording data from participant discussions on EEGs. Operations-based
exercises focus on issues affecting the operational execution of capabilities and critical tasks.
During operations-based exercises, evaluators collect and record participant actions, which form
the analytical basis for determining if critical tasks were successfully demonstrated and
capability targets were met.
Observation
Exercise evaluators should observe exercise activity in a non-attribution environment, in
accordance with the evaluation training and EEGs. Evaluators will generally be able to observe
many of the following topics:
•

Plans, policies, and procedures used during the exercise;

•

Legislative authorities used or implemented;

•

Roles and responsibilities of the government agencies and private organizations;

•

Pertinent decisions made, including information gathered to make decisions;

•

Activation or implementation of processes and procedures, requests for resources, use of
mutual aid agreements, etc.; and

•

How and what information is shared with other agencies and the public.

Data Collection
Evaluators should retain their notes and records of the exercise to support the development of the
AAR. As necessary, the lead evaluator may assign evaluators to collect supplemental data
during or immediately after the exercise. Such data is critical to fill in gaps identified during
exercise evaluation. For example, useful sources of supplemental evaluation data might include
records produced by automated systems or communication networks, and written records, such
as duty logs and message forms.

Homeland Security Exercise and Evaluation Program

5-4

Data Analysis
The goal of data analysis is to evaluate the ability of exercise participants to perform core
capabilities and to determine if exercise objectives were met. During data analysis, the
evaluation team consolidates the data collected during the exercise and determines whether
participants performed critical tasks and met capability targets.
Root-cause analysis
Evaluators consider participant performance against all targets to
involves not just
determine the overall ability to perform core capabilities.
identifying what issues
Additionally, the evaluation team takes notes on the course of
emerged, but rather
exercise play, demonstrated strengths, and areas for
discovering the root
improvement. This provides the evaluators with not only what
causes of those issues.
happened, but why events happened.
After this initial data analysis, evaluators examine each critical task not completed as expected
and each target not met, with the aim of identifying a root cause. A root cause is the source of or
underlying reason behind an identified issue toward which the evaluator can direct an
improvement. When conducting a root-cause analysis, the evaluator should attempt to trace the
origin of each event back to earlier events and their respective causes. Root-cause analysis may
also require the review and evaluation of an organization’s plans, policies, and procedures.
When completing the analysis, evaluators should consider the following questions:
•

Were the capability targets met? If the targets were not met, what factors contributed to
this result?

•

Did discussion or activities suggest the critical tasks were executed to meet capability
targets? If not, what was the impact or consequences?

•

Do current plans, policies, and procedures support critical tasks and capability targets?
Were participants familiar with these documents?

Analyzing events in this sequence will help evaluators determine the underlying cause of issues,
and inform an organization’s corrective actions to remedy the issue.

After-Action Report Draft
The AAR is the document that summarizes key information related to evaluation. The length,
format, and development timeframe of the AAR depend on the exercise type and scope. These
parameters should be determined by the exercise planning team based on the expectations of
elected and appointed officials as they develop the evaluation requirements in the design and
development process. The main focus of the AAR is the analysis of core capabilities. Generally,
AARs also include basic exercise information, such as the exercise name, type of exercise, dates,
location, participating organizations, mission area(s), specific threat or hazard, a brief scenario
description, and the name of the exercise sponsor and POC.
The AAR should include an overview of performance related to each exercise objective and
associated core capabilities, while highlighting strengths and areas for improvement. Therefore,
evaluators should review their evaluation notes and documentation to identify the strengths and
areas for improvement relevant to the participating organizations’ ability to meet exercise
objectives and demonstrate core capabilities.

Homeland Security Exercise and Evaluation Program

5-5

Upon completion, the evaluation team provides the draft AAR to the exercise sponsor, who
distributes it to participating organizations. Elected and appointed officials, or their designees,
review and confirm observations identified in the formal AAR, and determine which areas for
improvement require further action. Areas for improvement that require action are those that
will continue to seriously impede capability performance if left unresolved. As part of the
improvement planning process, elected and appointed officials identify corrective actions to
bring areas for improvement to resolution and determine the organization with responsibility for
those actions. This process is further described in Chapter 6: Improvement Planning.

Homeland Security Exercise and Evaluation Program

5-6

6. Improvement Planning
Overview
Exercises afford organizations the opportunity to evaluate capabilities and assess progress
toward meeting capability targets in a controlled, low-risk setting. After the evaluation phase
concludes, organizations should reach consensus on identified strengths and areas for
improvement and develop a set of improvements that directly addresses core capability gaps.
This information is recorded in the AAR/IP and resolved through the implementation of concrete
corrective actions, which are prioritized and tracked as part of a corrective action program. This
process constitutes the improvement planning phase and the final step in conducting an exercise.

Corrective Actions
Once exercise data are analyzed, organizations should perform an additional qualitative
assessment to identify potential corrective actions. Corrective actions are concrete, actionable
steps that are intended to resolve capability gaps and shortcomings identified in exercises or realworld events. In developing corrective actions, elected and appointed officials or their designees
should first review and revise the draft AAR, as needed,
Elected and appointed officials
prior to the After-Action Meeting (AAM) to confirm that
confirm that issues identified in
the issues identified by evaluators are valid and require
the draft AAR are valid and need
resolution. The reviewer then identifies which issues fall
resolution, and they determine
within their organization’s authority, and assume
appropriate corrective actions to
responsibility for taking action on those issues. Finally,
resolve those issues.
they determine an initial list of appropriate corrective
actions to resolve identified issues.
The organization’s reviewer should use the following questions to guide their discussion when
developing corrective actions:
•

What changes need to be made to plans and procedures to improve performance?

•

What changes need to be made to organizational structures to improve performance?

•

What changes need to be made to management processes to improve performance?

•

What changes to equipment or resources are needed to improve performance?

•

What training is needed to improve performance?

•

What are the lessons learned for approaching similar problems in the future?

After-Action Meeting
Once the organization’s reviewer has confirmed the draft areas for improvement and identified
initial corrective actions, a draft IP is developed for review at an AAM. AAMs serve as forums
to review the revised AAR and the draft IP. Prior to the AAM, as appropriate, the exercise
sponsor will distribute the revised AAR, which incorporates feedback on the strengths and areas
for improvement, and the draft IP to participants. Distributing these documents for review prior
to the meeting helps to ensure that all attendees are familiar with the content and are prepared to
discuss exercise results, identified areas for improvement, and corrective actions. The

Homeland Security Exercise and Evaluation Program

6-1

organization’s elected and appointed officials, or their designees, should attend the AAM along
with exercise planners to answer any questions or provide necessary details on the exercise itself.
During the AAM, participants should seek to reach final consensus on strengths and areas for
improvement, as well as revise and gain consensus on draft corrective actions. Additionally, as
appropriate, AAM participants should develop concrete deadlines for implementation of
corrective actions and identify specific corrective action owners/assignees. Participant
organizations are responsible for developing implementation processes and timelines, and
keeping their elected and appointed officials informed of the implementation status.

After-Action Report/Improvement Plan Finalization
Once all corrective actions have been consolidated in the final IP, the IP may be included as an
appendix to the AAR. The AAR/IP is then considered final, and may be distributed to exercise
planners, participants, and other preparedness stakeholders as appropriate.

Corrective Action Tracking and Implementation
Corrective actions captured in the AAR/IP should be tracked and continually reported on until
completion. Organizations should assign points of contact responsible for tracking and reporting
on their progress in implementing corrective actions. By tracking corrective actions to
completion, preparedness stakeholders are able to demonstrate that exercises have yielded
tangible improvements in preparedness. Stakeholders should also ensure there is a system in
place to validate previous corrective actions that have been successfully implemented. These
efforts should be considered part of a wider continuous improvement process that applies prior
to, during, and after an exercise is completed.
Conducting exercises and documenting the strengths, areas for improvement, and associated
corrective actions is an important part of the National Preparedness System, and contributes to
the strengthening of preparedness across the Whole Community and achievement of the National
Preparedness Goal. Over time, exercises should yield observable improvements in preparedness
for future exercises and real-world events.

Using Improvement Planning to Support Continuous Improvement
The identification of strengths, areas for improvement, and corrective actions that result from
exercises help organizations build capabilities as part of a larger continuous improvement
process. The principles of continuous improvement are:
•

Consistent Approach. Organizations should employ a consistent approach for
continuous improvement-related activities across applicable mission areas—prevention,
protection, mitigation, response, and recovery. This consistent approach enables a shared
understanding of key terminology, functions, processes, and tools. This approach also
fosters continuous improvement-related interoperability and collaboration across an
organization’s components.

•

Support National Preparedness. By conducting continuous improvement activities,
organizations support the development and sustainment of core capabilities across the
whole community. Continuous improvement activities also ensure that organizations are
able to support assessments of national preparedness in a timely, actionable, and
meaningful way.

Homeland Security Exercise and Evaluation Program

6-2

•

Effective Issue Resolution and Information Sharing. Through improvement planning,
organizations complete continuous improvement action items at the lowest level possible
while facilitating the sharing of strengths and areas for improvement.

•

Application across Operational Phases. The functions, processes, and tools apply to all
operational phases, including:
-

Near-real time collection and analysis during real-world events or exercises

-

Post-event/exercise analysis

-

Trend analysis across multiple events/exercises over time

Application of these principles and the conduct of improvement planning ultimately support the
program management phase of the HSEEP exercise cycle. By continually examining the
implementation of corrective actions, organizations can identify capability gaps, as well as
determine which corrective actions require validation through exercises. In this way,
improvement planning activities can help shape an organization’s exercise program priorities and
support continuous improvement in the building and sustaining of core capabilities.

Homeland Security Exercise and Evaluation Program

6-3

Glossary of Terms
This document provides descriptions for many of the commonly-used terms in HSEEP.
A
Term

Description

Actor

Actors are volunteers who simulate specific roles, such as disaster
casualty victims, in order to add realism to an exercise.

Actor Briefing

An Actor Briefing is generally conducted before the exercise and provides
actors with an overview of the exercise, including: exercise overview,
safety, real emergency procedures, symptomatology, acting instructions,
and schedule. Identification badges and symptomatology cards are
distributed before or during this briefing.

Administration/ Finance
Section

The Administration/Finance Section of the exercise planning team
provides financial management and administrative support throughout
exercise development, including exercise registration support and
scheduling.

After-Action Meeting
(AAM)

The AAM is a meeting held among elected and appointed officials or their
designees from the exercising organizations, as well as the lead evaluator
and members of the exercise planning team, to debrief the exercise and to
review and refine the draft AAR/IP. The AAM should be an interactive
session, providing attendees the opportunity to discuss and validate the
analytical findings and corrective actions in the draft AAR/IP.

After-Action Report
(AAR)

The AAR summarizes key exercise-related evaluation information,
including the exercise overview and analysis of objectives and core
capabilities. The AAR is usually developed in conjunction with an IP. The
lead evaluator and exercise planning team draft the AAR and submit it to
meeting participants before the AAM.
B

Best Practices

Best practices are peer-validated techniques, procedures, and solutions
that prove successful and are solidly grounded in actual experience in
operations, training, and exercises.
C

Capabilities-Based
Planning

Capabilities-based planning is defined as planning, under uncertainty, to
build capabilities suitable for a wide range of threats and hazards while
working within an economic framework that necessitates prioritization and
choice. Capabilities-based planning is the basis for guidance such as the
National Preparedness Goal.

Capability

A capability may be delivered with any combination of properly planned,
organized, equipped, trained, and exercised personnel to achieve an
intended target.

Capability Target

Capability targets are the performance thresholds for a core capability.
Capability targets may be derived from Threat and Hazard Identification
and Risk Assessments (THIRAs), subject-matter experts, national
guidance, or industry standards.

Concept and Objectives
(C&O) Meeting

A C&O Meeting is the formal beginning of the exercise planning process.
It is held to identify the scope and objectives of the exercise. For less
complex exercises and for organizations with limited resources, the C&O
Meeting can be conducted in conjunction with the IPM.

Homeland Security Exercise and Evaluation Program

Glossary-1

Contextual Inject

Contextual injects are introduced to a player by a controller to help build
the exercise operating environment and/or keep exercise play moving.
For example, if the exercise is designed to test information-sharing
capabilities, a MSEL inject can be developed to direct an actor to portray a
suspect by behaving suspiciously in front of a law enforcement player.

Contingency Inject

Contingency injects are events that a controller provides to a player if the
players get off track or do not take an action that is necessary for the
continuation of the exercise. This ensures that play moves forward, as
needed, to adequately evaluate performance of activities. For example, if
a simulated secondary device is placed at an incident scene during a
terrorism response exercise, but is not discovered, a controller may want
to prompt an actor to approach a player to say that he or she witnessed
suspicious activity close to the device location.

Control Staff
Instructions (COSIN)

The COSIN contains guidance that controllers, simulators, and evaluators
need concerning procedures and responsibilities for exercise control,
simulation, and support. The COSIN provides guidelines for control and
simulation support and establishes a management structure for these
activities.

Controller/Evaluator
(C/E) Briefing

The C/E Briefing is a pre-exercise overview for controllers, evaluators, and
the exercise administrative staff. The briefing summarizes the C/E
Handbook (or the COSIN and EvalPlan) and focuses on explaining the
roles and responsibilities of controllers and evaluators.

Controller/Evaluator
(C/E) Debriefing

The C/E Debriefing provides a forum for functional area controllers and
evaluators to review the exercise. The exercise planning team leader
facilitates this debriefing, which provides each controller and evaluator
with an opportunity to provide an overview of the functional area they
observed and to discuss both strengths and areas for improvement.
During the debriefing, controllers and evaluators complete and submit their
EEGs and their Participant Feedback Forms. Debriefing results are
captured for inclusion in the AAR/IP.

Controller/Evaluator
(C/E) Handbook

The C/E Handbook specifically describes the roles and responsibilities of
exercise controllers and evaluators and the procedures they must follow.
Because the C/E Handbook contains information about the scenario and
about exercise administration, it is distributed to only those individuals
specifically designated as controllers or evaluators.

Controllers

In operations-based and some complex discussion-based exercises,
controllers plan and manage exercise play, set up and operate the
exercise incident site, and possibly take the roles of individuals and
agencies not actually participating in the exercise. Controllers direct the
pace of exercise play, provide key data to players, and may prompt or
initiate certain player actions and injects to the players as described in the
MSEL to ensure exercise continuity. Controllers issue exercise materials
to players as required, monitor the exercise timeline, and supervise the
safety of all exercise participants. Controllers are the only participants
who should provide information or direction to players. All controllers
should be accountable to one senior controller.

Core Capabilities

Distinct critical elements necessary to achieve the National Preparedness
Goal.

Corrective Action

Corrective actions are the concrete, actionable steps outlined in an IP that
are intended to resolve preparedness gaps and shortcomings experienced
in exercises or real-world events.

Homeland Security Exercise and Evaluation Program

Glossary-2

Critical Tasks

Critical tasks are the distinct elements required to perform a core
capability. Critical tasks may be derived from Mission Area Frameworks,
organizational operations plans or SOPs, or discipline-specific standards.
D

Drill

A drill is a coordinated, supervised activity usually employed to validate a
specific operation or function in a single agency or organization. Drills are
commonly used to provide training on new equipment, develop or validate
new policies or procedures, or practice and maintain current skills.
E

End of Exercise (EndEx)

The official conclusion of an exercise.

Evaluation Plan
(EvalPlan)

The EvalPlan is typically used for exercises of a large scope and scale.
An EvalPlan provides evaluation staff with guidance and instructions on
evaluation or observation methodology to be used as well as essential
materials required to execute their specific functions.

Evaluation Team

The evaluation team consists of evaluators trained to observe and record
participant actions. These individuals should be familiar with the
exercising jurisdiction’s plans, policies, procedures, and agreements.

Evaluator

Evaluators, selected from participating agencies, are chosen based on
their expertise in the functional areas they will observe. Evaluators use
EEGs to measure and assess performance, capture unresolved issues,
and analyze exercise results. Evaluators passively assess and document
players’ performance against established emergency plans and exercise
evaluation criteria, in accordance with HSEEP standards and without
interfering with exercise flow.

Event

Within the MSEL, an event is an expected action that is anticipated to take
place during an exercise.

Exercise

An exercise is an instrument to train for, assess, practice, and improve
performance in prevention, protection, mitigation, response, and recovery
capabilities in a risk-free environment. Exercises can be used for testing
and validating policies, plans, procedures, training, equipment, and
interagency agreements; clarifying and training personnel in roles and
responsibilities; improving interagency coordination and communications;
improving individual performance; identifying gaps in resources; and
identifying opportunities for improvement.

Exercise Assembly
Area

The exercise assembly area is a gathering place for all deployable
resources that will be playing in an exercise. The purpose of the assembly
area is to gather all resources and personnel near the exercise site prior to
StartEx for safety briefings, weapons checks, and to ensure that resources
and personnel are transported in a safe and unhurried manner.

Exercise Assembly
Area Controller

The exercise assembly area controller is responsible for the logistical
organization of the exercise assembly area, including placement locations
for units entering the exercise assembly area, release of dispatched units
into the field, and coordination of pathways and overall safety within the
assembly area.

Exercise Director

The Exercise Director oversees all exercise functions during exercise
conduct; oversees and remains in contact with controllers and evaluators;
debriefs controllers and evaluators following the exercise; and oversees
setup and cleanup of the exercise as well as positioning of controllers and
evaluators.

Homeland Security Exercise and Evaluation Program

Glossary-3

Exercise Evaluation
Guide (EEG)

EEGs provide a template for observing and collecting exercise data in
relation to objectives and associated core capabilities. EEGs typically
identify targets and critical tasks for exercise objectives and core
capabilities and enable evaluators to capture structured and unstructured
data regarding exercise performance. Evaluators should develop and
customize EEGs to meet the unique objectives of their exercise and to
reflect jurisdiction-specific capability targets.

Exercise Plan (ExPlan)

An ExPlan is a general information document that helps operations-based
exercises run smoothly by providing participants with a synopsis of the
exercise. It is published and distributed to the participating organizations
following development of most of the critical elements of the exercise. In
addition to addressing exercise objectives and scope, an ExPlan assigns
activities and responsibilities for exercise planning, conduct, and
evaluation. The ExPlan is intended to be seen by the exercise players
and observers—therefore, it does not contain detailed scenario
information that may reduce the realism of the exercise.
The exercise planning team is responsible for the successful execution of
all aspects of an individual exercise. The planning team determines
exercise objectives and core capabilities, creates a realistic scenario to
achieve the exercise objectives, and develops documents to guide
exercise conduct and evaluation. The planning team’s organization and
management principles should include clearly defined roles and
responsibilities and a manageable span of control.
The exercise play area is the site or facility where the bulk of tactical
player activities and tasks are demonstrated during an exercise.

Exercise Planning Team

Exercise Play Area
Exercise Play Rules

Exercise play rules are the parameters that exercise participants follow
during the exercise. Exercise play rules describe appropriate exercise
behavior, particularly in the case of real-world emergencies.

Exercise Program
Management

Exercise program management is the process of overseeing a variety of
individual exercises and supporting activities sustained over time. An
effective exercise program helps whole community stakeholders maximize
efficiency, resources, time, and funding by ensuring that individual
exercises are part of a coordinated, integrated approach to building,
sustaining, and delivering core capabilities.

Exercise Program
Manager

The exercise program manager develops a self-sustaining exercise
program through program budget management oversight, exercise
conduct, and improvement tracking monitoring and reporting.

Exercise Project
Management

Exercise project management is the act of engaging in design and
development, conduct, evaluation, and improvement planning for an
individual exercise. Effective project management ensures that all aspects
of planning and executing an individual exercise are done efficiently and
are grounded in common approaches and best practices.

Expected Action Event

An expected action event serves as a holding place in the MSEL and
notifies controllers of when an expected action should occur.
F

Facilitated Discussion

A facilitated discussion is the focused discussion of specific issues through
a facilitator with functional area or subject-matter expertise.

Homeland Security Exercise and Evaluation Program

Glossary-4

Facilitator

During a discussion-based exercise, the facilitator(s) is responsible for
keeping participant discussions on track with exercise objectives and
ensuring all issues and objectives are explored as thoroughly as possible
within time constraints. If an exercise uses breakout groups, more than
one facilitator may be needed.

Final Planning Meeting
(FPM)

The FPM is the final forum for reviewing exercise processes and
procedures. An FPM should be conducted for all exercises to ensure that
all elements of the exercise are ready for conduct. Prior to the FPM, the
exercise planning team receives final drafts of all exercise materials. No
major changes to exercise’s design, scope, or supporting documentation
should take place at or following the FPM. The FPM ensures that all
logistical requirements have been met, outstanding issues have been
identified and resolved, and exercise products are ready for printing.

Full-Scale Exercise
(FSE)

FSEs are typically the most complex and resource-intensive type of
exercise. They involve multiple agencies, organizations, and jurisdictions
and validate many facets of preparedness. FSEs often include many
players operating under cooperative systems such as the Incident
Command System or Unified Command.

Functional Exercise
(FE)

Functional exercises are designed to validate and evaluate capabilities,
multiple functions and/or sub-functions, or interdependent groups of
functions. FEs are typically focused on exercising plans, policies,
procedures, and staff members involved in management, direction,
command, and control functions. In FEs, events are projected through an
exercise scenario with event updates that drive activity at the management
level. An FE is conducted in a realistic, real-time environment; however,
movement of personnel and equipment is usually simulated.
G

Game

A game is a simulation of operations that often involves two or more teams,
usually in a competitive environment, using rules, data, and procedures
designed to depict an actual or hypothetic situation. Games explore the
consequences of player decisions and actions and are therefore excellent
tools to use when validating or reinforcing plans and procedures or
evaluating resource requirements.

Ground Truth

The ground truth is comprised of the detailed elements of a prevention
exercise scenario that must remain consistent during exercise development
and conduct to ensure that realism is maintained and objectives may be
met in the unscripted move-countermove exercise environment. The
ground truth includes the scenario timeline, local threat environment,
simulated threat group, and individual adversary profiles and relationships.
Once composed, the ground truth is used as the basis for MSEL
development and red team operations planning, if applicable.

Ground Truth Advisor

In prevention exercises, the ground truth advisor tracks how the adversary
(i.e., red team) and exercise participant moves and countermoves change
the fabric of the exercise environment, potentially creating additional truths.
To ensure consistency, each unscripted MSEL inject should be vetted by
the ground truth advisor.
H

Homeland Security
Exercise and Evaluation
Program (HSEEP)

HSEEP is a program that provides a set of guiding principles for exercise
programs, as well as a common approach to exercise program
management, design and development, conduct, evaluation, and
improvement planning.

Homeland Security Exercise and Evaluation Program

Glossary-5

Hot Wash

A Hot Wash is a facilitated discussion held immediately after an exercise
among exercise players. It captures feedback about any issues, concerns,
or proposed improvements players may have about the exercise. The Hot
Wash is an opportunity for players to voice their opinions on the exercise
and their own performance.
I

Improvement Plan (IP)

The IP identifies specific corrective actions, assigns them to responsible
parties, and establishes target dates for their completion. The IP is
developed in conjunction with the After-Action Report.

Initial Planning Meeting
(IPM)

The IPM marks the beginning of the exercise development phase. An
IPM’s purpose is to determine exercise scope by gathering input from the
exercise planning team; design requirements and conditions (e.g.,
assumptions and artificialities); objectives; extent of play; and scenario
variables (e.g., time, location, hazard selection). The IPM is also used to
develop exercise documentation by obtaining the planning team’s input on
exercise location, schedule, duration, and other relevant details.

Inject

Injects are MSEL events that prompt players to implement the plans,
policies, and procedures that planners want the exercise to validate.
Exercise controllers provide injects to exercise players to drive exercise
play toward achievement of objectives. Injects can be written, oral,
televised, and/or transmitted via any means (e.g., fax, phone, e-mail,
voice, radio). Injects can be contextual or contingency.
L

Lead Evaluator

The lead evaluator should participate fully as a member of the exercise
planning team and should be a senior-level individual familiar with all
relevant issues associated with the exercise, including plans, policies, and
procedures of the exercising organizations; Incident Command and
decision-making processes of the exercising organizations; and
interagency and/or inter-jurisdictional coordination issues relevant to the
exercise. The lead evaluator should have the management skills needed
to oversee a team of evaluators over an extended process as well as the
knowledge and analytical skills to undertake a thorough and accurate
analysis of all capabilities being tested during an exercise.

Logistics Section

The Logistics Section of the exercise planning team provides the supplies,
materials, facilities, and services that enable the exercise to function
smoothly without outside interference or disruption. This section consists
of two subsections: service and support. The service subsection provides
transportation, barricading, signage, food and drinks, real-life medical
capability, and exercise security. The support subsection provides
communications, purchasing, general supplies, management of VIPs,
observer processing, and recruitment and management of actors.
M

Master Scenario Events
List (MSEL)

The MSEL is a chronological timeline of expected actions and scripted
events to be injected into exercise play by controllers to generate or
prompt player activity. It ensures necessary events happen so that all
objectives are met. Larger, more complex exercises may also use a
procedural flow, which differs from the MSEL in that it contains only
expected player actions or events. The MSEL links simulation to action,
enhances exercise experience for players, and reflects an incident or
activity meant to prompt players to action.

Homeland Security Exercise and Evaluation Program

Glossary-6

MSEL Meeting

The MSEL Meeting may be held in preparation for more complex
exercises to review the scenario timeline and focus on MSEL
development. A MSEL Meeting can be held in conjunction with or
separate from the MPM to review the scenario timeline for the exercise.

Midterm Planning
Meeting (MPM)

The MPM is a planning meeting for exercises. It is used to discuss
exercise organization and staffing concepts; scenario and timeline
development; and scheduling, logistics, and administrative requirements.
It is also a session to review draft documentation.

Mitigation

The capabilities necessary to reduce loss of life and property by lessening
the impact of disasters.

Moderated Discussion

A moderated discussion is a facilitated, discussion-based forum where a
representative from each functional area breakout presents to participants
a summary and results from a group’s earlier facilitated discussion. During
moderated discussions, spokespersons summarize the facilitated
discussion, present key findings and issues, and discuss any unresolved
issues or questions. At the end of the moderated discussion period, the
floor is open for questions.
N

National Exercise
Program (NEP)

The NEP’s mission is to serve as the principal exercise mechanism for
examining the preparedness and measuring the readiness of the United
States across the entire homeland security enterprise by designing,
coordinating, conducting, and evaluating a progressive cycle of exercises
that rigorously test the Nation’s ability to perform missions or functions that
prevent, protect against, respond to, recover from, and mitigate all
hazards.

National Incident
Management System
(NIMS)

The NIMS standard was designed to enhance the ability of the United
States to manage domestic incidents by establishing a single,
comprehensive system for incident management. It is a system mandated
by Homeland Security Presidential Directive 5 (HSPD-5) that provides a
consistent, nationwide approach for Federal, State, local, tribal, and
territorial governments; the private sector; and nongovernmental
organizations to work effectively and efficiently together to prepare for,
respond to, and recover from domestic incidents, regardless of cause,
size, or complexity.

National Preparedness
Goal (Goal)

The National Preparedness Goal defines the core capabilities necessary
to prepare for the specific types of incidents that pose the greatest risk to
the security of the Nation. The Goal emphasizes actions aimed at
achieving an integrated, layered, and all-of-Nation preparedness approach
that optimizes the use of available resources. Specifically, the Goal
defines success as: A secure and resilient Nation with the capabilities
required across the whole community to prevent, protect against, mitigate,
respond to, and recover from the threats and hazards that pose the
greatest risk.

National Preparedness
System

The National Preparedness System is an integrated set of guidance,
programs, and processes that will enable the Nation to meet the National
Preparedness Goal.

Homeland Security Exercise and Evaluation Program

Glossary-7

O
Objectives

Objectives are the distinct outcomes an organization wishes to achieve
during an individual exercise. Objectives should reflect the exercise
sponsor’s specific needs, environment, plans, and procedures, while
providing a framework for scenario development and a basis for
evaluation. Objectives can be based on outcomes from a THIRA, from
Homeland Security Strategies, and other preparedness documents.
Planners should create objectives that are specific, measurable,
achievable, relevant, and time-bound (SMART) and should limit the
number of exercise objectives to enable timely exercise conduct, facilitate
reasonable scenario design, and support successful evaluation.

Observer

Observers do not directly participate in the exercise; rather, they observe
selected segments of the exercise as it unfolds, while remaining separated
from player activities. Observers view the exercise from a designated
observation area and are asked to remain within the observation area
during the exercise. A dedicated controller or public information officer
should be assigned to manage these groups. In a discussion-based
exercise, observers may support the development of player responses to
the situation during the discussion by delivering messages or citing
references.

Observer Briefing

An Observer Briefing is generally conducted the day of an exercise and
informs observers and VIPs about program background, scenario,
schedule of events, observer limitations, and any other miscellaneous
information. Participant Handouts and ExPlans or SitMans, depending on
the type of exercise being conducted, are often distributed during this
briefing.

Observer/Media Area

This is a designated area that provides observers and real-world media
representatives with a view of the exercise but prevents them from
interfering with exercise play.

Operations Section

The Operations Section of the exercise planning team provides most of
the technical or functional expertise for both scenario development and
evaluation. This includes development of the MSEL.
P

Participant

Participants are the overarching group that includes all players, controllers,
evaluators, and staff members involved in conducting an exercise.

Participant Feedback
Form

Players and observers receive a Participant Feedback Form after the end
of the exercise that asks for input regarding observed strengths and areas
for improvement that players identified during the exercise. Providing
Participant Feedback Forms to players during the exercise Hot Wash
allows them to provide evaluators with their insights into decisions made
and actions taken. A Participant Feedback Form also provides players the
opportunity to provide constructive criticism about the design, control, or
logistics of the exercise to help enhance future exercises. Information
collected from feedback forms contributes to the issues, observations,
recommendations, and corrective actions in the After-Action
Report/Improvement Plan.

Homeland Security Exercise and Evaluation Program

Glossary-8

Planning Meetings

Effective exercise design and development involve a combination of
exercise planning meetings. These meetings bring together the full range
of exercise stakeholders to discuss and agree on key aspects of the
exercise’s design and development. Various factors—including exercise
scope, type, and complexity—inform the types of meetings needed, and
exercise planners should tailor the planning meeting schedule to suit the
particular nature of the exercise.

Planning Section

The Planning Section of the exercise planning team is responsible for
compiling and developing all exercise documentation. To accomplish this
effectively, the Planning Section also collects and reviews policies, plans,
and procedures that will be assessed in the exercise. This group is also
responsible for planning exercise evaluation. During the exercise, the
Planning Section may be responsible for developing simulated actions by
agencies not participating in the exercise and for setting up a SimCell as
required.

Player

Players have an active role in preventing, responding to, or recovering
from the risks and hazards presented in the scenario, by either discussing
or performing their regular roles and responsibilities. Players initiate
actions that will respond to and/or mitigate the simulated emergency.

Player Briefing

A Player Briefing is held immediately before an exercise and addresses
individual roles and responsibilities, exercise parameters, safety, badges,
and any other logistical items. For a drill or full-scale exercise, Player
Briefings typically occur in the exercise assembly area.

Preparedness

The actions taken to plan, organize, equip, train, and exercise to build and
sustain the capabilities necessary to prevent, protect against, mitigate the
effects of, respond to, and recover from those threats that pose the
greatest risk to the security of the Nation.

Prevention

The capabilities necessary to avoid, prevent, or stop a threatened or actual
act of terrorism.

Prevention Exercises

Prevention exercises may focus on issues that pertain to information and
intelligence sharing, credible threats, surveillance, and/or opposing forces.

Props

Props are nonfunctional replications of objects. The presence or discovery
of props requires certain actions by exercise players. Examples of props
include simulated bombs, bomb blast debris (shrapnel), mannequins or
body parts, and foam bricks and beams. Simulants that mimic the effects
of chemical or radiological hazards or that cause a positive reading of an
actual detection device are also considered props.

Protection

The capabilities necessary to secure the homeland against acts of
terrorism and manmade or natural disasters.
R

Recovery

The capabilities necessary to assist communities affected by an incident to
recover effectively.

Response

The capabilities necessary to save lives, protect property and the
environment, and meet basic human needs after an incident has occurred.

Rolling Summary
Report

The rolling summary report is an analysis of outcomes across a series of
exercises. The report includes an analysis of issues, trends, and key
outcomes from all exercises listed in an organization’s multi-year TEP. It
is developed and updated periodically (e.g., quarterly or biennially),
depending on the number of exercises conducted.

Homeland Security Exercise and Evaluation Program

Glossary-9

Root-Cause Analysis

When evaluating exercises, root-cause analysis involves not merely
identifying what issues emerged, but rather discovering the root causes of
those issues. Root-cause analysis enables exercise stakeholders to target
how best to address areas for improvement and close capability gaps.
S

Safety Controller

The safety controller is responsible for monitoring exercise safety during
exercise setup, conduct, and cleanup. All exercise controllers assist the
safety controller by reporting any safety concerns. The safety controller
should not be confused with the safety officer, who is identified by the
incident commander during exercise play.

Scenario

A scenario provides the storyline that drives an exercise to test objectives.
The scenario selected for an exercise should be informed by the actual
threats and hazards faced by the exercise stakeholders. The exercise
scenario should realistically stress the delivery of core capabilities,
providing a mechanism for testing objectives and assessing core capability
levels and gaps.

Scope

Scope is an indicator of extent of the exercise. The key elements in
defining exercise scope include exercise type, participation level, exercise
duration, exercise location, and exercise parameters.

Seminar

Seminars generally orient participants to, or provide an overview of,
authorities, strategies, plans, policies, procedures, protocols, resources,
concepts, and ideas. As a discussion-based exercise, seminars can be
valuable for entities that are developing or making major changes to
existing plans or procedures. Seminars can be similarly helpful when
attempting to gain awareness of, or assess, the capabilities of interagency
or inter-jurisdictional operations.

Senior Controller

The senior controller is responsible for the overall organization of the
exercise. The senior controller monitors exercise progress and
coordinates decisions regarding deviations or significant changes to the
scenario caused by unexpected developments during play. The senior
controller monitors actions by controllers and ensures they implement
designated and modified actions at the appropriate time. The senior
controller debriefs controllers and evaluators after the exercise and
oversees the setup and takedown of the exercise.

Simulation

(1) An electronic simulation is a method for predicting the results of
implementing a model over time, i.e., modeling and simulation. (2)
Simulation of nonparticipating personnel and agencies is a technique for
increasing realism in exercises.

Simulation Cell
(SimCell)

A SimCell is used to generate injects for, receive player responses for, and
provide information in place of nonparticipating organizations that would
likely participate actively if exercise events were real. Physically, the
SimCell is a working location for a number of qualified professionals who
portray these nonparticipating organizations.

Simulators

Simulators are control staff personnel who role play as nonparticipating
organizations or individuals. They most often operate out of the SimCell,
but they may occasionally have face-to-face contact with players.
Simulators function semi-independently under the supervision of SimCell
controllers, enacting roles in accordance with instructions provided in the
MSEL. All simulators are ultimately accountable to the Exercise Director
and senior controller.

Homeland Security Exercise and Evaluation Program

Glossary-10

Situation Manual
(SitMan)

A SitMan is provided for TTXs and games as the core documentation that
provides the textual background for a multimedia, facilitated exercise. The
SitMan supports the scenario narrative and serves as the primary
reference material for all participants during conduct.

Special Effects

Special effects are technical, mechanical, or electronic scenario
enhancements. Special effects typically require trained and licensed
personnel, special permission for use, and additional safety and/or security
precautions. Examples include use of pyrotechnics or explosives.

Sponsor

The sponsor is the primary funding organization for an exercise.

Start of Exercise
(StartEx)

The official beginning of an exercise.

Subject-Matter Expert
(SME)

SMEs add functional knowledge and expertise in a specific area or in
performing a specialized job, task, or skill to the exercise planning team.
They help make the scenario realistic and plausible and ensure
jurisdictions have the appropriate capabilities to respond. SMEs are ideal
for the positions of controllers and evaluators.

Support Staff

The exercise support staff includes individuals who are assigned
administrative and logistical support tasks during the exercise (e.g.,
registration, catering).

Symptomatology Card

Symptomatology cards are provided to each actor in a response-focused
exercise. Each card is unique, containing the signs and symptoms the
actor will portray as well as information for medical providers. At a
minimum, symptomatology cards should include vital signs; symptoms;
trauma injuries; acting instructions (e.g., disorientation, emotional
distress); and special needs (e.g., language barriers, physical limitations).
T

Tabletop Exercise (TTX)

A TTX is typically held in an informal setting intended to generate
discussion of various issues regarding a hypothetical, simulated
emergency. TTXs can be used to enhance general awareness, validate
plans and procedures, rehearse concepts, and/or assess the types of
systems needed to guide the prevention of, protection from, mitigation of,
response to, and recovery from a defined incident. Generally, TTXs are
aimed at facilitating conceptual understanding, identifying strengths and
areas for improvement, and/or achieving changes in attitudes.

Training and Exercise
Plan (TEP)

The TEP is the foundation document guiding a successful exercise
program. The TEP articulates overall exercise program priorities and
outlines a schedule of training and exercise activities designed to meet
those priorities.

Training and Exercise
Planning Workshop
(TEPW)

A TEPW is usually conducted to create a Multi-year TEP. At a TEPW,
stakeholders work together in a collaborative workshop environment to
identify and set exercise program priorities based on core capabilities.
Based on these program priorities, TEPW stakeholders develop a multiyear schedule of specific training and exercises.
V

Venue

A venue is the primary location of exercise conduct.

Homeland Security Exercise and Evaluation Program

Glossary-11

W
Whole Community

A focus on enabling the participation in national preparedness activities of
a wider range of stakeholders from the Federal, State, local, tribal, and
territorial government, the private and nonprofit sectors (including
nongovernmental organizations), and the general public in order to foster
better coordination and working relationships. Used interchangeably with
“all-of-Nation.”

Workshop

Although similar to seminars, workshops differ in two important aspects:
participant interaction is increased, and the focus is placed on achieving or
building a product. Effective workshops entail the broadest attendance by
relevant stakeholders. Products produced from a workshop can include
new standard operating procedures, emergency operations plans,
continuity of operations plans, and mutual aid agreements. To be
effective, workshops should focus on a specific issue, and the desired
objective, product, or goal must be clearly defined.

Homeland Security Exercise and Evaluation Program

Glossary-12

Acronyms and Abbreviations
Acronym/Abbreviation
A/V
AAM
AAR
C/E
C&O
COSIN
DHS
EEG
EndEx
EOC
EvalPlan
ExPlan
FE
FEMA
FPM
FSE
HSEEP
ICS
IP
IPM
MPM
MSEL
NEP
NGO
NIMS
POC
SimCell
SitMan
SMART
SME
SOP
StartEx
THIRA
TEP
TEPW
TTX
VIP
XPA

Description
Audio/Visual
After-Action Meeting
After-Action Report
Controller/Evaluator
Concept and Objectives
Control Staff Instructions
U.S. Department of Homeland Security
Exercise Evaluation Guide
End of Exercise
Emergency Operations Center
Evaluation Plan
Exercise Plan
Functional Exercise
Federal Emergency Management Agency
Final Planning Meeting
Full-Scale Exercise
Homeland Security Exercise and Evaluation Program
Incident Command System
Improvement Plan
Initial Planning Meeting
Midterm Planning Meeting
Master Scenario Events List
National Exercise Program
Nongovernmental Organization
National Incident Management System
Point of Contact
Simulation Cell
Situational Manual
Specific, Measurable, Achievable, Relevant, Time-bound
Subject-Matter Expert
Standard Operating Procedure
Start of Exercise
Threat and Hazard Identification and Risk Assessment
Training and Exercise Plan
Training and Exercise Planning Workshop
Tabletop Exercise
Very Important Person
Extent of Play Agreement

Homeland Security Exercise and Evaluation Program

Acronyms-1