2023 OSIE Summary

0704-0610_OSIE Summary 2023_Supplementary.pdf

On-Site Installation Evaluations

2023 OSIE Summary

OMB: 0704-0610

Document [pdf]

Download: pdf | pdf

2023 On-Site Installation
Evaluation Summary

2023 On-Site Installation Evaluation Summary

The estimated cost of this report or study for the
Department of Defense is approximately
$1,185,000 for the 2023 Fiscal Year. This
includes $689,000 in expenses and $495,000 in
DoD labor.
Generated on 2023Aug04

RefID: 0-A312807

Table of Contents
Executive Summary .............................................................................................................................................. 4
Introduction ........................................................................................................................................................... 7
Methodology ......................................................................................................................................................... 7
OSIE Results ...................................................................................................................................................... 15
Appendix A: Site Selection Methodology ........................................................................................................... 28
Appendix B: OSIE Process, Integrated Prevention Metric Development, Validation, and Scoring ................... 30
Appendix C: Acronyms List................................................................................................................................ 42
Appendix D: References .................................................................................................................................... 43

Executive Summary
On February 26, 2021, Secretary of Defense Austin directed an inaugural round of on-site installation
evaluations (OSIEs) at select installations with a focus on each installation’s prevention capabilities and ability
to effectively address risk for sexual assault (SA), sexual harassment (SH), suicide, and other harmful
behaviors. This initial round of OSIEs served as a pilot for the OSIE process and to ensure an enduring
evaluation capability that could be replicated in subsequent years.
On March 30, 2022, in response to the 2021 OSIE report and findings, Secretary Austin directed future OSIEs
be conducted on a biennial basis. In accordance with the Secretary’s direction and Department of Defense
Instruction (DoDI) 6400.11, “DoD Integrated Primary Prevention Policy for Prevention Workforce and Leaders,”
OSIEs were conducted in 2023 to better understand current gaps and best practices to provide the necessary
tools to enhance prevention of harmful behaviors, including SA, SH, suicide, retaliation, domestic abuse, and
child abuse and neglect.
The 2023 OSIEs built on the successes of the 2021 OSIEs and improved the processes and methods where
necessary and applicable. The thirteen sites and twelve ships selected for 2023 OSIEs were based on newly
developed risk index scores that captured leading indicators of harmful behaviors or mitigating factors informed
by survey data and other information. The site visits occurred between January and May 2023.
The 13 sites and 12 ships visited in 2023 were as follows (individual units assessed are found in Table 2
below). Unless noted as promising, the sites and ships were selected based on high scores on the risk index:

















Army Recruiting Company, Dearborn, MI
Army Reserve Center, Ashley, PA
Camp Carroll, South Korea
Camp Casey and Camp Hovey, South Korea
Fort Liberty, NC
Fort Carson, CO
Fort Cavazos, TX
Fort Stewart, GA
Los Angeles Air Force Base (AFB), CA (Promising)
Louisiana National Guard
Malmstrom AFB, MT
Marine Corps Support Facility (MCSF) Blount Island, FL (Promising)
Naval Air Station (NAS) Lemoore, CA
Ships homeported at Naval Station (NS) Mayport, FL
o USS Carney
o USS Donald Cook
o USS St. Louis (Promising)
Ships homeported at NS Norfolk, VA
o USS Bainbridge
o USS Fort Lauderdale (Promising)
o USS Gunston Hall
o USS Kearsarge
o USS Mahan
o USS Tortuga
Ships homeported at Commander Fleet Activities (CFA) Yokosuka, Japan
o USS Blue Ridge
o USS Ronald Reagan
4

USS Howard

Findings and Actions:
The 2023 OSIEs assessed prevention capabilities and climates of sites, ships, and units of interest using
standardized information collection procedures. This generated nine validated metrics that assessed the
priority, preparation, and quality implementation of protective environments, integrated prevention, and
stakeholder engagement. Like the 2021 OSIE findings, 2023 findings indicate prevention capabilities are in an
early phase of development and, in most cases, prevention efforts do not align with best practices. These
findings underscore certain preconditions must exist to achieve mature prevention capabilities and be poised to
effectively prevent harmful behaviors. First, primary prevention must be made a priority throughout the entire
chain of command and within prevention support offices and personnel. Then, leaders and personnel must be
adequately prepared, which is largely dependent on having local subject matter experts (e.g., the Integrated
Primary Prevention Workforce (IPPW)) in place. Once these two requisites are in place, research-based
primary prevention activities can be effectively implemented.
Many findings found across the Military Services during the 2023 OSIEs are currently being addressed through
implementation of existing Independent Review Commission on SA in the Military (IRC-SAM) and Suicide
Prevention and Response Independent Review Committee (SPRIRC) recommendations, as approved by the
Secretary of Defense; however, enhancements to these efforts can be made at the OSIE sites to address more
directly several of the findings.
In general, continued implementation of DoDI 6400.09, “DoD Policy on Integrated Primary Prevention of SelfDirected Harm and Prohibited Abuse or Harm,” and DoDI 6400.11, “DoD Integrated Primary Prevention Policy
for Prevention Workforce and Leaders,” will advance capabilities at OSIE sites, as many gaps identified at
OSIE sites are addressed by requirements in those policies. Implementation of these Department of Defense
(DoD) policies, in addition to the approved IRC-SAM and SPRIRC recommendations, will ensure significant
progress toward directly addressing many of the findings in this report.
The results of the 2023 OSIEs are broken down into four main areas detailed in the 2023 report and the
Department is initiating or continuing actions in these areas to address the findings:
1. Findings that reinforce SPRIRC and IRC-SAM findings
2. Institutional factors beyond a site’s/ship’s control
3. Lack of consistent prevention priority
4. Site-specific findings
In addition, 2023 visits identified best practices that foster healthy climates and prevention efforts. These
include:


Clear communication throughout the chain of command with feedback loops for Service members and
their families



Buy-in for prevention of harmful behaviors through support and integration of prevention personnel



Supportive leaders that understand the needs and concerns of the diverse Service members they lead
and actively work to address gaps in services

implement and to attain measurable benefits. The OSIEs are one tool that assess incremental improvements
and identify additional areas for improvement. As with the 2021 OSIEs, it was expected that sites and ships
would be early in their development of prevention capabilities. The 2023 OSIEs made clear that leadership at
all levels recognize the need for, and importance of, a strong prevention system; however, they are not
effectively or consistently communicating its priority due to challenges such as high operational tempo
(OPTEMPO), manning shortages, lack of Service prevention policies and regulations, and IPPW billets
remaining unfilled. Prevention must be a consistent and concerted effort. Lack of priority has cascading
effects, leaving leaders and prevention personnel at all levels lacking preparation and the necessary skills and
competencies to effectively implement prevention.
Prevention is essential to properly assist and care for Service members and to recognize, address, and reduce
harmful behaviors across the Total Force. The Department is currently taking steps to implement existing
SPRIRC and IRC-SAM recommendations, as approved by the Secretary of Defense, that address many OSIE
findings. The Department has taken unprecedented steps to develop and resource integrated prevention, but
to maximize those investments, complete buy-in from leadership at all levels is required. The Office of the
Secretary of Defense (OSD), Military Departments (MILDEPS), and National Guard Bureau (NGB) must
continue to reinforce the importance of prevention for the well-being of Service members. Prevention of
harmful behaviors is an imperative investment in the future of the Department, strengthening readiness and
resiliency across the Total Force.

Introduction
As Secretary of Defense Austin noted in his September 2022 memorandum, “Taking Care of our Service
Members and Families,” the Department has a sacred obligation to take care of its Service members. To meet
this obligation, the DoD is committed to creating safe environments where all Service members can thrive.
The DoD has conducted multiple independent reviews in the past five years. These include the “Report of the
Fort Hood Independent Review Committee,” the IRC-SAM, and the SPRIRC. These reviews highlighted the
need for continuing evaluations to ensure the Department is succeeding in its efforts. In a March 30, 2022,
memorandum, Secretary Austin directed OSIEs be conducted on a biennial basis. The OSIEs are coordinated
site visits to a select group of military sites/ships to assess the development of prevention capabilities and
support the enhancement of the Department’s command climate efforts. OSIEs allow the Department to better
identify and understand current gaps in prevention and prevention support efforts and provide
recommendations to enhance prevention capabilities and continually improve command climate.
The 2021 OSIEs served as a pilot for OSD to rollout the OSIE framework, methodology, and processes. One
of the largest successes of the initial OSIEs was the affirmation of the OSIE framework described below and in
Table 4. The framework was built around the three areas of priority, preparation, and implementation, and has
proven to be a solid and valuable means of understanding and assessing site/ship prevention capabilities and
climate. The 2021 and 2023 OSIEs show these three framework areas are clearly interconnected in ways that
make each necessary to achieve positive prevention outcomes.
Throughout the implementation of the 2023 OSIEs, the OSIE mission of oversight, detection of risk, assessing
priority, preparedness, and evaluating implementation has been highlighted as essential to identifying where
and how necessary change is failing to be implemented or is stalled. This summary seeks to highlight how the
Department must improve prevention efforts and offers change-oriented actions and strategies to continue to
care and provide a safe environment for its Service members.
From January to May 2023, OSIEs were conducted at 13 sites and 12 ships. For sites, the units with the
highest risk or protective scores within each site (Table 2) were evaluated. In addition, the helping agencies1
and prevention personnel that supported these units and ships were also assessed.

Methodology
The focus of the OSIEs is on integrated primary prevention for the military community. The following
definitions guided the methods used to identify sites/ships and develop evaluation metrics.
Table 1: OSIE Prevention Definitions

Primary
Prevention

Stopping harmful acts before they occur. Can be implemented for an entire
group or population without regard to risk (universal primary prevention) or
can be implemented for individuals, groups, or a population that is at risk
(selected primary prevention).
Primary prevention activities can target:
1. Influencers, such as leaders who set a climate and shape norms, but
may not be present when harmful acts occur
2. Bystanders, who may be present when harmful acts occur
3. Individuals who may commit harmful acts

Agencies assessed include Integrated Prevention, Equal Opportunity, Family Advocacy, Sexual Assault Prevention and Response, Substance Abuse
Prevention, Family Readiness, Chaplain, Judge Advocate, Inspector General, Military Police/Criminal Investigators, Suicide Prevention, and Behavioral
Health.

4. Individuals who may be affected by harmful acts
Integrated
Primary
Prevention

Prevention activities that simultaneously address multiple self-directed harm
and prohibited abusive or harmful acts or the inclusion of prevention activities
across self-directed harm and prohibited abusive or harmful acts into a
cohesive, comprehensive approach that promotes unity of effort, avoids
unnecessary duplication, and lessens training fatigue.

Military
Community

All individuals (e.g., Service members, DoD civilian employees, dependents)
who live and work together in the same geographic area, such as a DoD
installation.
Military community exists based on relationships and the potential to interact
with one another regardless of Service affiliation and chain of command.

Identification of OSIE Sites
For 2023 site selection, a five-factor risk index was used to assess a range of risk and protective factors across
the social ecology that may impact a site’s/ship’s risk for harmful behaviors.

Sites Identified
The following table summarizes the sites/ships identified by OSIEs based on their OSIE risk index score, the
risk index score for the overall site/ship and, for sites, the Defense Organizational Climate Survey (DEOCS)
risk and protective percentile score for each unit that participated in the OSIE. Ships homeported at NS
Mayport, NS Norfolk, and CFA Yokosuka were treated as stand-alone sites for the purposes of the OSIEs and,
thus, have a risk index score and not DEOCS protective/risk percentile scores for units of interest.
The scope of the site visits included units within each site that had the highest risk or protective percentile
scores on the DEOCS, as well as the helping agencies and leadership, typically at the installation level or
within a higher-level command, that supported those units’ prevention and response efforts. At large
sites/ships, site visits assessed only a small portion of the total military community. At small sites/ships, site
visits may have included most of the military community. In sites where assessed units included those with
high risk and high protective percentile scores, consistency of findings across these units provided clues about
whether highlighted findings were widespread across the site or localized within those units.
Factors contributing to risk and protective factors that informed site/ship and unit identification are varied and
complex. Assumptions about the environment, morale, leadership, etc. should be avoided, and a focus on the
OSIE findings is recommended.
Table 2: Risk Index Score and DEOCS Unit Protective and DEOCS Risk Percentile Scores for Participating Sites, Ships, and Units 2
0F

Site, Ships, and Units of Interest
Ashley, PA Reserve Center
340th Military Police Battalion* 4
362d Military Police Detachment
Camp Carroll
Headquarters and Headquarters Detachment,
25th Transportation Battalion
2F

Risk Index
Percentile 3

Protective
Percentile Score

Risk Percentile
Score

86
--81

-81
27
--

-27
69
--

Ships were treated as sites, so do not have DEOCS risk and protective percentile scores. The broader OSIE risk index was used to select ships of
interest.
3
Risk index percentiles are based on data as of August 3, 2023.
* Denotes site, ship, or unit that was identified as promising based on the risk index or risk/protective percentile scores.

95th Transportation Company
541st Quartermaster Company
B Battery, 2-1 Air Defense Artillery
Headquarters and Headquarters Battery, 2-1
Air Defense Artillery
Camp Casey
Camp Hovey
6-37 Field Artillery
1-38 Field Artillery
579th Forward Support Company
CFA Yokosuka
USS Blue Ridge
USS Ronald Reagan
USS Howard
Dearborn Recruiting Company
Dearborn Recruiting Company
Fort Carson
110th Military Police Company
A Company, 3-10 Special Forces*
B Battery, 3-29 Field Artillery
F Company, 704th Base Support Battalion
US Army Garrison-Fort Carson*
Fort Cavazos
74th Engineer Company
120th Quartermaster Company
E Company, 115th Base Support Battalion
Headquarters Support Company, 615th
Aviation Support Battalion
Rear Detachment, 64th Military Police
Company
Fort Liberty
F Company, 98th Civil Affairs Battalion*
D Company (Forward Support), 3d Special
Forces Group*
A Company, Womack Army Medical Center
25th Quartermaster Company
I Company, 407th Brigade Support Battalion
Fort Stewart
287th Quartermaster Company
A Company, 10th Engineer Battalion
C Company, 3-69 Armor Battalion
E Company, 3d Base Support Battalion
I Company, 703d Base Support Battalion
Louisiana National Guard
1-199 Training Regiment
A Company, 769th Engineer Battalion
B Company (Rear Detachment), 199th Base
Support Battalion
Forward Support Company, 527th Engineer
Battalion
Los Angeles AFB*
61st Communications Squadron (AFSPC)
9

----

20
10
13

75
97
77

80
89
---69
99
99
99
94
-95
-----95
----

--13
20
17
-----13
-10
91
7
3
91
-2
7
3

--91
82
83
-----78
-90
26
96
94
9
-93
93
92

93
---

-95
97

-17
16

--94
-----83
----

4
4
7
-2
9
8
4
8
-15
21
9

96
95
95
-97
93
94
97
93
-84
88
88

17
--

-60

-35

61st Civil Engineer and Logistic Squadron*
Space-Missile Systems (Enterprise Corps)
Space-Missile Systems
Space-Missile Systems
Malmstrom AFB
10th Missile Squadron
12th Missile Squadron*
341st Security Forces Group*
341st Security Forces Squadron
819th Red Horse Squadron
MCSF Blount Island*
Blount Island Command*
Blount Island Command*
NAS Lemoore
Center for Naval Aviation Technical Training
Unit-Lemoore *
Carrier Air Wing 9*
Strike Fighter Weapons School-PAC*
Strike Fighter Squadron 41
Strike Fighter Squadron 122
Strike Fighter Squadron 125
NS Norfolk
USS Bainbridge
USS Fort Lauderdale*
USS Gunston Hall
USS Kearsarge
USS Mahan
USS Tortuga
NS Mayport
USS Carney
USS Donald Cook
USS St. Louis*

----75
-----14

87
64
63
73
-77
88
83
3
10
--

10
35
37
20
-24
24
23
90
85
--

92
--

-77

-37

-----89
100
7
99
98
98
99
82
100
98
27

79
78
23
25
26
-------

31
25
75
82
79
-------

-----

OSIE Risk Index
Data sources and DEOCS factors were categorized into five levels or domains based on the social ecological
model. A social ecological model is a public health framework used to understand the complex interaction
between the individual, interpersonal, organizational, and community factors that affect a person’s overall
health and well-being.5 This framework enables preventionists to better understand the factors that contribute
to incidents of harm and abuse and opportunities to mitigate the harmful behaviors by addressing the
contributing factors. To create environments free from harm and abuse, it is necessary to enhance protective
factors and reduce risk factors at every level of the social ecological model. 6
4F

The traditional levels of the social ecological model were adapted to reflect a site’s/ship’s setting in which a
Service member is embedded in an existing chain of command or leadership structure. The social ecological

5
Bronfenbrenner, U. (2005). Ecological systems theory (1992). In U. Bronfenbrenner (Ed.), Making human beings human: Bioecological perspectives on
human development (pp. 106–173). Sage Publications Ltd.
6
The Centers for Disease Control and Prevention. (2022). “The Social-Ecological Model: A Framework for Prevention.”
https://www.cdc.gov/violenceprevention/about/social-ecologicalmodel.html

model used to produce the site/ship risk index reflects risk and protective factors at five different levels (see
Figure 1):
1. Individual (e.g., individual behaviors, attitudes)
2. Workplace (e.g., work peers, interpersonal teams)
3. Leadership (e.g., organizational factors controlled by the command team or supervisor)
4. Installation (e.g., installation historical prevalence or incidence rates)
5. Community (e.g., health and safety trends in the surrounding civilian community)
Figure 1: Five Domains of the OSIE Risk Index 7
5F

These five domains constitute a robust social ecological model tailored for the military environment. A “best fit”
approach was used that placed each risk and protective factor into a single level of the social ecological
framework. More information on the five domains can be found in Appendix B.
Each site profile in Appendix A includes the risk percentiles for the five domains. Higher percentiles denote
higher risk.

Data Sources
The following data sources were used to develop the OSIE risk index.
DEOCS 5.0
DEOCS 5.0 assesses 19 protective and risk factors that can impact a unit’s/organization’s climate and ability to
achieve their mission.
Protective factors are attitudes, beliefs, and behaviors associated with positive outcomes for units or
organizations. Higher favorable scores on protective factors are linked to a higher likelihood of positive
outcomes, such as improved performance, increased readiness, and higher retention, and are also linked to a
lower likelihood of negative outcomes, such as suicide, SH, and SA.

All acronyms in Figure 1 are spelled out in the Data Sources subsection below.

Risk factors are attitudes, beliefs, and behaviors associated with negative outcomes for units or organizations.
Higher unfavorable scores on risk factors are linked to a higher likelihood of negative outcomes, such as
suicide, SH, and SA, and are also linked to a lower likelihood of positive outcomes such as increased
performance, higher readiness, and elevated retention.
Certain leadership factors are measured for different leadership levels and were treated separately in this
analysis. Specifically, transformational leadership and passive leadership are measured for both the
unit/organization leader and the senior noncommissioned officer (NCO). Toxic leadership is measured for the
immediate supervisor and senior NCO. Leadership support is measured for the immediate supervisor only.
Risk and protective factors are shown in Table 3.
Table 3: Risk and Protective Factors from DEOCS 5.0

DEOCS 5.0 Risk Factors
Alcohol Impairing Memory
Binge Drinking
Stress
Passive Leadership
Toxic Leadership
Racially Harassing Behaviors
Sexually Harassing Behaviors
Sexist Behaviors
Workplace Hostility

DEOCS 5.0 Protective Factors
Cohesion
Connectedness
Engagement and Commitment
Fairness
Inclusion
Morale
Safe Storage for Lethal Means
Work-Life Balance
Leadership Support
Transformational Leadership

2018 Workplace and Gender Relations Survey of Active Duty Members (WGRA) – Contextual Analysis
Office of People Analytics’ (OPA’s) 2018 WGRA provides insights regarding the estimated prevalence and
characteristics of SA, SH, and gender discrimination in the Active Component based on Service members’ selfreported experiences, including perceptions of unit culture and climate.
2018 Defense SA Incident Database (DSAID)
DSAID is the Department’s authoritative, centralized database used to collect and maintain information about
SA cases involving members of the U.S. Armed Forces.
2020-2022 Defense Suicide Prevention Office (DSPO) Suicide Counts
DSPO provided the OSIE team with a record of every military suicide from 2020 through the first quarter of
2022
2021 Family Advocacy Program (FAP) Domestic and Child Abuse Counts
The OSIE team used aggregate counts of domestic abuse and child abuse and neglect incidents reported to
FAP in Fiscal Years 2019-2021 by installation. 8 Only data that met criteria for the Department’s definition of
abuse per DoDI 6400.01, “Family Advocacy Program (FAP),” were used. FAP is the DoD’s program
designated to prevent and respond to domestic abuse and child abuse and neglect.
8F

2022 U.S. County Health Rankings and Roadmaps (CHR&R)9
The CHR&R compiles U.S. local health data to help communities identify opportunities to improve their health.
The CHR&R spans several health focus areas: length of life, quality of life, tobacco use, diet and exercise,
8
9

Rates of domestic abuse, child abuse, and child neglect are not applicable to ships as those incidents are counted at the homeport installation.
Community refers to the community at large surrounding the installation and not just the military community.

alcohol and drug use, sexual activity, access to clinical care, quality of clinical care, education, employment,
income, family and social support, community safety, air and water quality, housing and transit, and
demographics.

On-Site Evaluation and Framework of Prevention Capability Assessment
OSIE teams conducted site visits that spanned several days and included focus groups, interviews, and
surveys across all helping agencies (e.g., integrated prevention personnel, prevention support personnel),
leadership teams, and Service members. These focus groups, interviews, and surveys collected data on
integrated prevention efforts related to harmful behaviors such as SA, harassment, retaliation, suicide,
domestic abuse, and child abuse and neglect.
The teams included Service members and civilian employees, which allowed for a mixture of military
perspectives and insight into Service and site/ship culture.

Prevention Capability Assessment Methods:
In 2021, OSD, in collaboration with RAND, used identified nine dimensions (see

Table 4) to guide the assessment of prevention capabilities for the OSIEs. These dimensions were identified
through an analysis of the focus areas not covered by existing DoD compliance checklists and DoD
assessment tools to enforce relevant prevention policies, as well as the OSIE framework outlined in the 2021
OSIE Report.
OSD prioritized three domains of focus for the development of new metrics:


Healthy and Protective Environment: Research shows that command climates can positively or
negatively impact behaviors such as SA and SH



Integrated Prevention: Effective prevention targets a mix of risk and protective factors that are both
common across issue areas as well as unique to specific harmful behaviors



Stakeholder Engagement: Outcomes can be improved when multiple stakeholders have genuine
involvement in prevention activities

Three additional domains were added from the OSIE framework:


Priority: Higher-level leadership sets the tone and sustains consistent focus on harmful behaviors



Preparation: Prevention personnel and intermediate leadership are equipped with the knowledge,
skills, and abilities (KSAs), and exist within a structure, that incentivize and support addressing harmful
behaviors



Implementation: Approach aligns with best practices and is done well (i.e., with high quality)

Crossing the three domains from the OSIE framework (columns in Table 4) with the three domains (i.e., focus
areas) in existing compliance checklists and assessment tools (rows in Table 4) yielded a matrix of nine
dimensions to be included in the assessment.

Table 4: Prevention Capabilities Assessed in OSIEs

OSIE FRAMEWORK AREA

FOCUS AREAS

HEALTHY AND
PROTECTIVE
ENVIRONMENT

INTEGRATED
PREVENTION

STAKEHOLDER
ENGAGEMENT

PRIORITY

PREPARATION

IMPLEMENTATION

(1)
Leaders prioritize
fostering a protective
environment by their
actions and
communications.

(2)
Leaders have the
requisite KSAs and
access to training to
develop those KSAs.

(3)
Leaders employ best
practices known to
support a protective
environment.

(4)
Leaders prioritize
prevention activities.

(5)
Leaders and
prevention personnel
have the requisite
KSAs to carry out
prevention
successfully.

(6)
Prevention activities
that target risk and
protective factors
across multiple harmful
behaviors are
evaluated.

(7)
Leaders prioritize
engaging stakeholders.

(8)
Prevention personnel
have the resources
and requisite KSAs to
engage stakeholders
effectively.

(9)
Stakeholders are
genuinely engaged in
prevention activities
across multiple
planning stages.

OSIE Results
Similar to the 2021 OSIE findings, 2023 findings indicate prevention capabilities are in an early phase of
development at the sites/ships visited and, in most cases, prevention efforts do not align with best practice.
These findings underscore certain preconditions must exist to achieve mature prevention capabilities and be
poised to effectively prevent harmful behaviors. First, primary prevention must be made a priority throughout
the entire chain of command and within prevention support offices and personnel. Then, leaders and
personnel must be adequately prepared, which OSIEs found is largely dependent on having local subject
matter experts (SMEs) (e.g., the IPPW) in place. Once these two requisites are in place, effective researchbased primary prevention activities can be implemented.
While institutional change can be slow and difficult, the estimated rates of SA, SH, and suicide as reported in
recent years necessitate a Department-wide effort to integrate meaningful change into policies, procedures,
and processes that may create or exacerbate risk factors. High OPTEMPO, manning shortages, and
communication challenges all serve as barriers to even the most committed leadership who prioritize Service
members receiving the care they need and deserve; however, it is imperative for leadership across the
Department to focus on and prioritize prevention to properly care for Service members.
Many findings found across Services during the 2023 OSIEs are currently being addressed through
implementation of existing IRC-SAM and SPRIRC recommendations, as approved by the Secretary of
Defense; however, enhancements to these efforts can be made at the OSIE sites to address more directly
several of the findings.
15

In general, continued implementation of DoDI 6400.09, “DoD Policy on Integrated Primary Prevention of SelfDirected Harm and Prohibited Abuse or Harm”, and DoDI 6400.11, “DoD Integrated Primary Prevention Policy
for Prevention Workforce and Leaders,” will advance capabilities at OSIE sites, as many gaps identified at
OSIE sites are addressed by requirements in those policies. Implementation of these policies, in addition to
the implementation of the approved IRC-SAM and SPRIRC recommendations, will ensure significant progress
toward directly addressing many of the findings in this report.
Therefore, taking into account current actions underway, results of the 2023 OSIEs are broken down into four
main areas below:
1. Findings that reinforce actions underway consistent with the approved SPRIRC and IRC-SAM
recommendations
o

Examples of findings in this domain include:
‒

Service members do not have confidence that complaints or reports are taken seriously due
to perceived lack of accountability.

‒

Challenges accessing behavioral health and lack of healthy options for social engagements
is exacerbating risk for harmful behaviors.

‒

Slow implementation of Department prevention policy is delaying positive changes in
prevention efforts, given that full-time personnel are needed to support leaders in command
climate assessments and implement research-based prevention efforts.

2. Institutional factors beyond a site’s/ship’s control
o

In addition to findings related to prevention and climate, OSIE teams often identify contextual or
institutional factors, outside of the site or ship’s control, that may exacerbate risk for harmful
behaviors or create barriers for effective prevention. Examples of findings in this domain include:
‒

Operational tempo and manning shortages across the force are driving increased stress,
impacting command climate and the prevention environment, which then have deleterious
effects on recruiting, readiness, and retention.

‒

Cost of living continues to impact remote areas with no military housing and limited
healthcare access.

‒

Lack of FAP resources for Reserve Component personnel and their families creates a gap
in services for seeking care and assistance before, during, or after incidents of violence or
neglect.

‒

Recruiters are under tremendous stress and operate in geographically dispersed and often
remote areas.

‒

Gaps in services and challenges with TRICARE Prime Remote create significant barriers
that keep recruiters and their families from receiving consistent physical and mental health
care.

‒

Rollout issues are common when deploying new personnel systems which negatively impact
Service members’ pay. Additional financial readiness and emergency loan education should
be highlighted by leadership prior to system changes.

‒

Adaptations to the prevention and response workforce models are needed to provide
prevention support and response services in deployed and geographically dispersed units.

‒

MILDEP and Service policies and regulations are needed that provide guidance and
authority for implementation of DoDIs 6400.09 and 6400.11, clearly delineate prevention
16

workforce roles within each Service, and take into consideration the unique needs of the
Reserve Component.
‒

Unique gaps in behavioral health care continue to exist on ships and in remote locations.

3. Lack of consistent prevention priority
o

As the Department is currently investing in integrated prevention, OSIE findings underscore a
critical and foundational first step is prioritizing prevention efforts to have long-term change.
Leadership sets the tone and focus for prevention of harmful behaviors; however, at the selected
OSIE sites, the consistent focus on prevention was lacking. Importantly, the OSIEs identified
many factors affecting the sites/ships that were out of their control; however, all leaders and OSIE
sites can immediately implement actions that prioritize prevention. Examples of findings in this
domain include:
‒

Senior leaders’ vision for healthy climates is not consistently communicated through the
chain of command, leading to perceived lack of emphasis on prevention of harmful
behaviors.

‒

Perceived barriers to help-seeking, such as limiting the times/days Service members can
make appointments, sends mixed messages about the priority placed on Service members’
well-being.

4. Site-specific findings
o

Examples of findings in this domain include:
‒

Lack of onboarding, initial orientation, and sponsorship efforts for Service members new to a
site result in decreased connection to the mission and already-onsite personnel.

‒

Lack of communication and awareness regarding a loss of a unit member decreased trust in
leaders and perceived value of Service members.

‒

Prevention personnel are actively working to establish collaborative relationships with NCOs
and program specialists but require leadership support to encourage and maintain
connections.

5. Characteristics of promising sites
o

In 2023, four promising sites/ships (Table 2) were visited, which are sites/ships with overall high
protective percentile scores. Several promising units (Table 2), which are units with overall high
protective percentile scores, were also visited.

There are several key characteristics that distinguish promising sites/ships and units.
Understanding characteristics associated with promising sites/ships helps to increase protective
factors and improve prevention activities and prevention support across the Department.

Clear Communication:

‒

Briefings and discussions are tailored to specific groups (e.g., commanding officer tailors
townhall by rank).

‒

Small group trainings reflect a positive culture of addressing harmful behaviors head-on.

‒

Leaders take the time and effort to explain the “why” behind a request, event, or requirement
to provide greater understanding of, and connection to, the mission.

‒

Generational differences are acknowledged, and clear, mutual understanding is sought.

Prevention Buy-In:
17

‒

Senior leaders are engaged and showcase high prioritization of prevention.

‒

Mid-level leaders are encouraged to have a vision regarding positive and safe work
environments, and are given “white space” to build them.

‒

Junior enlisted members understand the importance of prevention and actively seek to
become engaged.

‒

IPPW is effectively integrated with prevention support services.

Supportive Leaders:
‒

Service members are fully supported regardless of gender, role in unit, or other discerning
features.

‒

Change is accepted, managed, and supported instead of being viewed as a feared
disruption.

‒

Leaders at all levels are encouraged to participate in community/prevention meetings and
are encouraged to apply the data presented to develop interventions at the unit level.

‒

Gaps in services are quickly identified and actively managed to limit disruption to quality of
life.

‒

Leaders ensure subordinates and peers have a meaningful work-life balance.

DoD Actions to Address Findings that Align to IRC-SAM and SPRIRC
To address findings that aligned to approved IRC-SAM and SPRIRC recommendations, the Department will
continue its swift and thoughtful implementation of those efforts.

MILDEP and NGB Actions to Address Institutional Factors
[MILDEPS/NGB] Promote DoD SPARX Connection Prevention Community of Practice among prevention
workforce supporting Reserve Component locations to ensure National Guard and Reserve prevention
program managers collaborate, share information about resources, and identify needs and sources of data.
(Integrated Prevention)
[MILDEPS/NGB] Collaborate with servicing acquisition offices to explore options relating to contractors
working on a site or ship, if any. (Healthy and Protective Environment)
[Department of Army] Explore and implement, as appropriate, additional data/metrics to evaluate recruiters'
workload and performance, highlight best practices per region, and better understand how taking care of the
recruiters will impact the mission. (Healthy and Protective Environment)
[Department of Army] Evaluate recruiting specific pre-command training to ensure it adequately addresses
strategies to provide support to personnel in geographically dispersed locations. (Stakeholder Engagement)

Actions to Address Lack of Consistent Prevention Priority
Several requirements included in DoDI 6400.11, “DoD Integrated Primary Prevention Policy for Prevention
Workforce and Leaders,” still pending implementation, such as professional military education and other
appropriate leadership development opportunities that prepare individuals to lead and support integrated
primary prevention, are designed to help improve prevention priority. In addition, the following actions apply to
military leaders at all sites/ships with a specific focus on improving priority scores.


Complete DoD SPARX Knowledge Part 1: A one-hour developmental course for prevention personnel
and leaders to build foundational prevention knowledge and skills.



Review DoD SPARX Leadership Action Toolkit (available on www.prevention.mil) and DoD SPARX
Knowledge Nuggets: Tools for leaders to develop a deeper understanding of harmful behaviors and
how to be engaged prevention partners.



Following completion of DoD SPARX tools, host quarterly small group discussions with subordinate
leaders to communicate the importance of integrated prevention (e.g., on addressing shared risk and
protective factors for multiple harmful behaviors) and discuss opportunities for prevention of harmful
behaviors and methods to facilitate communication up and down the chain of command, enhance
healthy command climates, and ensure leadership’s vision and intent is carried throughout the
organization consistently and over time.

Site-Specific Findings
Certain OSIE findings were unique to a site or applied to only a few sites. Following are those findings broken
down by each site and grouped based on the three integrated prevention areas of Protective Environments,
Integrated Prevention, and Stakeholder Engagement.
Table 5: Ashley Army Reserve Center Findings

Protective Environments
 Increase efforts to strengthen connectedness such as enhancing sponsorship programs and created
standardized in-processing that encourages unit cohesion
Integrated Prevention
 Enhance collaboration between prevention program managers and both internal and external
collaborators
Stakeholder Engagement
 Provide Soldiers with updated, easily accessible contact data for critical services and, where possible,
provide resource information local to the Soldier’s home residence
Table 6: Camp Carroll Findings

Protective Environments
 Foster use of “This is My Squad” (TIMS), which allows Soldiers to talk to leadership about any topic other
than work
o TIMS helps breakdown personal barriers and offers opportunities to discuss new programs
o Junior Soldiers who may not want to talk with behavioral health personnel may be more willing to
present concerns to unit leaders
 Implement on-site “office hours” for distant support services at Camp Carroll so Soldiers do not need to
travel to another location, and to help foster a “drop in and talk” mentality
Integrated Prevention
 Encourage the use of guest speakers at the end of formations to build awareness of resources and to
reinforce the importance of prevention
 Foster development of healthy relationship skills (e.g., communications, problem recognition skills) so
Soldiers can recognize potential problems and act
Stakeholder Engagement

 Build faith in the chain of command through action; demonstrate leadership’s vision for prevention by
engaging with small groups and taking on problem behaviors head-on
 Foster reinvigoration of the Better Opportunities for Single Soldiers (BOSS) program, which was seen as a
valuable resource by junior officers who recognized NCO involvement in BOSS as building positive
command climates
Table 7: Camp Casey and Camp Hovey Findings

Protective Environments
 Identify opportunities and best practices to quickly assimilate new arrivals
 Evaluate Morale, Welfare, and Recreation (MWR) leisure and travel opportunities for alignment with junior
Soldiers’ interests (e.g., e-sports venues vs. city tours)
 Assess the breadth and depth of marital challenges and available support resources. Subsequently, FAP
should develop research-based prevention activities tailored to the population
 Develop an orientation process at Camp Casey that highlights local garrison resources
Integrated Prevention
 Incoming leaders should tap into civilian personnel who offer extensive and enduring knowledge on
unique local problems
 Develop a plan for short- and longer-term engagement with the integrated prevention team at Camp
Humphreys
 Identify opportunities to share lessons learned and insights from other areas within Korea
Stakeholder Engagement
 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members
Table 85: CFA Yokosuka Findings

Protective Environments
 Promote and expand access to healthy leisure activities as a prevention measure against harmful
behaviors
 Identify additional research-based measures to manage stress for Sailors in departments disproportionally
impacted by high OPTEMPO
 Review the administrative assignment process and revise as needed to ensure gender-neutral
assignments
 Implement prevention activities into daily activities to provide training in small doses and to consistently
communicate the importance of prevention
Integrated Prevention
 Promote collaboration and enhance communication between shore side resources and ship assets to
ensure continuity and integration of prevention efforts
 Develop evidence-informed processes and opportunities for shore side and other resources to collaborate
and integrate prevention activities
Stakeholder Engagement

 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members
 Enhance peer support networks while maintaining communication channels remain open and available
 Review current communication methods and employ diversified social media platform use to ensure
effectiveness of communication to those Sailors in high-risk categories
 Identify mechanisms to sustain and institutionalize recognition and reward of Sailors’ achievements and
good conduct through traditional and creative means (e.g., music choice on Friday)
Table 9: Dearborn Recruiting Company Findings

Protective Environments
 Prioritize care-seeking over routine recruiting tasks
Integrated Prevention
 Fully staff the Soldier and Family Assistance roles to include the assistant position
o Clarify position duties, encourage boundaries, and create connections to share responsibilities
 Maintain and train on a digital Sexual Harassment/Assault Response and Prevention continuity tool that
leverages capabilities and information in the “WeCare” app to increase access to support
 Work with other local Services (e.g., National Guard, Army Reserve, Navy Reserve) to identify MWR and
support resources for each station to establish broad community connections in alignment with the
approved SPRIRC recommendation 5.27
Stakeholder Engagement
 Provide regular updates to Soldiers on command climate findings and actions taken to address concerns,
to increase accountability
 Leverage U.S. Army Recruiting Command (USAREC) and brigade leaders in planned messaging that
acknowledges challenges and provides support resources (e.g., USAREC “WeCare” app)
 Provide a quarterly resource update that addresses trending topics and provides solutions
Table 10: Fort Carson Findings

Protective Environments
 Prioritize care-seeking over routine maintenance and other non-critical tasks
 Identify ways to increase access to community resources and safety through after-hours shuttles or oncall designated drivers
Integrated Prevention
 Increase technical assistance for evaluation of prevention efforts to measure quality and impact
Stakeholder Engagement
 Engage Soldiers and increase awareness of prevention activities and resources before there is a crisis
 Use the leadership competencies in DoDI 6400.11 to provide training and mentorship for mid-level leaders
so they become more effective in understanding their Soldiers’ stressors
 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members

Table 11: Fort Cavazos Findings

Protective Environments
 Prioritize care-seeking over routine maintenance and non-critical tasks
 Minimize negative repercussions for scheduling care appointments when services are needed and
available
Integrated Prevention
 Continue to expand the use of the People First Center beyond vignette-based training and create a onestop shop for all things prevention and response
Stakeholder Engagement
 Leverage generational differences in knowledge and skills (i.e., technological expertise) to enhance
innovation within career fields/daily tasks and foster fresh ideas
Table 12: Fort Liberty Findings

Protective Environments


Conduct an internal review of practices and local policies regarding when Service members with
suicidal ideations may be required to participate in duty-related situations that pose an increased risk
for self-harm or increased access to lethal means



Institute “Kitchen Police” rotations to mitigate the impact of manning shortages and shift work



Ensure all tenant organizations (e.g., Special Operations Command units) are sharing appropriate
risk/protective factor data and are aware of community resources and are encouraged to use them
Integrated Prevention

 Establish service sharing agreements and support personnel office hours at geographically dispersed
locations to ensure greater access to support
Stakeholder Engagement
 Leverage affinity groups and unit-level engagement opportunities to better connect with unit members
Table 13: Fort Stewart Findings

Protective Environments
 Review barracks work orders related to physical security to assess process efficiency and completion
timelines
 Ensure Soldiers are aware of applicable privately owned weapons/ammunition policies and safe-handling
procedures
Integrated Prevention
 Analyze IPPW and program specialist requirements in DoDI 6400.11 and clarify roles and relationships
between the IPPW and program specialists
Stakeholder Engagement
 Review newcomer orientation and other opportunities and apply findings to inform Service members and
leaders of all available prevention and prevention support resources and how to access them
Table 14: Louisiana National Guard Findings

Protective Environments
22

 Promote a battle rhythm for weekend drill efforts that constructively uses time to enhance knowledge,
skills, and abilities
 Prioritize and equip members to make it to drill
o Facilitate alternate drill days or assist with identifying carpooling options/resources and childcare
arrangements, when possible
o Local commands should engage Service Member and Family Programs (SMFP) to leverage resources
they have (e.g., financial and family readiness) to assist with difficulties Service members have during
drill or training
o Evaluate the feasibility of expanding the Camp Beauregard childcare support model to other locations
across the state and consider early adoption of the National Guard Weekend Drill Childcare Pilot
Program
o The Adjutant General should request inclusion into the Weekend Drill Childcare Pilot Program prior to
the full program rollout in Fiscal Year 2025
o States should leverage the Army Fee Assistance Program when eligible and consider partnerships with
organizations under a Memorandum of Agreement construct for childcare, when possible
Integrated Prevention
 IPPW should continue to participate in needs assessment training, continue utilizing the Integrated
Primary Prevention Tool, and leverage national level support from NGB to inform prevention activities
 Actively promote the newly developed SMFP mental health support capabilities across the state
 Expand the SMFP footprint to increase availability of services across the state
 Empower NCOs to participate in and collaborate with the prevention workforce
 Complete a new standard operating procedure to:
o Raise awareness of available resources
o Reinforce opportunities for prevention personnel engagement
o Establish clear communication, understanding, and transparency between prevention support services
o Implement measurement standards to evaluate performance
o Use SMEs to facilitate trainings
Stakeholder Engagement
 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members
 Redefine “handle things at the lowest level” to encourage information sharing vice information harboring
Table 15: Los Angeles AFB Findings

Protective Environments
 Review newcomer orientation and other opportunities and apply findings to inform Service members and
leaders of all available prevention and prevention support resources and how to access them
Integrated Prevention
 Encourage mid-level leadership to lay out the vision for positive and safe work environments
 Review the implementation and delivery impact of the Space Force receiving prevention support through
the Air Force
23

Stakeholder Engagement
 Identify and address the different needs or concerns of long-term civilian employees vs. newly arrived
Service members (e.g., housing costs, commuting distances, childcare options)
 Review policies and plan for insights on prevention implementation at locations with a significant civilian
employee presence

Table 166: Malmstrom AFB Findings

Protective Environments
 Coordinate with Air Force to ensure awareness materials are current
 Ensure Military and Family Life Counselors are clearly visible, and that Airmen understand that there is no
“chain of command” to access them
 Leverage the established interfaith alliance to build a community-based coalition of services and
encourage Airmen involvement in off-base activities
Integrated Prevention
 Use SMEs to facilitate trainings
 Evaluate outcomes of secondary trauma training outcomes and report lessons learned
 Ensure Community Action Team/Community Action Board (CAT/B) membership aligns with Air Force
guidance
o CAT/B chairs should evaluate adding special victims counsel and Inspector General as members as
allowed by Air Force Instruction 90-5001
 Analyze information shared, audiences, frequency, etc. from various data sharing meetings to determine
potential duplication and opportunities for efficiency
Stakeholder Engagement
 Leverage Wing leadership’s visibility to have discussions about what “dignity and respect” means to them
as well as the Airmen, families, and civilians throughout the community
o Capture those messages and share them widely
Table 17: MCSF Blount Island Findings

Protective Environments
 Mentor young Marines to become future leaders that uphold the values of current leadership
 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members
Integrated Prevention
 Ensure collateral duty prevention staff have the necessary resources to be successful
 Examine ways to improve access to prevention support services (e.g., a shuttle or bringing prevention
support personnel to Blount Island on a periodic but regular basis)
Stakeholder Engagement
 Improve relationship between civilian employees and Service members
o Invite DoD civilian managers and supervisors to be a part of the Force Preservation Council
o Ensure visibility on the availability of civilian-specific prevention resources (i.e., Employee Assistance
Programs) on the installation
Table 18: NAS Lemoore Findings

Protective Environments
 Evaluate the impact of variable bonuses for Sailors assigned and make recommendations to the
Department of Navy, if necessary
25

 Prioritize improvements in quality of life (e.g., transportation, housing, food, and parking)
Integrated Prevention
 Prioritize units with high numbers of limited duty personnel for support from integrated prevention staff
Stakeholder Engagement
 To ensure no gaps in services, and to further clarify the "warm handoff" process, formalize processes that
enable chaplains to fully integrate into community wide prevention activities and formally establish
relationships with prevention support agencies as appropriate and to the extent permitted by law and
regulation
Table 19: NS Mayport Findings

Protective Environments
 Provide increased opportunities for prevention and prevention support organizations to bring those
services and conduct outreach/education events on the ships
 Invite family members to military community events to build connectedness and increase knowledge of
resources and hold events during work hours to not interfere with work-life balance
 Prioritize care-seeking over routine maintenance and non-critical tasks despite OPTEMPO
 To the extent possible, extend childcare hours at the locations available to Sailors
Integrated Prevention
 Empower and encourage NCOs to participate in and collaborate with the prevention workforce
Stakeholder Engagement
 Implement a system that confirms care appointments (for accountability) rather than requiring proof of the
medical issue or concern
 Leverage female leaders to increase mentorship and support opportunities for junior female Service
members
Table 70: NS Norfolk Findings

Protective Environments
 Engage with the Commander, Navy Installations Command Programming to support Sailors and mission
readiness across the installation and tenant units
 Increase care for the caregivers and provide them specific and additional resources
 Evaluate compliance with the Comprehensive Crew Endurance Management Policy to ensure it is being
consistently followed
 Prioritize public-private venture housing for sailors in the shipyard and maintenance phase
Integrated Prevention
 Clarify that Deployed Resiliency Counselors (DRCs) can serve the support strike group at the pier
 Expand mental health resources and prevention personnel, such as DRCs and Embedded Integrated
Prevention Coordinators, in all ships regardless of size with a focus on continuity of care and ease of
access
Stakeholder Engagement
 Examine how negative perceptions of females, especially pregnant females, is detrimental to commands,
the community, and the Navy
 Hold regular meetings between tenant units and site command to identify gaps
26

 Increase the availability of mental health providers by outsourcing mental health care

Conclusion
Over the past several years, the Department has initiated unprecedented actions to prevent harmful behaviors
and improve climate. Many of these changes require institutional and structural changes that take time to
implement and attain measurable benefits. The OSIEs are one tool that assess incremental improvements
and identify additional areas for improvement. As with the 2021 OSIEs, it was expected that sites and ships
would be early in their development of prevention capabilities. The 2023 OSIEs made clear that leadership at
all levels recognize the need for and importance of a strong prevention system; however, they are not
effectively or consistently communicating its priority due to challenges such as high OPTEMPO, manning
shortages, lack of Service prevention policies and regulations, and IPPW billets remaining unfilled. Prevention
must be a consistent and concerted effort. Lack of priority has cascading effects, leaving leaders and
prevention personnel at all levels lacking preparation and the necessary skills and competencies to effectively
implement prevention.
Prevention is essential to properly assist and care for Service members and to recognize, address, and reduce
harmful behaviors across the force. The Department is currently taking steps to implement approved SPRIRC
and IRC-SAM recommendations that address many OSIE findings, and will reassess 2021 OSIE sites in 2024
to measure progress and gather lessons learned.
The Department has taken unprecedented steps to resource integrated prevention, but to maximize those
investments, complete buy-in from leadership at all levels is required. OSD, the MILDEPS, and NGB must
continue to reinforce the importance of prevention for the well-being of Service members. Prevention of
harmful behaviors is an imperative investment in the future of the Department, strengthening readiness and
resiliency across the Total Force.

Appendix A: Site Selection Methodology
Background
On February 26, 2021, Secretary of Defense Lloyd Austin issued the memorandum, “Immediate Actions to
Counter SA and SH and the Establishment of a 90-Day IRC-SAM,” which directed immediate actions to
address SA and SH. Immediate Action 2 directed OSD to conduct OSIEs and to provide quarterly command
climate updates.
To support identification of sites/ships for the 2021 evaluations, Under Secretary of Defense (Personnel and
Readiness) (USD(P&R)) directed the completion of a force-wide DEOCS. The DEOCS was selected as the
primary data source for the 2021 installation evaluations because it serves as the most timely and sensitive
Defense-wide measure of command climate and because other relevant data were delayed due to COVID. In
2022, command climate updates employed a multi-measure approach to better capture the many facets of site
risk.

Data Sources
2021-2022 DEOCS
Designed by OPA, the DEOCS 5.0 assesses 19 protective and risk factors that can impact a
unit’s/organization’s climate and ability to achieve their mission. The DEOCS 5.0 measures nine risk factors
and 10 protective factors.
Protective factors are attitudes, beliefs, and behaviors associated with positive outcomes for organizations or
units. Transformational leadership10 ratings for the unit/organization leader and the senior non-commissioned
officer/senior enlisted leader (senior NCO/SEL), if applicable, are treated as two separate factors.
Risk factors are attitudes, beliefs, and behaviors associated with negative outcomes for organizations or units.
Passive leadership11 is measured for both the unit/organization leader and the senior non-commissioned
officer (senior NCO/SEL).
2018 WGRA – Contextual Analysis:
OPA’s 2018 WGRA provides insights regarding the estimated prevalence and characteristics of SA, SH, and
gender discrimination in the Active Component; Service members’ experiences with reporting these types of
incidents; and perceptions of unit culture and climate.
2018 DSAID:
DSAID is the Department’s authoritative, centralized database used to collect and maintain information about
SA cases involving members of the U.S. Armed Forces.
2020-2022 DSPO Suicide Counts:
DSPO provided the OSIE team with military suicide data from 2020 through the first quarter of 2022. DSPO is
part of the Office of the Under Secretary of Defense for Personnel and Readiness (OUSD(P&R)) and is the
authoritative source for suicide data in the DoD.
2021 FAP Domestic and Child Abuse and Neglect Counts:
The Office of the Deputy Assistant Secretary of Defense for Military Community and Family Policy provided the
OSIE team with aggregate counts of domestic abuse and child abuse and neglect incidents reported to FAP in

10
Transformational leadership is a leadership style that inspires staff by providing motivation and meaning to their work, giving attention to individuals’
unique needs, and directing their focus to higher goals, such as those of the mission.
11
Passive leadership is a leadership style that avoids and neglects mistakes or problems until they can no longer be ignored.

Fiscal Years 2019-2021. Only data that met criteria for the Department's definition of abuse per DoDI 6400.01
were provided. FAP is the DoD’s program designated to prevent and respond to domestic abuse and child
abuse and neglect.
2022 U.S. CHR&R:
The CHR&R is a program of the University of Wisconsin Population Health Institute that compiles U.S. local
health data to help communities identify opportunities to improve their health. The CHR&R spans several
health focus areas: length of life, quality of life, tobacco use, diet and exercise, alcohol and drug use, sexual
activity, access to clinical care, quality of clinical care, education, employment, income, family and social
support, community safety, air and water quality, housing and transit, and demographics.

Data Ingestion and Merging
Each data source informing the OSIE risk index was ingested into Advana for visualization and analysis.

Appendix B: OSIE Process, Integrated Prevention Metric
Development, Validation, and Scoring
Measures
Based on an analysis of the requirements in DoDI 6400.09 and the elements of the OSIE framework (priority,
preparation, and implementation), OSD, in coordination with RAND, developed nine new metrics to assess
prevention capabilities associated with specific focus areas in DoDI 6400.09: healthy and protective
environments, integrated prevention, and stakeholder engagement. 12 Three main tools (focus group
discussion protocols, tabletop exercises, and surveys) were used to collect data to inform maturity scores
which capture an site/ship’s overall capacity for integrated prevention. This section presents the dimensions
on which sites/ships were scored and the data collection tools used.
49F

Nine dimensions were used to guide the assessment of prevention capabilities for the OSIE report.
The OSD prioritized three domains of focus:
1) Healthy and Protective Environment: Research shows that command climates can positively or
negatively impact behaviors such as SA and SH.
2) Integrated Prevention: Effective prevention targets a mix of risk and protective factors that are both
common across problem areas as well as unique to specific harmful behaviors.
3) Stakeholder Engagement: Outcomes can be improved when multiple stakeholders have genuine
involvement in prevention activities.
Three additional domains are also included in OSIE framework:
1) Priority: Higher-level leadership sets the tone and sustains consistent focus on harmful behaviors.
2) Preparation: Prevention personnel and intermediate leadership are equipped with the ability, and exist
within a structure, that incentivizes and supports addressing harmful behaviors.
3) Implementation: Approach aligns with best practices and is done well (i.e., with high quality).

Information collection for these metrics were approved by Office of Management and Budget (OMB Control Number 0704-0610).

Table 21: Prevention Capabilities Assessed at OSIEs

OSIE FRAMEWORK AREA

FOCUS AREAS

HEALTHY AND
PROTECTIVE
ENVIRONMENT

INTEGRATED
PREVENTION

STAKEHOLDER
ENGAGEMENT

PRIORITY

PREPARATION

IMPLEMENTATION

(1)
Leaders prioritize
fostering a protective
environment by their
actions and
communications.

(2)
Leaders have the
requisite KSAs and
access to training to
develop those KSAs.

(3)
Leaders employ best
practices known to
support a protective
environment.

(4)
Leaders prioritize
prevention activities.

(5)
Leaders and
prevention personnel
have the requisite
KSAs to carry out
prevention
successfully.

(6)
Prevention activities
that target risk and
protective factors
across multiple harmful
behaviors are
evaluated.

(7)
Leaders prioritize
engaging stakeholders.

(8)
Prevention personnel
have the resources
and requisite KSAs to
engage stakeholders
effectively.

(9)
Stakeholders are
genuinely engaged in
prevention activities
across multiple
planning stages.

These areas are referred to as core dimensions. Given the breadth of these nine dimensions, each was
divided into multiple subdimensions, which are narrower in focus. Subdimensions were worded as positive
statements (e.g., “Leaders consistently deter harmful behaviors”) to represent high-quality standards to which
sites/ships should aspire. Within each subdimension are even narrower “data elements,” which serve as the
foundation for the dimension scores.
Each data element, also worded as a positive standard to achieve, is judged to be “present” or “absent” by
considering multiple data sources collected at the site. A scoring rubric was created to specify the number of
data elements that must be “present” within a subdimension for the subdimension to be considered “present” at
the site. To be rated “present,” evidence supporting the subdimensions must be visible across the site/ship
(e.g., within each unit and at multiple levels of the chain of command). As the number of data elements varies
by subdimension, the number of “present” data elements required for a subdimension to be rated as “present”
also varies across subdimensions. Figure 22 shows an example for Core Dimension 1 (Healthy and Protective
Environment – Priority) and its subdimensions. This dimension has five subdimensions and the two data
elements are shown for subdimension 1.2. In the scoring rubric, both data elements (1.2.1 and 1.2.2) must be
rated as present for Subdimension 1.2 to be rated as present.

Figure 2: Example of the Link between Data Elements, Subdimensions, and Core Dimensions

Once it is determined which subdimensions are present and absent, then a maturity score is used to determine
the final score for the core dimension. Table 22 below shows the maturity scoring for each core dimension.
Although a six-point scale is used to reflect the range of maturity, the exact makeup of the scoring rubric for
each core dimension varies by the number of subdimensions. Typically, the highest level of maturity not only
has all the subdimensions present, but also an additional requirement for a more robust presence of those
subdimensions.

Background on Maturity Scoring
RAND developed a structured maturity scoring system tailored to each core dimension. In its simplest form, a
maturity model is a set of characteristics, attributes, indicators, or patterns that represent progression and
achievement in a particular domain or discipline. The artifacts that make up the model are typically agreed
upon by the domain or discipline and are validated through application and iterative recalibration. A maturity
model allows an organization or industry to have its practices, processes, and methods evaluated against a
clear set of artifacts that establish a benchmark. These artifacts typically represent best practice and may
incorporate standards or other codes of practice that are important in a particular domain or discipline. By
having the ability to benchmark, organizations can use maturity models to determine their current level of
achievement or capability and then apply these models over time to drive improvement. However, when used
in a broader sense, maturity models can also help organizations benchmark their performance against other
organizations in their domain or industry, and help an industry determine how well it is performing by
examining the achievement or capability of its member organizations. Architecturally, maturity models typically
have “levels” along an evolutionary scale that defines measurable transitions from one level to another. The
corresponding attributes define each level; in other words, if an organization demonstrates these attributes, it is
said to have achieved both that level and the capabilities that the level represents. Having measurable
transition states between the levels enables an organization to use the scaling to:




Define its current state
Determine its future, more “mature” state; and
Identify the attributes it must attain to reach that future state

RAND tailored the general maturity approach, developing a specific scoring method for each individual
dimension (see Table 22). Thus, rather than one overall, generic scoring system, the maturity approach
32

focused on the specifics of each dimension. This approach was based on an assessment process OSD and
RAND used in a DoD project rating the SA and SH prevention capabilities of U.S. Military Service Academies
(Acosta et al., 2022).
In general, for each dimension, a higher maturity rating indicated a greater number of subdimensions that were
rated as present (which were driven by the number of data elements present). For example, there are five
subdimensions for Dimension 1 (Healthy and Protective Environment – Priority). A site could achieve a
maturity score of 2 by having any three subdimensions present. This scoring method was chosen because it
assigns a higher score for more subdimensions present, while also allowing sites to express their level of
maturity in different ways. For many of the dimensions, to obtain the highest score, a site needs to show
consistent evidence that the subdimensions (and their underlying data elements) have been maintained over
the past two or more years despite competing priorities.
As implemented, the maturity model can serve three purposes: it will allow DoD and others to understand
sites’ current capabilities, it may help sites identify ways to strengthen their prevention efforts, and it may
permit comparison within and across sites.
Table 22: Link between Data Elements, Subdimensions, and Maturity Scoring
Subdimensions
Dimension Maturity Scoring

(# of data elements needed to rate Subdimension as
‘present’/total # data elements)

1. Healthy and Protective Environment – Priority
Maturity Score:
5-Present in all 5 and consistent evidence that presence has been
maintained over the past two years despite competing priorities
4-Present in all 5 subdimensions

1.1. Consistently emphasize the importance of a healthy
protective environment (3/4)
1.2. Consistently deters negative behaviors (2/2)
1.3. Leaders hold subordinates accountable for timely
action (2/2)

3-Present in 4 out of 5 subdimensions

1.4. Leaders reinforce positive behaviors (1/1)

2-Present in 3 out 5 subdimensions

1.5. Leader’s role model positive behaviors (1/1)

1-Present in 1 or 2 out of 5 subdimensions
0-None are Present
2. Integrated Prevention – Priority
Maturity Score:

2.1. Leaders see integrated primary prevention as a
consistent and enduring priority and communicate it to
subordinates (2/2)

5-Present in all 4 subdimensions and consistent evidence that
sufficiency has been maintained over time despite competing
priorities

2.2. Leaders hold prevention personnel accountable for
sustained integrated prevention (2/2)

4-Present in all 4 subdimensions

2.3. Leaders reinforce best practice prevention processes
(i.e., sufficient dose, theory-based, evaluated, trained
deliverers, interactive content) (2/2)

3-Present in 3 out of 4 subdimensions
2-Present in 2 out 4 subdimensions

2.4. Leaders prioritize data and evaluation related to
prevention (2/2)

1-Present in 1 out of 4 subdimensions
3. Stakeholder Engagement – Priority
Maturity Score:

3.1. Leaders and prevention personnel use stakeholder
engagement to inform priorities (1/1)

5-Present in 3 out of 3 subdimensions, including support from the
data call, and consistent evidence that presence has been
maintained over time despite competing priorities
4-Present in 3 out of 3, including support from the data call

3.2. Leader communications stress the importance of
stakeholder engagement (1/1)
3.3. Leaders and prevention personnel provide positive
reinforcement for stakeholder engagement (2/2)

3-Present in 3 out of 3 subdimensions
2-Present in 2 out of 3 subdimensions
1-Present in 1 out of 3 subdimensions
0-None are Present
4. Healthy and Protective Environment – Preparation
Maturity Score:
5-Present in all 4 subdimensions, plus mean of data element 2.1.1 is
greater than 4.0
4-Present in 4 out 4 subdimensions

4.1. Leaders are knowledgeable and skilled in building a
protective environment**
4.2. Established or systematic processes/structure to support
healthy climate
4.3. Leaders and subordinates maintain present connections
(3/4)

3-Present in 3 out of 4 subdimensions
2-Present in 2 out of 4 subdimensions

4.4 Leaders monitor climate-related efforts and behaviors
and consider them in performance evaluations (2/2)

1-Present in 1 out of 4 subdimensions

**This data element is scored via a survey = overall mean
score above 3.0 for the eleven leader survey items

0-None are Present
5. Integrated Prevention – Preparation
Maturity Score:

5.1. Prevention personnel receive ongoing and systematic
training and professional development to continually
improve their approach to integrated prevention (2/3)

5-Present in all 5 subdimensions
4-Present in 4 of the 5 subdimensions

5.2. Leaders are knowledgeable and skilled in primary
prevention**

3-Present in 3 out of 5 subdimensions

5.3. Prevention personnel are dedicated, knowledgeable and
skilled in primary prevention (2/2)

2-Present in 2 out of 5 subdimensions
1-Present in 1 out of 5 subdimensions

5.4. Collaborative structure exists to support integrated
primary prevention (2/2)

0-None are Present

5.5. Continuity of prevention staff and effective prevention
activities are maintained over time (2/2)
**This data element is scored via a survey = overall mean
score above 3.0 for the eight leader survey items
6. Stakeholder Engagement - Preparation
Maturity Score:
5-Present in all 4 subdimensions and mean of 8.1 OR 8.2 is greater
than 4
4-Present in all 4 subdimensions

6.1. Leaders have the knowledge and skills needed to
conduct stakeholder engagement**
6.2. Prevention personnel are dedicated, knowledgeable and
skilled in conducting stakeholder engagement^^
6.3. Stakeholders are knowledgeable about prevention (2/2)

3-Present in 3 out of 4 subdimensions

6.4. Sufficient resources exist to conduct stakeholder
engagement (1/1)

2-Present in 2 out of 4 subdimensions
1-Present in 1 out of 4 subdimensions

0-None are Present

**This data element is scored via a survey = overall mean
score above 3.0 for the four leader survey items
^^ This data element is scored via a survey = overall mean
score above 3.0 for the six prevention survey items

7. Healthy and Protective Environment – Implementation
Maturity Score:

7.1. Subordinates and peers are referred to appropriate
resources when at-risk for harmful behaviors (2/2)

5-Present in all 5 subdimensions

7.2. Leaders clearly communicate expectations for
benchmarks, roles, and responsibilities for
improving/maintaining protective environments to
subordinates (2/3)

4-Present in 4 of the 5 subdimensions
3-Present in 3 out of 5 subdimensions
2-Present in 2 out of 5 subdimensions

7.3. Leaders proactively monitor the stress levels of
subordinates (2/2)

1-Present in 1 out of 5 subdimensions

7.4. Leaders and Service members are held accountable for
harmful behaviors in a consistent manner (e.g., through
standard operating procedure) (2/2)

0-None are Present

7.5. Positive behaviors are rewarded/recognized (1/1)
8. Integrated Prevention - Implementation
Maturity Score:
5-Present in all 5 subdimensions

8.1. Prevention approach is integrated (use common
messages, consistent collaboration, common operating
procedures) (3/4)

4-Present in 4 of the 5 subdimensions

8.2. Prevention approach is comprehensive (3/4)

3-Present in 3 out of 5 subdimensions

8.3. Prevention approach is evaluated (3/3)

2-Present in 2 out of 5 subdimensions

8.4. Prevention approach is continuously improved (2/2)

1-Present in 1 out of 5 subdimensions

8.5. Resistance to the prevention approach is monitored and
addressed (2/3)

0-None are Present
9. Stakeholder Engagement - Implementation
Maturity Score:

9.1. Level of collaboration

Score based on the following scale:









NONE (0): Feedback from stakeholders is neither sought
nor used by leaders or prevention personnel.
INFORM (1): Leaders and prevention personnel share
information in a variety of ways with key stakeholder groups
(“We will keep you informed”). No effort is made to get
input.
INVOLVE (2): Leaders and prevention personnel seek
input from stakeholders AFTER decisions are made.
PARTICIPATE (3): Leaders and prevention personnel see
input BEFORE decisions are made.
COLLABORATE (4): Leaders and prevention personnel
work with stakeholders to jointly frame the problem and the
solutions. Leaders and prevention personnel regularly
circle back with stakeholders to update them on progress.
COLLABORATE PLUS (5): Leaders and prevention
personnel work with stakeholders to jointly frame the
problem and the solutions, using a standing group of
stakeholders. This includes leaders and prevention

personnel regularly circling back with stakeholders to
update the group on progress.

Subdimensions
Core dimensions were designed to be broad categories. In contrast, subdimensions were designed to address
narrower topics. Striking a balance between breadth and simplicity, each core dimension contains three to five
subdimensions, except for core dimension nine (Stakeholder Engagement – Implementation) which contains
one subdimension. Subdimensions were chosen for their theoretical connection to the dimension, their
support in the research literature, and their focus on a narrower aspect of the core dimension. Below is a
summary of the subdimensions used to assess each of the nine core dimensions and relevant references
supporting their inclusion.

Subdimensions for Dimension 1: Healthy and Protective Environment – Priority
This dimension contains five subdimensions that aim to assess the extent to which leaders prioritize a healthy
and protective environment and set the tone to sustain a focus on a protective environment.
Subdimensions

References

1.1

Leaders consistently emphasize the
importance of a healthy protective environment

Crittendon & Hope, 2017, pp.18-21; Hoover,
Randolph, Elig, & Klein, 2001, pp. 31-33;
Ratcliff, Key-Roberts, Simmons, & JiménezRodríguez, 2018, pp. 4-18

1.2

Leaders consistently deter negative behaviors

Cook, Jones, Lipari, & Lancaster, 2005; Ratcliff,
Key-Roberts, Simmons, & Jiménez-Rodríguez,
2018, pp. 4-16

1.3

Leaders hold subordinates accountable for
timely action

Jones & Bullis, 2003, pp. 24-25

1.4

Leaders reinforce positive behaviors

Jones & Bullis, 2003, pp. 21-40

1.5

Leaders role model positive behaviors

Ratcliff, Key-Roberts, Simmons, & JiménezRodríguez, 2018, pp.2

Subdimensions for Dimension 2: Integrated Prevention – Priority
This dimension contains four subdimensions that aim to assess the extent to which leaders prioritize integrated
primary prevention and set the tone to sustain a focus on a prevention.
Subdimensions

References

2.1

Leaders see integrated primary prevention as a
consistent and enduring priority and
communicate it to subordinates

Noonan et al., 2009; Kreuter, Lezin, & Young,
2000; McCartan, Kemshall, & Tabachnick, 2015;
Campbell & Wasco, 2005; Patton, 2010

2.2

Leaders hold prevention staff accountable for
sustained integrated prevention

Thompson, Taplin, McAfee, Mandelson, &
Smith, 1995; Nation et al., 2003; McIntosh, Filter,
Bennett, Ryan, & Sugai, 2010

2.3

Leaders reinforce best practice prevention
processes (sufficient dose, theory-based,

Kratochwill, Volpiansky, Clements, & Ball, 2007;
Hawkins, Shapiro, & Fagan, 2010; Mihalic &
Irwin, 2003; McDonald, Charlesworth, &
36

2.4

evaluated, trained deliverers, interactive
content)

Graham, 2015; Murnieks, Allen, & Ferrante,
2011

Leaders prioritize data and evaluation related to
prevention

DeGue et al., 2012; Brubaker, 2009; Provost &
Fawcett, 2013; Mandinach, 2012; Sable, Danis,
Mauzy, & Gallagher, 2006

Subdimensions for Dimension 3: Stakeholder Engagement – Priority
This dimension contains three subdimensions that aim to assess the extent to which leaders prioritize
stakeholder engagement and set the tone to sustain a focus on stakeholder engagement to inform primary
prevention.
Subdimensions

References

3.1

Leaders and prevention personnel use
stakeholder engagement to inform priorities

Ahmed & Palermo, 2010; Dills, Fowler, & Payne,
2016; Goodman et al., 2017; Hood et al., 2010

3.2

Leader communications stress the importance
of stakeholder engagement

Ahmed & Palermo, 2010; Jolibert & Wesselink,
2012

3.3

Leaders and prevention staff provide positive
reinforcement for stakeholder engagement

Hood et al., 2010

Subdimensions for Dimension 4: Healthy and Protective Environment – Preparation
This dimension contains four subdimensions that aim to assess the extent to which leaders and prevention
staff are equipped—with skills and knowledge—and empowered with a clear line of sight across the chain of
command to maintain a healthy and protective environment.
Subdimensions

References

4.1

Leaders are knowledgeable about and skilled at
building a protective environment

Cook, Jones, Lipari, & Lancaster, 2005, pp. 9-10

4.2

Established or systematic processes/structure
support a protective environment

Crittendon & Hope, 2017, pp. 20-29

4.3

Leaders and subordinates maintain sufficient
connections

Ratcliff, Key-Roberts, Simmons, & JiménezRodríguez, 2018, pp. 4 & 17

4.4

Leaders monitor climate-related efforts and
behaviors and consider them in performance
evaluations

Hoover, Randolph, Elig, & Klein, 2001, pp. 32-33

Subdimensions for Dimension 5: Integrated Prevention – Preparation
This dimension contains five subdimensions that aim to assess the extent to which leaders and prevention staff
are equipped—with skills and knowledge—and empowered with a clear line of sight across the chain of
command to sustain high-quality integrated primary prevention.
Subdimensions

References
37

5.1

Prevention personnel receive ongoing and
systematic training and professional
development to continually improve their
approach to integrated prevention

Kratochwill, Volpiansky, Clements, & Ball, 2007;
Hawkins, Shapiro, & Fagan, 2010; Mihalic &
Irwin, 2003; McDonald, Charlesworth, & Graham,
2015; Murnieks, Allen, & Ferrante, 2011

5.2

Leaders are knowledgeable and skilled at
primary prevention

Kratochwill, Volpiansky, Clements, & Ball, 2007;
Hawkins, Shapiro, & Fagan, 2010; Mihalic &
Irwin, 2003; McDonald, Charlesworth, & Graham,
2015; Murnieks, Allen, & Ferrante, 2011

5.3

Prevention personnel are dedicated,
knowledgeable and skilled in primary prevention

Kratochwill, Volpiansky, Clements, & Ball, 2007;
Hawkins, Shapiro, & Fagan, 2010; Mihalic &
Irwin, 2003; McDonald, Charlesworth, & Graham,
2015; Murnieks, Allen, & Ferrante, 2011

5.4

Collaborative structure exists to support
integrated primary prevention

DeGue et al., 2012; Brubaker, 2009; Provost &
Fawcett, 2013; Mandinach, 2012; Sable, Danis,
Mauzy, & Gallagher, 2006

5.5

Continuity of prevention staff and effective
prevention activities are maintained over time

Dills, Fowler, & Payne, 2016; Wandersman &
Florin, 2003; Lundgren & Amin, 2015; Bond &
Hauf, 2004; McMahon, Postmus, & Koenick,
2011

Subdimensions for Dimension 6: Stakeholder Engagement – Preparation
This dimension contains four subdimensions that aim to assess the extent to which leaders and prevention
staff are equipped—with skills and knowledge—and empowered with a clear line of sight across the chain of
command to sustain stakeholder engagement efforts to inform primary prevention.
Subdimensions

References

6.1

Leaders have the skills and knowledge needed
to conduct stakeholder engagement

SAMHSA, 2021

6.2

Prevention staff are dedicated, knowledgeable
and skilled in conducting stakeholder
engagement

Scaccia et al., 2015; Powell et al., 2015;
SAMHSA, 2021

6.3

Stakeholders are knowledgeable about
prevention

Desai, 2017

6.4

Sufficient resources exist to conduct
stakeholder engagement

Noonan et al., 2009; Krug, Mercy, Dahlberg, &
Zwi, 2002; García-Moreno et al., 2015; Hawkins,
Shapiro, & Fagan, 2010

Subdimensions for Dimension 7: Healthy and Protective Environment – Implementation
This dimension contains five subdimensions that aim to assess the extent to which actions taken by leaders
and prevention staff are aligned with best practices for building a healthy and protective environment and are
done well (i.e., with high quality).
38

Subdimensions

References

7.1

Subordinates and peers are referred to
appropriate resources when at-risk for harmful
behaviors

Crittendon & Hope, 2017, pp.18-21

7.2

Leaders clearly communicate expectations for
benchmarks, roles, and responsibilities for
improving/maintaining protective environments
to subordinates

Ratcliff, Key-Roberts, Simmons, & JiménezRodríguez, 2018, pp.4-16, 18

7.3

Leaders proactively monitor the stress level of
subordinates

Hoover, Randolph, Elig, & Klein, 2001, pp. 4

7.4

Leaders and Service members are held
accountable for harmful behaviors in a
consistent manner (e.g., through standard
operating procedure)

Cook, Jones, Lipari, & Lancaster, 2005

Positive behaviors are rewarded/recognized

Jones & Bullis, 2003, pp. 21-40

7.5

Ratcliff, Key-Roberts, Simmons, & JiménezRodríguez, 2018

Subdimensions for Dimension 8: Integrated Prevention – Implementation
This dimension contains five subdimensions that aim to assess the extent to which actions taken by leaders
and prevention staff are aligned with best practices for integrated primary prevention and are done well (i.e.,
with high quality).
Subdimensions

References

8.1

Prevention approach is integrated (use common
messages, consistent collaboration, common
operating procedures)

Gidycz et al., 2018.

8.2

Prevention approach is comprehensive

Bronfenbrenner, 1992, 2005; Casey & Lindhorst,
2009; Banyard, Eckstein, & Moynihan, 2010;
Prochaska & Prochaska, 2011; Vladutiu, Martin,
& Macy, 2011

8.3

Prevention approach is evaluated

Chinman et al., 2016; 2018; Francisco, Paine, &
Fawcett, 1993

8.4

Prevention approach is continuously improved

Chinman et al., 2016; 2018; Francisco, Paine, &
Fawcett, 1993

8.5

Resistance to the prevention approach is
monitored and addressed

Nation et al., 2003; Rich, Utley, Janke, &
Moldoveanu, 2010

Subdimension for Dimension 9: Stakeholder Engagement – Implementation
This dimension contains one subdimension that aims to assess the extent to which actions taken by leaders
and prevention staff are aligned with best practices for stakeholder engagement and are done well (i.e., with
high quality).

Subdimensions
9.1

References

Level of collaboration ranging from none, to inform (sharing
information, lowest level) to collaborate (sharing decision
making and implementation, highest level)

International Association for Public
Participation, 2018

Focus Group Discussion Protocols (DPs)
In the initial OSIE process, seven DPs were developed to measure integrated prevention and prevention
capacity. DPs were designed to guide discussions among certain stakeholders.
Table 23: Discussion Protocols and Target Participants

Discussion Protocol
DP1
DP2
DP3
DP413
DP5
DP6
DP7

Target Participants
Command
Service Members – E1-E4, O1-O3, W1-W2
Service Members – E1-E4, E5-E6
Leaders – O3-O4, W3-W4
Leaders – E7-E9
Prevention Personnel
Prevention Support Personnel

Surveys
The OSIEs administered surveys to Service member personnel. Surveys were generally completed using pen
and paper.

Tabletop Exercise (TTX)
The TTX is an activity that prompts prevention personnel to complete a prevention-related activity. The TTX
used at most military installations usually consists of an exercise involving deployment/redeployment
prevention readiness based off a real-world scenario.

Data Collection
OSIE site visits were conducted by multi-disciplinary teams which were led by a Senior Executive or GS-15
and included seven staff members representing OUSD(P&R) (Office of Force Resiliency; Office of Diversity,
Equity, and Inclusion; DSPO; SAPRO; Military Community Advocacy, Office of People Analytics; and the
Diversity Management Operation Center). The teams also included representation from the Military Services
(military and civilian employees) and the NGB. Teams were comprised of both Service members and civilian
employees. The Service members assigned to each OSIE team acted as a senior subject matter expert. This
allowed for a mixture of military perspectives and insight into service and site/ship culture. Research
assistants were also present during focus group interviews to record interviewee responses, allowing team
members to engage with focus group participants in a fluid manner.
Several improvements and modifications were made from the 2021 OSIEs in advance of the 2023 OSIEs.
These improvements included adjusting the language and flow of the discussion protocols to improve clarity
and simplicity. Improvements were also made to scoring and reporting, such as the development of a process
for organizing notes to better facilitate scoring.
At the end of the each OSIE site visit, research assistants from Miami University (through an Intergovernmental
Personnel Act agreement with DoD) compiled responses from the DPs and TTX into a single document using
13

Also used with officers above O4/W4 if not in command position.

the Qualtrics (2023) online secure survey platform. The compiled document and survey mean scores were
then sent to the site visit team for scoring. Individually, team members used the data to derive a maturity score
for each dimension. Individual scores were provided to the team leader. To ensure inter-rater reliability, the
individual scores were provided to the team leader to measure the consistency of scoring across team
members. The team lead was responsible for validating scores. When team member scores differed, the
team conferred to assign and validate scores. Only validated team scores were used for the final assessment
of a site/ship.

Appendix C: Acronyms List
AFB
BOSS
CFA
CHR&R
DEOCS
DoD
DoDI
DP
DRC
DSAID
DSPO
FAP
FIPS
IPPW
IRC-SAM
KSAs
MCSF
MILDEPS
MWR
NAS
NCO
NGB
NS
OPA
OPTEMPO
OSD
OSIE
OUSD(P&R)
SA
SAPRO
SEL
SH
SME
SMFP
SPRIRC
TIMS
TTX
USD(P&R)
WGRA
Append

Air Force Base
Better Opportunities for Single Soldiers
Commander Fleet Activities
U.S. County Health Rankings and Roadmaps
Defense Organizational Climate Survey
Department of Defense
Department of Defense Instruction
Discussion Protocol
Deployed Resiliency Counselor
Defense Sexual Assault Incident Database
Defense Suicide Prevention Office
Family Advocacy Program
Federal Information Processing Standards
Integrated Primary Prevention Workforce
Independent Review Commission on Sexual Assault in the Military
knowledge, skills, and abilities
Marine Corps Support Facility
Military Departments
Morale, Welfare, and Recreation
Naval Air Station
noncommissioned officer
National Guard Bureau
Naval Station
Office of People Analytics
operational tempo
Office of the Secretary of Defense
on-site installation evaluation
Office of the Under Secretary of Defense for Personnel and Readiness
sexual assault
Sexual Assault Prevention and Response Office
senior enlisted leader
sexual harassment
subject matter expert
Service Member and Family Programs
Suicide Prevention and Response Independent Review Committee
This is My Squad
tabletop exercise
Under Secretary of Defense for Personnel and Readiness
Workplace and Gender Relations Survey of Active Duty Members

Appendix D: References
Bronfenbrenner, U. (1992). Ecological systems theory. Jessica Kingsley Publishers.
Bronfenbrenner, U. (Ed.). (2005). Making human beings human: Bioecological perspectives on human
development. Sage Publications Ltd.
Chinman, M., Acosta, J., Ebener, P., Malone, P., & Slaughter, M. (2016). Can implementation support help
community-based settings better deliver evidence-based sexual health promotion programs? A randomized
trial of Getting To Outcomes®. Implementation Science, 11(78). https://doi.org/10.1186/s13012-016-0446-y
Chinman, M., Ebener, P., Malone, P. S., Cannon, J., D'Amico, E. J., & Acosta, J. (2018). Testing
implementation support for evidence-based programs in community settings: a replication cluster-randomized
trial of Getting To Outcomes®. Implementation Science: IS, 13(1), 131. https://doi.org/10.1186/s13012-0180825-7
Cook, P., Jones, A., Lipari, R. & Lancaster, A. (2005). Service academy 2005 sexual harassment and assault
survey. Defense Manpower Data Center, Arlington, VA: Survey and Program Evaluation Division.
Crittendon, D. & Hope, R.O. (2017). An assessment of FY2016 locally developed questions from the DEOMI
Organizational Climate Survey: Recommendations and potential implications, No. 10-17, 2017, Defense Equal
Opportunity Management Institute, Patrick Air Force Base, Florida.
DeGue, S., Holt, M. K., Massetti, G. M., Matjasko, J. L., Tharp, A. T., & Valle, L. A. (2012). Looking ahead
toward community-level strategies to prevent sexual violence. Journal of Women's Health, 21(1), 1-3.
Dills, J., Fowler, D., & Payne, G. (2016). Sexual violence on campus: Strategies for prevention. National
Center for Injury Prevention and Control (U.S.), Division of Violence Prevention.
Desai, V. M. (2017). Collaborative stakeholder engagement: An integration between theories of organizational
legitimacy and learning. Academy of Management Journal, 61, 2018, 220-244.
Francisco, V. T., Paine, A., & Fawcett, S. B. (1993). A methodology for monitoring and evaluating community
health coalitions. Health Education Research: Theory and Practice, 8, 403-416.
García-Moreno, C., Zimmerman, C., Morris-Gehring, A., Heise, L., Amin, A., Abrahams, N., Oswaldo, M.,
Bhate-Deosthali, P., Nduku, K., & Watts, C. (2015). Addressing violence against women: a call to action. The
Lancet, 385(9978), 1685-1695.
Gidycz, C.A., Wyatt, J., Galbreath, N.W., Axelrad, S. & McCone, D.R. (2018). SA prevention in the military:
Key issues and recommendations, Military Psychology, 30(3), 240-251.
Goodman, M. S., Thompson, V. L. S., Arroyo Johnson, C., Gennarelli, R., Drake, B. F., Bajwa, P.,
Witherspoon, M. & Bowen, D. (2017). Evaluating community engagement in research: quantitative measure
development. Journal of Community Psychology, 45(1), 17-32.
Hawkins, J. D., Shapiro, V. B., & Fagan, A. A. (2010). Disseminating effective community prevention practices:
Opportunities for social work education. Research on Social Work Practice, 20(5), 518-527.
Hood, N. E., Brewer, T., Jackson, R., & Wewers, M. E. (2010). Survey of community engagement in NIHfunded research. Clinical and Translational Science, 3(1), 19-22.
Hoover, E. C., Randolph, J.S., Elig, T.W. & Klein, P.M. (2001). Overview of the 2000 Military Exit Survey, No.2
2001-001. Defense Manpower Data Center, Arlington, VA: Survey and Program Evaluation Division.

International Association for Public Participation. (2018). Public Participation Pillars. Available online at
www.iap2.org.
Jolibert, C., & Wesselink, A. (2012). Research impacts and impact on research in biodiversity conservation:
The influence of stakeholder engagement. Environmental Science & Policy, 22, 100-111.
Jones, S. M., & Bullis, C. (2003). Improving Accountability for Effective Command Climate: A Strategic
Imperative. United States Army War Colleges, Carlisle, Pennsylvania.
Kratochwill, T. R., Volpiansky, P., Clements, M., & Ball, C. (2007). Professional development in implementing
and sustaining multitier prevention models: Implications for response to intervention. School Psychology
Review, 36(4).
Kreuter, M.W., Lezin, N.A., & Young, L.A. (2000). Evaluating community-based collaborative mechanisms:
Implications for practitioners. Health Promotion Practice, 1, 49-63.
Krug, E. G., Mercy, J. A., Dahlberg, L. L., & Zwi, A. B. (2002). The world report on violence and health. The
Lancet, 360(9339), 1083-1088.
Lundgren, R., & Amin, A. (2015). Addressing intimate partner violence and sexual violence among
adolescents: emerging evidence of effectiveness. Journal of Adolescent Health, 56(1), S42-S50.
Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice.
Educational Psychologist, 47(2), 71-85.
McIntosh, K., Filter, K. J., Bennett, J. L., Ryan, C., & Sugai, G. (2010). Principles of sustainable prevention:
Designing scale-up of school-wide positive behavior support to promote durable systems. Psychology in the
Schools, 47(1), 5-21.
McCartan, K. F., Kemshall, H., & Tabachnick, J. (2015). The construction of community understandings of
sexual violence: Rethinking public, practitioner, and policy discourses. Journal of Sexual Aggression, 21(1),
100-116.
McDonald, P., Charlesworth, S., & Graham, T. (2015). Developing a framework of effective prevention and
response strategies in workplace SH. Asia Pacific Journal of Human Resources, 53(1), 41-58.
McMahon, S., Postmus, J. L., & Koenick, R. A. (2011). Conceptualizing the engaging bystander approach to
sexual violence prevention on college campuses. Journal of College Student Development, 52(1), 115-130.
Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—
factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1(4),
307-329.
Noonan, R. K., Emshoff, J. G., Mooss, A., Armstrong, M., Weinberg, J., & Ball, B. (2009). Adoption, adaptation,
and fidelity of implementation of sexual violence prevention programs. Health Promotion Practice, 10(1_suppl),
59S-70S.
Patton, M. Q. (2010). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and
Use. Guilford Press.
Powell, A., Watson, J., Staley, P., Patrick, S., Horn, M., Fetzer, L., Oglesby, J., & Verma, S. (2015). Blending
Learning: The Evolution of Online and Face-to-Face Education from 2008-2015. Promising Practices in
Blended and Online Learning Series. International association for K-12 online learning.
Prochaska, J. J., & Prochaska, J. O. (2011). A review of multiple health behavior change interventions for
primary prevention. American Journal of Lifestyle Medicine, 5(3), 208-221.
44

Provost, F., & Fawcett, T. (2013). Data science and its relationship to big data and data-driven decision
making. Big Data, 1(1), 51-59.
Ratcliff, N. J., Key-Roberts, M., Simmons, M.J & Jiménez-Rodríguez, M. (2018). Inclusive Leadership Survey
Item Development, No. 2018-03. Consortium of Universities, Washington DC.
Rich, M. D., Utley, E.A., Janke, K., & Moldoveanu, M. (2010). I'd rather be doing something else: male
resistance to rape prevention programs, The Journal of Men’s Studies 18, 268-288.
Sable, M. R., Danis, F., Mauzy, D. L., & Gallagher, S. K. (2006). Barriers to reporting SA for women and men:
Perspectives of college students. Journal of American College Health, 55(3), 157-162.
Scaccia, J. P., Cook, B. S., Lamont, A., Wandersman, A., Castellow, J., Katz, J., & Beidas, R. S. (2015). A
practical implementation science heuristic for organizational readiness: R = MC2. Journal of Community
Psychology, 43(4), 484–501. https://doi.org/10.1002/jcop.21698
Thompson, R. S., Taplin, S. H., McAfee, T. A., Mandelson, M. T., & Smith, A. E. (1995). Primary and
secondary prevention services in clinical practice: twenty years' experience in development, implementation,
and evaluation. JAMA, 273(14), 1130-1135.
Vladutiu, C. J., Martin, S. L., & Macy, R. J. (2011). College-or university-based SA prevention programs: A
review of program outcomes, characteristics, and recommendations. Trauma, Violence, & Abuse, 12(2), 67-86.
Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention. American
Psychologist, 58(6-7), 441.

File Type	application/pdf
File Title	Microsoft Word - TAB C - OSIE Summary 2023_v6
Author	HilberAJ
File Modified	2023-11-14
File Created	2023-11-14