SUPPORTING STATEMENT - PART A
On-Site Installation Evaluations – OMB 0704-0610
Summary of Changes from Previously Approved Collection
|
1. Need for the Information Collection
These information collections continue to support a high-visibility requirement directed in Secretary of Defense Memorandum, “Immediate Actions to Counter Sexual Assault and Harassment and the Establishment of a 90-Day Independent Review Commission on Sexual Assault in the Military,” February 26, 2021. Immediate Action 2 directs the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) to develop a plan of action and milestones to conduct high risk installation evaluations. Memorandum “Plan of Action and Milestones for High Risk Installation Evaluations,” March 30, 2021 USD(P&R) approved the plan of action and milestones.
These information collections, conducted at on-site visits at high risk and promising installations, are in support of those directed actions. The initial authorization provided the opportunity to conduct information collections from 20 installations from June 2021-January 2022. The purpose of the site visits was to determine, using standardized metrics, if the installations’ prevention capabilities were ready and able to address risk for interpersonal and self-directed violence at the unit and installation level. Where site visits determined that the installations’ prevention capabilities were ready and able, the evaluations highlighted actions that could be replicated elsewhere, and where the installations’ prevention capabilities were not ready and able, results enabled Office of Force Resilience to offer recommendations for concrete actions that can be taken to strengthen prevention activities to reduce risk and enhance protective factors.
The initial 20 sites served as a pilot for standardized metrics and methods developed for On-Site Installation Evaluations (OSIEs). Beginning 2023, biennial information collections will be used to evaluate 30 DoD installations each year where the military community is at increased or decreased risk for destructive behaviors based on the revised methods and metrics refined through the initial pilot. Effective prevention decreases risk factors and increases protective factors. On-site evaluations will answer: Are the installation’s prevention capabilities poised to address the risk detected on the command climate assessment? If so, what is going right that could be replicated elsewhere? If not, what concrete actions could be taken to improve the installation’s ability to address risk and enhance promise?
A report based on the first wave of OSIE site visits was submitted to the Secretary of Defense in Oct 2021 (with an addendum of updated results scheduled for submission in early 2022). The pilot and enduring evaluation assess the implementation of DoD Instruction 6400.09, “DoD Policy on Integrated Primary Prevention of Self-Directed Harm and Prohibited Abuse or Harm,” September 11, 2020.
DoD Office of Force Resiliency (OFR) initially identified 20 DoD units and installations of risk and promise to take part in the pilot assessment. Sites were identified based on analysis of their 2021 command climate assessments. Sites with extreme scores were selected for site visits. Methods for the selection of participating sites in future years of this evaluation will follow a similar identification process as in the pilot (using a more robust assessment of risk and promise), and these methods will be provided in TAB A for the renewal period.
At each site, in addition to assessing installation assets (e.g., prevention personnel that provide services to the entire installation) data will be collected from the 1-3 units that drive the installation’s risk and promise score. There will be three data sources: 1) responding to a “request for information”; 2) participating in discussions during the site visit; and 3) completing a survey. The discussions and survey constitute the information collections covered in this request.
2. Use of the Information
The respondents will be a cross section of personnel at the selected installations that fall into two general categories. First, are personnel that are specific to prevention and intervention activities relating to a variety of negative behaviors (e.g., alcohol use, suicide, sexual harassment). These include: Sexual assault response coordinators, victim advocates, Family Advocacy Program staff, MEO/EEO staff, Mental Health Professionals, Enlisted medical personnel (e.g., medics, corpsmen), Inspectors general and misdemeanor and felony-level law enforcement representatives, Chaplains, MWR and community/ support services staff, and Physical health professionals. We are collecting data from these individuals because they have first-hand knowledge of how prevention activities are carried out at the installation. In most cases, given the size of the identified sites, samples of participants will represent a census of the personnel at these locations.
The second category are general personnel that will be important to talk to for their perceptions of how prevention is prioritized and experienced at the installation and will include (the target sample is in parentheses): Installation commander, E1-E4 Men (10), E1-E4 Women (10), O1-O3 Men (10), O1-O3 Women (10), E5-E6 Men (10), E5-E6 Women (10), O4-O5 Men (10), O4-O5 Women (10), E7-E9 Men (10), E7-E9 Women (10), O6 Men (10), O6 Women (10), First sergeants (5 or more). These individuals will be selected from the specific units that drive command climate risk or promise at the site. We will use personnel data to determine how representative these respondents are of the total base population.
Respondents’ participation in these information collections is voluntary.
There will be three data sources: 1) responding to a “request for information”; 2) participating in discussions during the site visit; and 3) completing a survey. Data source 1 and 2 will be the same questions. The request for information will be sent ahead via an emailed Word document to a point of contact (POC) at each installation. The POC will arrange to have the relevant individual(s) answer the questions in the Word document and send back by email to the government representative leading the OSIE team. In our experience, these types of requests for information are often returned incomplete, unclear, or inaccurate. Thus, at the site visits, site visit staff will follow up to confirm the information and fill in any missing data. During the site visit, various groups of individuals specified above will be organized into a series of discussion groups in one to one and a half hour slots. The questions for 1 and 2 will be open-ended questions about the prevention activities. The discussions will NOT be recorded, but a note taker will be part of the site visit team and will take detailed notes. Those notes will be hand carried back to the agency to be organized and used for analysis. Data source 3 will be a survey of individual competencies to carry out prevention activities. Individuals will complete the survey at their respective discussion group slot during the site visit. The survey will be paper and pencil and collected by site visit staff who will hand carry the completed forms back to the agency for data entry and analysis.
The end result of the data collection will be a series of ratings for each installation to characterize the maturity of prevention at each installation (i.e., how consistently prevention is prioritized, how consistently people are prepared to conduct needed prevention, and how consistently prevention is done well). The ratings will be on a 1-5 scale (1 = rarely or never, 5 = consistently). There will be 9 ratings for each installation. The 9 ratings come from three infrastructure elements being applied to four content domains (see Table 1). A detailed description of the metrics development, validation, and scoring will be found at TAB B. The information collection tools will be found at TABs C and D. A manuscript detailing the development and validation of the survey will be found at TAB E.
Table 1. Rating domains for each installation
|
PRIORITY |
PREPARATION |
IMPLEMENTATION |
PROTECTIVE ENVIRONMENT |
1 |
2 |
3 |
Leaders demonstrate priority for fostering a protective environment by their actions (e.g., role modeling, taking timely action to address negative behaviors and reinforce positive ones) and communications. |
Leaders need to have the requisite knowledge, skills, abilities (KSAs) and access to training to develop those KSAs. Structures and processes need to be in place that support a protective environment (e.g., appropriate socialization opportunities; providing leaders’ professional feedback and support for difficult tasks). |
Leaders employ practices known to support a protective environment (e.g., model appropriate behavior, encourage appropriate socialization, refer service members to resources when needed, encourage open and respectful communication).
Multiple actions are taken to ensure that these practices are done well, including setting expectations and clearly delineating roles and responsibilities; ensuring subordinates have appropriate workloads and are rewarded for high performance; all service members are held responsible for negative behaviors with either appropriate sanctions (i.e., sexual assault) or resources (substance abuse, suicidal ideation). |
|
INTEGRATED PRIMARY PREVENTION |
4 |
5 |
6 |
Leaders make conducting prevention activities a priority and hold subordinates accountable for its completion. |
Leaders provide prevention personnel the resources they need to conduct prevention. Leaders and prevention personnel have the knowledge and skills they need the carry out prevention successfully. |
Prevention activities are carried out and evaluated that target risk and protective factors that cut across multiple negative behaviors. |
|
STAKEHOLDER ENGAGEMENT |
7 |
8 |
9 |
Leaders convey that engaging stakeholder involvement is a priority and hold subordinates accountable for doing so. |
Prevention personnel have the resources, knowledge, and skills to engage stakeholders effectively. |
Stakeholders are regularly and genuinely engaged in prevention activities across multiple stages (i.e., problem conceptualization, prevention activity implementation, evaluation). |
3. Use of Information Technology
This data will be collected during in person site visits as much as possible. In rare circumstances (e.g., when a key stakeholder is on leave during the site visit), a follow-up phone call may be used.
4. Non-duplication
The pre-visit request for information will obtain all existing information in support of the evaluations, such that the data collected on site will be unique and not duplicate existing information collections.
The information obtained through this collection is unique and is not already available for use or adaptation from another cleared source.
5. Burden on Small Businesses
This information collection does not impose a significant economic impact on a substantial number of small businesses or entities.
6. Less Frequent Collection
On-site information collection will take place at 30 installations identified as high and low risk on a biennial basis. Different installations will likely be selected for each evaluation cycle which will minimize burden of these information collections on the respondents.
7. Paperwork Reduction Act Guidelines
This collection of information does not require collection to be conducted in a manner inconsistent with the guidelines delineated in 5 CFR 1320.5(d)(2).
8. Consultation and Public Comments
Part A: PUBLIC NOTICE
A 60-Day Federal Register Notice (FRN) for the collection published on Thursday, November 4, 2021. The 60-Day FRN citation is 86 FR 60804 FRN 60804.
No comments were received during the 60-Day Comment Period.
A 30-Day Federal Register Notice for the collection published on Friday, January 14, 2022. The 30-Day FRN citation is 87 FR 2424 FRN 2424.
Part B: CONSULTATION
DoD contracted with the RAND Corporation to support the development of the methods and metrics for the pilot on-site installation evaluations (OSIEs) and to collect and analyze pilot data. RAND subcontracted with the Wandersman Center to supplement its data collection capability. Given the personnel and manpower hours necessary to support the number of engagements scheduled for each biennial site visit and the amount of data analysis to be completed after each visit, DoD will continue to contract with a third party (TBD) to ensure the rigor of all products.
9. Gifts or Payment
No payments or gifts are being offered to respondents as an incentive to participate in the collection.
10. Confidentiality
A statement of confidentiality and limits of confidentiality (e.g., self/other harm) will be included in the survey as well as in the introduction of the discussions.
A Privacy Act Statement is not required for this collection because we are not requesting individuals to furnish personal information for a system of records.
A System of Record Notice (SORN) is not required for this collection because records are not retrievable by PII.
A Privacy Impact Assessment (PIA) is not required for this collection because PII is
Not being collected electronically.
Any files collected by OFR will be securely transferred from OFR to the third party (TBD) using the secure system of OFR’s choice.
All data management processes and privacy standards will be enforced per DoD policy. Only members of the OSIE team (DoD, contractors, and approved subcontractors) will have access to the data. Project team members who are working with raw data may have copies on encrypted, password-protected computers or servers with access limited by system level permissions (depending on data analysis needs). Data will not be transmitted by email, but only by upload to a secure server to which only team members have access.
There will be no hardcopy materials or printouts of analysis results. Any printouts of discussion notes will be stored and subsequently destroyed per DoD policy.
11. Sensitive Questions
No questions considered sensitive are being asked in this collection.
12. Respondent Burden and its Labor Costs
Part A: ESTIMATION OF RESPONDENT BURDEN
Collection Instrument(s)
[On-site Interview Questions]
Number of Respondents: 2700
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 2700
Response Time: 1 hour
Respondent Burden Hours: 2700 hours
[Survey: Competency Assessment for Sexual Assault Prevention Practitioners (CASAPP)]
Number of Respondents: 1400
Number of Responses Per Respondent: 1
Number of Total Annual Responses: 1400
Response Time: 10 minutes
Respondent Burden Hours: 233 hours
Total Submission Burden (Summation or average based on collection)
Total Number of Respondents: 4100
Total Number of Annual Responses: 4100
Total Respondent Burden Hours: 2933 hours
Part B: LABOR COST OF RESPONDENT BURDEN
Collection Instrument(s)
[On-site Interview Questions]
Number of Total Annual Responses: 2700
Response Time: 1 hour
Respondent Hourly Wage: $28.22
Labor Burden per Response: $28.22
Total Labor Burden: $76,194
[Survey: Competency Assessment for Sexual Assault Prevention Practitioners (CASAPP)]
Number of Total Annual Responses: 1400
Response Time: 10 minutes
Respondent Hourly Wage: $28.22
Labor Burden per Response: $2.82
Total Labor Burden: $3948
An average DoD hourly wage was calculated from payscale.com, a compensation aggregation web service.
Overall Labor Burden
Total Number of Annual Responses: 4100
Total Labor Burden: $80,142
The Respondent hourly wage was determined by using the [Department of Labor Wage Website] ([http://www.dol.gov/dol/topic/wages/index.htm])
13. Respondent Costs Other Than Burden Hour Costs
There are no annualized costs to respondents other than the labor burden costs addressed in Section 12 of this document to complete this collection.
14. Cost to the Federal Government
Part A: LABOR COST TO THE FEDERAL GOVERNMENT
Please note, RAND through their contract ($2,000,000) with OFR will assume the burden for data processing of the two instruments.
Collection Instrument(s)
[On-site Interview Questions]
Number of Total Annual Responses: 4100
Processing Time per Response: 0 hours
Hourly Wage of Worker(s) Processing Responses: $0
Cost to Process Each Response: $0
Total Cost to Process Responses: $0
Collection Instrument(s) [Competency Assessment for Sexual Assault Prevention Practitioners (CASAPP)]
Number of Total Annual Responses: 4100
Processing Time per Response: 0 hours
Hourly Wage of Worker(s) Processing Responses: $0
Cost to Process Each Response: $0
Total Cost to Process Responses: $0
Overall Labor Burden to the Federal Government
Total Number of Annual Responses: 8800
Total Labor Burden: $0
Part B: OPERATIONAL AND MAINTENANCE COSTS
Please note, RAND through their contract ($2,000,000) with OFR will assume the burden for data processing of the two instruments.
Cost Categories
Equipment: $0
Printing: $0
Postage: $0
Software Purchases: $0
Licensing Costs: $0
Other: $0
Total Operational and Maintenance Cost: $0
Part C: TOTAL COST TO THE FEDERAL GOVERNMENT
Total Labor Cost to the Federal Government: $2,000,000
Total Operational and Maintenance Costs: $0
Total Cost to the Federal Government: $2,000,000
15. Reasons for Change in Burden
There has been both an increase and decrease in burden. The burden has increased since the previous approval for “On-Site Interview Questions” due to an increase in site evaluations. Beginning 2023, biennial information collections will be used to evaluate 30 DoD installations each year where the military community is at increased or decreased risk for destructive behaviors based on the revised methods and metrics refined through the initial pilot. The initial burden was estimated at 20 sites. There was also a decrease in burden for “Competency Assessment for Sexual Assault Prevention Practitioners (CASAPP)” as the information collection protocols have been streamlined and the time to complete the instruments were revised based on the first round of evaluations. These changes resulted in a slight decrease in burden.
The results of this information collection will be provided as a report to the Secretary of Defense and will not be published outside of DoD unless directed by the Secretary.
17. Non-Display of OMB Expiration Date
We are not seeking approval to omit the display of the expiration date of the OMB approval on the collection instrument.
18. Exceptions to “Certification for Paperwork Reduction Submissions”
We are not requesting any exemptions to the provisions stated in 5 CFR 1320.9.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kaitlin Chiarelli |
| File Modified | 0000-00-00 |
| File Created | 2022-01-25 |