BHMC Supporting_Statement_PartB_05.08.2018

Supporting Statement Outline – Sample

NOTE: Complete Part B for Survey ICR Requests

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

If the collection of information employs statistical methods, it should be indicated in Item 17 of OMB Form 83-I, and the following information should be provided in this Supporting Statement:

1.  Description of the Activity

The Building Healthy Military Communities (BHMC) pilot is a multi-year initiative that aims to better understand unique challenges faced by geographically dispersed Service members and their families that may impact force readiness and well-being. The Office of the Under Secretary of Defense for Readiness (OUSD(R)) BHMC pilot is conducting a Rapid Needs Assessment (RNA) to establish a baseline assessment of readiness requirements and available resources to support these requirements at a state level, as well as to identify current gaps in resources for Service members across all components. The pilot will be conducted in three phases over the course of three years: 1) Planning, 2) Implementation, 3) Evaluation. The pilot is currently in the planning phase. Phases 2 and 3 will be conducted iteratively to allow for periodic evaluation and adjustment of interventions, as necessary.

2.  Procedures for the Collection of Information

Describe any of the following if they are used in the collection of information:

1. Statistical methodologies for stratification and sample selection;

Interviews selection: Nonprobability sampling, the resource providers were selected on the basis of their availability (e.g., because they were found through public searches and they volunteered when asked) or personal judgment that they are representative. The consequence is that an unknown portion of this population is excluded (e.g., those who did not volunteer or could not be found). For this convenience sample, we chose interviewees from Service member-dense geographic sections of each pilot state in order to maximize our travel budget and human resources. In addition to soliciting names of interviewees from BHMC team members, we asked State Coordinators, State Family Program Directors and other colleagues in our seven pilot states for suggestions of Resource Managers they felt would be important to interview. We have paid special attention to including struggling programs to identify gaps and challenges which must be focused on.

Interview analysis:

The following methodology for analyzing interview data affords the research evaluation team the ability to collect, manage, and code large amounts of qualitative data during the Needs Assessment and verify it with members of the team conducting the field interviews. A dedicated notetaker will take notes in a formatted template during each of the qualitative interviews according to the project’s protocol. After verifying content with interviewers, notetakers will then upload the de-identified notes onto a secure file sharing site (Box.com) on a daily basis. One of the researcher team members will then import the qualitative interview notes into Dedoose, a secure password-protected web-based qualitative data analysis application (Dedoose Version 7.0.23, Los Angeles, CA: SocioCultural Research Consultants, LLC). The research team will conduct all analyses within this data management and analysis system. Using a codebook of qualitative theming codes and sub-codes created for this project, the research team will analyze a subset of interviews together and once a substantial and acceptable inter-rater reliability (i.e.. Kappa > .70) is achieved, coders will analyze the remaining interviews individually. The research team will present the themed data to the group of interviewers and notetakers for verification and discussion. A report synthesizing the key codes and subcodes will be compiled for each of the seven BHMC pilot states.

Social Media Data Mining:

This purpose of this activity is to illuminate how military personnel and family members in seven pilot states discuss health & wellness topics on social media. It establishes a baseline for measuring the impact (on volume and sentiment) of future BHMC campaigns. Synoptos, a media intelligence company, has been contracted to perform this activity.

The BHMC team, in conjunction with Synoptos, developed a list of keywords related to health, readiness, and fitness, and focusing on the Total Force Fitness domains. Using publicly available postings on social media
(primarily Twitter and Instagram), Syntopos will isolate users who self-identify as having a military connection (Active or Reserve component SM in any Service; spouse or family etc.) and search for postings that match any of the BHMC keywords. They will also examine postings of Institutional Accounts — publicly available military social media accounts, or an account representing a military-connected institution. They will analyze both the number of posts and sentiment of those posts.

Sentiment Analysis is a subjective reading of the poster’s expressed attitude toward Total Force Fitness goals and can be labeled in 3 ways: 1) “Supportive” — Supportive of or concerned about Total Force Fitness goals, i.e. for a healthy lifestyle; 2) “Contrary” — Contrary to or dismissive of Total Force Fitness goals, i.e. against a healthy lifestyle; and 3)
“Neutral” — Neutral to Total Force Fitness goals, or no opinion expressed. This activity will be conducted for a baseline report and at multiple time points (TBD) throughout the pilot.

The BHMC team anticipates that this information will be used by State Coordinators to assess needs and gaps in their states and to help inform their Strategic Plans. This data will also be used to assess changes in the attitudes of SMs and families related to health, wellness, and readiness topics throughout the course of the pilot.

b.  Estimation procedures;

N/A

c.  Degree of accuracy needed for the Purpose discussed in the justification;

N/A

d.  Unusual problems requiring specialized sampling procedures; and

N/A

e.  Use of periodic or cyclical data collections to reduce respondent burden.

N/A – Each participant will be interviewed once as part of the Needs Assessment phase. The Social Media Data Mining activity uses only retrospective and publicly available postings on social media.

3.  Maximization of Response Rates, Non-response, and Reliability

Discuss methods used to maximize response rates and to deal with instances of non-response.  Describe any techniques used to ensure the accuracy and reliability of responses is adequate for intended purposes.  Additionally, if the collection is based on sampling, ensure that the data can be generalized to the universe under study.  If not, provide special justification.

Qualitative Interviews – As these are qualitative face-to-face interviews, we simply sending calendar invitations to increase response rates. The interview will be prefaced with letting interviewees know they are free to decline to respond to any question. Interviews are scheduled at a time that is convenient for interviewees and a reminder will be sent the proceeding week. The interviewees were selected purposefully because of their expertise and knowledge regarding needs of SMs and theirs families and for their opinions our proposed interventions in later phases of the project.

Social Media Data Mining – We are pulling publicly postings on social media sites of people who self-identify as having a connection to the military (e.g. active duty, National Guard, Reserves, spouses etc.) and cannot be generalized beyond those groups. Maximization of response rates/non-response is not relevant. This is one of many sources of information SCs will use to determine needs and gaps and to inform their Strategic Plans.

4.  Tests of Procedures

Describe any tests of procedures or methods to be undertaken.  Testing of potential respondents (9 or fewer) is encouraged as a means of refining proposed collections to reduce respondent burden, as well as to improve the collection instrument utility.  These tests check for internal consistency and the effectiveness of previous similar collection activities.

Qualitative Interviews- We have conducted a qualitative interview training session (December 2016) with team members and beta tested the interview questions with project team members who have current or former military experience for comprehensibility and appropriateness and made chances as necessary.

Social Media Data Mining – N/A – this is publicly available retrospective data.

5.  Statistical Consultation and Information Analysis

a. Provide names and telephone number of individual(s) consulted on statistical aspects of the design.

Qualitative Interviews: Salvatore Libretto, PhD, Uniformed Services University of the Health Sciences – 703-772-4845

Social Media Data Mining: Mark Malseed, Synoptos – 202-360-2268

b. Provide name and organization of person(s) who will actually collect and analyze the collected information.

USU/CHAMP: Salvatore Libretto, Ashleigh Simon, Sheryl Hoehner

TLIF: Alyson Raynor, Charmagne Paat, Cindy Crawford, Courtney Boyd, Dawn Bellanti, Erika Stoerkel, Mac Beckner

NGB: Colleen Bush, Darla Siegel, Esther Sablan, Jason Mathewson, LTC Larry Dismore, Tim Hawley, Anthony Wickham

 

Deloitte Consulting: Eric Goldman, Kari Harris, Maddie Soskin, Otto Snell, Emily Broyles

OUSD(P&R): CAPT Kimberly Elenberg, Leonard Litton

DHA: LTC Todd Yosick

5