Supporting Statement A_2900-0730_Updated 120715

Development of the

Deployment Risk and Resilience Inventory (DRRI)

Follow-Up Study, VA Form 10-21087

OMB 2900-0730

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify legal or administrative requirements that necessitate the collection of information.

The need to validate measures for use with the newest deployment cohort, including Veterans of the wars in Iraq and Afghanistan, has been identified as a critical need by both Veterans Affairs (VA) and the Department of Defense (DoD). The current request for a revision to OMB 2900-0730 is responsive to this identified need by proposing additional data collection with a sample of Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans for the purpose of validating updated scales for assessing deployment-related risk and resilience factors that have documented implications for posttraumatic stress disorder (PTSD) and other mental health problems. The originally approved OMB project (VA Form 10-21087) involved collecting data from Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) veterans to further refine and validate updated Deployment Risk and Resilience Inventory (DRRI) scales with respect to mental health outcomes. The purpose of the present request for a revision to this OMB-approved project is to conduct additional data collections with OEF/OIF Veterans who participated in the original survey for the purpose of further exploring the construct validity of these scales. Specifically, the goal of this follow-up study is to examine deployment-related factors assessed in the DRRI as they relate to subsequently assessed occupational and family outcomes, as well as VA service use. The long-term goal of this project is to provide a suite of scales that will be optimally useful to researchers and clinicians interested in studying factors that increase or reduce risk for PTSD and other health problems among Veteran and military samples.

Legal authority for this data collection is found under 38 USC, Part I, Chapter 5, Section 527 that authorizes the collection of data that will allow measurement and evaluation of the Department of Veterans Affairs Programs, the goal of which is improved health care for veterans.

2. Indicate how, by whom, and for what purposes the information is to be used; indicate actual use the agency has made of the information received from current collection.

This project is responsive to identified research priorities within VA and DoD. This work will build on a data collection for a recently funded project (by VA Health Services Research & Development) to update and validate DRRI-2 scales, which was approved under OMB 2900-0730. Objectives for this recently completed study were to administer DRRI scales to a national sample of OEF/OIF veterans, conduct psychometric analyses, and use this information to finalize the survey instrument and provide evidence for the validity of these scales in terms of their associations with post deployment health outcomes. The objectives of the upcoming data collection with the same sample are to: administer scales measuring functioning, mental health, and use of VA services in follow-up surveys of OEF/OIF veterans from the prior study to further examine the construct validity of the DRRI-2 with respect to the occupational and family functioning of both female and male veterans and implications for VA service use. The forthcoming data collection is particularly concerned with the predictive validity of the DRRI-2 in these functional contexts. The project will provide information to assist military leaders to better prepare personnel for future deployments and DoD and Veterans Health Administration (VHA) health-care policy-makers and practitioners to plan and implement more effective prevention and treatment programs, as well as services for returning veterans. The information will also be used to publish articles that will increase the knowledge base about the physical and mental health status and needs of OEF/OIF service members. The researchers will disseminate the findings to organizations with interest in active duty service members and in mental health and functioning among veterans. These organizations will include the Department of Defense, the Office of Seamless Transition, Veterans Health Administration, Quality Enhancement Research Initiative programs, the National Center for Post-Traumatic Stress Disorder, the VHA Mental Illness, Research, Education and Clinical Care Centers, the Women Veterans Health Strategic Healthcare Group, VA Family Services, and the Women’s Mental Health and Military Sexual Trauma Group within the Office of Mental Health Services.

The information will be provided through reports, presentations at national conferences, and publications in scientific peer-reviewed journals. It will also be provided to VHA clinical mental health service managers, clinical program leaders, and administrators and policy makers. Several VHA medical centers are interested in developing appropriate programs to address the mental health and functioning needs of the post-deployed population. This data collection will assist those efforts.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

This project involves surveys which will be administered in a mailed paper-and-pencil format. No information will be collected through automated, electronic, mechanical, or other technological means. Conducting the survey electronically would mean that Veterans without access to computers or the Internet would be excluded from the study and preclude the ability to randomly sample a representative population of Veterans.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The information that will be collected is not readily available and cannot be obtained in any way other than by self-report. Given that the lead Principal Investigator (PI) on this project is an author of the DRRI, there is confidence that these scales have not been validated in other work. In addition, with respect to the aims of the follow-up data collection, this will be the first study to examine the predictive validity of the DRRI-2. It will also be the first to examine gender differences in the effects of deployment stressors and their associated mental health sequelae on veterans’ post deployment functioning in both work and family domains, as well as their need for and use of a broad range of VA programs and services (i.e., health-care, employment, and education services).

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

No small businesses or other small entities are impacted by this information collection.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

The initially approved data collection activity occurred one time only. The follow-up data collection that is included in this request for a revision to the originally approved OMB clearance involves administering a survey assessing occupational and family functioning, mental health symptoms, and use of VA services to the same participants twice over a 3-year period. Data for the follow-up surveys will be collected on OEF/OIF service members previously assessed on deployment factors related to risk and resilience, as well as physical and mental health following deployment. This activity will allow the VHA to be responsive to a very important population of veterans and to develop interventions that will meet their needs. VHA would not be able to be as responsive to the needs of this population of patients if the data collection is not conducted.

7. Explain any special circumstances that would cause an information collection to be conducted more often than quarterly or require respondents to prepare written responses to a collection of information in fewer than 30 days after receipt of it; submit more than an original and two copies of any document; retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years; in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study and require the use of a statistical data classification that has not been reviewed and approved by OMB.

There are no such special circumstances.

8. a. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the sponsor’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the sponsor in responses to these comments. Specifically address comments received on cost and hour burden.

The notice of Proposed Information Collection Activity was initially published in the Federal Register on July 24, 2015 (Volume 80, Number 44200, pages 44200 - 44201). No comments have been received in response to this notice.

b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, clarity of instructions and recordkeeping, disclosure or reporting format, and on the data elements to be recorded, disclosed or reported. Explain any circumstances which preclude consultation every three years with representatives of those from whom information is to be obtained.

To determine the data elements to be developed and collected, the investigators consulted with researchers within the VHA. In addition to conducting a thorough review of all documents, both those distributed by the Federal Government and those published in the scientific literature, the Principal Investigator and research team have engaged in lengthy discussions with other researchers and experts on what data elements are needed regarding psychosocial risk and resiliency factors and measures of postdeployment functioning, as well as how these data elements should be queried. Although DoD does some screening of post-deployed service members, they are not using the comprehensive standardized measures proposed for use in this research study. Consequently, these data are not available from any other source. Outside consultation will be conducted with the public through the 60- and 30-day Federal Register notices.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

All potential study participants will receive a small token of appreciation in the amount of $25 in the first mailing of the survey for both of the forthcoming data collections, just as they did for the initial phase of data collection. The decision to include this token is a response to the finding that response rates are better when incentives are used. High response rates are important to ensure the generalizability of study results.

10. Describe any assurance of privacy to the extent permitted by law, provided to respondents and the basis for the assurance in statue, regulation, or agency policy.

Participants will be assured that all data will be kept private as permitted by law, and that no identifying information will be used in any dissemination activities. Study participants will review an information sheet that contains all the elements of an informed consent form. Participants are not required to complete any portion of the consent page. Completion of the survey implies consent. All data collection procedures will be approved by the Institutional Review Board (IRB) at the VA Boston Healthcare System.

11. Provide additional justification for any questions of a sensitive nature (Information that, with a reasonable degree of medical certainty, is likely to have a serious adverse effect on an individual's mental or physical health if revealed to him or her), such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private; include specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

The initial data collection instrument did not include questions that would be likely to have a serious adverse effect on an individual's mental or physical health. There were some questions that may have been sensitive or elicited emotional responses for particular individuals. This includes items from the DRRI scales: Sexual and General Harassment; Combat Experiences; and Aftermath of Battle. Additionally, some items on the Post-traumatic Stress Disorder questionnaire may have been sensitive for certain individuals. Although there will also be some sensitive questions in the follow-up data collection (e.g., questions about mental health symptoms), individuals are aware of these issues if they apply to them and will be electing to reveal the information. In addition, in order to understand risk and resilience factors for post-deployment veterans, these data are crucial. As described previously, a number of safeguards have been put in place to protect the privacy and confidentiality of participants. In addition to these strategies, respondents are provided the opportunity to talk to the clinical contact, Dr. Karen Mitchell (a licensed and trained Clinical Psychologist in the Women’s Health Sciences Division of the National Center for PTSD) if they should become distressed. Finally, information will not be obtained from anyone other than the survey respondent.

12. Estimate of the hour burden of the collection of information:

The study questionnaires are sent to potential participants in multiple steps via standard U.S. mail. A cover letter is included with the questionnaire, which details the purpose of the research, assures that responses will be kept private as allowed by law, emphasizes the voluntary nature of participation, states an estimated time to complete the survey instrument, emphasizes that we are interested in group data and not a particular person’s individual standing, and provides information on risks and benefits. This letter conforms to all standards for the protection of human subjects. An opt-out form allowing potential participants to indicate that they did not want to be contacted again is also included in this mailing, which can be returned in the provided postage-paid envelope.

Data collection for the initial study was split into two waves to minimize the number of items each participant is asked to complete. Less than half of the sample (n = 463), that which is required to ensure adequate power for hypothesis testing, were asked to complete original and revised versions of DRRI items to allow for an examination of incremental validity as well as a brief measure of Posttraumatic Stress Disorder. The other half of the sample (n = 1044) completed only new DRRI scales along with several measures of mental and physical health. The time necessary to complete the initial data collection was estimated to be 60 minutes for the first wave, and 50 minutes for the second. Given an estimate that participants can complete 5-10 items per minute, the range of time to complete the surveys for the follow-up data collections that are proposed in this revision is estimated to be no more than 45 minutes, as there are approximately 230 items in the survey. Data collection for the follow-up assessments will be split into two data collection periods with the same sample over a three-year period to allow for the examination of mechanisms related to post deployment functioning over time.

The estimated burden hours for the follow-up data collection proposed in this revision is 754 respondents x 45 min. per respondent / 60 = 566 annual respondent hours

The annual burden is estimated to be 566 hours.

b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB 83-I.

The request covers only one form. The follow-up request is for the revised form, which will be administered for the data collections.

c. Provide estimates of annual cost to respondents for the hour burdens for collections of information. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14 of the OMB 83-I.

There will be no cost to the respondent.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

There will be no costs to respondents or record keepers.

14. Provide estimates of annual cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operation expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The estimated total budget for the follow-up data collections is $782,853, including staff support, collection and analysis of survey data, reporting of results, dissemination of results, etc.

FY 14: Salary = $154,755; Survey contractor/participant payments/consultants = $112,467;

Total = $267,222

FY 15: Salary = $154,755; Survey contractor/participant payments/consultants = $112,467; travel = $713;

Total = $267,935

FY 16: Salary = $154,755; Consultants = $7425; travel = $713;

Total: $162, 893

FY 17: Salary = $77,378; Consultants = $7425;

Total = $ 84,803

15. Explain the reason for any burden hour changes since the last submission.

There is no change in burden.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The initial study involved the administration of DRRI scales, as well as measures to assess PTSD and other health problems and potential confounders. Additional measures were included for the purpose of examining discriminant and criterion-related validity of the DRRI-2, as well as conducting secondary analyses to address substantive questions about relationships between deployment risk and resilience factors and measures of physical and mental health. With respect to criterion-related validity analyses, the primary hypothesis asserted that multiple dimensions of risk and resilience would be associated with PTSD and other health measures, such that OEF/OIF veterans who report greater exposure to risk factors (e.g., combat exposure) and less access to resilience factors (e.g., post-deployment social support) endorse more symptoms of PTSD and other mental and physical health problems.

Both Classical Test Theory (CTT) and Item Response Theory (IRT) analyses were conducted to inform the psychometric evaluation of DRRI scales and the development of abbreviated item scales. CTT analyses employed the Statistical Package for the Social Sciences (SPSS); IRT analyses were conducted using the PARSCALE program and Winsteps statistical software, as appropriate. As expected based on prior studies, missing data did not result in the loss of more than 5% of the sample in the initial study.

The frequencies of distribution and probabilities of endorsement for each DRRI item were examined to confirm the quality of selected items, and internal consistency reliability estimates for each finalized scale have been computed.

To evaluate Hypothesis 1 (which posited main effects of risk and resilience factors on PTSD and other health measures), and to provide critical evidence for the criterion-related validity of DRRI scales, bivariate correlations were calculated between each of the DRRI scales and scores on PTSD and associated health problems. To account for any multicollinearity among deployment risk and resilience factors within deployment timeframes, this set of analyses was followed by simultaneous regression analyses in which each of the health measures is regressed on each sets of pre-deployment, deployment, and post-deployment risk and resilience factors, in turn. Consistent with prior work (Vogt et al., 2005), deployment factors were split into two separate sets of mission-related and interpersonal stressors, and analyses were conducted separately for these two sets of variables. Significant partial regression coefficients (serving as estimates of effect size) provide evidence for the unique predictive validity of each of the deployment factors in accounting for scores on PTSD and other health problems.

A final set of hierarchical regressions was conducted to account for any multicollinearity across deployment timeframes. For these analyses, significant pre-deployment, deployment, and post-deployment predictors from the previous set of analyses were entered into a series of regression analyses predicting each of the health measures, in turn. For each health measure, the set of significant pre-deployment factors was entered first, followed by the set of significant deployment factors, and then by the set of significant post-deployment factors (preceded by measures of self-report bias if necessary). To the extent that each set of variables was determined to account for significant variance in the outcome, partial regression coefficients were examined for each predictor.

Confirmatory IRT analyses were conducted on the DRRI scales and this information was used to identify final sets of abbreviated items for each of the DRRI scales following recommended procedures in the literature (Embretson & Reise, 2000; Hambleton et al., 1991).

The follow-up data collection will be conducted to assess how DRRI scales relate to functioning (e.g., work and family functioning), as well as use of VA services and programs. Additional measures are included for the purpose of examining predictive validity of the DRRI-2, as well as conducting secondary analyses to address substantive questions about relationships between deployment risk and resilience factors, and measures of physical and mental health, functioning, and VA service use. The figure below presents the conceptual framework that guided the follow-up study design.

Proposed Conceptual Model Examining the Predictive Validity of the DRRI-2

Structural equation modeling (SEM) will be the primary approach applied in testing study hypotheses, supplemented by chi square tests, t-tests, and regression analyses when appropriate. SEM was chosen as the primary method of analysis because it presents important advantages over traditional regression. First, all paths in the tested models are estimated simultaneously, thereby making use of all available data and increasing the accuracy of parameter estimates. Second, because SEM allows for the modeling of measurement error, standard errors are minimized, resulting in estimates of relationships among variables that are as close to the true values as possible. Third, models will be estimated in Mplus, which uses all available raw data to estimate model parameters via full information maximum likelihood (Muthén & Muthén, 2004).

Consistent with recommendations, measurement models will be assessed prior to testing structural models for all applications of SEM in this study. The aim of the measurement component is to clarify the operationalization of latent variables in terms of their manifest indicators (i.e., confirmatory factor analysis). A matrix of variances and covariances will be analyzed, whereby items will be specified to load on only one factor, and residuals will be constrained to be orthogonal (McArdle, 1996). Consistent with convention, maximum likelihood (ML) estimation will be applied. A number of fit indices will be consulted to determine the adequacy of the resulting factor solution to reproduce the variances and covariances among observed scores, including, but not limited to, the root mean square error of approximation (RMSEA; Steiger, 1990) and the comparative fit index (CFI; Bentler, 1990). Close fit, according to contemporary criteria and standards, will provide endorsement for the latent structure of the variables. After establishing the fit of the measurement model, structural paths will be added to estimate the relationships among latent variables. Evidence for the importance of proposed associations will be found in the significance of the path coefficients, as well as overall fit of the model. Mediation will be tested using the chi-square difference test, as fully and partially mediated models are nested within one another (see descriptions below). A non-significant chi-square provides evidence that the more parsimonious model (i.e., the fully mediated model) should be retained. In order to further examine mediation, the significance of the indirect effects of a given independent variable (IV) on a given dependent variable (DV) will be tested using the “effects” option within Mplus, which calculates the products of all paths between two variables and standard errors to test their significance. Importantly, a multi-group SEM approach will be applied to conduct gender comparisons in all SEM-based analyses. In multi-group SEM, two models are compared—one with paths estimating freely for both groups and one with paths constrained to be equal across groups. As these models are nested within one another, the chi square difference test will be used to determine whether the models are equivalent for women and men. A significant chi square indicates that the models are not equivalent. If significant, the equivalence of specific paths will be tested.

We will attend to both statistical significance and effect sizes for all analyses. To protect against an inflated Type I error rate associated with multiple tests in regression analyses, we will apply a sequential Bonferroni-type procedure to control the false discovery rate should this be necessary (Benjamini & Hochberg, 1995). In addition, if incomplete data rates exceed 5%, we will apply a full-information maximum likelihood estimation procedure (Graham, Hofer, Donaldson, MacKinnon, & Schafer, 1997; Little & Rubin, 2002) to achieve reduced standard errors and more precise parameter estimates (Arbuckle, 1996; McArdle & Bell, 2000). Finally, potential confounders of study relationships will be considered, as appropriate. For example, it may be necessary to control for physical health status in the examination of associations between family functioning and mental health service use.

The study sample is currently available, as we will be following up with the same study sample used in the original study. For the initial study, approval was obtained to draw a national random sample of OEF/OIF veterans from Defense Manpower Data Center (DMDC), which maintains automated files of military personnel with the necessary demographic information to support the study. The study panel of participants from the initial data collection includes approximately 1,044 Veterans. All participants had separated from service, and had recently returned from deployment. Women were oversampled relative to their proportion in the population of Veterans to allow for systematic testing of gender differences, resulting in a final sample with 54% female and 46% male. Veterans deployed from the National Guard/Reserves were also oversampled to allow for supplemental analyses of subgroup differences between Active Duty and National Guard/Reservist personnel, with 44% of the final initial sample identified as National Guard/Reservist personnel. All participants from this initial sample who agreed to be re-contacted for follow-up studies will be invited to participate in both follow-up data collections.

Initial Data Collection

Figure 1. Activity	Year 1 Year 2						Year 3
	Jul-Nov 2009	Dec 2009- Apr 2010	May-Sept 2010	Oct-Feb 2010	Mar 2011-July 2011	Aug- 2011-Dec 2012		Jan-May 2012	Jun 2012-Oct 2012
Pre-investigation Tasks
Recruit and hire research assistants	X
Seek IRB approval for study	X
Convene planning meetings with co-investigators	X
Confer with project consultants	X
Finalize survey instrument for study	X
Wave I
Prepare survey materials for multi-stage data collection	X
Create survey tracking program	X
Develop data management system	X
Coordinate with DMDC to select sample	X
Secure veteran list from DMDC	X
Multi-stage survey administration – Phase I		X
Data entry – Wave I		X	X
Compute initial item and scale properties			X
Compute initial item response theory analyses			X
Refine scales as needed			X	X
Wave II
Prepare survey materials				X
Multi-stage survey administration – Wave II					X	X		X
Data entry – Part II								X
Conduct final psychometric analyses on full scales								X
Compute final item response theory analyses								X
Confer with project consultants								X
Finalize all scales								X	X
Prepare conference presentations								X	X
Prepare manuscripts for publication								X	X
Develop manual to accompany DRRI and abbreviated scales								X	X

Follow-up Data Collections

Activity

Year 1

Year 2

Year 3

Year 4

Oct-Dec 2013

Jan- Mar 2014

Apr-Jun 2014

July-Sep 2014

Oct-Dec 2014

Jan-Mar 2015

Apr-Jun 2015

Jul-Sep 2015

Oct- Dec 2015

Jan-Mar 2016

Apr- Jun 2016

Jul- Sep 2016

Oct- Dec 2016

Jan-Mar 2017

Infrastructure Preparation

Recruit, hire and train research staff

X

X

Secure contract with survey research firm

X

X

Finalize study materials

X

X

Submit materials for IRB approval

X

X

Attain OMB approval

X

Convene planning meetings with Co-Investigators & Project Consultants

X

Submit participant roster to an IRS address search

X

Time 2 Data Collection

Prepare mail survey materials

X

Develop data management system

X

Six-stage mailing to 894 potential study participants

X

X

X

Data entry and scoring

X

X

Calculate descriptive statistics on variables in T2 dataset

X

Confer with Project Consultants

X

Hypothesis testing and other study analyses

X

X

X

Prepare summaries of findings for broader dissemination

X

X

Work with operational partners within VA to translate findings into enhanced VA services

X

X

Preparation of conference submissions and presentations

X

X

Preparation of manuscripts for publication

X

X

Time 3 Data Collection

Prepare mail survey materials

X

Six-stage mailing to 894 potential study participants

X

X

X

Data entry and scoring

X

X

Calculate descriptive statistics on variables in T3 dataset

X

Confer with Project Consultants

X

X

Hypothesis testing and other study analyses

X

X

Prepare summaries of findings for broader dissemination

X

X

Work with operational partners within VA to translate findings into enhanced VA services

X

X

Preparation of conference submissions and presentations

X

X

Preparation of manuscripts for publication

X

X

Prepare final report for funding agency

X

*Time 3 data collection to begin in June 2016

17. If seeking approval to omit the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

There are no requests for approval to omit the expiration date for the OMB approval of the information collection.

18. Explain each exception to the certification statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB 83-I.

There are no exceptions.

Page 1