SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION
Prospective Studies of US Military Forces: The Millennium Cohort Study
OMB Control Number 0720 – 0029
JUSTIFICATION
Needs and Use
The concept and design manifest in the Millennium Cohort Study was recommended in the 1998 Institute of Medicine (IOM) Report “The Gulf War Veterans: Measuring Health”. Under the subheading “Strategies to Protect the Health of Deployed US Forces”, IOM recommended that prospective investigations be planned to evaluate multi-dimensional factors relevant to health and health change so that these factors can be assessed over the lifetime of the service member.
Section 743 of the Strom Thurmond National Defense Authorization Act for FY1999
authorized the Secretary of Defense to “…establish a center devoted to a longitudinal study to evaluate data on the health conditions of members of the Armed Forces upon return from deployment on military operations for purposes of ensuring rapid identification of any trends in diseases, illnesses or injuries among such members as a result of such operation.”
Language in the Floyd D. Spence FY2001 National Defense Appropriations Act “…longitudinal studies of military personnel before they are deployed to potentially hostile situations and after their return”, funded the activity.
The Millennium Cohort Study was designed in response to the IOM recommendation and to Congress’ authorization and funding, as a prospective, 21-year-long, multi-panel and wave, cohort investigation.
The main objectives of the undertaking are (1) to develop a long-term profile of health change among current and former members of the Armed Forces, especially in relation to individual deployment experience, and (2) to better define the nature of and risk factors for the development of post war illness among US military personnel. These objectives will be accomplished by joining electronic healthcare utilization, deployment, exposure, and demographic data available from other Department of Defense (DoD) sources for all participants, with self-reported health status information collected from the study participants. A survey instrument is used to collect self-reported baseline data as well as a series of follow-up surveys that are collected in 3-year intervals through 2022. For approximately 25% of the population, we are able to capture pre- and post-deployment metrics due to the timing of deployments. Participants are encouraged to complete follow-up surveys even after they separate from military service.
The first enrollment group, Panel 1, was randomly selected, probability-based, and oversampled for deployment from 1998-2000 to Southwest Asia, Bosnia or Kosovo, women, and Reserve/Guard. The second and third enrollment groups, Panels 2 and 3, were randomly selected, probability-based, and oversampled for women and Marines. These groups were oversampled to ensure sufficient statistical power to reveal findings if they exist in these smaller subgroups of the study population. All potential participants must be actively serving in the Armed Forces at the time of initial contact. Please see the following publication: Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, Corbeil TE, Hooper TI, for the Millennium Cohort Study Team. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. Journal of Clinical Epidemiology. 2007 Feb;60(2):181-91.
Since the study is currently in the process of finalizing the cohort for Panel 3, the following numbers should be understood as estimates:
By July 1, 2003, 77,047 (36%) had returned a Panel 1 baseline questionnaire. Panel 1 targeted 256,400 service members of whom 213,949 had valid addresses allowing for study contact attempt.
Among Panel 1 participants, 55,021 have submitted the first follow-up survey (71%), and approximately 54,790 have submitted the second follow-up survey (71%). Participation in the second follow-up survey consisted of 84% web and 16% paper responders. Of those that did not submit their first follow-up survey, 8,259 submitted their second follow-up survey. 46,439 Panel 1 participants have responded to the enrollment and two follow-up surveys.
By February 15, 2006, 31,110 (25%) had returned a Panel 2 baseline questionnaire. Panel 2 targeted 150,000 of whom 122,410 were determined to have valid addresses allowing for study contact attempt.
Among Panel 2 participants, 17,151 have submitted a first follow-up survey (55%), and participation consisted of 83% web and 17% paper responders.
Panel 3 enrollment targeted 200,000 service members of whom 153,650 had valid addresses, in order to enroll an additional 40,000 individuals. By December 31, 2008, 43,440 had returned a baseline questionnaire with 93% of responses occurring via web and 7% via paper.
During the 2010 survey cycle, we will attempt to contact 250,000 service members to enroll 50,000 individuals in Panel 4.
Since enrollment began in 2001, more than 151,000 service members have consented and submitted a baseline survey (36.0%, 25.4%, and 28.3% baseline response rates for Panels 1, 2 and 3 respectively).
To date, a total of 521 deaths have occurred within the Panel 1 responder group. A total of 89 deaths have occurred within the Panel 2 responder group, and a total of 22 deaths have occurred within the Panel 3 responder group.
Approximately 22,375 of the 77,047 Panel 1 responders had left military service and transitioned to civilian life by the time they submitted their third follow-up survey and approximately 5,299 of Panel 2 responders had left military service and transitioned to civilian life by the time they submitted their second follow-up survey. We estimate that slightly more than a total of 36,599 participants from Panels 1, 2 and 3 will have separated by the time they are contacted for their next follow-up surveys in early 2010.
At the end of this fourth enrollment phase, we expect to have enrolled a total of 200,000 service members. As of our first OMB approval in September 2003 and throughout the course of the study, proportions of military versus civilian participants will shift in favor of civilians.
Purpose and Users of the Information
As noted in the response to item 1, overall, the purpose of the information collected by the Millennium Cohort Study is to assist US policy makers understand long-term health outcomes that may be associated with military service, especially of deployment and service in hostile operations. Specifically, the study seeks to track the development of major chronic, physical, and psychiatric illnesses among veterans, and establish risk factors through temporal sequence of exposure and health outcome.
DoD and Department of Veterans Affairs (DVA) policy makers and researchers will use these prospective strategic analyses to develop prevention and treatment strategies that will maintain and improve the well-being of service members.
Information Collection Techniques
The military maintains electronic records pertaining to inpatient and outpatient healthcare utilization, immunization, demographic and deployment status for all personnel. Through data use agreements, we are able to access these data for study participants and link this information to self-reported data. Self-reported health status information is obtained both during and after military service when participants complete either a paper or web-based health status questionnaire, every 3 years, through 2022.
Approximately 46,000 of 77,047 Panel 1 responders, 27,000 of 31,110 Panel 2 responders, and 40,335 of the 43,705 Panel 3 responders elected to fill out the web-based version of the Millennium Cohort Study questionnaire. The combined entry of the subject's randomly assigned study ID and last 4 digits of their SSN are verified prior to allowing access to the questionnaire or entry of any survey responses. All exchanges between survey participant web browsers and web server software are made over secure (128-bit encrypted) connections based upon well-established and widely accepted Secure Sockets Layer (SSL) technology.
The designers of the Millennium Cohort Study have not made 100% web-based participation an objective or a goal. Realizing the benefits of allowing both web and paper-based submissions, an approach that employs bi-modal submission was offered to all participants. For more information on objectives related to web enrollment and comparisons between web and paper responders please see Smith B, Smith TC, Gray GC, Ryan MAK, for the Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. American Journal of Epidemiology. 2007 Nov;166(11):1345-54.
The first telephone survey of nonresponders was performed after the Millennium Cohort Pilot Study and focused on survey content and reasons for nonresponse. More recently, in 2005, a telephone survey of nonresponders was conducted during the 2004-2006 data collection effort. The attached report, “MilCohort NonResponse Study Final Report”, describes the Millennium Cohort Telephone Study of 3,000 nonresponders conducted by the Research Triangle Institute (RTI). This substudy consisted of telephone calls as well as moderate tracing of nonresponders that could not be reached by telephone. The telephone survey was administered to Panel 1 participants who had not completed a 2004-2006 questionnaire and resulted in a 31% response rate. In addition to asking questions regarding reasons for nonresponse, the phone survey asked about incentives, participant contact, and collected information on health status.
Please see Chapter 3 of RTI's “MilCohort NonResponse Study Final Report”, for telephone questionnaire results and recommendations. Overall recommendations covered six main areas: study materials, panel maintenance, tracing sample members, incentives, telephone prompting, and future non-response studies. Many of these suggestions were incorporated in the 2007-08 survey cycle effort, including tailoring messages for service specific and separated participants, emphasizing surveys can be completed on the web or by paper, tracing participant mailing addresses via US Postal Service services (i.e. National Change of Address program, Address Change Service), offering new incentives this past survey cycle, and testing automated telephone messaging to consented participants. As a result of these additional marketing tactics, 8,259 Panel 1 participants completed their second follow-up survey that did not submit their first follow-up survey.
Response and retention rates are of utmost importance to Millennium Cohort investigators. Much effort has been focused on investigation of the type and the use of incentives, wording of invitations, email contact, and twice yearly contact on Veteran’s Day and Memorial Day. Incentives include specially designed t-shirts and hats with study logos, phone cards, and Millennium Cohort coins. Invitations and email contact are designed specifically for service branch, separation status, and other demographics and vetted through study team members, co-investigators, and members of the Millennium Cohort Scientific Steering and Advisory Committee. Twice yearly contact using Veteran’s Day and Memorial Day serves to keep the service members engaged as well as solicit change of contact information. Retention centers around a website with sections dedicated to keeping those who have left military service in the cohort. Additionally, this enrollment cycle deployed a “welcome to the cohort” campaign that sent welcome cards to enrolling cohort members describing the scope and length of the study once again.
Since it is essential to know the biases that may exist in a study, we have proactively initiated 17 analysis plans dedicated to investigating potential biases and the generalizability of the Cohort. To date, investigations of potential biases in the Millennium Cohort Study have found no determinants for enrollment bias based on differences in health, established the reliability of self-reported data, determined high internal consistency for standardized instruments included within the questionnaires, and demonstrated Cohort members represent the US military. Please see the following manuscripts:
Wells TS, Jacobson IG, Smith TC, Spooner CN, Smith B, Reed RJ, Amoroso PJ, Ryan MAK, for the Millennium Cohort Study Team. Prior health care utilization as a determinant to enrollment in a 22-year prospective study, the Millennium Cohort Study. European Journal Of Epidemiology. 2008 Feb;23(2):79-87.
Smith B, Wingard DL, Ryan MAK, Macera CA, Patterson TL, Slymen DJ, for the Millennium Cohort Study Team. US military deployment during 2001-2006: comparison of subjective and objective data sources in a large prospective health study. Annals of Epidemiology. 2007 Dec;17(12):976-82.
LeardMann CA, Smith B, Smith TC, Wells TS, Ryan MAK, for the Millennium Cohort Study Team. Smallpox vaccination: comparison of self-reported and electronic vaccine records in the Millennium Cohort Study. Human Vaccines. 2007 Nov/Dec;3(6):245-51.
Smith B, Smith TC, Gray GC, Ryan MAK, for the Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. American Journal of Epidemiology. 2007 Nov;166(11):1345-54.
Smith TC, Jacobson IG, Smith B, Hooper TI, Ryan MAK, for the Millennium Cohort Study Team. The occupational role of women in military service: validation of occupation and prevalence of exposures in the Millennium Cohort Study. International Journal of Environmental Health Research. 2007 Aug;17(4):271-84.
Smith TC, Smith B, Jacobson IG, Corbeil TE, Ryan MAK, for the Millennium Cohort Study Team. Reliability of standard health assessment instruments in a large, population-based cohort study. Annals of Epidemiology. 2007 Jul;17(7):525-32.
Smith B, Leard CA, Smith TC, Reed RJ, Ryan MAK, for the Millennium Cohort Study Team. Anthrax vaccination in the Millennium Cohort: validation and measures of health. American Journal of Preventive Medicine. 2007 Apr;32(4):347-53.
Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, Corbeil TE, Hooper TI, for the Millennium Cohort Study Team. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. Journal of Clinical Epidemiology. 2007 Feb;60(2):181-91.
Riddle JR, Smith TC, Smith B, Corbeil TE, Engel CC, Wells TS, Hoge CW, Adkins J, Zamorski M, Blazer D, for the Millennium Cohort Study Team. Millennium Cohort: the 2001-2003 baseline prevalence of mental disorders in the US military. Journal of Clinical Epidemiology. 2007 Feb;60(2):192-201.
Chretien JP, Chu LK, Smith TC, Smith B, Ryan MAK, for the Millennium Cohort Study Team. Demographic and occupational predictors of early response to a mailed invitation to enroll in a longitudinal health study. Biomed Central Medical Research Methodology. 2007 Jan;7:6.
4. Duplication and Similar Information
Independent high echelon reviews conducted by the Defense Technical Objectives Board prior to the beginning and since completion of Panel 1 (see MD.25) indicate that the work of the Millennium Cohort Study is not being duplicated anywhere else in DoD, or indeed, across the Federal Government. Regular reviews by the independent American Institute of Biological Sciences (AIBS) similarly have reported no duplication of effort. On-going oversight by the Millennium Cohort Scientific Steering and Advisory Committee, made up of high-level civilian and military science professionals continues to report that this investigation remains unique among government funded military investigations. Lastly, yearly review by the DoD Defense Health Board further indicates no duplications of efforts with any other federal agency. In sum, at this time the Millennium Cohort Study does not duplicate any other federally sponsored military data collection effort.
5. Small Business
This collection of information does not involve small businesses or other small entities.
6. Less Frequent Collections
7. Special Circumstances
There have been (and we continue to anticipate) no special circumstances requiring the collection to be conducted in a manner inconsistent with the guidelines in 5 CFR 1320.5 (d) (2).
8. Federal Register Notice/Consultations
The Federal Register Notice for this collection was published in Volume 73 Number, 251 on 31 December 2008. There have been no inquiries or comments.
Study coinvestigators have been selected from all services and the DVA and include physicians, statisticians, and epidemiologists that meet monthly to review direction and progress. A Scientific Steering & Advisory Committee (SSAC) consisting of eight civilian and military experts in epidemiology, preventive medicine, statistics, questionnaire design, survey methodology, and veterans affairs, oversees the study on an on-going basis. This committee meets annually to review progress and recommend course corrections when needed. In addition, the Study’s staff confers regularly with experts in the nature and availability of demographic, deployment, and healthcare data from the Defense Manpower Data Center, Monterey CA.
Other consultation and oversight of the Millennium Cohort Study includes annual reviews by the NHRC internal Institutional Review Board, the Health Affairs’ Institutional Review Board, American Institute of Biological Sciences (AIBS), Office of Budget and Management (OMB), and Defense Health Board.
9. Payment/Gift to Respondents
An investigation into whether response rates differed by incentives offered prior to enrollment was conducted during the pilot phase of the Millennium Cohort Study. No differences in response rates were found including those who did not receive an incentive. Following this research, the Millennium Cohort Team chose to offer cost savings initiatives to motivate survey response via the internet. Cost-saving initiatives, while offered in return to those who complete the survey via the web, are unconditional and may be obtained upon request by any participant.
Based on findings from the RTI nonresponse study, during the 2007-2008 cycle several other incentivizing tactics were tested on consented participants, including sending automated voice messages, adding a $5 Starbucks gift card to the survey packet, and offering a hat incentive up front to encourage response. An increase in number of response was seen from these various avenues explored when compared with regular mail strategies (see attached graphs titled “Nonresponse pilot studies”). Of the Panel 1 participants that did not submit their first follow-up survey, 8,259 submitted their second follow-up survey.
Monetary incentives have been and continue to be considered as a viable option to increase response rate. However, existing military regulations prohibit the use of cash as reimbursement to participate in a study. The use of monetary incentives may be revisited in the future as more Millennium Cohort participants will separate from military service. A question will be added to the next survey instrument that asks if a participant is a Federal employee as this information can be used to allow monetary incentives in the future.
10. Confidentiality
The Privacy Act, as defined under Title 5, US Code 136, DoD Regulations, Executive Order 9396, and in DoD RCS#DD-HA(AR)2106 is printed on the front page of the paper copy of the Millennium Cohort Questionnaire. The surveys will also include the OMB control number, expiration date, and the Public Burden Statement. This document specifies the Authority supporting the request for information, the purpose for its collection, the routine uses to which it will be put, the scope of anonymity in the use of personal identifiers and the voluntary nature of participation.
There were no firmly established procedures for on-line informed consent for the field of human subjects research as this protocol was being reviewed and initially implemented. However, a report sponsored by the American Association for the Advancement of Science released in November 1999, focused on internet-based human subject research. The report, titled, “Ethical and Legal Aspects of Human Subjects Research on the Internet,” states that the three principles of protecting human subjects: autonomy, beneficence, and justice, must be applied in on-line informed consent documents. The informed consent document used for the Millennium Cohort Study meets these principles, outlining risks for both on-line and paper-based questionnaire completion and is closely reviewed by the NHRC IRB. Further, regarding the issue of a participant’s signature, a main concern for on-line consent focuses on the validity of the consent with respect to the researcher not knowing the “age, competency, or comprehension” of the participant. These concerns are mitigated for the Millennium Cohort Study since the study population is drawn from a known source, rather than soliciting unknown responders. Additionally, a validity check at the very beginning of the survey process and at the end of the informed consent document will confirm that the participant is part of the known sample population.
In addition, the Principal Investigator and all co-investigators are held responsible for performing and monitoring the research under the protocol titled, "Prospective Studies of US Military Forces: The Millennium Cohort Study". They have read and understood the provisions of Title 32 Code of Federal Regulations Part 219 (Protection of Human Subjects), DoD Directive 3216.2 (Protection of Human Subjects in DoD‑Supported Research), Secretary of the Navy Instruction (SECNAVINST) 3900.39B (Protection of Human Subjects), Naval Medical Command Instruction (NAVMEDCOMINST) 6710.4 "Use of Investigational Agents in Human Beings" ‑ if applicable), and Naval Medical Research and Development Command Instruction (NMRDCINST) 3900.2 (Protection of Human Research Volunteers from Research Risks), SECNAVINST 5370.2H (Standards of Conduct) (and local instructions, as applicable). They have agreed to abide by all applicable laws and regulations, and agreed that in all cases, the most restrictive regulation related to a given aspect of research involving protection of research volunteers will be followed. In the event that they have a question regarding their obligations during the conduct of this DoD-sponsored project, they have ready access to each of these regulations, as either a personal copy or as available on file from the Chair, Committee for the Protection of Human Subjects at the NHRC, San Diego. They understand that their immediate resource for clarification of any issues related to the protection of research volunteers is the Chair, Committee for the Protection of Human Subjects.
11. Sensitive Questions
It is understood by the Study’s Principal and co-investigators, and by the Study’s staff that all questions regarding health can be understood as sensitive in nature. It is for this reason that assiduous attention is paid every day by all who are connected with the study to maintain the participant’s privacy and confidentiality.
In regard to particularly sensitive topics, no questions concerning religious beliefs are included in the survey. Additionally, no questions concerning sexual attitudes are included. There are two questions, (questions 69f & 69g in the 2007-08 baseline survey and questions 67f & 67g in the 2007-08 follow-up survey) addressing potential exposure to traumatic life events, in which participants are asked whether they have ever been (a) forced into sexual relations, i.e., been sexually assaulted, or (b) sexually harassed. Both questions came from the National Health Survey of Persian Gulf War Era Veterans. (US Department of Veterans Affairs, Veterans Health Administration. See OMB # 2900-0558 - Expiration Date 9/98; Q9a21 & Q9a22). Also included is one question, (question 27c on the baseline survey and question 26c on the follow-up survey), asking participants whether in the past four weeks they have had little or no sexual desire, or taken little pleasure in sex. This question was derived from the Patient Health Questionnaire (PHQ) (Spitzer R, Williams J, Kroenke K, et al. (Q12)), which is framed to provide insight into a participant’s recent experience with depressive symptoms. Furthermore, these questions are commonly asked as part of clinical psychiatric assessments. Response from these questions provide useful insight into an individual’s current and historical quality of life and into possible clinical psychiatric status. In addition, deployment-specific exposures have been added to differentiate stressful exposers during deployment. Several questions, including (m), “…being responsible for the death of a non-combatant,” are recognized as potentially sensitive. Please note that these questions are a subset of the Walter Reed Army Institute of Research (WRAIR) developed Mental Health Assessment Tool (MHAT), recommended by MHAT leaders (COL Charles Hoge and COL Carl Castro), and selected to specifically exclude incriminating queries (such as unnecessary use of force against non-combatants).
12. Burden Estimated (hours)
Sample consists of 36,599 respondents.
The estimated total annualized cost to the respondents for this collection is $474,872.
13. Cost to Respondents
There are no additional costs to respondents other than those provided in item 12, above.
14. Cost to Federal Government
The estimated total annualized cost to the Federal Government for this collection is $757,791.
The total annualized cost was calculated by taking the cost of the Millennium Cohort Study’s eighteen month survey cycle, which is approximately $2,525,971 (which includes printing, postage, services, equipment purchases, travel and salaries), and multiplying that by 30% (the proportion of Panel 1, 2, and 3 participants we estimate will be separated by 2010).
15. Change in Burden
During the 2007-2008 data collection phase approximately 29,441 civilian members of the Cohort completed and submitted a questionnaire (22,375 from Panel 1, 5,299 from Panel 2, and 1,766 from Panel 3). This amounted to a civilian burden of 22,081 hours. Given an estimated civilian burden for the 2010 data collection phase of 27,450 hours there is a change in burden of positive 5,369 hours.
The change in total annualized cost to the Federal Government is positive $572,659. This increase attributes in direct proportion to the increase in numbers of civilian-former military participants in this third data collection wave.
16. Publication/Tabulation
The DoD Center for Deployment Health Research, as the lead agent for implementing the Millennium Cohort Study, has responsibility for all data collection and management, all data security, the maintenance of all assurances including but not limited to human subjects protection, and other Privacy Act considerations. As part of these responsibilities the Center has defined a set of parameters for the maintenance of data security and integrity, a process for submission and review of collaborative research requests, and a set of requirements and guidelines, with which collaborators must comply during the investigative process.
Millennium Cohort Study researchers have published or have in press 25 peer-reviewed publications. A complete list of Millennium Cohort publications to date include:
LeardMann CA, Smith TC, Smith B, Wells TS, Ryan MAK, for the Millennium Cohort Study Team. Baseline self-reported functional health predicts vulnerability to posttraumatic stress disorder following combat deployment: prospective US military cohort study. British Medical Journal, 2009; In press.
Welch KE, LeardMann CA, Jacobson IG, Speigle SJ, Smith B, Smith TC, Ryan MA, for the Millennium Cohort Study Team. Postcards encourage participant updates. Epidemiology, 2009 Mar;20(2):313-4.
Jacobson IG, Smith TC, Smith B, Keel PK, Amoroso PJ, Wells TS, Bathalon GP, Boyko EJ, Ryan MAK for the Millennium Cohort Study Team. Disordered eating and weight changes after deployment: longitudinal assessment of a large US military cohort. American Journal of Epidemiology, 2009 Feb;169(4):415-27.
Smith TC, Wingard DL, Ryan MAK, Kritz-Silverstein D, Slymen DJ, Sallis JF, for the Millennium Cohort Study Team. PTSD prevalence, associated exposures, and functional health outcomes in a large, population-based military cohort. Public Health Reports, 2009 Jan;124:90-102.
Smith B, Ryan MAK, Wingard DL, Patterson TL, Slymen DJ, Macera CA, for the Millennium Cohort Study Team. Cigarette smoking and military deployment: a prospective evaluation. American Journal of Preventive Medicine, 2008 Dec;35(6):539-46.
Jacobson IG, Smith TC, Bell NS. Military combat deployment and alcohol use reply. Journal of the American Medical Association. 2008 Dec;300(22):2607.
Jacobson IG, Ryan MAK, Hooper TI, Smith TC, Amoroso PJ, Boyko EJ, Gackstetter GD, Wells TS, Bell NS, for the Millennium Cohort Study Team. Alcohol use and alcohol-related problems before and after military combat deployment. Journal of the American Medical Association, 2008 Aug;300(6):663-75.
Smith B, Chu LK, Smith TC, Amoroso PJ, Boyko EJ, Hooper TI, Gackstetter GD, Ryan MAK, for the Millennium Cohort Study Team. Challenges of self-reported medical conditions and electronic medical records among members of a large military cohort. BMC Medical Research Methodology, 2008 Jun;8:37.
Smith TC, Wingard DL, Ryan MAK, Kritz-Silverstein D, Slymen DJ, Sallis JF, for the Millennium Cohort Study Team. Prior assault and posttraumatic stress disorder after combat deployment. Epidemiology, 2008 May;19(3):505-12.
Wells TS, LeardMann CA, Smith TC, Smith B, Jacobson IG, Reed RJ, Ryan MAK, for the Millennium Cohort Study Team. Self-reported adverse health events following smallpox vaccination in a large prospective study of US military service members. Human Vaccines. 2008 Mar/Apr;4(2):127-33.
Wells TS, Jacobson IG, Smith TC, Spooner CN, Smith B, Reed RJ, Amoroso PJ, Ryan MAK, for the Millennium Cohort Study Team. Prior health care utilization as a determinant to enrollment in a 22-year prospective study, the Millennium Cohort Study. European Journal Of Epidemiology. 2008 Feb;23(2):79-87.
Smith TC, Ryan MAK, Wingard DL, Slymen DJ, Sallis JF, Kritz-Silverstein D, for the Millennium Cohort Study Team. New onset and persistent symptoms of posttraumatic stress disorder self-reported after deployment and combat exposures: prospective population-based US military cohort study. British Medical Journal. 2008 Feb;336(7640):366-71.
Smith B, Wingard DL, Ryan MAK, Macera CA, Patterson TL, Slymen DJ, for the Millennium Cohort Study Team. US military deployment during 2001-2006: comparison of subjective and objective data sources in a large prospective health study. Annals of Epidemiology. 2007 Dec;17(12):976-82.
LeardMann CA, Smith B, Smith TC, Wells TS, Ryan MAK, for the Millennium Cohort Study Team. Smallpox vaccination: comparison of self-reported and electronic vaccine records in the Millennium Cohort Study. Human Vaccines. 2007 Nov/Dec;3(6):245-51.
Smith TC, Zamorski M, Smith B, Riddle JR, LeardMann CA, Wells TS, Engel CC, Hoge CW, Adkins J, Blazer D, for the Millennium Cohort Study Team. The physical and mental health of a large military cohort: baseline functional health status of the Millennium Cohort. BMC Public Health. 2007 Nov;7(147):340.
Smith B, Smith TC, Gray GC, Ryan MAK, for the Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. American Journal of Epidemiology. 2007 Nov;166(11):1345-54.
Smith TC, Jacobson IG, Smith B, Hooper TI, Ryan MAK, for the Millennium Cohort Study Team. The occupational role of women in military service: validation of occupation and prevalence of exposures in the Millennium Cohort Study. International Journal of Environmental Health Research. 2007 Aug;17(4):271-84.
Smith TC, Smith B, Jacobson IG, Corbeil TE, Ryan MAK, for the Millennium Cohort Study Team. Reliability of standard health assessment instruments in a large, population-based cohort study. Annals of Epidemiology. 2007 Jul;17(7):525-32.
Smith B, Leard CA, Smith TC, Reed RJ, Ryan MAK, for the Millennium Cohort Study Team. Anthrax vaccination in the Millennium Cohort: validation and measures of health. American Journal of Preventive Medicine. 2007 Apr;32(4):347-53.
Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, Corbeil TE, Hooper TI, for the Millennium Cohort Study Team. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. Journal of Clinical Epidemiology. 2007 Feb;60(2):181-91.
Riddle JR, Smith TC, Smith B, Corbeil TE, Engel CC, Wells TS, Hoge CW, Adkins J, Zamorski M, Blazer D, for the Millennium Cohort Study Team. Millennium Cohort: the 2001-2003 baseline prevalence of mental disorders in the US military. Journal of Clinical Epidemiology. 2007 Feb;60(2):192-201.
Chretien JP, Chu LK, Smith TC, Smith B, Ryan MAK, for the Millennium Cohort Study Team. Demographic and occupational predictors of early response to a mailed invitation to enroll in a longitudinal health study. Biomed Central Medical Research Methodology. 2007 Jan;7:6.
Smith TC, Smith B, Corbeil TE, Ryan MAK, Riddle JR, for the Millennium Cohort Study Team. Impact of terrorism on caffeine and tobacco use [letter in response to "Self-reported mental health among US military personnel, prior and subsequent to the terrorist attacks of September 11, 2001"]. Journal of Occupational and Environmental Medicine. 2004 Dec;46(12):1194-5.
Smith TC, Smith B, Corbeil TE, Riddle JR, and Ryan MAK, for the Millennium Cohort Study Team. Self-reported mental health among US military personnel, prior and subsequent to the terrorist attacks of September 11, 2001. Journal of Occupational and Environmental Medicine. 2004 Aug;46(8):775-82.
Gray GC, Chesbrough KB, Ryan MAK, Amoroso P, Boyko EJ, Gackstetter GD, Hooper TI, Riddle JR, for the Millennium Cohort Study Group. The Millennium Cohort Study: A 21-year prospective cohort study of 140,000 military personnel. Military Medicine. 2002 Jun;167(6):483-8.
17. Expiration Date
DoD is not seeking an exception to displaying the expiration date of this information collection.
18. Exceptions to Certification Statement in item 19 of OMB Form 83-I
There are no exceptions to the certification statement in Item 19 of OMB Form 83-1.
COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.
The responder universe consists of a probability-based sample of active-duty and Reserve/Guard members of the US military, identified through service rosters as of October 1, 2000 (Panel 1), October 1, 2003 (Panel 2), October, 1 2006 (Panel 3), and October 1, 2009 (Panel 4). In Panel 1 military personnel who had served in Southwest Asia, Bosnia, and Kosovo after 1997 were over-sampled such that they matched 1:1 with personnel who did not deploy to those areas. Additionally, Reserve, National Guard, and female service personnel were over-sampled to assure sufficient statistical power to investigate hypotheses in these smaller subgroups of the military population. In Panels 2 and 3, military personnel who were Marines or females were oversampled also to assure sufficient statistical power to investigate hypotheses in these smaller subgroups. The entire Cohort, after Panel 4 enrollment, will be population-based and represent approximately 8% of US service membes from all services and components. The 36,599 personnel projected to become civilians during the 2010 effort after becoming participants while in military service, are expected to follow a random attrition process from military service and are expected to be similar in makeup to the responders in general.
Since the study is currently in the process of finalizing the cohort for Panel 3, the following numbers should be understood as estimates:
By July 1, 2003, 77,047 (36%) had returned a Panel 1 baseline questionnaire. Panel 1 targeted 256,400 service members of whom 213,949 had valid addresses allowing for study contact attempt.
Among Panel 1 participants, 55,021 have submitted the first follow-up survey (71%), and approximately 54,790 have submitted the second follow-up survey (71%). Participation in the second follow-up survey consisted of 84% web and 16% paper responders. Of those that did not submit their first follow-up survey, 8,259 submitted their second follow-up survey. 46,439 Panel 1 participants have responded to the enrollment and two follow-up surveys.
By February 15, 2006, 31,110 (25%) had returned a Panel 2 baseline questionnaire. Panel 2 targeted 150,000 of whom 122,410 had valid addresses allowing for study contact attempt.
Among Panel 2 participants, 17,151 have submitted a first follow-up survey (55%), and participation consisted of 83% web and 17% paper responders.
Panel 3 enrollment targeted 200,000 service members of whom 153,650 had valid addresses, in order to enroll an additional 40,000 individuals. By December 31, 2008, 43,440 had returned a baseline questionnaire with 93% of responses occurring via web and 7% via paper.
During the 2010 survey cycle, we will attempt to initiate contact with 250,000 service members to enroll 50,000 individuals in Panel 4.
Since enrollment began in 2001, more than 151,000 service members have consented and submitted a baseline survey (36.0%, 25.4%, and 28.3% baseline response rates for Panels 1, 2 and 3 respectively).
To date, a total of 521 deaths have occurred within the Panel 1 responder group. A total of 89 deaths have occurred within the Panel 2 responder group, and a total of 22 deaths have occurred within the Panel 3 responder group.
Approximately 22,375 of the 77,047 Panel 1 responders had left military service and transitioned to civilian life by the time they submitted their third follow-up survey and approximately 5,299 of Panel 2 responders had left military service and transitioned to civilian life by the time they submitted their second follow-up survey. We estimate that slightly more than a total of 36,599 participants from Panels 1, 2 and 3 will have separated by the time they are contacted for their next follow-up surveys in early 2010.
At the end of this fourth enrollment phase, we expect to have enrolled a total of 200,000 service members. As of our first OMB approval in September 2003 and throughout the course of the study, proportions of military versus civilian participants will shift in favor of civilians.
2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.
A stratified random sampling process was employed to reduce bias, allow for external validity after weighting, and ensure enough statistical power to address small subgroups of the population reasonably well in a population-based setting. The Millennium Cohort Study questionnaire is sent to participants every 3 years allowing information to be acquired without burdening participants with yearly questionnaires.
3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.
Much effort is focused into response and retention rates. Response rates are maximized principally through employment of modified Dillman Mail and Electronic Survey Methods9. Each participant will be sent three paper questionnaires or three email requests to participate depending on whether or not s/he has responded to the last questionnaire/email request sent. Each questionnaire mailing/email-request-to-participate is followed two weeks later by a reminder postcard/reminder email. If a participant has not responded after 3 such invitations and we have not received notice that the address at which we attempted to reach them was not valid we infer that the questionnaire/email-request was received by the participant and that they have chosen not to participate. Such individuals are classified at that time as nonresponders. Protocol requires that 3 attempts be made to rectify “bad” addresses. New or “good” addresses are sought from the Internal Revenue Service through contract with the National Institute for Occupational Safety and Health (NIOSH).
The vast majority of questions comprising the Millennium Cohort Study Questionnaire are derived from instruments previously vetted in military and non-military populations including the Recruit Assessment Program (RAP) 2003 Survey, Seabee Health Survey, Patient Health Questionnaire, National Health Survey of Persian Gulf War Era Veterans, Medical Outcomes Study Short Form 36-Item Health Survey for Veterans (SF-36V), Patient Health Questionnaire (PHQ), Complementary and Alternative Medicines Questionnaire, CAGE Questionnaire for Detecting Alcoholism, NIAAA Task Force on Recommended Alcohol Questions-National Council on Alcohol Abuse and Alcoholism Recommended Sets of Alcohol Consumption (2003), National Health Interview Survey, National Health and Nutrition Examination Survey (NHANES), National Health and Nutrition Examination Survey (NHANES) Reproductive Health Questionnaire (RHQ) 2003-04, 2001 National Health Interview Survey (NHIS) and Health Evaluation Assessment Review (HEAR), Occupational Conversion Index, and answer choices for #78 on the follow-up survey derived from DMDC's Human Resources Strategic Assessment Program (HRSAP) Survey Report, Gender Differences in Officers’ Reasons for Leaving Active Duty, Note No. 2004-04, January 2004.
Paper questionnaires are printed as scannable Teleform documents. Quality control evaluation of data from Panel 1 shows that participants are entering both paper and web-based questionnaire data coherently and reliably with an average of fewer than one unreadable entry per questionnaire in our most recent sampling.
Demographic data on all invited personnel have been examined to determine differences in distributions among responders and nonresponders. Investigation showed responders to be demographically representative of the invited sample1. This has allowed us to determine when statistical weighting techniques may be appropriate. An investigation of reliability of data found substantial test-retest reliability of survey responses and high internal consistency of health metrics on the questionnaire10. We also found few health differences between responders and nonresponders in terms of health care utilization in the 12 months preceding study invitation11. In addition, several investigations were conducted to assess the reliability of exposure reporting and found that self-reported exposures were quite reliable when compared with electronic military data12-16.
Additionally, based on findings from the RTI nonresponse study, during the 2007-2008 cycle several other incentivizing tactics were tested on consented participants, including sending automated voice messages, adding a $5 Starbucks gift card to the survey packet, and offering a hat incentive up front to encourage response. An increase in number of response was seen from these various avenues explored when compared with regular mail strategies (see attached graphs titled “Nonresponse pilot studies”). Of the Panel 1 participants that did not submit their first follow-up survey, 8,259 submitted their second follow-up survey.
Monetary incentives have been considered, however, existing military regulations prohibit the use of cash as reimbursement for military members to participate in this study. The use of monetary incentives will be regularly revisited in the future as more Millennium Cohort participants separate from military service and information regarding federal employment after separation from military service is assessed. A question will be added to the next survey instrument that asks if a participant is a Federal employee as this information can be used to leverage monetary incentives in the future.
4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.
Following preliminary focus group evaluations of the draft questionnaire conducted in late 1999 with military enlisted and officer groups of less than 10 people, a pilot study was conducted on a 1% sample of military personnel in the spring of 2000 as a means of testing the utility of the instrument. Following this pilot, corrections were made to produce the final survey instrument.
Along with nonresponse testing as described earlier in the report, we will conduct focus group testing. The purpose of the focus group testing is to determine effective strategies to maximize participation rates in populations with similar demographics to Millennium Cohort participants, ensure the clarity of our contact materials, including the text and overall visual format of emails, postcards, and survey packets, and lastly, to obtain feedback on the cost-saving initiatives currently offered.
5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
tyler.c.smith@med.navy.mil (619) 553-7593
Edward J. Boyko, M.D. MPH, VAMC Puget Sound
eboyko@u.washington.edu (206) 764-2830
paul.amoroso@us.army.mil (253) 968-1160
Gary D. Gacksetter, D.V.M., MPH, PhD, Analytic Services Inc. (ANSER)
gary.gackstetter@anser.org (703) 416-3405
Tomoko Hooper, MD, MPH, FACM, USUHS
thooper@usuhs.mil (301) 295-1975
Margaret A.K. Ryan, MD, MPH, Naval Hospital Camp Pendleton
margaret.ryan@med.navy.mil (760) 725-0906
Timothy S. Wells, DVM, PhD AFRL/HEPA WPAFB
timothy.wells@wpafb.mil (937) 255-3931
**************************
The Research Team at the Department of Defense Center for Deployment Health Research, Naval Health Research Center, San Diego CA 92106-3521
NHRC-Dept164List@med.navy.mil (619) 553-7027
References
Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, Corbeil TE, Hooper TI, for the Millennium Cohort Study Team. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. Journal of Clinical Epidemiology. 2007 Feb;60(2):181-91.
Samet JM, Munoz A. Evolution of the cohort study. Epidemiol Rev 1998;20:1-14.
Xu X, Laird N, Dockery DW, Schouten JP, Rijcken B, Weiss ST. Age, period, and cohort effects on pulmonary function in a 24-year longitudinal study. Am J Epidemiol 1995;141:554-66.
Rijcken B, Schouten JP, Xu X, Rosner B, Weiss ST. Airway hyperresponsiveness to histamine associated with accelerated decline in FEV1. Am J Respir Crit Care Med 1995;151:1377-82.
Vasan RS, Pencina MJ, Cobain M, Freiberg MS, D'Agostino RB. Estimated risks for developing obesity in the Framingham Heart Study. Ann Intern Med 2005;143:473-80.
Sytkowski PA, D'Agostino RB, Belanger AJ, Kannel WB. Secular trends in long-term sustained hypertension, long-term treatment, and cardiovascular mortality. The Framingham Heart Study 1950 to 1990. Circulation 1996;93:697-703.
Colditz GA, Manson JE, Hankinson SE. The Nurses' Health Study: 20-year contribution to the understanding of health among women. J Womens Health 1997;6:49-62.
Hammond EC. Smoking in relation to the death rates of one million men and women. Natl Cancer Inst Monogr 1966;19:127-204.
Dillman DA. Mail and Telephone Surveys: The Total Design Method. New York: Wiley; 1978. xvi, 325.
Smith TC, Smith B, Jacobson IG, Corbeil TE, Ryan MAK, for the Millennium Cohort Study Team. Reliability of standard health assessment instruments in a large, population-based cohort study. Annals of Epidemiology. 2007 Jul;17(7):525-32.
Wells TS, Jacobson IG, Smith TC, Spooner CN, Smith B, Reed RJ, Amoroso PJ, Ryan MAK, for the Millennium Cohort Study Team. Prior health care utilization as a determinant to enrollment in a 22-year prospective study, the Millennium Cohort Study. European Journal Of Epidemiology. 2008 Feb;23(2):79-87.
Smith B, Wingard DL, Ryan MAK, Macera CA, Patterson TL, Slymen DJ, for the Millennium Cohort Study Team. US military deployment during 2001-2006: comparison of subjective and objective data sources in a large prospective health study. Annals of Epidemiology. 2007 Dec;17(12):976-82.
LeardMann CA, Smith B, Smith TC, Wells TS, Ryan MAK, for the Millennium Cohort Study Team. Smallpox vaccination: comparison of self-reported and electronic vaccine records in the Millennium Cohort Study. Human Vaccines. 2007 Nov/Dec;3(6):245-51.
Smith TC, Jacobson IG, Smith B, Hooper TI, Ryan MAK, for the Millennium Cohort Study Team. The occupational role of women in military service: validation of occupation and prevalence of exposures in the Millennium Cohort Study. International Journal of Environmental Health Research. 2007 Aug;17(4):271-84.
Smith B, Leard CA, Smith TC, Reed RJ, Ryan MAK, for the Millennium Cohort Study Team. Anthrax vaccination in the Millennium Cohort: validation and measures of health. American Journal of Preventive Medicine. 2007 Apr;32(4):347-53.
Riddle JR, Smith TC, Smith B, Corbeil TE, Engel CC, Wells TS, Hoge CW, Adkins J, Zamorski M, Blazer D, for the Millennium Cohort Study Team. Millennium Cohort: the 2001-2003 baseline prevalence of mental disorders in the US military. Journal of Clinical Epidemiology. 2007 Feb;60(2):192-201.
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION |
| Author | Terri Tanielian |
| Last Modified By | Patricia Toppings |
| File Modified | 2009-06-30 |
| File Created | 2009-03-17 |