Supporting Statement B
Revision Request for Clearance
NATIONAL HEALTH INTERVIEW SURVEY
OMB No. 0920-0214
(expires 01/31/2019)
Contact Information:
Marcie Cynamon
Division of Health Interview Statistics
National Center for Health Statistics/CDC
3311 Toledo Road
Hyattsville, MD 20782
301.458.4174 (voice)
301.458.4035 (fax)
November 14, 2016
1.Respondent Universe and Sampling Methods 3
2.Procedures for the Collection of Information 4
3.Methods to Maximize Response Rates and Deal with Nonresponse 5
4.Tests of Procedures or Methods to be Undertaken 6
5.Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 7
B. Collection of Information Employing Statistical Methods
The NHIS is a cross‑sectional household interview survey. The sampling plan follows a multistage probability design. Approximately every ten years, the NHIS sampling plan is revised following the decennial census of the population. From 2006-2015, the sampling plan was based on the 2000 decennial census. In 2016, a new sampling plan was implemented to keep the sample current with population distribution changes over the decade.
The 2016 NHIS sample design is structured with a nationally-focused design as its core, and contains large reserve samples that can be used to achieve state or minority objectives. The sample design takes into account demographic shifts in the U.S. civilian noninstitutionalized population, and allows for additions and contractions to reflect funding availability and to meet estimation goals. The base sample consists of approximately 35,000 completed household interviews annually. To balance the precision of national and state-based estimates, most of the sample (approximately 25,000 completed interviews) are allocated proportionally to the state population to maximize the precision of national-level estimates. A smaller portion of the sample (approximately 10,000 completed interviews) are shifted to increase sample in the 10 least populous states, enabling state-level estimates of key variables to be produced for all 50 states and DC by pooling 3 years of data. This flexibility reflects the increasing demand for state-level health outcomes, in particular support of the recent focus on state-level heath care.
While the sampling frame for the NHIS had traditionally used field listing by the Census Bureau, in order to contain costs, the frame used from 2016 onward employs a commercial list that covers addresses within all 50 states and the District of Columbia. Supplementary field listing is undertaken to improve coverage in rural areas with poor addresses, in high density areas with addresses that are too general (as drop boxes for apartment buildings), and of university housing units. Because on-campus college dormitories are not included in the commercial address database, a separate sample was created for these residences (OMB# 0920-0222, expires 07/31/2018). The sampled colleges were contacted and a listing was made of dormitories. From this list individual dormitories are selected for inclusion. When a university is sampled, a field representative works with university housing officials to obtain a roster of beds/units within the sampled dormitories from which sampled units are selected. Prior to accessing university housing units to complete the interviews, university residence offices are notified via advance letter (Attachment 8e) that a Census interviewer will be on the premises; and a thank you letter (Attachment 8f) is mailed to the same office once interviews have been completed (this is in addition to the main thank you letter (Attachment 8c) that is mailed to interview participants directly).
To date, the sample design implemented in 2016 has not been found to affect estimates generated using NHIS data, compared to previous years. To monitor the design’s performance, NHIS analysts perform monthly checks as part of routine data review. As in the past, each year results from the January sample are compared to the previous year to determine whether the results are consistent. In addition to comparing the unweighted and weighted frequencies, the input and output specifications are reviewed, and the flowcharts are compared to the skip instructions and universes for each question. If a difference is found, steps are taken to determine whether the change is legitimate or whether there is a factor other than the programming of the questionnaire such as the location or context of the question in the questionnaire. If a difference persists, the paradata are reviewed to determine whether there are changes in the mean or median time spent on that question, whether interviewers had a high rate of backing up to return to that question, and whether other questions in that battery were similarly affected. Persistent differences are examined to determine whether there is any other interviewer effect, such as differences between newly-hired and experienced interviewers or newly-added primary sampling units compared to continuing primary sampling units. In addition, national estimates on the key set of indicators that are released in a quarterly report as part of the Early Release program are monitored by NHIS analysts.
The U.S. Bureau of the Census is responsible for drawing the final sample and for performing the necessary field procedures related to data collection and initial processing. Specifications for the field operations are provided by the Division of Health Interview Statistics (DHIS) staff at NCHS.
DHIS staff provide specifications for the sample design, specific content of the questionnaire, detailed instructions for the administration of the interview, and procedures to carry out quality control measures, such as reinterview and paradata analysis. The Census Bureau, in addition to drawing the sample, performs supervisor and interviewer training and conducts the field operations. These operations include first contacting all households via an advance letter (Attachment 8a), followed by a personal visit. In some cases, contact via telephone is also used to follow up on respondents who were unable to be contacted in person or to complete the interview during a personal visit.
DHIS staff monitor the field activities through observation and communication with Census during all phases of data collection and through the analysis of paradata such as audit trails, contact history, and item timing. Frequent status meetings are held to assess progress toward data collection goals.
All data are weighted to provide national estimates using the following four components: 1) The reciprocal of the probability of selection; 2) a household nonresponse adjustment within segment; 3) a first-stage ratio adjustment; and 4) a second stage ratio (or post stratification) adjustment to the U.S. population by age, sex, and race-ethnicity.
Standard errors may be calculated using a Taylor linearization approach as applied in SUDAAN variance software. (See: Research Triangle Institute. SUDAAN Language Manual; Release 11.0. Research Triangle Park, NC: Research Triangle Institute. 2012.)
A small sample of respondents is reinterviewed by the Census Bureau to ensure that interviewers are not submitting falsified information. NHIS reinterviews are conducted primarily by telephone, by staff at one of the Census Bureau’s centralized call centers. The reinterview is very brief and verifies that the original interview was completed. Typically, the NHIS reinterview is conducted within two to three weeks of the main survey with the same respondent who originally participated in the NHIS. The reinterview questionnaire is shown in Attachment 5.
Additional technical details on routine survey execution can be found in the National Center for Health Statistics (2014) Survey Description Document available at ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2014/srvydesc.pdf.
A sample of adult respondents that is not part of the sample set aside for MEPS (OMB# 0935-0118, expires 12/31/2018) may be selected to participate in follow-back surveys and special methodological projects. For instance, in 2017, NHIS respondents may be included as participants in short, web-based methodological and cognitive testing activities. These small-scale assessment projects will build on and integrate the findings from previous NHIS follow-back surveys and special methodological projects to inform the 2018 questionnaire redesign. Additional details about these follow-back activities are provided in Supporting Statement A.
As 2016 NHIS data collection is still underway and 2015 data files have not yet been released, the latest year of available data is from the 2014 survey. In 2014, the final household response rate was 73.8 percent. This rate is calculated by dividing the number of completed household interviews by the number of assigned, in-scope households. The sample child component was completed in 91.2 percent of participating households in 2014, for an overall response rate of 66.6 percent. The sample adult component was completed in 80.5 percent of participating households in 2014, for an overall response rate of 58.9 percent.
The NHIS, like most surveys, has witnessed steadily declining response rates. Over the past ten years, response rates have fallen by more than ten percentage points, from 86.5% in 2005 to 73.8% to the latest available in 2014. Reasons for declining response rates are unclear but may include increased survey length, general mistrust of the government, growing time constraints, improvements in privacy screening technology for telephones, and other reasons.
To provide respondents with advance notification of the interview in an attempt to maximize response rates, an advance letter is sent to all sample households prior to the interviewer's arrival (Attachment 8a). The letter legitimizes and justifies the survey, increasing the probability that the respondent will cooperate. It references the authorizing legislation of the survey, a statement of confidentiality and an explanation of how the data will be used, as well as the voluntary nature of the survey and other elements for informed consent. The letter further explains the purpose of and need for the survey and tells the respondent that there is some chance that they may be contacted more than once. If at the time of the initial contact the interviewer is told that the letter was not received, another letter is provided prior to the interview and time is allowed for the person to read it before proceeding. Additional written materials have been developed in recent years to supplement the advance letter, most recently the new informational brochure that was launched in 2016 to visually brand the survey and engage respondents in fulfilling the important mission of the NHIS (Attachment 8b). In addition, targeted interviewer training modules on improving respondent cooperation (such as gaining cooperation, accessing respondents through gatekeepers, and averting refusals) are presented at initial training for new interviewers and at least once a year during their annual refresher training.
If the time of contact is inconvenient for a respondent, interviewers offer to schedule an appointment for a more convenient time. If the respondent declines the interview with one interviewer, the field work supervisor often reassigns the case to an interviewer with more experience at converting reluctant respondents. Although face-to-face interviews are preferred, interviewers are allowed to substitute telephone interviews if attempts to get a face-to-face interview are not successful.
To explore the utility of employing monetary incentives in the NHIS, an incentive experiment was conducted in the summer of 2015. Fielded from May through July in the states included in three Census Regional Offices (New York, Philadelphia, and Denver), the experiment was designed to assess the impact of respondent incentives on response rates, survey costs, and data quality. Described in greater detail in Supporting Statement A, preliminary results from this experiment indicate, however, that offering incentives across the board to all respondents has limited potential to improve survey outcomes, and that incentives may be better suited for application in a targeted manner when they are deemed to be beneficial to the study design.
In the third quarter of 2016, an adaptive design experiment (approved as a nonsubstantive change under the 2016 NHIS clearance on 6/9/2016, and described in Attachment 10) was carried out to test the impact of adaptive case prioritization on sample representativeness and nonresponse bias, while maintaining survey costs and minimizing any possible negative effect on the overall response rate. Pending results from the ongoing evaluation of the experiment, case prioritization may be further tested and/or implemented during 2017 NHIS data collection. Case prioritization would affect only interviewer activities, and would neither affect nor alter public burden from conducting the NHIS.
The ongoing test of alternate methods for assessing disability continues, comprising one component of a larger testing effort to develop and adopt a standard set of disability questions to be used with multiple surveys in multiple countries. At the end of the family interview a random half of families receives six supplemental disability questions originally developed for use on the American Community Survey (ACS). The other random half of families receives the Adult Functioning and Disability Supplement (AFD), part of an international project to develop and test improved measures of functioning. Randomization of families into one of the two treatment groups will occur at the time the family roster is established during the Household Module, based on a software-generated pre-assigned table of random numbers, a method developed by Kish (1949)1.
For 2017, a series of small-scale projects is planned to inform the impending questionnaire redesign, building on and extending the findings from prior follow-back surveys and methodological experiments described in Supporting Statement A. These projects will involve short, web-based methodological and cognitive assessments to pilot test new and/or updated questionnaire items, evaluate the impact of different categorical response option formats on answer choices, and measure respondent comprehension of health care-related terms and concepts.
Other developmental work related to the NHIS questionnaire is conducted by the NCHS Center for Questionnaire Design and Evaluation Research (CQDER) under their clearance (OMB No. 0920-0222, expires 07/31/18).
The following person was consulted in the statistical aspects of the design and collection of the NHIS:
Van L. Parsons, Ph.D.
Statistical Research and Survey Design Staff
Office of Research and Methodology
National Center for Health Statistics
(301)458-4421
The following person is responsible for collection of the data:
Anne Kearney, Ph.D.
Survey Director, National Health Interview Survey
Demographic Surveys Division
U.S. Bureau of the Census
Suitland, MD
(301)763-6780
Anne.Theresa.Kearney@census.gov
The following person is responsible for analysis of the NHIS data:
Stephen Blumberg, Ph.D.
Associate Director for Science
Division of Health Interview Statistics
National Center for Health Statistics
(301)458-4107
sblumberg@cdc.gov
List of Attachments
Attachment 1 Applicable Laws and Regulations: NHIS Legislative Mandate (42 USC 242K)
Attachment 2a Family Questionnaire
Attachment 2b Sample Adult Questionnaire
Attachment 2c Sample Child Questionnaire
Attachment 2d Need for Core Modules
Attachment 2e OMB Statement and Screener
Attachment 2f Additions to the Core NHIS
Attachment 3a Supplement Questions
Attachment 3b Description of Supplements
Attachment 3c Core and Supplement Changes Summary Table
Attachment 4 Flashcards
Attachment 5 Reinterview Questionnaire
Attachment 6 Methodological Projects Description
Attachment 7a 60 Day Federal Register Notice
Attachment 7b Public Comments
Attachment 8a Advance Letter
Attachment 8b NHIS Brochure
Attachment 8c Thank You Letter
Attachment 8d Supplemental Mailings
Attachment 8e University Advance Letter
Attachment 8f University Thank You Letter
Attachment 9 Research Ethics Review Board Approval
Attachment 10 Adaptive Design Experiment Description
1 Kish, L. (1949). A Procedure for Objective Respondent Selection within the Household. Journal of the American Statistical Association, 44 (247), 380–387.
| File Type | application/msword |
| Author | Sarah S. Joestl |
| Last Modified By | Buie, Verita (CDC/OPHSS/NCHS) |
| File Modified | 2016-11-15 |
| File Created | 2016-11-15 |