Attachment 8
National Health Interview Survey
2013 Web/Computer-Assisted Telephone Interview (CATI) National Health Care Interview Survey Project Overview
A Multi-mode Followback Survey to the NHIS
February 12, March 1, 2013
This revision request is for proposed changes to a currently approved collection (National Health Interview Survey or NHIS, OMB control number 0920-0214). It describes a pilot follow-back project with web and telephone data collection in 2013 based on the 2012 NHIS sample frame.
This project, entitled the 2013 National Health Care Interview Survey (NHCIS), is funded by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the Department of Health and Human Services (DHHS) and Patient Protection and Affordable Care Act (ACA) funds. The primary purpose of the study is to evaluate the ability to effectively recontact a panel of NHIS respondents for a multimode survey to monitor aspects of health care reform at two points in time. Respondents will be interviewed via telephone or internet. A secondary goal is to conduct methodological research to assess the impact of restricting such a followback to online response only. and to demonstrate the utility of this methodology for monitoring aspects of health care reform at two points in time. The funding covers one followback interview.
Beginning in 2011, questions were added to the NHIS to provide additional information on ACA relevant topics such as the use of preventive services, changes in insurance coverage, financial barriers to medical care, and emergency room use. Also Iin 2012, questions were added to the NHIS about frequency of use of the internet and email. In this followback survey, we will attempt to recontact approximately 12,000 NHIS respondents from 2012 about one year after their initial interview and invite them to respond by telephone or internet to the NHCIS. Data collection will be conducted by the US Census Bureau, which has extensive experience with telephone and online surveys, in partnership with NCHS.
The goals of the NHCIS are methodological and substantive and methodological. With respect to substantive goals, data from the initial interview in 2012 and the follow-back interview in 2013 will be examined for changes in health status, health insurance coverage, health care utilization, access to care, and unmet needs. Because this is a probability sample of the civilian, noninstitutionalized US population, and by using both internet and telephone to conduct interviews, response rates and coverage of the population should be high and the results should be generalizable to the civilian, noninstitutionalized adult population of the US.
From a methodological standpoint, we will evaluate completion rates, nonresponse biases, and data quality by mode to determine whether reliable data can be collected from NHIS respondents after one year. By systematically varying recruitment mode (telephone or email recontact) and response mode (encouraging telephone or internet data collection), we will identify the most effective methods for conducting followback surveys and whether the most effective method differs by participants’ email and internet usage characteristics. The varying response modes may also shed light on the susceptibility of NHIS questions to mode effects, as well as the nature of the limitations of internet panels in general.. Also, data from web responders will be reweighted to determine how restricting a followback to this mode only would impact the estimates.
This project also supports the objectives and goals of the HHS Data Strategy to improve HHS data collection systems, particularly, to improve the speed and efficiency of data collection, reduce the time in the field and improve time for data release to the public. Project results will improve responsiveness of HHS data systems to policy needs in terms of faster data collection, administrative and design flexibility, and efficiency. Through the HHS Data Council and the work groups developing the HHS Data Strategy, this project has been presented to agencies across HHS. There is much interest in creating an internet panel mechanism that avoids the shortcomings of commercially available products with regard to coverage, response rates, and concerns about panel members becoming professional survey takers.
Online data collection offers numerous advantages, including lower costs relative to computer assisted personal or telephone interviewing; the ability to more easily and responsively adapt individual questions and instrument sections compared to mail surveys; and higher data quality due to built-in skips and edits. However, those advantages cannot compensate for having nationally representative data. As such, one of the important purposes of the study is to assess the limitations of the proposed approach.
Significant methodological work is required to address challenges for HHS in using the web for conducting national health surveys of the US population, from ensuring a representative sample, covering small populations, and variations in survey response rates. This project will continue developmental work and provide a systematic investigation of the use of the web as an option for HHS survey research in an era of limited funds and declining response rates.
With respect to substantive goals, provided that the methodological investigation discussed above suggests that longitudinal comparisons would be sufficiently robust, Ddata from the initial interview in 2012 and the follow-back interview in 2013 wouldwill be examined for the probability of changes in health status, health insurance coverage, health care utilization, access to care, and unmet needs. For instance, we may be able to xxxx
Background/Rationale
Creating a longitudinal panel based on the NHIS addresses HHS interest in monitoring the impact of programs implemented as a result of the Affordable Care Act. The depth of health related information obtained from the NHIS lays the foundation for understanding change. While it has been proven that the NHIS frame can be reused as is the case for MEPS, we have not used it to administer the same questions to the same individuals over a period of time. This study involves one followback interview of one sample adult per family as an efficient way to enhance our knowledge to inform decisions during periods of rapid development.
Faced with increasing costs to conduct household surveys and accompanied by decreasing response rates by the public, innovative technologies must be considered for HHS surveys to ensure sufficient sample participation and maintain statistical rigor. The use of the web for data collection is quick and inexpensive and could be a useful tool to improve the efficiency of HHS surveys and timeliness of data collection and reporting. The use of the NHIS as a frame for other surveys has several advantages. The NHIS interviewed sample is relatively large—between 35,000 and 41,000 households annually—and includes an oversample of households with black, Hispanic, and Asian persons. The NHIS has relatively high response rates and already includes content on health care reform that could provide baseline measurements in analyses of change over time. It also offers a rich source of covariates with its flexible content and format.
Although the U.S. population has increasingly become more “connected” via the Internet and electronic devices, many households do not have ready access to computers and others have little interest in using computers or other smart communication tools that would enable this type of data collection. For this segment of the population alternative modes of contact must be considered. Combining web with other data collection modes is a methodological challenge that will be explored with this project.
This project will study the willingness of household survey participants to respond to future surveys via alternative modes and the quality of the data. Beginning in January 2012, the NHIS added questions on Internet access and e-mail use to the sample adult section. A personal e-mail address will be sought from the sample adult as well. This adult is the focus of the detailed NHIS questionnaire and will be the focus of subsequent data collection. This technique establishes a web-based panel drawn from a representative sample of the U.S. population.
A high-quality, quick-turnaround survey mechanism such as subsample of a nationally representative study population that has demonstratedanel willingness to complete internet and telephone surveys would be a useful tool for cost-effectively monitoring health systems change, provided that there is not they type of response bias that often characterizes internet panels. . The NHIS, a large continuous national survey and the principal source of health information for the civilian non-institutionalized US population, includes questions on a wide range of topics of interest to public health policymakers and researchers. The NHIS is not nimble, but its previous participants could serve as a sampling frame for additional quick-turnaround surveys.
The use of the NHIS as a frame for other surveys has several advantages. The NHIS interviewed sample is relatively large—between 35,000 and 41,000 households annually—and includes an oversample of households with black, Hispanic, and Asian persons. The NHIS has relatively high response rates and already includes content on health care reform that could provide baseline measurements in analyses of change over time. It also offers a rich source of covariates with its flexible content and format. While the NHIS sample frame has been used for years to create a frame for the Medical Expenditure Panel Survey (MEPS), it has never been used as a frame for an online followback survey.
Online data collection offers numerous advantages, including lower costs relative to computer assisted personal or telephone interviewing; the ability to more easily and responsively adapt individual questions and instrument sections compared to mail surveys; and higher data quality due to built-in skips and edits. However, those advantages cannot compensate for having nationally representative data. As such, one of the important purposes of the pilot is to assess the limitations of the proposed approach.
By following back NHIS respondents a year later, we have created a rich source of information concerning change in health care access and other key indicators related to health care reform. The NHCIS will take about 15 minutes to complete (instrument attached as Part AAttachment 3g). The NHCIS questionnaire covers numerous topics that could change over time as a result of health care reform, such as financial burden of medical care, health status, access and use of health care, health insurance coverage, preventive service use, health behaviors, and satisfaction with care. Additional questions are asked about family food security, socioeconomic status, and demographics.
Almost all of the follow-back survey questions were taken from the 2012 NHIS, with the exception of a few questions on satisfaction with care and alcohol screening. The satisfaction with care questions included in the NHCIS instrument were slightly adapted from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) questions to accommodate the mode difference. These questions are widely used in many surveys. Additionally, three questions on screening for alcohol use by a health care provider are included from a module of the Behavioral Risk Factor Surveillance Survey (BFRSS) that is being phased in. Specifically, . Sstarting in 2014, these BRFSS questions will be offered to states as an optional module. The NHCIS will take about 15 minutes to complete (instrument attached as Part A).
The NHIS interview has several components. One part of the interview is about all family members. The remaining sections pertain to one randomly selected adult and, if there are children in the family, one randomly selected child. Most of the NHIS interview is about the sample adult (SA), and this sample adult will be the subject of the NHCIS. A sSample of 12,000 adults who are not part of the MEPs subsample will be split into monthly subsamples of approximately 1,300 cases each so that interviews occur about 12 months after the 2012 NHIS interview.
As an in-person survey, the NHIS has addresses for all households and telephone numbers for approximately 95% of all cases. As mentioned previously, questions about the SA’s internet and email access, use, and frequency of use were added in 2012 so that this information would be available to use to recontact participants and to stratify the NHCIS sample. At the conclusion of the NHIS interview, approximately 30% of SAs gave a face-valid email address; however, not all reported that they checked email regularly or daily. Cases will be divided into three groups based on their 2012 responses to the internet and email use questions:
Email group (respondents who gave an email address and check it regularly or daily);
Internet group (respondents who use the internet regularly but did not give an email address or do not check email regularly); and
Phone group (respondents who do not regularly use the internet or email).
Sample allocation across the groups, illustrated in Part the attached flowchartB, will be approximately 32% email (3,840 total cases); 30% internet (3,600 cases), and 38% phone (4,560 cases). The distribution of these strata reflects slight redistribution of cases in favor of the email group to ensure adequate sample size. SAs who did not have a phone and did not use the internet in 2012 will still receive an advance letter because we have 100% of participant addresses; they may have obtained a phone and/or internet access in the past year. They have the option of participating either by web or by contacting us via the toll-free number provided in the letter.
While the questionnaire can be completed via internet or telephone, invitations to participate will be offered via letter, telephone, and email. By random assignment, half of the email group will be notified of the survey by email (all of whom will be directed to the online survey), and half will receive a mailed letter containing either web or phone instructions. Cases in the internet and phone groups will receive postal mail notification, which will provide respondents with either a URL, username and password with which to access the web survey, or a toll-free number to call to complete the survey by telephone (group flowchart attached as Part B).
Data Collection
The NHCIS sample frame is drawn from consists of 12,000 nationally representative respondent cases that completed the NHIS SA interview from April through December 2012, and were not included in the MEPS sample frame. They will be recruited for the NHCIS by email or letter as described in the previously.
paragraph.
The first two sections of the telephone and web instrument will be used to locate and verify the identity of the 2012 NHIS SA. Respondents’ answers must match the existing NHIS values on at least two of the variables listed below:
Name (first name, last name, or alias)
Date of birth, where at least 2 of the 3 fields match (i.e., day and month; day and year; month and year)
Age (this will be the 2012 AGE variable plus one year to make it current), and
Gender.
The interview will end if the respondent ‘fails’ and is considered a non-match. No PII from 2012 is revealed to respondents. Following respondent verification, the informed consent section is administered.
The Census Bureau has extensive experience conducting both telephone and web surveys, and has been the data collection agent for the NHIS since the start of the study in 1957. The Bureau has experience collecting data online for over 30 surveys using a secure network, and will host the survey-specific website used to collect data online. They have also conducted usability testing to ensure that the screens and instructions are easily understood.
This project will use professional telephone interviewers at the Jeffersonville, Indiana National Telephone Calling Center. All telephone numbers will be hand-dialed. Deaf or hearing-impaired respondents will be able to use a TTY to complete the NHCIS interview. The Bureau also has experience collecting data online for over 30 surveys using a secure network, and will host the survey-specific website used to collect data online. Respondents will have a unique username and password to protect their identity and responses and will be asked to verify their identity when logging on. The web instrument will be optimized to the extent possible to accommodate different types of operation systems and devices (iPad, tablets, desktops, laptops, mobile telephones, etc.). Respondents who receive the web instrument will be able to stop the survey at any time and access it at a later date to pick up where they left off or switch from web to telephone.
After the emails and letters are sent, the NHCIS recruits will be given approximately one week to complete the online or phone interview. If the interview is not completed after the first week, a reminder email or letter will be sent. In order to maximize response rate, cases that remain non-responsive after being sent a reminder notification will be followed up by telephone, regardless of initial mode assignment. Respondents will have the option of switching modes, should they so desire. An extensive paradata file will capture a full range of outcomes, including mode of completion, cross-overs, breakoffs, administration and item times.
Based on MEPS findings, we expect to achieve a similar response rate of around 60%. Much can be learned about the characteristics of the nonrespondents from their 2012 NHIS interview data.
Substantive longitudinal analyses
Subject to quality control procedures and discussion with OMB about reweighting approacheevaluation, s , NCHS maywill release a NHCIS public use file (PUF) and related documentation. Such public releases of tThe NCHIS microdata file would will be releasedbe dependent upon following experiences gained through processing, editing, weighting, and clearance by the NCHS Disclosure Review Board. A means to link the NHCIS and NHIS microdata wouldill be provided to analysts. The initial report would ill include survey methods, design, and operations specifics, along with a discussion of any differences related to recruitment mode, interview response mode, and internet and email use. Data quality and response bias analyses permitting, eEnsuing analyses wouldill describe changes in health outcomes and possibly identify predictors of such change.
This linked NHIS-NHCIS file will greatly enhance the NHCIS as an analytic resource. Changes between the initial interview in 2012 to the follow-back interview in 2013 will be examined for measures including health status, health insurance coverage and coverage stability among the insured and previously insured, having a usual source of medical care, use of selected preventive services, provider contact, types of providers seen, communications with one’s regular source of medical care, medication cost avoidance strategies, unmet medical needs due to cost, and other access to care measures.
5.2 Methodological analyses
More specifically, detailed paradata about the interview process will be examined to better understand response propensities, amount of effort needed to obtain responses from various types of respondents, the time lag between letters, telephone calls, and reminder emails and interview completion. To assess nonresponse, we will compare nonresponse rates between the two modes of administration, as well as employ record linkage to match follow-up nonresponders to 2012 data to describe them according to demographic, socioeconomic, and health characteristics. In addition, we will examine response rates and administration and item times in each mode of administration for variation along demographic, socioeconomic, and other criteria. Further paradata examinations will include number and point of breakoffs across modes, characteristics of respondents who cross over from one mode into the other, any reports of problems with the web instrument and the number of times help screens were accessed.
Recognizing that both mode of notification and mode of interview completion represent potential sources of bias to estimates, embedded in the multi-mode survey is an experimental component that will allow both response bias and mode effects to be examined. By systematically varying recruitment mode (telephone or email recontact) and response mode (encouraging telephone or internet data collection), we also hope to identify the most effective methods for conducting future followback surveys and whether the most effective method differs by participants’ email and internet usage characteristics. The varying response modes may also shed light on the susceptibility of NHIS questions to mode effects. To maintain adequate power in light of this embedded experiment, we will collapse groups analytically as appropriate. To further assess the ability to use this panel for faster and less costly internet only surveys, we will explore whether there is a basis for reweighting the internet group to represent the US population. Reweighting the internet portion of the sample is a statistical exercise for qualitative research only. It is not intended for this to replace in the public use file the entire sample of respondents using both modes.
While the questionnaire can be completed via internet or telephone, invitations to participate will be offered via letter, telephone, and email. By random assignment, half of the email group will be notified of the survey by email (all of whom will be directed to the online survey), and half will receive a mailed letter containing either web or phone instructions. Cases in the internet and phone groups will receive postal mail notification, which will provide respondents with either a URL, username and password with which to access the web survey, or a toll-free number to call to complete the survey by telephone (group flowchart attached as Part B).
In early March 2013, a brief test of sample verification procedures, case management and systems tests, and instrument programming will be conducted with thirty cases drawn from selected Quarter 1/2012 NHIS sample. This is a debugging process; the questionnaire should not undergo any changes as a result of this test. The main implementation is scheduled to start in late April or May 1, 2013, and continue through early 2014.
Incentives
Declining response rates are a great concern for survey research. Asking respondents to participate in a followback interview is a burden that might be met more favorably with a token of appreciation. We do not assume that remuneration will impact response rates, and have not used them with the NHIS, but recognize that a systematic test would be a valuable addition to this research. A review of the literature provides a vast array of research findings but no concrete recommendations. We propose to randomly assign cases to one of three treatment groups: no promise of remuneration, a promise of $10, and a promise of $20 upon completion of the interview. Mention of this will be in the advance letter sent to all households via mail or internet (email). This amount is less than that offered by other surveys that expect participation over a longer period of time. A report of the impact of remuneration on response rates will be produced.A $10 cash token of appreciation will be provided upon interview completion. This amount was selected based on findings from an incentive experiment embedded in a recent telephone module of the State and Local Area Integrated Telephone Survey (SLAITS). Incentive-eligible SA respondents who complete the interview but decline to confirm or provide address information cannot be mailed the incentive payment. The Thank You letter, sent upon completion of the interview, is attached as Part C.
Page
| File Type | application/msword |
| File Title | NCHS RESEARCH ETHICS REVIEW BOARD (RESEARCH ERB) |
| Author | kdo7 |
| Last Modified By | Marcie Cynamon |
| File Modified | 2013-03-01 |
| File Created | 2013-03-01 |