Details of Methodological Projects

Attachment 6 - NHIS Methodological Projects for 2017
The Division of Health Interview Statistics (DHIS) is in the process of redesigning the NHIS questionnaire, for a
planned implementation of the new survey instrument beginning in January 2018. To inform and support
decision making, a number of short, web- and/or mail-based, quick-turnaround methodological, cognitive and
field testing projects are anticipated for 2017. Collectively, these projects are represented in line 5 of the burden
table.
The aims of these projects are to test new and updated questionnaire items, evaluate the impact of different
response options on answer choices, and measure respondents’ comprehension of health care-related terms
and concepts. To this end, these projects will build on information learned in prior NHIS follow-back surveys and
methodological research. We outline research below.
Research using a commercial web panel
Respondents from commercial web panels can be used to test conceptual understanding of key terms in
surveys. DHIS previously tested web panel respondents’ comprehension of key terms related to health insurance
and the Affordable Care Act (ACA) through a brief web questionnaire that took approximately 10 minutes to
complete. For example, respondents were asked questions to measure their understanding of deductibles, copays, and premiums. DHIS also asked respondents questions about their knowledge of and familiarity with
different aspects of the ACA including the “Health Insurance Marketplace” and healthcare.gov website. The
findings from the web panel were used to improve questions on health insurance. We would like to continue to
use this method to aid in the development of new and revised content in the redesigned questionnaire.
Follow-back surveys with NHIS respondents
DHIS has obtained e-mail addresses from respondents who completed the NHIS. This information can be used to
recontact NHIS respondents to participate in additional follow-back surveys where we can test new versions of
survey questions. This would involve testing components of the full NHIS instrument with new or revised
content. Follow-back surveys of NHIS participants have been used in the past to test survey questions related to
the ACA. We would use follow-back surveys to explore the following research questions:
1. How does alternate question wording affect respondent comprehension of the constructs to be
measured?
2. What is the test/retest reliability of newly-developed questions?
3. What are preferred answer choices for categorical questions?
4. What probes (e.g., time frames or date ranges) work best to promote recall of medical visit dates?
The sample for these projects will consist of current- and prior-year (2013-2017) NHIS respondents who
provided a valid email address and are not in the MEPS sample. NCHS has several thousand email addresses.
Lab and field pretesting
Cognitive testing may be used to understand how respondents interpret new and revised survey questions for
the redesigned survey. DHIS can use the cognitive lab through the Collaborating Center for Question Design and
Evaluation Research (CCQDER) at NCHS to conduct this testing. These projects typically involve recruiting 20-30
research participants through advertisements to visit the lab. An interviewer administers the questions and

observes the participants’ answers. The interviewer will then ask follow-up questions to understand how the
participants processed the question and any problems that the respondent had answering the survey questions.
A dress-rehearsal field pretest may also be used to test components of the redesigned questionnaire. A field
pretest gives DHIS an opportunity to test the design and operational aspects of the survey prior to full-scale
implementation of the redesigned survey. DHIS staff would examine a number of data points from the pretest
such as the input/output specifications, item distributions, skip patterns, and item nonresponse to understand
how well the survey is working. Respondent and interviewer debriefings may also be used to identify any
problems with the survey questions or flow of the instrument.