Annex E - Cognitive test report

ANNEX E

REPORT ON COGNITIVE INTERVIEWS

Report on cognitive interviews provided by Iowa State SBRS, March 16, 2012
To assist with the development of the Provider and Stakeholder survey instruments for this
project, a series of seven cognitive interviews were conducted in February and March of 2012.
Participants. Participants for the seven cognitive interviews were recruited by word-of-mouth
from small communities in Iowa that are not among the project’s sampled communities. Each
cognitive interview participant fulfills a role in his/her community that is similar to eligible
respondents in the actual project. Four of the participants are stakeholders and three are health
care providers in their communities. The providers included a physician (MD), a nurse
practitioner (ARNP), and a dentist (DDS). The stakeholder participants included an Economic
Development Director, the Chief Operating and Compliance Officer of a small medical center, a
Hospital Assistant Administrator, and a Chamber of Commerce Director. The seven cognitive
interview participants included four men and three women.
Methodology. The cognitive interviews were conducted over the telephone by one of three
SBRS staff using paper documents. A written script was used to introduce each interview and to
provide consent information emphasizing the confidentiality and voluntary nature of
participation. The cognitive interview participants were given basic background information
about the project that was similar to the information that actual project participants will receive.
Respondents were asked to think aloud as they considered and processed the survey questions.
They were instructed to ask the interviewer if there was anything unclear or that they did not
know how to answer. One practice question (“How many places have you lived in since you
were born?”) was read to the respondents to illustrate the reason for doing cognitive interviewing
and to help them practice “thinking aloud.”
Throughout the interview, each survey question was read to the respondent and answers were
recorded following standard interviewing procedures. Respondents were also reminded
periodically to “think aloud” as they considered their answers and to ask questions if they were
uncertain about the meaning or intent of specific items. Interviewers made notes to record any
respondent comments or questions.
The final component of the cognitive interview consisted of specific requests for feedback. After
all questions were answered, the interviewer probed for anything that might have caused the
respondent difficulty, for any pertinent issues that were not included, and for general comments
or suggestions.
After completing the cognitive interview process, participants were sent a $50 gift card for a
local grocery store to thank them for their time and cooperation. For documentation purposes
they were also sent a receipt to sign and return to SBRS in an enclosed pre-addressed postagepaid envelope.
Survey Revisions. The series of cognitive interviews identified no major problems with the
surveys. However, numerous minor issues were raised. Potential revisions were discussed and
carefully considered by the research group before changes were incorporated. The goal was to

make the surveys clear and consistently understood for as many people as possible, and
clarifying a text for one person could make it more obscure for others.
A variety of revisions were made as a result of the cognitive interview response. Several question
transitions and introductions were revised for clarity. Minor edits were made to several
questions, also for clarity. Two items identified as redundant were deleted, and three new items
were added. In the provider survey, questions relating to type of practice and residency or
internship location were changed significantly both to clarify and to focus more effectively on
the question intent. In addition to the survey revisions, the cognitive interviewing process also
highlighted the need for certain resource documentation to be available for telephone research
interviewers during project data collection.
Conclusion. The cognitive interviews conducted by SBRS staff proved to be a valuable tool in
the refinement of the survey instruments to be used for this study. Several clarifications and
revisions were made that will increase the effectiveness and accuracy of the instruments. The
research staff felt that the time and effort put into the series of cognitive interviews was very
worthwhile and will result in greater consistency and accuracy of data.