GenIC - 10-Day Letter

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

June 30, 2016

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, exp. 07/31/2018) plans to design and evaluate a set of questions on the trust, acceptance, and confidence in vaccines and vaccination schedules. Currently, a number of different sets of attitudinal questions about vaccination are used across the Centers for Disease Control and Prevention (CDC) and the Department of Health and Human Development (DHHS)’s surveys. This project represents a collaborative effort by both NCHS and the National Center for Immunization and Respiratory Diseases (NCIRD) to harmonize these various instrument and produce a series of short question sets that can be inserted into ongoing and periodic public health surveillance surveys. We will use an iterative series of focus groups and cognitive interviews to conduct this research

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about the Design and Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical survey design and cognitive testing research. Focus groups are a widely-used method to understand how a population of interest thinks and speaks about topics of interest. Findings from the focus groups can be used, in conjunction with other sources (such as previously-fielded questionnaires) to develop a draft questionnaire that can then be evaluated through cognitive testing. As you know, the purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: Vaccination acceptance/confidence survey questionnaire development

Currently, there are no standard sets of vaccination acceptance or confidence questions across the federal government’s surveys (e.g. the National Immunization Survey), and as such estimates of vaccination rates (and the reasons behind refusals) vary widely by survey. In an effort to harmonize its measures, NCIRD hopes to develop and deploy fit-for-purpose, standard sets of vaccination acceptance and confidence questions. Their goal is to develop sets of questions that take either one, three, or five minutes of respondent burden, but which have high levels of construct and measurement reliability. One of these three sets could then be embedded in CDC or HHS sponsored surveys, depending on the needs and length of the survey. NCHS will work with NCIRD staff to develop these 1/3/5 minute vaccination acceptance sets and a core set of vaccination acceptance/confidence questions for use across multiple modes.

NCHS will conduct and analyze both focus groups and cognitive interviews in successive rounds. Analysis of the focus groups, in combination with data from NCIRD and input from the DHHS’ National Vaccine Program Office (NVPO), will allow NCHS to develop a core questionnaire and the three short sets. Following their design, cognitive interviews will be used to determine the types of experiences or perceptions that respondents include in their answers, difficulties experienced by respondents when answering the questions, preferred question language, as well as identify potential response error and potential differences in responses by mode.

The focus group moderator guide is included in Attachment 1. A set of vaccination acceptance/confidence domains we plan on evaluating through the cognitive interviews is found in Attachment 2. Please note that we are not providing specific questions, as they will be developed following the analysis of the first phase of this project, but this final questionnaire will be limited to the domains outlined in Attachment 2. The testing procedure conforms to the cognitive interviewing techniques that have been described in our generic clearance package (OMB No. 0920-0222).

For the independent questionnaire development, focus groups, cognitive testing, and analysis of vaccination acceptance/confidence questions, CQDER’s goal is to recruit a maximum of 90 participants/respondents (aged 18 and over). Recruiting will focus on obtaining a sample of focus group participants and cognitive interviewing respondents who are parents with elementary school children and younger. A maximum of five focus groups (with as many as 8 participants) and approximately 50 cognitive interviews are planned, and may take place in the Washington DC area, or any other location that the research team at NCHS and NCIRD considers useful for the study design. In addition to the Washington DC area, other potential sites include (but are not limited to) Southwest Washington State and the Atlanta, GA area. The exact number of both focus groups and cognitive interviews will depend on the iterative study design and the need to conduct groups or interviews in locations other than Washington DC. For example, if after three focus groups, the research team decides that it has reached theoretical saturation, additional cognitive interviews may be conducted instead of the other planned focus groups.

Focus group participants and cognitive interview respondents will be recruited through a variety of methods, including newspaper advertisements, online ads, word-of-mouth, and flyers. The 5 minute screeners used to determine eligibility of individuals responding to the newspaper advertisements/flyers for the focus groups and cognitive interviews are shown in Attachments 3 and 4, respectively. Note that the wording of the templates have been approved and are contained within our umbrella package. Only project specific information has been added to the documents. It is anticipated that 72 individuals will need to be screened in order to recruit 40 focus group participants. It is anticipated that 96 individuals will need to be screened in order to recruit 50 cognitive interview respondents.

Focus groups and cognitive interviews will be conducted by CQDER staff members with English speaking respondents for up to 90 minutes per focus group and up to 60 minutes per cognitive interview (including the completion of a Participant/Respondent Data Collection Sheet). Focus groups and cognitive interviews will be conducted in the Questionnaire Design and Evaluation Research Laboratory as well as at off-site locations. All focus groups/cognitive interviews conducted in the Questionnaire Design and Evaluation Research Laboratory will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. Focus groups/cognitive interviews conducted off-site will only be audio recorded. These recordings will allow researchers to ensure the quality of their interview notes. In the rare case that a cognitive interview respondent initially agrees to audio recording during the telephone screening, but changes their mind and checks “no” to allowing the interview to be recorded on the informed consent document the interview will proceed without audio recording. In this case the interviewer will depend on their handwritten notes when conducting analysis. In addition, cognitive interview respondents who select “yes” for allowing the audio recording on the informed consent form, but “no” for retaining the recording for future research will be allowed to participate in the study.

After participants/respondents have been briefed on the purpose of the study and the procedures that CQDER routinely takes to protect human subjects, participants/respondents will be asked to read and sign an Informed Consent document (Attachments 5 and 6 for the focus groups and cognitive interviews, respectively). Only project specific information has been added to the document. Participants/Respondents will also be asked to fill in their demographic characteristics on the Participant/Respondent Data Collection Sheet (Attachment 7). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The focus group moderator or cognitive interviewer will then ask the participant(s)/respondent to confirm that they understand the information in the Informed Consent. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

After the focus group/cognitive interview, participants/respondents will be given the thank-you letter (document contained in umbrella package) signed by the Director of NCHS, a copy of the consent form, and $50 for focus group/$40 for cognitive interview.

After the focus group/cognitive interview, respondents will be asked to read the Special Consent for Expanded Use of Video and Audio Recordings (Attachments 8 and 9 for focus groups and cognitive interviews, respectively). There will be no coercion and participants/respondents will be told that they can call and reverse the decision at any time if they change their minds. If participants/respondents do sign the special consent form they will be given a copy of that as well.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

In total, for this project, the maximum respondent burden will be 124 hours. A burden table for this project is shown below:

Information Collection Name	Number of Participants	Number of Responses per Participant	Average Hours per Response	Response Burden in Hours
Focus Group Screener	72	1	5/60	6
Cog Interview Screener	96	1	5/60	8
Focus Groups	40	1	1.5	60
Cognitive Interview	50	1	1	50

Attachments (94)

cc:

V. Buie

T. Richardson

DHHS RCO

3 | Page