Project Description and Justification

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

May xx, 2019

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, exp. 08/31/2021) plans to conduct a cognitive interviewing study to evaluate American Community Survey (ACS) questions on disability (OMB No. 0607-0810 Exp. Date 06/31/2021; OMB No. 0607-0936, Exp. Date 12/31/2009) and National Health Interview Survey (NHIS) questions on child behaviors, including injuries, sleep, screen time, and physical activities (OMB No. 0920-0214, Exp. Date 12/31/2020). The ultimate goals are to evaluate the impacts made from adapting the Washington Group (WG) short-set of questions on disability to the ACS survey format, and to evaluate new and revised questions on child behaviors in preparation for the redesign of the 2020 NHIS.

Study Backgrounds and Purposes:

ACS Questions on Child and Adult Disability

To improve ACS data on disability, it is proposed that the ACS adopt the Washington Group (WG) short set of questions on disability. The WG short set includes six questions on hearing, seeing, walking, cognition, self-care, and communicating. To be included, several changes are required of the WG questions to accommodate the format of the ACS. The purpose of this study is to evaluate any impact of those changes on the performance of the WG questions.

A primary change for the WG questions is the establishment of one set of questions for all household members, regardless of age. (Currently, the WG has specific sets of disability questions for adults, for children ages 5-17, and for children ages 2-4.) Additionally, wording will be changed from “you,” “your child,” or “[name]”to “this person” as is consistent with the ACS format.

Finally, cognitive testing will evaluate the overall usability of WG questions as self-administered questions. For this study, the administration of the ACS questions will closely resemble an abridged version of the ACS instrument, with household demographics questions appearing before the disability questions. Participants respond to all questions for each household member, including themselves.

NHIS Questions on Child Behaviors

The NHIS questions on child behaviors are being tested in preparation for the upcoming redesign of the 2020 National Health Interview Survey. Rationales for testing and inclusion are detailed below:

1. Child Injury

Injuries are the leading cause of death in children. Non-fatal injuries are also a serious health problem for children and adolescents. The proposed child-injury questions are adapted from existing NHIS adult-injury questions. Specifically, the questions being tested are designed to assess incidence, severity, and type of injury occurring among children in the past 3 months. The questions will also examine where injuries occur.

2. Child Sleep

Insufficient sleep, poor sleep quality, and sleepiness are common problems in children and adolescents and translate to both health problems and academic difficulties. Five questions on sleep have been selected for cognitive testing which borrow from the Pediatric Sleep Questionnaire and are designed to determine whether the child experiences sleepiness during the day, the quality of the child’s sleep, and any difficulties breathing while sleeping. Cognitive testing will help determine if parents may struggle answering these new questions.

3. Child Screen Time

As technology has evolved, the use of screened devices has become more common among children and new questions that capture the various ways children utilize screens are necessary.  One question has been selected for cognitive testing which borrows from the National Survey of Children’s Health, which asks whether the child engages in screen time (outside of schoolwork) for 60 minutes in a given day. Previous cognitive testing indicated that parents struggle to provide an exact number of hours a child uses screens on a typical school day. This question will help determine if providing a distinct cut-off helps to produce more reliable parental responses.

4. Child Physical Education

Regular physical activity has been shown to not only help children and adolescents improve their cardiorespiratory fitness, but also reduce their likelihood of developing mental health and other chronic health conditions. A set of four questions has been selected for cognitive testing that explore a child’s level of physical activity, as well as their potential for physical activity.  Questions have been derived from the Youth Risk Behavior Survey and the National Household Travel Survey.

Study Protocol:

The ACS questions on disability and the NHIS questions on child behaviors are included as Attachment 1. Interviewers will first hand participants the attached ACS question instrument. After participants complete the instrument, they hand it back to their interviewer. The interviewer then probes on the ACS questions, focusing on any of the participant’s children with disabilities (response of “Yes”). Once finished probing on the ACS questions, interviewers then continue with the NHIS questions, which are interviewer administered.

The testing procedure conforms to the cognitive interviewing techniques that have been described in CCQDER’s generic OMB clearance package (OMB No. 0920-0222, Exp. Date 08/31/2021). Staff will adhere to a protocol designed for previous projects when screening, selecting, and acquiring informed consent for adults.

We propose to recruit at least 20 English-speaking adults (aged 18 and over) who are parents/guardians of children ages 2-17. The recruits will range in their experiences with the following criteria: (1) are parents/guardians of children ages 2-17 who have had an injury during past 3 months, (2) are parents/guardians of children ages 2-17 who have difficulty seeing, (3) are parents/guardians of children ages 2-17 who have difficulty hearing, (4) are parents/guardians of children ages 5-17 who have difficulty walking, remembering or concentrating, with self-care, or communicating. Recruitment of individuals will be guided first by the criteria listed above. Participants may meet any number of the criteria, including no criteria, and there are no specific combinations of criteria. Each of the criteria is aligned with a set of questions on the instrument. Also, because qualitative sampling is based on theoretical relevance more than equal cell sizes, on-going analysis may reveal the need to test specific sets of questions more than others. We will recruit up to 20 more English-speaking adults if on-going analysis suggests a need for additional testing.

Secondarily, we aim to recruit respondents with a roughly even mix of age, race, and educational attainment. The initial goal is to recruit groups in equal proportion, to the extent possible; that is, within the constraints of those willing to participate in the study.  However, as explained above, qualitative sampling is based on theoretical relevance and on-going analysis may determine a need to recruit more from one group than others.  The PI and recruiters will determine if there is a need for more participants of particular age, race, or educational attainment after the first round of testing.

Recruitment will be carried out through a combination of a newspaper advertisement(s), flyers, word-of-mouth, and the CCQDER Respondent Database. The newspaper advertisements/flyers used to recruit respondents are shown in Attachments 2a and 2b. The 5-minute screener used to determine eligibility of individuals responding to the newspaper advertisements/flyers is shown in Attachment 3a. The 5-minute screener used to determine eligibility of individuals from the CCQDER Respondent Database is shown in Attachment 3b. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 80 individuals may need to be screened in order to recruit 40 participants. Within these constraints, we plan to recruit participants with some demographic variety (particularly in terms of gender, education, and race/ethnicity).

Interviews will be conducted by CCQDER staff members with English speaking respondents for close to 60 minutes per interview. Interviews will be conducted in the Questionnaire Design and Evaluation Research Laboratory as well as at off-site locations. All interviews conducted in the Questionnaire Design and Evaluation Research Laboratory and off-site will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. These recordings will allow researchers to ensure the quality of their interview notes. Recordings will only be used by researchers from CCQDER and NCHS who are working on the NHIS project as well as Census researchers who are working on the ACS project. Recordings will remain under CCQDER staff control. There will be no external sharing of the recordings. NCHS and Census researchers will have read and signed a non-disclosure affidavit before viewing/listening to recordings in the QDRL under CCQDER supervision.

Video or audio recording is required for this project except in the rare case that a study participant initially agrees to be video recorded during the telephone screening, but changes their mind. In that case, they will be asked if they agree to be audio recorded. If they decline to be audio recorded the interview will proceed without recording. In this case the interviewer will depend on their handwritten notes when conducting analysis.

NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted off-site. Due to the size of the video recordings, the internal drive of the encrypted laptop is not sufficient for storage of the recordings. Recordings will be saved to an NCHS government issued encrypted flash drive. The encrypted flash drive is FIPS 140-2 compliant and approved for use by OCISO. CCQDER staff will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes. Within 24 hours, a CCQDER staff member will review ACS Questions on Disabilities and NHIS Questions on Child Behaviors interview notes and will delete any direct or indirect personal identifiable information (PII) if found.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.

After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about adult and child function, child injury, child sleep, child screen time, and child physical education. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

After the interview, respondents will be given the thank-you letter (document contained in umbrella package) signed by the Director of NCHS (Attachment 6), a copy of the informed consent document, and $40. Special Consent for Expanded Use of Video and Audio Recordings will NOT be asked of respondents because they are answering questions about their children (child injury, child sleep, child screen time, child physical education, and child disabilities).

We propose giving respondents $40 incentives, which is our standard incentive. In total, for this project, the maximum respondent burden will be 46 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses per Participants	Average hours per response	Response Burden (in minutes)
Screener (recruited from newspaper/flyer)	40	1	5/60	3
Screener (recruited from CCQDER Respondent Database	40	1	5/60	3
Questionnaire	40	1	55/60	37
Respondent Data Collection Sheets	40	1	5/60	3
Total				46

Attachments (6)

cc:

V. Buie

J. Zirger

DHHS RCO

Page 6 of 6