2015 National Survey of Children's Health Pretest:
Request for OMB Review
Supporting Statement B
January 15, 2015
OMB Control No. 0607-XXXX
Since the beginning, the purpose of the National Survey of Children’s Health (NSCH) has been to estimate national and state-level prevalence for a variety of physical, emotional, and behavioral child health indicators in combination with information on the child’s family context and neighborhood environment. Conversely, the purpose of the National Survey of Children with Special Health Care Needs (NS-CSHCN) was to estimate the prevalence and number of children with special health care needs (CSHCN) in the state population and to estimate the percent of households with children having one or more CSHCN age 17 or younger. Recently, decreasing response rates required the Maternal and Child Health Bureau (MCHB) to redesign the NSCH and NS-CSHCN. This redesign includes combining the original NSCH and NS-CSHCN into a single annual survey that will be known solely as the NSCH with the first year being a Pretest. Another large part of the redesign involves changing the sampling frame from a list-assisted Random Digit Dial (RDD) to an Address-Based Sample (ABS) frame. Two new modes of data collection will be evaluated during the NSCH Pretest: Mail (self-administered) and Web (self-administered) which is different from the telephone, interviewer-administered mode that was originally used.
The NSCH Pretest will use an address-based sample derived from non-Title 13 sources and covering the 50 states and the District of Columbia. The pretest will be conducted from June 2015 through December 2015. Households will be randomly sampled as described in section B.1.1 and a screening questionnaire will be sent to each sampled household. Information on the presence of children within the household, child demographic information, as well as basic questions about each child’s health provided on the screener will be used to determine whether the household is eligible for one of the three age-based surveys: 0 to 5 year old children, 6 to 11 year old children, or 12 to 17 year old children. This screener information is also used for the subsampling selection of a specific child within the household based on an oversampling of CSHCN at 35%. In order to limit respondent burden, regardless of the number of eligible children, no more than one child per household will be sampled for the age-based topical surveys, which means no more than one topical survey, will be administered in any given household. The target population for the NSCH survey consists of children age 17 or younger.
For the NSCH Pretest, a sample frame of 16,000 household addresses will be used. The sample file was provided by the Census Bureau’s Center for Administrative Records Research and Applications (CARRA). As background, CARRA is an interdisciplinary group in the Research and Methodology Directorate and is charged with the strategic re-use of administrative data from federal, state, and commercial providers. Information is combined from multiple sources to create new data products that would be impossible to produce using single data sets.
Through their record linkage and statistical matching, Census Bureau operations are enhanced, demographic and socioeconomic research capabilities are extended, data quality is improved, data collection and processing costs are reduced, and respondent burden is lowered.
The following CARRA information is pertinent to the NSCH Pretest:
Depending on the universe, CARRA can provide phone numbers for approximately 84% of households in the U.S.; they can be identified as landline or cell, and can be ranked according to quality when there are multiple numbers per address.
Two data sources were used for CARRA’s creation of the NSCH Pretest sample:
Data Source 1
88.59 percent coverage of all residential units in the United States.
A variable exists which indicates presence and number of children in a household (22.62 percent of households indicated having at least one child).
Variables exist for ages of the children, 0-5, 6-10, 11-15, 16-17; for the NSCH Pretest, the last two categories were collapsed to ages 11-17.
Data Source 2
93.7 percent coverage of all residential units in the United States.
A variable exists which indicates presence of children.
It was also a requirement that at least one phone number had to be present on each address record.
CARRA provided this sample frame of 16,000 households, allocated across the 50 states and the District of Columbia, and stratified by state and the age group of children present in the household. The strata were mutually exclusive, prioritized by the age group of children present in the household (age group 0-5 first). See Table B.1.1.A. The sample sizes were allocated based on the number of records per state for each age group. For example, if Arizona has 2 percent of the records in the sample universe for children in the age group 0-5 (Stratum S1), Arizona has 2 percent of the 0-5 age group sample.
Table B.1.1.A: CARRA’s Addressed-Based Sample for the NSCH Pretest
Stratum |
Age Group of Children |
Explanation |
Stratum Size |
S1 |
0-5 |
|
4,000 |
S2 |
6-10 |
…but excluding addresses in S1 |
4,000 |
S3 |
11-17 |
…but excluding addresses in S1 and S2 |
4,000 |
S4 |
Other |
…all remaining addresses; do not identify as having children |
4,000 |
The following Table B.1.1.B is a summary of the comparisons that will be performed on the mode and incentive treatment groups based on the screener response rates. The Fisher Exact Test is used to derive power.
Table B.1.1.B: Mode and Incentive Treatment Group Comparison
Treatment Group |
Initial Cases |
Mode |
Screener Incentive |
Topical Incentive |
Incentive Comparison |
Mode Comparison |
1 |
7,120 |
$5 |
$5 |
n/a |
1 vs. 2 p=0.05, power≈1.00 p=0.10, power≈1.00 p=0.20, power≈1.00 |
|
2 |
3,560 |
Web Push+Mail |
$5 |
n/a |
2 vs. 3 p=0.05, power=0.94 p=0.10, power=0.97 p=0.20, power=0.99 |
n/a |
3 |
3,560 |
Web Push+Mail |
$10 |
n/a |
Eligible children within households that have a completed screener will be sampled for one of the three age-based topical surveys: 0 to 5 year old children, 6 to 11 year old children, or 12 to 17 year old children. Only one child per household will be selected for a topical questionnaire in an effort to minimize respondent burden.
In order to select the sample child from a household, it must first be determined whether each eligible child is a Child with Special Health Care Needs (CSHCN) or a Child without Special Health Care Needs (Non-CSHCN). This is determined by how specific health questions are answered on the screener questionnaire for each child. After this determination is made, each household will fall within a specific Household Type (HHTYP). See Table B.1.2.A for the strategies used for selecting the NSCH Pretest Sample Child.
Table B.1.2.A: Strategies for Selecting the NSCH Pretest Sample Child
Household Type (HHTYP)
|
Number of Eligible Children in Household |
Number of Eligible Non-CSHCN, CSHCN |
% Probability of Selection for Non-CSHCN |
|
% Probability of Selection for CSHCN |
Notes |
|
1 |
0 or ‘blank’ |
0,0 |
0 |
No eligible children in household |
|||
2 |
1 |
1,0 or 0,1 |
100% |
Single child is always selected |
|||
3 |
2 |
2,0 or 0,2 |
Each child has an equal 50% probability of selection |
Simple random selection of 1 of the 2 Non-CSHCN or 1 of the 2 CSHCN |
|||
4 |
2 |
1,1 |
10% |
|
90% |
Includes 80% oversampling of CSHCN |
|
5 |
3 |
3,0 or 0,3 |
Each child has an equal 33.3% probability of selection |
Simple random selection of 1 of 3 Non-CSHCN or 1 of 3 CSHCN |
|||
6 |
3 |
2,1 |
40% |
|
60% |
Includes 80% oversampling of CSHCN |
|
7 |
3 |
1,2 |
0% |
|
100% |
Includes 80% oversampling of CSHCN |
|
8 |
4 |
Any combination |
Before the sort, each of the first 4 children has an equal 25% probability of selection |
Simple random selection of 1 of the first 4 (sorted) children, regardless of Non-CSHCN or CSHCN. |
|||
Each household will be pre-assigned a value for each of the eight Household Types that corresponds with the oversampling criteria in the Probability of Selection column in the table above. This value denotes the order of the child (0, 1, 2, 3, or 4) that should be selected after the proper sorting of eligible children has occurred. For HHTYP 1 and HHTYP 2, no sorting occurs because there are either no eligible children or one eligible child who will always be selected. For HHTYP 3 through HHTYP 8, children will be sorted by their special needs status (CSHCN children first followed by Non-CSHCN) and then sorted by age (youngest to oldest). Finally, HHTYP 8 children will be sorted by their special needs status (CSHCN children first followed by Non-CSHCN), then sorted by name, and then sorted by age (youngest to oldest).
The respondent universe for the NSCH Pretest is adults age 18 or older who live in the U.S. and who are parents or guardians of at least one child who is age 17 or younger living in the same household. Those households that do not have any infants or children are asked to mark “No” to the first question on the screener which asks, “Are there any youth or children age 17 or younger who usually live or stay at this address?” and would then screen-out of the remaining survey questions.
The initial sample size for the NSCH Pretest is 16,000 unique households nationwide. These households are then split among three different treatment groups, which differ in mode (Mail Only or Web Push + Mail), and/or incentive amount ($5 or $10).
Assumed response rates for the screener, the percent eligible for the topical, and the response rates for the topical are critical to the calculation of the various expected sample sizes for the NSCH Pretest1. These percentages are included in Table B.1.3.A, resulting in the three analysis treatment groups in the last column.
The random assignment of mode and incentive for each household will be made according to the percentages and assigned randomly across each of the strata provided by the CARRA dataset. These assignments will be entered for each record on the 16,000 sample input file, before any data is collected.
Table B.1.3.A: Expected Sample Sizes of NSCH Pretest Treatment Groups
Initial Sample |
Valid Sample (89%)2 |
Census Assigned Screener Mode |
Respondent Choice of Screener Mode |
Screener Incentive Response Rate % |
Sample after Screener
|
Sample Eligible for Topical (40%)3 |
% Topical Incentive4: sample Response Rate % |
Sample after Topical |
Treatment Group |
16,000
S1: 4,000 S2: 4,000 S3: 4,000 S4: 4,000 |
14,240 |
50% Mail: 7,120 |
100% $5: 7,120 70% |
4,984 |
1,994 |
100% $5: 1,994 70% |
1,396 |
1 |
|
50% Web Push+Mail: 7,120 |
55% Web: 3,916 |
50% $5: 1,958 66% |
1,292 |
517 |
100% $0: 517 95% |
491 |
2 |
||
50% $10: 1,958 70% |
1,371 |
548 |
100% $0: 548 95% |
521 |
3 |
||||
45% Mail: 3,204 |
50% $5: 1,602 56% |
897 |
359 |
100% $0: 359 65% |
233 |
2 |
|||
50% $10: 1,602 60% |
961 |
384 |
100% $0: 384 65% |
250 |
3 |
||||
Totals |
9,505 |
3,802 |
|
2,891 |
|
||||
There will be written specifications for weighting the data that will have been collected in the NSCH-FS Pretest for people in households selected from the frame obtained through the Census Bureau's own Center for Administrative Records Research and Applications (CARRA) database. Instructions for computing adjustments (e.g., under coverage and nonresponse) and several final household and person-level weights will be provided.
Standard 1.3 of the OMB Standards and Guidelines for Statistical Surveys (2006) states that “Agencies must design the survey to achieve the highest practical rates of response, commensurate with the importance of survey uses, respondent burden, and data collection costs, to ensure that survey results are representative of the target population so that they can be used with confidence to inform decisions.” Implicit in this standard is the assumption that the frame variables used at the design stage are sufficiently predictive of the collection variables for this to be feasible. Under this assumption, standard nonresponse bias analyses techniques can and will be applied to frame data variables to study potential areas of nonresponse bias (both item and unit) in the survey estimates.
This section describes the data collection procedures that will be used in the NSCH Pretest. Data will be collected from respondents via a Mail or Web survey. Contact will be made to the 16,000 households by one of those two modes (8,000 by Mail and 8,000 by Mail with the Web URL, username, and password included in the invitation letter).
Mail respondents will receive a two-phase self-administered survey. In the first phase, households will be screened to determine if there are any children age 17 or younger who usually live or stay at that address. Those households that meet the eligibility criteria go on to roster the children living at that address and answer questions to determine the special needs status of each child (up to 4 children). Detailed information will be collected for Child 1 – Child 4, while basic information (name, age, and sex) will be collected for Child 5 – Child 10. In the second phase, households that are deemed to have eligible children will be mailed one of the three age-based topical questionnaires that request more information about one specific child living at that address.
Web respondents will receive a mailed request to complete a single-phase self-administered survey. This means that the screener and the topical are combined into a single online Web instrument. These respondents would receive the same “screening” questions as the Mail respondents, but would instead be requested to complete detailed information for as many children as they have living or staying at that address. Web respondents would still only have one child selected for an age-based topical questionnaire and would be taken directly into it after reviewing their child roster on the child dashboard screen.
The topical survey (for both Mail and Web) will cover the following content areas: child’s health and functional status; the child as an infant; health care services; experience with child’s health care providers; child’s health insurance coverage and experience of uninsured children in low income families; providing for the child’s health; the child’s learning, schooling, and activities; family functioning; parental health; neighborhood and community characteristics; and adult demographics.
The NSCH Pretest employs multiple contacts with households to maximize response. These include an advance letter, up to four questionnaire mailings at each phase, as well as one reminder postcard. The questionnaires and letters can be found in Appendix B.
In designing the various modes of the NSCH Pretest screener and topical questionnaires, attention will be placed on the following design elements to help increase cooperation by prospective respondents.
In developing and refining specific questions, the goal will be to create a logical, clear questionnaire with concrete question wording and simple grammar
The Mail and Web versions of the questionnaire will be attractive with clear and simple instructions on how to complete specific questions
Questions will be grouped according to subject areas
Questionnaire formatting will maximize readability, including appropriate question spacing, font type and size, and easy to follow skip instructions
Questionnaire formatting considerations will also include the use of color and pictures to enhance respondent comprehension
Respondent contact strategies and letters have been carefully put together to grab the attention of the respondent and peek interest in the subject matter
In addition to the methods above, we have designed a multi-wave incentive experiment to evaluate the efficacy of incentives as a means of increasing respondent cooperation with the NSCH Pretest. See Section A.9 of the supporting statement for more information on incentives.
Data collection for the NSCH Pretest will involve a series of mailings and nonresponse follow-up activities, emphasizing questionnaire completion. Our proposed approach to data collection and nonresponse follow-up is based on previous project experience and recommendations made by Dillman and colleagues (2009)5.
Advance Letter and Invitation Letter for Paper and Web. For both mode conditions, potential respondents will first receive an advance letter providing details on the study, a brief explanation of the importance of their cooperation, along with the notification that they will be receiving a small token of appreciation for their participation. The invitation letter for the paper mode will be accompanied by a screener questionnaire and return envelope. For the Web mode, the invitation letter will include the URL along with the username and password for accessing the Web version of the questionnaire (which combines the screener and topical into a consolidated instrument). Both the Mail and Web mode invitation letters will include an incentive: paper modes having a $5 cash incentive, while the Web is either a $5 or $10 cash incentive. Both versions of the invitation letter (Mail and Web) will include a toll-free number for the individual to call if there are questions or comments. The letters will be printed on official Census Bureau letterhead and will be signed by the director, John H. Thompson.
Reminder postcard. Subsequent to the invitation letter mailing, the Census Bureau will send a reminder postcard to all non-responding individuals assigned to the Mail and Web modes. The postcard is sent within a week after the initial invitation and encourages those who have not completed and returned their survey or those who have not completed the questionnaire online to do so right away. It will contain the NSCH toll-free number and e-mail address so that respondents can contact the Census Bureau with questions or if they need assistance.
Hardcopy questionnaire mailing. For Mail mode cases, the topical questionnaire and accompanying cover letter will be personalized to fill in the sample child’s name and other identifying information to ensure that the survey is completed for the correct child. This level of personalization in the questionnaire improves data quality by reducing the opportunity for skip logic errors. It also results in a questionnaire that is as short as possible for the selected child. The shorter the questionnaire, the more likely the respondent is to complete it.
Replacement questionnaire/invitation letter mailings (and FedEx Mail delivery). Subsequent to the previous reminder postcard mailing, the Census Bureau will send all remaining Mail nonrespondents a second hardcopy screener questionnaire and all Web nonrespondents a second invitation letter. Following the second invitation mailing, all nonrespondents (Mail and Web) will receive a FedEx Mail delivery. This package will contain a personalized letter that urges the household to respond as well as a paper screener questionnaire and return envelope. The packaging, mode of delivery, and speed by which it will be delivered will further stress the importance and urgency for response. Respondents will also receive a pre-recorded call via a phone tree a few days prior to the FedEx mailing to alert them of the arrival of the package. Finally, a fourth mailing will be sent to all nonrespondents containing one last invitation letter, hardcopy screener questionnaire, and return envelope. All Mail topical nonrespondents will receive a second, third, and fourth invitation letter along with a hardcopy topical questionnaire and return envelope.
Nonresponse Follow-Up. All nonrespondents who have not completed a screener after the fourth invitation mailing will be sent to Telephone Follow Up (TFU). These cases will be called over a period of about a month in order to attempt to gain cooperation over the phone. Census Bureau call centers will track these calls in the WebCATI system and, if requested, will collect the respondent’s answers over the phone.
These operational strategies will both facilitate response rates and reduce the likelihood of differential non-response by mode, which may introduce substantial bias in the resulting estimates.
The Census Bureau will collect the information on behalf of MCHB. Contact information for the Census Bureau’s principal staff on the project are listed below:
Jason Fields
Survey Director
U.S. Census Bureau, ADDP, HQ-7H069
4600 Silver Hill Road
Washington, DC 20233-0001
Phone: (301) 763-2465
Email: jason.m.fields@census.gov
Daniel Doyle
Assistant Survey Director
U.S. Census Bureau, ADDP, HQ-6H242A
4600 Silver Hill Road
Washington, DC 20233-0001
Phone: (301) 763-5304
Email: daniel.p.doyle@census.gov
Leah Meyer
Survey Team Member
U.S. Census Bureau, ADDP, HQ-6H158A
4600 Silver Hill Road
Washington, DC 20233-0001
Phone: (301) 763-7174
Email: leah.meyer@census.gov
1 These percentages will be analyzed and adjusted, as needed, for production.
2 The 89% valid rate is based on NHES-12 results.
3 For MSG addresses that indicated to have at least one child present, via their child identifiers, only 40 percent of the time did the NHES actually find a child.
4 Since the web instrument is a combined screener and topical module, a web respondent would never be mailed any follow-up correspondence regarding the topical with a $0 incentive (unless they requested a paper version of the questionnaire); instead, they would complete the survey in its entirety.
5 Dillman, Don A., Jolene D. Smyth, and Leah Melani Christian. 2009. Internet, mail and mixed-mode surveys: The tailored design method, 3rd edition. Hoboken, NJ: John Wiley & Sons.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |