1024-0254 Part B-9-27-2016.NPS

Supporting Statement B

National Park Service Centennial National Household Survey

Pretests: Cognitive Interviews and Household Survey

OMB Control Number 1024-0254

Terms of Clearance: None

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The sample for the proposed pretesting will consist of a total of 1,600 phone numbers drawn from all phone numbers belonging to residents of the 50 states and the District of Columbia. A dual sampling frame will be used, to include both land line and cell phone numbers. There will be no stratification of the sample for the purpose of the pretesting because neither the purpose, nor the sizes of the sample call for such. No cognitive interviews will be conducted with children. The pretest of the Youth Engagement segment of the survey will be conducted within the context of the Household survey. There will be no independent sample drawn for that purpose. We expect about 20% of the households completing the survey to have children between the ages of 12 and 17 living with them. We further expect that not less than half will agree for their child to complete the Youth Engagement segment of the survey. We expect that about 20% of those who do not agree to complete the entire survey will agree to answer just a couple of questions, which will be used for the non-response bias analysis of the final survey.

The Cognitive Interviews will be conducted using a simple random sample of telephone numbers. A dual sampling frame will be used, consisting of land line and cell phone numbers in a proportion reflective of the prevalence of cellphones based on most current estimates available at the time the sample is drawn.

The Household Survey will also be conducted by using a simple random sample of telephone numbers which will include both landline and cell phone numbers in a proportion reflective of the prevalence of cell phones based on the most current estimates available at the time the sample is drawn.

Table 1.

Pretest Elements	Sample size	Expected Raw Response Rate (%)	Total number of Completed Responses
Cognitive Interviews	400	7.5%	30
Household Survey	1,200	7.5%	90
Youth Engagement Survey	18	50%	9
Non-respondent Survey (Household Survey only)	120	20%	24
Total	1,600

This collection also includes a Youth Engagement survey. A set of questions will be asked of youth (kids aged 12-to17 living in the same household with the adult responding to the survey). This survey will be administered only in household that respond affirmatively having children under the age of 18 living with them. Upon the completion of the Household survey the adult will be asked if they would allow their child to participate in the Youth Engagement survey. Upon approval, the parents/adult will be provided information about the scope, purpose and intent of the survey and that the burden on the child will be less than 5 minutes. The child will be provided the same information before the start of the survey process.

Finally a survey of non-respondents (those with whom contact is established, but who do not agree to complete the entire survey) will be conducted to determine if the non-response burden will be deemed appropriate for this process. The non-response survey will not be used at this point to perform any statistical analysis to determine non-response bias.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

A dual sampling frame will be used to include both landline and cell phone Random Digit Dialing (RDD) generated numbers. The probability samples of telephone numbers will be purchased from the Marketing Systems Group (M.S. G.), one of the leading national vendors specializing in the generation of scientific samples. M.S.G. will draw simple random samples from the two sampling frames. For the purpose of this collection (the pretest), there will be no stratification of the sample for statistical purposes. The intent is to validate the survey questions and to determine the respondent burden.

Landline RDD – this sample will include randomly generated numbers within residential area code and exchanges. These exchanges are restricted to 100-series banks known to contain households. The landline RDD database contains all residential landline exchanges in the U.S. Including the District of Columbia. RDD samples can be defined by a variety of geographies, including ZIP Codes and Counties on up to National geography. Additionally, each exchange contains a demographic profile reflective of the area it serves. This auxiliary information may be used in the non-response bias analysis of the results of the final survey

Cellular RDD - this telephone sample will be randomly drawn from a database containing all cellular dedicated thousand series blocks (first seven digits) in the country. Cellular RDD samples can be defined by a variety of geographies, including area code, state, and national geography. Cellular RDD samples will contain working, non-working and unassigned numbers but it also insures each telephone number has an equal probability of selection.

The telephone numbers will be crudely pre-screened by M.S.G. to eliminate, insofar as possible, disconnects, businesses, and other known ineligibles. Any ineligibles not identified through pre-screening process will be further screened during the survey calling process.

This telephone survey will employ the following protocols:

• Use probability samples of phone numbers

• Use a combination of land line and cell phone RDD samples in a proportion reflective of the prevalence of cell phone only and cell phone mostly households at the time the samples are drawn

• Phone numbers are called up to 8 times if previous attempts did not results in a completed survey, an irate refusal, or an otherwise ineligible number.

• Soft refusals are called back in an attempt at refusal conversion.

• Calls are made during calling sessions Sundays through Thursdays from 5 to 9 PM respondent time and on Friday and Saturday afternoons.

• Potential respondents who do not agree to complete the full survey are solicited to answer a few demographic questions as well as two substantive questions. The information will be used in the non-response bias analysis.

The level of accuracy that would be achieved as a result of the protocols described should be quite sufficient for the purposes of testing the respondent burden. Our previous experience with similar surveys, shows that after 50 completed surveys the average duration of the interviews changes very little if at all.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Protocols to be employed for maximizing response rates to reduce non-response bias and obtain reliable estimates generalizable to the population of interest will be addressed in the methodology for the final survey. The methodology to be used to assess non-response bias will also be described in the supporting statements to the ICR for the final survey.

The results of this collection will only be used to refine survey content, analyze question constructs, response choice wording and respondent burden. WYSAC typically uses 12 call backs in an attempt to connect with a respondent, however, in this case up to 8 call back attempts will be made, since achieving the highest possible response rate is not of the essence for the purpose of this collection.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

This is a request to test the instruments and methods that will be used in the final version of the NPS Centennial National Household Survey. The overall intent of this request is to determine the utility of the survey instruments as well as to identify the following:

• the average respondent burden

• the need to reduce burden

• if there are ways to reduce burden

• any programming errors or technical issues with the CATI (computer assisted telephone interviewing) system

• any training issues that may be unique to this survey that will need to be addressed prior to the final survey

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

A thorough literature review was conducted prior to designing the scripts for the NPS Centennial National Household Survey and the proposed pretesting (cognitive interviews and Household Survey). We contacted 5 experts (this includes the three non-federal employees mentioned in Part A-Table 2) in the field of survey development and design to ask them to provide feedback on the clarity of questions, questionnaire design and sampling procedures. The experts provided editorial comments and suggestions that were used to improve the design and order of questions in the current versions.

The following individuals will be responsible for the design, collection and analysis of the pretest:

Bistra Anatchkova, Ph.D

Survey Research Manager

Wyoming Survey & Analysis Center (WYSAC)

University of Wyoming

307-760-3459

Nanette Nelson

Associate Research Scientist

Wyoming Survey & Analysis Center

University of Wyoming

307-399-0496

4