Supporting_Statement_PartB_VSC_7-26-08

Supporting Statement Part B for Paperwork Reduction Act Submission for National Park Service Visitor Survey Card

Reinstatement of OMB Control Number 1024-0216

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The respondent universe is all visitors, 16 years and older, to approximately 330 National Park System units during the survey periods. Visitor Survey Cards will be distributed to approximately 132,000 individuals (approximately 400 cards in each of approximately 330 units of the National Park System). Based on the experience from previous years, 39,960 responses are expected (29% response rate). This response rate is at the highest end of rates achieved for customer service evaluations in the private sector (10% to 30%).

In its first year in FY 1998, the VSC had an overall response rate of 24%. This increased to 26% in FY 2005, and to 29% in FY 2007. We attribute this upward trend to two factors: 1) greater familiarity since 1998 with the VSC among NPS staff; and 2) an increase since 2004 in the number of parks that use on-site drop boxes as a response option. As funding and logistics allow, we will increase the use of drop boxes in as many units as it is practical to do so. This effort should yield yet higher response rates.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

All parks participating in the VSC are assigned a survey month between February 1 and August 31. In all cases, this is a month during the park’s peak visitation season. Each park receives 400 surveys to distribute during its assigned month. With a response rate of 29%, an average of 116 surveys is returned per park. The associated margin of error is +/- 9% at the 95% confidence level. This level of accuracy is sufficient because—as noted in Part A—a consistently low rating for any item signals the need for additional systematic investigation into potential problems with the provision of that item. This can be done through staff debriefings, observation of program or facility use, focus groups of visitors, or more comprehensive visitor surveys.

The corresponding margin of error for the national dataset is less than +/- 1%. This is desirable for reporting System-level performance with respect to national GPRA goals.

The survey cards are distributed at staffed entrance stations and other locations with high concentrations of visitors. Following procedures outlined in the VSC Workbook (http://www.psu.uidaho.edu/vsc.workbook.htm), the survey coordinator in each park randomly schedules a minimum of eight survey days (each a four-hour time period) during the month. (Parks with low visitation may schedule more days.) Sampling days are stratified so that four are weekdays and four are weekends. In addition, equal numbers of mornings and afternoons are included in the schedule.

Initial visitor contacts are made by NPS employees in each of the NPS units. This contact takes approximately one minute. Interviewers use the following contact script as a guide:

“Hello! My name is (name), and (full park name) is conducting an important survey. We are asking visitors for their opinions about the park’s services. Your participation in the survey is voluntary and anonymous and you can complete the card in less than three minutes when it is convenient to you. The survey card has its own preaddressed, postage-paid envelope. Just drop it in a US mailbox or a convenient drop-box in the park when you are through with it. Your opinion is important! Would you be willing to complete the survey card and mail it back to us?”

Visitors who agree to participate are given a postage-paid, addressed, and scannable survey card. They are encouraged to complete the survey and mail it back at the end of their park visit or, in some cases, deposit the completed survey in a locked metal drop box on-site. The survey card takes approximately two minutes to complete.

Visitor Survey Cards returned by mail go to the University of Idaho Park Studies Unit in an attached, pre-addressed, postage-paid envelope. Drop boxes are shipped to the University of Idaho Park Studies Unit. Only then are the boxes unlocked. Returned cards are scanned and the data used to create reports for each individual park. Park cluster, regional, and national data reports are created by combining the park data for each cluster, region, and the entire National Park System. Parks are advised that they should use caution when interpreting responses to any question with a sample size of less than 30. Graphs of individual measures with samples of less than 30 are flagged with a bold caution warning. In addition, data from parks with less than 30 returned surveys are omitted from the Systemwide report. Finally, parks with discrepancies in their data-collection methods (as determined by the University of Idaho) do not receive individual reports, and their results are not used in preparing the Systemwide report.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The VSC response rate of 29% is high compared to customer service evaluations in the private sector. In FY 2005, the VSC had an overall response rate of 26%. This increased to 28% by FY 2006 and to 29% in FY 2007. As funding and logistics allow, VSC staff will continue to employ the use of drop boxes in as many units as possible. This effort should continue to yield yet higher response rates. In addition, beginning with the 2009 VSC, the following two steps are planned to further increase response rates: 1) the contact script has been updated to assure the participants that their responses are anonymous, and 2) a statement referring to personal data being disclosed to the Department of Justice has been removed from the PRA statement on the survey instrument. In the past, the statement, “Data collected through visitor surveys may be disclosed to the Department of Justice when relevant to litigation or anticipated litigation, or to appropriate Federal, State, local or foreign agencies responsible for investigating or prosecuting violation of law” has been included; however, since no personal information is being collected, the statement is not applicable and is therefore being removed. The included survey instrument has the statement red-lined, but for the final, printed survey the statement will be deleted entirely.

Non-response bias will be examined by comparing Visitor Survey Card results with the more detailed Visitor Services Project surveys to be conducted in approximately 15 selected National Park System units annually. The overall satisfaction question is included on both survey questionnaires, so that responses can be compared. In addition, based upon the expected national response rate at or around 29%, the following language will be added to the national report: “Low survey response rates increase the probability of non-response bias. Non-response bias occurs when those who choose to participate in a survey differ substantially and systematically from those who choose not to participate. If these differences are related to GPRA measures, the results may be unreliable.”

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The VSC has been used annually since FY 1998. In 2004, a pilot test was conducted (OMB #1024-0224, NPS #04-049) to examine the addition of a question concerning value for entrance fee paid. The test was conducted at Mount Rainier National Park and Klondike Gold Rush National Historical Park in Washington state. Test results found that changing the survey questions had no effect on the response rate or GPRA scores generated by the survey. The proposed VSC is the same card that was approved in 2005 and examined in the 2004 pilot test, with the exception of a line added after “commercial services in parks,” so that individuals can specify which service(s) they are rating.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Jennifer Hoger Russell, University of Idaho, is the Visitor Services Project Director, and thus, is responsible for organization of data collection, analysis, and reporting. (contact number: 208-885-4806)

Dr. Steven Hollenhorst, University of Idaho, is the director of the Park Studies Unit. He is responsible for overseeing the Park Studies Unit that is responsible for both the VSC and the Visitor Services Project (VSP). (contact number: 208-885-7911)

Dr. James Gramann, Texas A & M University and the National Park Service Social Science Program, is involved in the oversight and design of the VSC. (contact number: 979-845-4920)

Dr. Lena Le, who has an advanced degree in statistics, was consulted on the statistical aspects of the survey. Dr. Le is at the University of Idaho and can be contacted at 208-885-2585.

4