1024-0224 NPS Programmatic Review CASA & FOMA

OMB Control Number: 1024-0224 Current Expiration Date: 8-31-XXXX

National Park Service

U.S. Department of the Interior

Programmatic Review and Clearance Process

for NPS-Sponsored Public Surveys

The scope of the Programmatic Review and Clearance Process for NPS-Sponsored Public Surveys is limited and will only include individual surveys of park visitors, potential park visitors, and residents of communities near parks. Use of the programmatic review will be limited to non-controversial surveys of park visitors, potential park visitors, and/or residents of communities near parks that are not likely to include topics of significant interest in the review process. Additionally, this process is limited to non-controversial information collections that do not attract attention to significant, sensitive, or political issues. Examples of significant, sensitive, or political issues include: seeking opinions regarding political figures; obtaining citizen feedback related to high-visibility or high-impact issues like the reintroduction of wolves in Yellowstone National Park, the delisting of specific Endangered Species, or drilling in the Arctic National Wildlife Refuge.

				Submission Date:					12/21/2016
Project Title: Castillo de San Marcos NM and Fort Matanzas NM Visitor Experience Studies
Abstract (not to exceed 150 words)
Two onsite questionnaires will be used to collect information about visitor use at Castillo de San Marcos National Monument (CASA) and Fort Matanzas National Monument (FOMA). The focus will be user characteristics, visitor experience, and perceptions of crowding impacts on park interpretive facilities, programs, and services. The surveys will be conducted in spring 2017 and will be used to inform park planning and management, and improve interpretive operations to meet visitors’ needs and expectations.
Principal Investigator Contact Information
Name: Steve Lawson
Title: Director, Public Lands Planning and Management
Affiliation: Resource Systems Group, Inc. (RSG)
Address: 55 Railroad Row White River Junction, VT 05001
Phone: 802-295-4999
Email: steve.lawson@rsginc.com
Park or Program Liaison Contact Information
Name: Steve Roberts
Title: Chief of Interpretation and Education
Park: Castillo de San Marcos & Fort Matanzas National Monuments
Address: 1 South Castillo Drive St. Augustine, FL 32084
Phone: 904-829-6506
Email: steven_j_roberts@nps.gov
	Project Information
	Where will the collection take place? (Name of NPS Site)				Castillo de San Marcos (CASA) and Fort Matanzas (FOMA)
	Sampling Period	Start Date: April 1,2017			End Date: April 30, 2017
	Type of Information Collection Instrument (Check ALL that Apply)
	 Mail-Back Questionnaire		 Face-to-Face Interview					 Focus Groups
	 On-Site Questionnaire		 Telephone Survey
	 Other (list)
	Will an electronic device be used to collect information?  No  Yes - type of device
	Survey Justification:
	Social science research in support of park planning and management is mandated in the NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS pursues a policy that facilitates social science studies in support of the NPS mission to protect resources and enhance the enjoyment of present and future generations (National Park Service Act of 1916, 38 Stat 535, 16 USC 1, et seq.). NPS policy mandates that social science research will be used to provide an understanding of park visitors, the non-visiting public, gateway communities and regions, and human interactions with park resources. Such studies are needed to provide a scientific basis for park planning and development. The park management teams at Castillo de San Marcos National Monument (CASA) and Fort Matanzas National Monument (FOMA) are interested in learning how interpretive programs are affected during periods of high visitation. According to CASA staff, during periods of high visitation, combined use by on-site interpretive programs, organized groups, and general visitors may result in conditions that impact how visitors use areas in the parks to learn and for other experiences. With an annual increase in visitation, CASA staff is interested in understanding the impacts of high visitation on the current interpretive services/programs to determine if they are best structured to meet visitor expectations commensurate with different levels of use. FOMA staff have additional concerns about visitors being unable to take the ferry trip to the Fort due to ticket availability, the quality of interpretive/educational programs at the park, and the quality of the visitor experience overall. Results of the information collected in this study will assist park managers in informing, evaluating, and/or improving the following: Interpretive, wayfinding, and informational signage, programs, and materials based on visitors’ reported needs and desires for such facilities and services, Management of historic resources and park experiences, especially experiential quality, Park planning, specifically efforts to enhance the park’s relevance and service to its visitors, by collecting information about visitor characteristics, activities, needs, and evaluations of facilities and services.

Survey Methodology

1. Respondent Universe:

The respondent universe for both surveys (CASA and FOMA) will be all recreational visitors, age 18 or older, contacted at designated intercept locations in the parks during the sampling period.

2. Sampling Plan/Procedures:

The sampling plan described below will be used to survey visitors at both CASA and FOMA.

This will be an on-site survey that is expected to be completed and returned before leaving the sampling area. Visitor groups will be asked to participate in the survey end of their visit. A “first-after-last completed” sampling approach will be used to generate a representative sample. For example, at the beginning of each sampling day at each park, the first visitor groups to exit the CASA Castillo or the FOMA Visitor Center will be asked if their visit to the park has concluded for the day. If yes, they will be asked to participate in the study. If the visitor agrees to participate the survey administrator will go through the process of administering the survey. Upon completion, the survey administrator will approach the very next visitor group exiting the CASA Castillo or the FOMA Visitor Center will be asked if their visit to the park has concluded for the day. If yes, they will be asked to participate. This process will continue until the sampling day concludes. The survey administrator will be trained on every aspect of on-site surveying, including: administering the questionnaires, avoiding sampling bias, and handling all types of interviewing situations, especially safety of visitors and the administrator. Quality control will be ensured by monitoring the survey administrator in the field, and by checking their paperwork at the end of each survey day.

Sampling will occur on weekends and weekdays. On each sampling day, sampling will be conducted 7 hours per day during the hours of operation for both sites (CASA 8:45 am – 5:15 pm and FOMA 9:00 am – 5:30 pm), Sampling locations will be staffed with the appropriate number of surveyors to ensure daily targets are met on weekends and weekdays.

Location	Total number of visitors contacted per site during the sampling period
	F	S	Su	M	T	W	Th	F	S	Su	TOTAL
CASA – Entrance	50	50	50	50	50	50	50	50	50	50	500
FOMA – Visitor center	50	50	50	50	50	50	50	50	50	50	500
TOTAL	100	100	100	100	100	100	100	100	100	100	1000

3. Instrument Administration:

The initial contact with all visitor groups at both locations will be used to explain the study and determine any interest in participating. When a group is encountered, the survey administrator will approach the closest person in the group and ask if they are at least 18 years old (if it is not apparently obvious) and if their visit to the park has concluded. Once established, the adult with the closest birthday in the group will be asked to participate. The initial contact is not expected to take more than one minute

The identified respondent will be given a questionnaire, provided instructions and will be informed that the questionnaire is designed to take no more than 12 minutes to complete and that no personally identifiable information will be collected. The respondents will be asked to complete and return the questionnaire to the surveyor before leaving the sampling area.

For the CASA questionnaire (only) the respondents will be asked to review a series of photographs used to depict various levels of crowding in an exhibit room in the fort. A total of six photos will be presented one at a time in random order. After each photo, the respondent will be asked to indicate (yes or no) if they would feel crowded in the room with the number of people shown in the photo. There are four additional questions in this section that the photos will be used to provide responses. At the end of this section the surveyor will leave the respondent to complete the survey. Upon completion the respondent will return the survey to the surveyor before leaving the sampling area.

Non-response survey

All visitors who refuse to participate after the initial contact will be asked if they would be willing to answer the four questions that will serve as the non-response survey for this collection. If they refuse they will be thanked for their consideration and observational data (group size, gender, and time of day) will be recorded. If the visitor agrees, the surveyor will record their responses and observational data (listed above). Afterwards they will be thanked for their time. The number of refusals will be recorded and used to calculate the overall response rate for the collection at the park.

(d) Expected Response Rate/Confidence Levels:

Approximately 500 visitors will be contacted at each location CASA (n= 500) and FOMA (n=500). Based upon similar studies conducted by Resource Systems Group, Inc. (RSG) for the National Park Service, using the same prescribed sampling protocols, we propose a response rate of at least 70% for this collection. We anticipate we will receive 700 completed surveys from all visitors contacted at both locations (CASA n= 350) and FOMA n=350).

The number of refusals will be recorded and reported, and will be used in calculating the response rate. Based on the expected number of responses at each park, there will be 95% confidence that the survey findings will be accurate to approximately 5 percentage points for each park.

An experienced on-site field supervisor will track survey response rates on a daily basis and will provide additional training and support at locations with response rates lower than anticipated.

	Response rates based upon total estimated number of all visitor contacted
Intercept Location	Initial Contacts	Acceptance 70%	Non-respondents (Soft refusals) 30%	Non-response survey 80%	Hard Refusals 20%
CASA	500	350	150	120	30
FOMA	500	350	150	120	30
Total	1000	700	300	240	60
				% of soft refusals

(e) Strategies for dealing with potential non-response bias:

The following four questions will be used in a non-response bias analysis. These questions will only be asked to visitors who refused to participate when initially contacted and did not give a “hard refusal” (refuse to participate in the study, and refuse to answer the non-response bias questions), and to respondents in groups who agreed to participate. Responses to these questions will be recorded by the survey administrator on the survey contact log form

1. How much of a problem did you have viewing/reading exhibits due to crowds today? Would you say it was a  Big Problem  Small Problem or  Not a Problem

2. Including yourself, how many people are in your personal group on this visit to Castillo de San Marcos NM/Fort Matanzas NM?

3. On this visit to Castillo de San Marcos NM/Fort Matanzas NM, what type of group are you with?

4. What is your state or country of residence (if not US)?

Responses to these questions will be recorded and the results will be reported. Any implications for applicability of survey results to generalizations about the study population will be discussed.

(f) Description of any pre-testing and peer review of the methods and/or instrument

All of the survey questions are taken from the currently approved NPS Pool of Known Questions (1024-0224). These questions have been used in other studies completed by RSG, where question clarity and understanding from respondents has been demonstrated. Variations of these questions have been reviewed by NPS managers, PhD-level and MS-level NPS survey research consultants at RSG.

Pretesting occurred with individuals in RSG’s Portland office. The respondents commented that the wording of the questions was clear. A review of their responses reflected their understanding. Questionnaire completion times were observed and are incorporated into the burden estimate below.

Burden Estimates

The combined total burden for this collection is estimated to be 176 hours. For both survey efforts, we have estimated respondent burden as followed:

• Initial contact time: 17 hours.

One minute will be used to explain the survey to all visitors contacted and to request participation (n=1,000).

• Instructions: 23 hours.

Two minutes will be used to provide instructions to all visitors that agree to participate (n=700)

• non-response bias check: 8 hours

Two minutes will used to conduct the non-response bias check with all non-respondents (n=240).

• CASA Survey: 70 hours.

12 minutes to complete and return the on-site questionnaire (n= 350)

• FOMA Survey: 58 Hours

10 minutes to complete and return the on-site questionnaire (n=350)

It is expected that the remaining 60 visitors (6% of all visitors contacted) will completely refuse to participate and for those individuals there will be no calculation of burden, however, the surveyor will record any reason for refusal and the observational data listed above. Training and supervision of the

Combined Total Burden Estimates for CASA and FOMA Surveys

Estimated Total Number

Estimation of Time (minutes)

Estimation of Burden (hours)

Initial Contacts

1000

Initial Contact

1

Initial Contact

17

Instructions

Nonresponse Survey

700

240

Instructions

Nonresponse Survey

2

2

Nonresponse Survey

23

8

Completed Responses

CASA

FOMA

350

350

To complete response CASA

FOMA

12

10

To complete response CASA

FOMA

70

58

Total Hours

176

Reporting Plan

The study results will be presented in an internal report for park managers. Results of statistical analyses and summary statistics will be compiled (e.g. response frequencies, measures of central tendency, correlations, Chi-square, analysis of variance, factor analysis, and scale reliability analysis, as appropriate). Final reporting will be delivered to park managers in hard copy and electronic formats and posted as a Natural Resource Data Series in the NPS Data Store (https://irma.nps.gov/DataStore/Reference/Profile/) as required by the NPS Programmatic Review Process.