1024-0224 Programmatic Review Form - SAFRA

NPS Form 10-201 (Rev. 09/2016) OMB Control No. 1024-0224

National Park Service Expiration Date XX/XX/XXXX

PROGRAMMATIC REVIEW AND CLEARANCE PROCESS

FOR NPS-SPONSORED PUBLIC SURVEYS

The scope of the Programmatic Review and Clearance Process for NPS-Sponsored Public Surveys is limited and will only include individual surveys of park visitors, potential park visitors, and residents of communities near parks. Use of the programmatic review will be limited to non-controversial surveys of park visitors, potential park visitors, and/or residents of communities near parks that are not likely to include topics of significant interest in the review process. Additionally, this process is limited to non-controversial information collections that do not attract attention to significant, sensitive, or political issues. Examples of significant, sensitive, or political issues include: seeking opinions regarding political figures; obtaining citizen feedback related to high-visibility or high-impact issues like the reintroduction of wolves in Yellowstone National Park, the delisting of specific Endangered Species, or drilling in the Arctic National Wildlife Refuge.

SUBMISSION DATE: 5.11.2017

PROJECT TITLE: San Francisco Maritime National Historical Park Visitor Use Study

ABSTRACT: (not to exceed 150 words)

The purpose of this collection is to inform park planning and management decisions related to upgrading the San Francisco Maritime National Historical Park (SAFR) Visitor Center and Maritime Museum, including developing interpretive programs and materials based on visitors’ reported needs and desires for such facilities and services. Visitation to the Visitor Center and Maritime Museum has increased dramatically in recent years, and the results of a past visitor study, conducted in 2005, no longer adequately represent current visitor demographics, expectations, and preferences for interpretive experiences. Correspondingly, the SAFR management team does not possess adequate information on current visitor demographics, preferences, and expectations to inform the development of a plan to update the Visitor Center and Maritime Museum. A visitor survey will be conducted in summer 2017 to collect information about visitor use, experience, and attitudes toward park management, and their expectations for park services and facilities.

PRINCIPAL INVESTIGATOR CONTACT INFORMATION:

Name: Steve Lawson, Senior Director, Public Lands Planning and Management

Affiliation: Resource Systems Group, Inc. (RSG)

Address: 55 Railroad Row, White River Junction, VT 05001

Phone: 802-295-4999

Email: steve.lawson@rsginc.com

PARK OR PROGRAM LIAISON CONTACT INFORMATION:

Name: Morgan Smith, Chief of Interpretation

Affiliation: San Francisco Maritime National Historical Park

Address: 2 Marina Boulevard, Building E, 2nd Floor, San Francisco, CA 94123

Phone: 415-859-6797

Email: morgan_smith@nps.gov

PROJECT INFORMATION:

Where will the collection take place? San Francisco Maritime National Historical Park (SAFR)

Sampling Period Start Date: July 28, 2017 Sampling Period End Date: August 7, 2017

Type of Information Collection Instrument: (Check ALL that Apply)

 Mail-Back Questionnaire  Face-to-Face Interview  Focus Groups

 On-Site Questionnaire  Telephone Survey

Other (List)

Will an electronic device be used to collect information?  No  Yes – Type of Device:      

SURVEY JUSTIFICATION:

Social science research in support of park planning and management is mandated in the NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS pursues a policy that facilitates social science studies in support of the NPS mission to protect resources and enhance the enjoyment of present and future generations (National Park Service Act of 1916, 38 Stat 535, 16 USC 1, et seq.). NPS policy mandates that social science research will be used to provide an understanding of park visitors, the non-visiting public, gateway communities and regions, and human interactions with park resources. Such studies are needed to provide a scientific basis for park planning and development.

The San Francisco Maritime National Historical Park (SAFR) management team is undergoing a planning process to update the park’s Visitor Center and Maritime Museum. The Visitor Center and Maritime Museum at SAFR provide interactive interpretive experiences, where visitors can learn about the maritime history of the San Francisco Bay Area. Demand for these interpretive services at SAFR has increased considerably in recent years. In 2016, there were a total of 521,370 visits to the Visitor Center and Maritime Museum, a 19% increase from 2015 (439,058 visits) and 69% increase from 2013 (308,855 visits).

To develop an effective plan for updating the Visitor Center and Maritime Museum in the face of increasing demand, SAFR park managers need to understand a) who is visiting the park, and b) what visitors expect to experience and learn from the park’s interpretive opportunities. The last visitor study at SAFR was conducted in 2005, a year in which visitation to the Visitor Center and Maritime Museum was 64% lower than in 2016. Correspondingly, the SAFR management team does not possess adequate information on current visitor demographics, preferences, and expectations to inform the development of a plan to update the visitor center and museum. As such, the SAFR management team requested this visitor study.

Visitor groups will be recruited on-site to participate in a mail-back survey. The survey instrument is designed to collect information about trip motivations and planning, preferences for management, visitor use of services and facilities, and visitor demographics. Results of the information collected will provide park managers with data that will be used during the planning process.

SURVEY METHODOLOGY:

1. Respondent Universe:

Based upon NPS Visitor Use Statistics, there were more than 1.1 million recreational visits to SAFR in July (n=573,671) and August (n=588,161) 2016. The respondent universe for this collection will be all recreational visitors, age 18 or older, visiting designated intercept locations within San Francisco Maritime NHP during the sampling period. To provide context for the expected visitor flows at each intercept location for this study, the monthly visitation rates for 2016 were as follows:

Table 1. 2016 Survey Interception Location Recreation Visits, By Month (Source: NPS Visitor Use Statistics)

	2016
INTERCEPT LOCATION	July	August
Visitor Center / Maritime Museum	39,670 / 14,457	36,110 / 33,654
Hyde Street Pier	118,657	117,673
TOTAL	172,784	187,437

2. Sampling Plan / Procedures:

The sampling plan described below will be used to sample visitors at the Visitor Center/Maritime Museum and the Hyde Street Pier. These sites were selected because results from the 2005 SAFR Visitor Services Project (VSP) survey indicate that these locations should have the highest visitation among park locations during the study period, maximizing sampling efficiency (Hyde Street Pier 71%, Visitor Center/Maritime Museum 43%/46%), indicating. Sampling will take place for 9 days, over an 8-hour period (approximately 9:00 AM to 5:00 PM) at each location, during operating hours for the location (Table 2). There will be one survey administrator at each location.

Table 2. Sampling Days Per Site: On-site Intercepts

	Sampling Days Per Site
	JULY		AUGUST		TOTAL
INTERCEPT LOCATION	WEEK DAYS	WEEKEND DAYS	WEEK DAYS	WEEKEND DAYS
Visitor Center/Maritime Museum	1	1	5	2	9
Hyde Street Pier	1	1	5	2	9
TOTAL	2	2	10	4	18*
*Intercept locations will be sampled concurrently, resulting in 9 total sample days for the study period.

A self-administered mail-back survey will be distributed on-site. The survey packets will include a booklet-sized questionnaire and a pre-addressed envelope affixed with a U.S. first class postage stamp.

A “timed-interval” approach will be used to randomly select visitor groups for participation. For a “timed-interval” approach, the survey administrator will attempt to sample one visitor group every N minutes of the day, where N is the time interval. This approach allows for the distribution of questionnaires to be evenly spread out through the sampling period (across days) and throughout each sampling day (across hours). For this study, the timed interval for contacting visitor groups will be roughly 8-minutes at both intercept locations. The interval was calculated using the following formula: minutes per sample day (480) / contacts per sample day (n=55).

Table 3. Estimated Number of Visitor Contacts during Sampling Period

	Estimated Number of Visitor Contacts Each Day
	Sampling Day
INTERCEPT LOCATION	1	2	3	4	5	6	7	8	9	TOTAL
Visitor Center/Maritime Museum	56	55	56	55	56	55	56	55	56	500
Hyde Street Pier	55	56	55	57	55	56	55	56	55	500
Total	111	111	111	112	111	111	111	111	111	1000

The following protocol will be used to implement the “timed-interval” approach to participant selection and questionnaire distribution:

1. The survey administrator will contact the first visitor group to enter the survey area at the start of the sampling period and ask them to participate in the survey.

2. If the group agrees to participate, they the administrator will ask the non-response bias questions, provide a survey packet and give the instructions for completing and mailing back the survey instrument. Once this process ends, the survey administrator will wait until the start of the next time interval to contact the next visitor group.

3. If the visitor group refuses to participate, the group will be asked to complete the non-response bias questions and thanked for their time.

4. After completing a contact with a refusal group, the survey administrator will contact the very next visitor group to enter the survey area and ask them to participate in the survey. The survey administrator will continue this process until a visitor group agrees to participate. At the end of the process (as described in #2 above), the survey administrator will wait until the start of the next time interval to contact the next visitor group.

The survey administrator will repeat this sequence of steps throughout the sampling day.

All visitors contacted will be asked to answer the non-response bias question.

3. Instrument Administration:

At the start of the sampling period, the first visitor group to enter the sampling area will be greeted, introduced to the purpose of the study, and asked if the visitor group is willing to participate. If a visitor group agrees to participate, they will be asked which member of the group (at least 18 years old) has the next birthday; this will be done to randomize selection of the individual within the group to complete the questionnaire. The initial contact is not expected to take more than one minute.

The individual with the next birthday will be asked to take the mail-back questionnaire packet, complete the questionnaire at the end of their trip, and return the completed questionnaire via any U.S. postal service mailbox. This individual will also be asked to provide their name and mailing address to be used to send a “thank you” postcard and replacement questionnaire within 11 working days (and then 21 days) after the on-site contact. Individuals will be assured that their addresses will only be used for purposes of the collection and will be destroyed when the study is complete. After collecting the address information, the survey administrator will thank the individual for their time. The following script will be used during the initial contact:

“Hello, my name is _________. I am conducting a survey for San Francisco Maritime National Historical Park to help managers and planners understand your opinions about the interpretive programs and services offered here. Your participation is voluntary and all responses will be kept anonymous. Would you be willing to take a questionnaire and mail it back – at the end of your visit - using the postage-paid envelope?”

 If YES – then ask (if not obvious), “Is there at least one adult group member at least 18 years of age or older?”

 If NO (no adult group members) – then say, “Ok. Thank you for your time and have a great day.”

 If YES – then ask, “Has any member of your group already participated in this survey?”

 If YES (previously agreed to participate) then, “Thank you for agreeing to participate in this study, we hope that you will return the questionnaire soon if you have not already. Have a great day.”

 If NO (have not previously participated) then, “Thank you for agreeing to participate. Most of our questions are the in mail-back survey, but I do have a few questions I need to ask you now. Of the adults in your group, whose birthday is next?”

[Survey administrator identifies the visitor and directs next portion of contact to this visitor.]

“Great, I’d like you to complete this questionnaire at the end of your visitor to San Francisco Maritime National Historical Park [survey administrator hands the individual the mail-back packet]. The questionnaire is inside this stamped, addressed envelope.”

“Additionally, we’ll need your name and mailing address so that we can send you a thank-you postcard and a replacement questionnaire in case you lose or do not return this one. This information will only be used to send these two mailings, and will be destroyed after the study is completed.”

[The survey administrator will ask the non-response bias questions. The responses will be recorded and afterwards the participant will be thanked for their time.]

 If NO – (soft refusal) then, “That’s fine. We won’t bother you with the mail-back survey. But would you be willing to take just one more minute and answer a couple of questions for me now, to help us be sure our sample is reliable?”

[The survey administrator will ask the non-response bias questions (see Section E: Strategies for Dealing with Potential Non-Response Bias). The responses will be recorded by the survey administrator in spaces provided on the survey log and non-response bias form. After completing the non-response bias question interview, the survey administrator will thank the participant for their time.]

 If NO – (hard refusal) then, “Thank you for your time. Have a great day.”

All survey administrators will be trained on every aspect of on-site surveying, including: survey administration, avoiding sampling bias, and handling all types of interview situations, especially safety of visitors and the administrator. Quality control will be ensured by monitoring interviewers in the field, and by checking their paperwork at the end of each survey day. In the event that on-site acceptance rates for participation in the visitor study are lower than the expected 80% acceptance rate during the sampling period, an experienced on-site field supervisor will provide additional training and support for the survey administration process, including:

• Working with survey administrators to ensure that survey administration protocols are being followed properly,

• Recommending the adjustment of the specific bounds of the survey administration area to better capture flows of visitors,

• Recommending the reduction of the time interval.

(D) Expected Response Rate / Confidence Level:

Approximately 1,000 visitors will be contacted for the study (Visitor Center/Maritime Museum n=500, and Hyde Street Pier n=500). The VSP 2005 SAFR mail-back survey had an on-site acceptance rate of 68% and a mail back response rate of 58%. In 2016, Resource System Group, Inc. (RSG) conducted similar studies across 15 units of the National Park Service, using similar prescribed sampling methods for on-site recruitment for mail-back surveys. These collections had an average onsite acceptance rate of 80% and average response rate of 43%. Based on these past studies, we propose an acceptance rate of at least 80% and a response rate of at least 50% for this collection.

Table 4. Expected Acceptance Rate

Intercept Location	Initial Contacts	Acceptance 80%	Non-respondents 20%	Non-respondents
				Non-response survey (80%)	Hard Refusals (20%)
Visitor Center / Maritime Museum	500	400	100	80	20
Hyde Street Pier	500	400	100	80	20
Total	1,000	800	200	160	40

The target number of 800 questionnaires (n= 400 per intercept location) will be distributed to ensure that an adequate sample will be generated so that there will be 95% confidence that the survey findings will be accurate to approximately 5 percentage points for the overall sample (based on the formula where confidence level = 1 ÷ √N, where N = target distribution number). Given an assumed onsite acceptance rate of 80% (yielding a target number of 800 distributed questionnaires) and an assumed response rate of 50% for returned, completed questionnaires, the total number of visitor contacts will be 1000 visitors to yield at least 400 completed, returned questionnaires for analysis.

5. Strategies for dealing with potential non-response bias:

The following five questions will be used in a non-response bias analysis:

1. How many adults, 18 and older, and children are in your personal group?

2. What type of group are you with? Friends, Family, Friends and Family, Some other type of group?

3. How long is this trip to San Francisco Maritime National Historical Park in total, from beginning to end?

4. What is your state of residence (if you live in the US) or country of residence (if you do not live in the US)?

5. Did you and your personal group attend any informational or interpretive programs at San Francisco Maritime National Historical Park today? (Yes or No)

Responses to non-response bias questions will be recorded for every visitor contact, expect “hard refusals” (refuse to participate in the study, and refuse to answer the non-response bias questions). For all “hard refusals” observational data (group size, gender, and time of day) will be recorded. The number of refusals will be recorded and used to calculate the overall response rate for the collection at the park. Results of the non-response bias check will be reported and any implications for applicability of survey results to generalizations about the study population will be discussed.

5. Description of any pre-testing and peer review of the methods and/or instrument:

All of the questions in this survey were taken from the currently approved NPS Pool of Known Questions (1024-0224). These questions have been used in other studies completed by RSG, where the clarity of survey questions and understanding from respondents has been demonstrated. Variations of these questions have been reviewed by NPS managers, and PhD-level and MS-level NPS survey research consultants at RSG. Pre-testing was conducted with RSG administrative staff (e.g., H.R. representatives, accountants), who do not possess experience with or knowledge of survey research or park management. The respondents commented that overall, the wording of the questions was clear, and review of their responses on their questionnaires reflected their understanding. Questionnaire completion times were observed and are incorporated into the burden estimate below.

BURDEN ESTIMATES:

The combined total burden for this collection is estimated to be 176 hours (Table 6). We have estimated respondent burden as follows:

• On-site contact – (Instructions/Address Card/Non-Response Bias Check) : 40 hours

Two minute will be used to provide instructions to all visitors that agree to participate (n=800)

One minute will used to conduct the non-response bias check with respondents while they fill out their contact information on an address card (n=800).

• On-site Non-response bias check (non-respondents): 3 hours

One minute will used to conduct the non-response bias check with all non-respondents that do not provide hard refusals for all further contact (n=160)

• Visitor mail-back survey: 133 hours

It will take an average of 20 minutes to complete and return the mail-back questionnaire (n=400)

It is expected that the remaining 40 visitors (approximately 4% of all visitors contacted) will completely refuse to participate and for those individuals there will be no calculation of additional burden outside of the 1-minute for initial contact; however, the surveyor will record any reason for refusal and the observational data.

Table 6. Burden Estimates

Respondents	Responses	Completion Time * (minutes)	Burden Hours
On-site Contacts Instructions/Address Card/Non-Response Check (Respondents)	800	3	40
Non-respondents Non-response check	160	1	3
Returned/Completed Surveys	400	20	133
Total burden requested under this ICR:	1,360	Contact time added to completion time	176

REPORTING PLAN:

The study results will be presented in an internal report for park managers. Results of statistical analyses and summary statistics will be compiled (e.g. response frequencies, measures of central tendency, correlations, Chi-square, analysis of variance, factor analysis, and scale reliability analysis, as appropriate). Final reporting will be delivered to park managers in hard copy and electronic formats, and posted as a Natural Resource Data Series in the NPS Data Store (https://irma.nps.gov/DataStore/Reference/Profile/) as required by the NPS Programmatic Review Process.

NOTICES

Privacy Act Statement

General: This information is provided pursuant to Public Law 93-579 (Privacy Act of 1974), December 21, 1984, for individuals completing this form.

Authority: National Park Service Research mandate (54 USC 100702)

Purpose and Uses: This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service.

Effects of Nondisclosure: Providing information is mandatory to submit Information Collection Requests to Programmatic Review Process.

Paperwork Reduction Act Statement

We are collecting this information subject to the Paperwork Reduction Act (44 U.S.C. 3501) and is authorized by the National Park Service Research mandate (54 USC 100702). This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service. All parts of the form must be completed in order for your request to be considered. We may not conduct or sponsor and you are not required to respond to, this or any other Federal agency-sponsored information collection unless it displays a currently valid OMB control number. OMB has reviewed and approved The National Park Service Programmatic Review Process and assigned OMB Control Number 1024-0224.

Estimated Burden Statement

Public Reporting burden for this form is estimated to average 60 minutes per collection, including the time it takes for reviewing instructions, gathering information and completing and reviewing the form. This time does not include the editorial time required to finalize the submission. Comments regarding this burden estimate or any aspect of this form should be sent to the Information Collection Clearance Coordinator, National Park Service, 1201 Oakridge Dr., Fort Collins, CO 80525.

RECORDS RETENTION - PERMANENT. Transfer all permanent records to NARA 15 years after closure. (NPS Records Schedule, Resource Page 1 of 10

Management And Lands (Item 1.A.2) (N1-79-08-1)).