Recreation Visitor Use Surveys
Bureau of Reclamation
The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When Item 18 on the OMB Form 83-I is checked "Yes", the following documentation should be included in the Supporting Statement to the extent that it applies to the methods proposed:
Generally, respondents to Bureau of Reclamation administered surveys will be customers directly engaged in recreational activities on Reclamation lands at the location where the survey is being administered. These respondents are considered to be the target population for information collection purposes. Visitors typically come from the local area where the surveys are being administered; however, many visitors come from large metropolitan areas within a 100 mile radius and from adjoining States. Visitors will be sampled and surveyed using probability sampling. Several of the surveys target specific users engaged in specific activities such as river rafting on a specific river, visitors camping at a developed campground, or visitors boating at a specific reservoir.
A response rate of 80 percent is desired. It is anticipated that response rates will be higher as most respondents, particularly those participating in intercept interviews, are highly involved and interested in the survey subject matter. In addition, surveyors will, for the most part, collect completed survey questionnaires on site which will increase the response rate. For 11 of the 12 surveys, including the customized surveys, it is anticipated that 348 survey questionnaires will have to be handed out per area to achieve the desired return of 278 surveys (80 percent response rate). Anticipating an 80 percent response rate for the remaining Fee Survey, 500 surveys will need to be handed out to achieve the desired return of 400 surveys. For a typical year, approximately 9,200 surveys will be handed out to achieve the desired return of 7,350 surveys annually and an 80 percent response rate. It is Reclamation’s intent to continue the information collection effort until the target number of responses are collected.
Reclamation feels that this collection strategy is sufficient to properly evaluate and analyze the results to achieve program goals and objectives.
2. Describe the procedures for Collection of information including:
2a. Statistical methodology for stratification and sample selection
Surveys will be distributed to visitors to specific recreation areas at Bureau of Reclamation Federal water projects. The sampling frame is a defined geographic area consisting of a reservoir and its immediate surrounding land base or a defined river corridor that is an integral part of a Reclamation project area. These geographic areas/sampling frames are relatively stagnate and will likely not change during the 3-year OMB approval period. However, Reclamation will modify and update the sampling frame if necessary to respond to each Reclamation offices specific needs related to each survey collection. Any updates or changes will immediately be made available to all surveyors participating in the survey collection.
Pre-testing of all the questions in the River Instream Flow Survey, Reservoir Preferred Water Level Survey, and the Water Level Impact on Recreation Boating Use Survey will occur prior to administering the survey instrument to the general public. Refer to Section 4 of Part B of the justification package for further guidance on pre-testing of these three surveys and customized surveys.
The approved surveys will be administered on site through the use of personal interviewing techniques. Completion of the surveys by respondents is voluntary. Under the supervision of a Project Leader, employees, contractors, or volunteers will ask the questions verbatim from the pre-approved questionnaires and will record responses or hand out questionnaires and wait for individual respondents to return them immediately upon completion. For those individuals staying in the area, arrangements may be made for surveyors, camp hosts, or volunteers to pick up completed surveys at a later time or have the individuals deposit them in a predetermined location at a campground, entrance station, visitor center, or survey drop box.
The sampling design that will be used is called Stratified Cluster Sampling (Sample Survey). The results of a sample survey should be more representative of the target population and provide less biased estimates of the populations than other types of surveys such as the census survey. This technique is used where you do not know in advance who is in the population (e.g. you do not have a list of names from a phone book from which to randomly select from). Stratified Cluster Sampling is especially useful when sampling recreation areas where there is no way to determine who will be visiting. This allows sampling at different field locations based upon time blocks that include days of the week and hours of the day that is further based upon an estimate of the percent of visitation at each of the locations and times. The time blocks are then randomly selected and assigned for survey sampling. Visitors are then surveyed in these locations during these time blocks. Typically, sampling times will include on 8-10 day period with a random start in June and three four-day periods randomly selected during July and August. Note: If the recreation area has a longer recreation season than Memorial Day to Labor Day, then the time period for administering the survey may be established to correlate to the proper length of the recreation season. Appropriate time blocks will be established for those areas. Whatever the length of recreation season, sample periods will be selected to ensure a balance of weekend and weekday periods and an appropriate distribution across the entire summer season.
In order to get input from a representative sample of all users to individual Reclamation water projects, every 5th group consisting of one or more individuals will be surveyed. For groups of more than one individual, the surveyor will solicit input from the person that will have the next birthday of those that are in the group. In all cases, those individuals surveyed will be 16 years of age or older. For the purposes of this survey collection process, Reclamation has not attempted to differentiate between a personal group and an organized user group. Valuable information can be obtained from both types of groups; therefore, both types of groups will be surveyed.
On selected sampling days, interviewers will introduce themselves as researchers working for the Bureau of Reclamation. They will alert the potential respondents that participation is voluntary and that anonymity will be protected. The surveyor will mention the length of time that it will take to fill out the survey and briefly describe what the survey is about. Visitor contacts will occur based on a pre-designed systematic schedule established by the Project Leader. Sampling will begin with the first available group during the sample time. Once a surveyor has finished with one group, he/she will move to the next eligible group that arrives at the survey site. If a group refuses to participate, the surveyor will then contact the next eligible group, adhering to the established sampling schedule of intercepting every 5th group.
There may be visitors that refuse to participate in the survey collection (e.g. a day use visitor that arrives late in the day and needs to participate in his or her activity and return home shortly after arrival). All surveyors will keep a log and record the disposition of every contact, including refusals and the observable characteristics of both the non-participants and the participants for use in a non-response bias analysis. Some of the items or characteristics that can be entered in the log for both respondents and non-respondents are the date, time and place where the survey was handed out; group size; activity that the respondent was participating in; approximate age of respondent; and whether the respondent was a male or female or an adult or child.
Private contractors, managing partners, or universities will also use the survey instruments to collect data for Reclamation. This will typically be completed under the terms of an acquisition document and the collected information will be used to supplement studies and analyses of Reclamation sites or resources. Contractual requirements will limit the use of the data to only support Reclamation programs and research efforts.
All individuals administering the approved survey questionnaires will receive orientation and training in the use of survey techniques by the Project Leader. In this manner, they will be sensitized to the need to assist respondents, ensure accuracy of the information being solicited, the need to reduce the burden on respondents, and the value of following the sampling plan.
Once the surveys are collected, the data are entered into a Statistical Package for Social Science (SPSS) program. Statistical programs are run on the data using SPSS. In all cases, non-parametric statistics are run. These include frequencies, means, and standard deviations. Depending on the research questions to be answered, parametric statistics are run. There are many possibilities, but typically a test of significance or analysis of variance such as a t test or chi-square would be run. The statistics are then interpreted and the findings documented in a findings report and presented to mangers for decision-making purposes.
Reclamation will not exceed the burden hours anticipated and the aggregate burden hours outlined in the Burden Hour Tables in Section A, 12b of this request and will not administer any survey instrument until OMB approval is received. In addition, except for minor adjustments in the format for clarity purposes, Reclamation will not make any changes to the survey once they have been approved. However, Reclamation may delete a specific question or questions after approval by OMB, if necessary. The deletion of certain items in specific questions is further explained in the Question Justification section (i.e., certain items that are not applicable to an area being surveys may be deleted, as appropriate).
More detailed instructions for administering surveys and a summary of the information collection process can be found in the Code of Conduct and Instruction for Surveyors document contained in these OMB package.
2b. Estimation procedure
In most cases there will be a need to employ a statistical methodology to identify the actual sample size once a survey is determined to be necessary. For the purpose of this information collection, a statistical methodology has been applied to determine the sample size based on a number of parameters. It is assumed that there are approximately 700,000 visitors annually at the site where the Fee Survey will be administered and 30,000 average visitors at the remaining areas that will be surveyed (source of information is Reclamation’s Visitor Use Data Report which is a database that, among other things, tracks annual visitation to Reclamation recreation areas.). The average sample size has been determined by statistical formula to be 400 for the Fee Survey and 278 for the remaining surveys. These sample sizes will achieve the desired confidence level of 95 and 90 percent respectively with a sampling error of ±5 percentage points.
Sample Size Methodology Used
(1) Selected a desired confidence level (i.e., probability that the results will be for a given level of precision):
Confidence Level Standard Deviations from the Mean
80% 1.28
90% 1.64
95% 1.96
(2) Determined an acceptable level of accuracy for results in terms of standard errors (e.g., Reclamation wanted to be accurate within 5% at the 95% confidence level):
Standard Error =
Standard Deviation with Confidence Level of Interest
= 0.025 (or 2.5%)
=
1.96
(3) Approximated the variance in the population: Because the variance was unknown, Reclamation assumed the highest possible proportional variance that occurs when 50% of the sample possesses the characteristic of interest and 50% do not.
(4) Calculated Sample size with a 95% confidence level: n = sample size and p = proportion of population having characteristic of interest (i.e., 50%/50% from #3 above). When surveying people, the approximate size of the population must be determined that in most instances is over 1,000 people. The sample size rarely exceeds 400 respondents; therefore, the following formula was used because the population (i.e., number of visitors to the area) being sampled is equal to or greater than 1,000.
(a) n =
(Standard error)2
n =
=
= 400
(0.025)2 .000625
The example above only includes the calculations for a 95% level with a ±5% sampling error. The calculations for a 90% confidence level were determined the same way.
The purpose of this proposal is to submit a request to the Office of Management and Budget (OMB) to authorize Reclamation to sponsor surveys for the collection of recreation data from users of Reclamation lands, rivers, and reservoirs, ensuring compliance with the requirements of the Paperwork Reduction Act. These data collections would be pursued on an as needed basis over a 3-year period to aid in proposed project evaluations and would be paid for using project funds. To meet its needs for the collection of visitor use data, Reclamation is requesting OMB to authorize a two-part request. The first part of the request provides Reclamation with a set of 11 pre-approved recreation visitor use surveys to be administered in their entirety and as approved by OMB. Please note that the majority of the questions contained in 9 of the 11 surveys being submitted for OMB approval under this request are questions that have been approved previously by OMB as part of an information collection request by the Corps of Engineers (COE) (OMB Control No. 0710-0001). However, two of the nine COE survey titles have been changed by Reclamation to better reflect the purpose of the survey (i.e., Boat Survey has been changed to Marina Survey and Recreation Value Survey has been changed to Recreation Development Survey). The remaining two surveys are composed of similar questions and are formatted consistent with two Reclamation surveys approved by OMB (OMB Control No. 1006-0026).
The second part of the request consists of OMB and the Department of the Interior (DOI) and Reclamation agreeing upon a process whereby Reclamation custom designs a specific survey instrument by taking questions from the pre-approved survey questionnaires to fit a specific situation or area and then obtain OMB expedited approval of a specific customized survey.
Request Consists
of 2
Parts
Pre-Approved
Surveys
Customized Surveys
OMB approval of the submitted surveys contained in this information collection request will be used for a period not to exceed 3 years. Reclamation will monitor burden hours so as not to exceed the allocated annual burden hours. Information on individual surveys will be collected into a database from which reports can be generated to reflect the amount of work being conducted under this information collection program. Once Reclamation has used the total number of OMB authorized annual burden hours for any given year, it will stop administering surveys in these topical areas.
The approved surveys will be administered on site through the use of personal interviewing techniques. Completion of the surveys by respondents is voluntary. Under the supervision of a Project Leader, employees, contractors, or volunteers will ask the questions verbatim from the pre-approved questionnaires and will record responses or hand out questionnaires and wait for individual respondents to return them immediately upon completion. For those individuals staying in the area, arrangements may be made for surveyors, camp hosts, or volunteers to pick up completed surveys at a later time or have the individuals deposit them in a predetermined location at a campground, entrance station, visitor center, or survey drop box.
A stratified-cluster sampling technique will be used to both identify potential respondents and obtain survey information. The stratified cluster sampling technique allows sampling at different field locations, days of the week, and hours of the day based upon the percent of use at each of the locations and times. All people are then surveyed at these locations during these times.
Private contractors, managing partners, or universities will also use the survey instruments to collect data for Reclamation. This will typically be completed under the terms of an acquisition document and the collected information will be used to supplement studies and analyses of Reclamation sites or resources. Contractual requirements will limit the use of the data to only support Reclamation programs and research efforts.
All individuals administering the approved survey questionnaires will receive orientation and training in the use of survey techniques. In this manner, they will be sensitized to the need to assist respondents, ensure accuracy of the information being solicited, and of the need to reduce the burden on respondents.
Once the surveys are collected, the data are entered into a Statistical Package for Social Science (SPSS) program. Statistical programs are run on the data using SPSS. In all cases, non-parametric statistics are run. These include frequencies, means, and standard deviations. Depending on the research questions to be answered, parametric statistics are run. There are many possibilities, but typically a test of significance or analysis of variance such as a t test or chi-square would be run. The statistics are then interpreted and the findings documented in a findings report and presented to mangers for decision-making purposes.
Reclamation will not exceed the burden hours anticipated and the aggregate burden hours outlined in the Burden Hour Table in Section A, 12b of this request and will not administer any survey instrument until OMB approval is received. In addition, Reclamation will not make any changes to the survey once they have been approved. However, Reclamation may delete a specific question or questions after approval by OMB, if necessary. The deletion of certain items in specific questions is further explained in the Question Justification section (i.e., certain items that are not applicable to an area being surveys may be deleted, as appropriate).
Note: This expedited approval request follows the same process that the Department of the Interior (DOI) is currently using to conduct customized generic Customer Satisfaction Surveys that has been approved under OMB Control No. 1040-0001. The DOI was allowed to create its own questions as long as the questions fit the intent of 11 different topic areas that were approved by OMB. Reclamation’s request differs in this respect because only OMB pre-approved questions will be used; however, the process is consistent with the process that was used by the DOI in customizing generic surveys and obtaining expedited OMB approval.
Occasionally a specific need or situation will dictate having to customize a survey to address special requirements. A Project Leader will be responsible for initiating the actions necessary to obtain OMB approval of the customized survey and overseeing administration of the survey. The following discussion assumes that OMB agrees to the expedited process by approving all aspects of this two-part information collection request (i.e., approval of pre-approved surveys and the concept of expedited approval of a customized survey outlined in this section).
When customizing a survey instrument for OMB expedited approval, OMB, DOI and Reclamation will follow the process listed below. Note: As with the DOI generic Customized Satisfaction Survey process, Reclamation, DOI, and OMB agree that a 60-day and 30-day Federal Register Notice is not necessary due to the fact that Reclamation has already complied with those requirements by announcing this two-part information collection to the public for review and comment.
Bureau of Reclamation:
(1) Will select a Project Leader that will check available data sources to determine if existing information can meet the needs of the field office without the use of pre-approved or a customized survey. If pertinent information is not readily available and cannot be adequately obtained from administering a pre-approved survey, the Project Leader will initiate the process to obtain OMB approval of a customized survey. To meet the needs of the field office, the survey questions will be selected from the pre-approved surveys so that syntax, sentence structure, and format will approximate generally accepted survey methods.
(2) Will calculate the desired number of respondents needed to achieve a ±5 percent sampling error, 90 percent confidence level, and a response rate of no less than 80 percent. The sample size required will be based on the actual visitation at the site/area to be surveyed and all the procedures and survey methodologies outlined throughout this request package will be followed.
(3) Will submit the customized survey that is comprised of a pre-determined set of questions along with the OMB package to Reclamation’s Information Collection Officer in Denver, Colorado for review. The justification package will include references to this agreed upon process and sufficient documentation that will allow OMB to process the customized survey information collection in an expedited manner. The OMB package will be sent to the DOI well in advance of the actual time when the field office plans on administering the survey instrument to the public. Field offices will contact Reclamation’s Information Collection Office to determine the appropriate timeframe needed by the DOI for review keeping in mind the concept of an expedited approval process. Upon concurrence, the Information Collection Officer will forward the customized survey and the completed justification package to DOI’s Information Collection Coordinator in Washington, D.C., for review and concurrence.
(4) Will have the field office and the Denver Office work with the DOI and OMB until all outstanding issues and concerns are resolved.
Department of the Interior – Information Collection Coordinator:
(1) Will provide an administrative and technical review of the submitted materials. In order to achieve an expedited review of the customized survey package, a reasonable timeframe for review will be established that is mutually agreed upon between Reclamation and the DOI.
(2) Will transmit the package to OMB once it is approved.
(3) Will notify Reclamation upon OMB approval.
Office of Management and Budget:
(1) Will contact the DOI if additional information or revisions are required before OMB will approve the package.
(2) Will approve the customized survey instrument in a timely manner once all revisions and concerns are addressed and in the spirit of an expedited process.
(3) Notify the DOI and Reclamation once the package has been approved.
2c. Degree of accuracy needed for the purpose described in the justification
One of the surveys is required to achieve a 95 percent confidence level and 11 of the surveys are required to achieve a 90 percent confidence level. All surveys have a sampling error of ±5 percentage points with an anticipated response rate of at least 80 percent.
2d. Unusual problems requiring specialized sampling procedures, and
There are none.
2e. Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The surveys will be used by different field offices throughout the 17 Western States. Except for the Fee Survey, the surveys will not be conducted annually at the same location but only periodically at different locations on an as needed basis.
Participation in the recreation user surveys is always voluntary. In our experience, recreation users who are directly engaged in the experience for which information is being solicited are enthusiastic about the opportunity to render opinions affecting future management, custodial responsibilities, and care of the area. Therefore, voluntary compliance is most often high and thus, nonresponse rates are low.
Response rates will be maximized through careful consideration to detail in the development, pretesting, and implementation of the survey process. Additionally, surveyors will explain to respondents the use of the respondent’s information and benefits that may result from use of collected information.
To help in maximizing the response rate, surveyors will follow a code of conduct and specific instructions on how to administer the survey instruments. In addition, if it is determined that there is a sufficient number of non-English speaking visitors to Reclamation lands, Reclamation will provide surveys in the identified non-English language. This will increase the response rate
and help alleviate some nonresponse bias. If it was determined to be necessary after initial field work was initiated, Reclamation would have the appropriate surveys printed in the non-English language and take the necessary steps to ensure that the non-English speaking respondents understand the nature of the information collection. This will be accomplished by having brief instructions printed in the non-English language explaining how to complete the survey and the reasons for collecting the information. Professional judgment of the Project Leader in consultation with Reclamation’s Area Manager would be used to determine when a non-English survey would be needed.
In general, our experience with surveying recreationists has shown non-responsiveness to be a minor issue. However, if a respondent is not willing to participate in filling out a survey, Reclamation will not assume a zero value for those respondents to prevent a non-response bias to be integrated into the survey results. Reclamation will track the total number of surveys distributed against the number of completed surveys. The final report will describe the nonresponse bias as the number of surveys that were not completed or returned. If certain questions within a survey instrument are not answered, professional judgment will be used to explain why an answer to a specific question was left blank.
In addition, Reclamation’s survey Project Leader will ensure that surveyors will not only track the number of surveys that were not completed or returned by respondents, but, it will also attempt to ask the respondent why he/she refused to participate. If the respondent provides a reason for not participating in the information collection, Reclamation will record the responses in a log to be kept by the surveyors. Refer to Section 2a of Part B for a further description of information and respondent characteristics that the surveyor will possibly enter into a log.
There are enough questions in the River Instream Flow Survey, Reservoir Preferred Water Level Survey, and the Water Level Impact on Recreation Boating Use Survey that will require pre-testing of the entire surveys prior to administering the survey instrument to the general public. These three surveys plus the remaining 8 surveys in the OMB package have previously been pre-tested using in-house personnel (i.e., managers, outdoor recreation planners, and economists), as well as the academic community outside the agency. Testing outside the agency involved less that 10 individuals. Reclamation feels that the questions contained in 8 of the 11 surveys are straight forward questions that the general public will be capable of answering without assistance and without further testing and revision. Please note that all the questionnaires are intercept surveys, and the surveyor will be available to answer any minor questions that the respondent may have concerning individual questions. This should facilitate the completion of all the questions in the survey instrument.
When pre-testing the three survey instruments mentioned above, Reclamation will seek additional testing beyond those individuals that reviewed the initial 11 surveys. Pre-testing will occur at the site where any one of the 3 surveys will be administered. It is anticipated that approximately 20 individuals will have to be given the survey instruments for pre-testing purposes before Reclamation is confident that the surveys are ready to be given to the general public. The surveys will be randomly handed out to individuals and/or groups at the targeted recreation areas. If a group of more than one individual is asked to participate in testing the survey, the nearest birthday method will be used to select the person that will test the survey. All individuals completing a survey will be 16 years of age or older.
Reclamation or its contractors will look for comments and concerns from the individuals pre-testing the surveys that are related to the clarity of the questions, relevance, and format. Half way through the testing period (i.e., once half of the individuals have commented on the survey), Reclamation will adjust/revise appropriate questions, if necessary, and again conduct further testing (i.e., make the revised questionnaire available to the remaining 10 individuals). Reclamation will again make changes to questions, if necessary. Based on pre-testing, Reclamation may change the structure of the question and/or the format of a question, but will not change the intent or meaning of the question. Reclamation will not add any new questions to a survey but may delete a question after pre-testing is complete, if appropriate.
When developing a customized survey from the pre-approved surveys and prior to any submittals to OMB for approval, qualified Reclamation personnel will test drafts of their proposed survey instruments on small samples of students, colleagues, or others, as appropriate. Note: Questions contained in the customized survey will only include questions that have been approved by OMB as part of this information collection approval package. Other than those individuals described in this paragraph, field testing of a customized survey will not be undertaken prior to submittal to OMB. The number of individuals sampling the surveys from outside the agency will not exceed 10. The peer review comments and test results will be used to provide insight on probable visitor perspectives about the survey instrument. The peer review comments will also be used to identify any trouble spots in the proposed questionnaires regarding layout and design and will guide revising the questionnaire in ways that will minimize the burden to the public.
In addition to the information provided in Part A, Question 8a., the recreation-related questionnaires in use by the Corps of Engineers were originally developed, tested, and improved in cooperation with experts in survey research. Experts assisting the Corps included: Dr. Chilman, Associate Professor, Forestry Department, Southern Illinois University; Dr. William Hammitt, Professor, PRTM Department, Clemson University; and Dr. Daniel Stynes, Professor, Parks and Recreation Department, Michigan State University.
As previously mentioned, Reclamation recently secured the services of Dr. Glenn Haas, Dr. Robert Aukerman, and Dr. Marcella Wells of Aukerman and Haas and Associates, LLC to review and provide feedback on the clarity of the questions, relevance, and format of the survey questionnaire. Dr. Haas and Ackerman are both Professors’ Emeritus at Colorado State University, Fort Collins, Colorado, and have extensive experience in the field of recreation and tourism. Dr. Wells is a former professor at Colorado State University in the field of recreation and tourism specializing in interpretive planning.
Dr. Robert Aukerman
729 Duke Square
Fort Collins, CO. 80525
970-484-2028
Dr. Glenn Haas
P.O. Box 105
Como, CO 80432
970-691-2641
Dr. Marcella Wells
3403 Green Wing Court
Fort Collins CO 80524
970-498-9350
The following internal sources were consulted during the development of the survey questionnaires:
Name and Title |
Agency |
Telephone Number |
Jon Platt, Resource Economist |
Bureau of Reclamation |
303-445-2737 |
Chad DeVore, Outdoor Recreation Planner |
Bureau of Reclamation |
303-445-3689 |
Vernon Lovejoy, Recreation Program Analyst |
Bureau of Reclamation |
303-445-2913 |
Richard Rizzi, Manager, Lands Resources Office |
Bureau of Reclamation |
303-445-2900 |
Darrell Welch, Recreation Program Analyst |
Bureau of Reclamation |
303-445-2711 |
File Type | application/msword |
File Title | Supporting Statement |
Author | Darrell Welch |
Last Modified By | Frey_N |
File Modified | 2008-07-29 |
File Created | 2008-07-29 |