Supplemental and Part B Questions

Survey of Great Lakes Beach Visitors Related to Beach Hazards

Supplemental Questions for DOC/NOAA Customer Survey Clearance
(OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

This data collection is being conducted by NOAA’s Coastal Services Center (CSC) to obtain information from likely visitors to Great Lakes beaches about their sources of beach conditions information, awareness and understanding of hazardous beach conditions, and perceptions of risk, in order to develop improved outreach to the public on hazardous conditions in the swim zone.

NOAA’s National Weather Service (NWS) generates forecasts of beach conditions, such as wave height and rip current strength, and distributes them to the public through multiple outlets such as weather-focused websites (i.e., www.weather.gov); weather radio; local TV, radio, and newspaper weather forecasts, and mobile alerts. Other outreach campaigns have developed succinct, easily remembered messages—such as “Break the Grip of the Rip” and “Flip, Float, Follow”—but have not always targeted specific audiences. Furthermore, the Great Lakes feature a variety of beach hazards often lumped into the category of rip currents, including structural currents, channel currents, seiches, and the choppy, low-period wave action specific to Lake Michigan. This has often stymied outreach and education measures, and reduced the effectiveness of drowning-prevention campaigns.

Working with NWS and Minnesota Sea Grant, CSC is planning to conduct a pair of surveys described here. The purpose of the first survey (web-based) is to better understand how likely visitors to Great Lakes beaches obtain information about beach conditions and use that information to make decisions about entering the water, as well as the extent of their awareness and understanding of existing outreach and information about beach hazards. The results of the web-based survey will inform the development of new outreach messages which will be tested through a second survey (beach intercept) to gain feedback from beachgoers on their understanding of the messages and awareness and perception of beach risks. In addition to providing feedback on the outreach messages, the respondents will benefit from considering the risks associated with beach hazards.

The results of the beach intercept survey will be used by CSC and partners to craft more effective outreach messages about beach hazards and how to avoid or escape from strong currents and waves occurring in the swim zone in the Great Lakes.

2. Explain how this survey was developed. With whom did you consult during the development of this survey on content? statistics? What suggestions did you get about improving the survey?

The CSC team consulted with Eastern Research Group, Inc. (ERG) on the development of the survey instruments. ERG has significant experience assessing technical assistance provided by federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer satisfaction with services. The development of the web-based survey was informed by reviewing relevant background materials on previous beach communication campaigns, current beach safety awareness programs, expert interviews, and a literature review. The draft survey instrument was also circulated to Michigan Sea Grant (MI SG) staff actively working on a parallel project in the region.

The beach intercept survey was developed in parallel to the Web-based survey using the same process, with placeholders in the survey for the outreach messages that will be developed based on results from the Web-based survey and a set of interviews with experts (fewer than 9 non-federal interviewees) in beach hazards outreach.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

CSC will be conducting two surveys, a web-based survey of awareness of the risks of Great Lakes currents and waves in likely beach goers and an intercept survey of beach goers on their awareness and understanding of outreach messages on the risks of currents and waves.

Web-based Survey

ERG will obtain a list of email addresses for use in a survey using a reputable survey email list vendor. The list will include only email addresses whose owners have agreed to receive further emails on topics such as this. ERG will select a sample from this list and send a series of emails to the potential respondents: pre-notification, email survey link, follow-up reminder. The set of emails to be sent are further detailed in Section B below. The survey will be a link within the email sent. ERG will host the survey on its secure Web-based survey Web site running Vovici™ survey software. All responses will be anonymous.

The email list being used by ERG to select the sample contains self-reported information on the email owners’ locations. ERG will use the location information on the email list to select a sample that includes all individuals who reside in the area likely to travel to the Great Lakes. This includes the states of Illinois, Indiana, Michigan, Minnesota, Ohio, Wisconsin, and the western parts of Pennsylvania, and New York. Respondents will be selected at the zip code or county level based on geographic proximity to the Great Lakes and discussions with tourism bureaus about the regions that tend to visit the Great Lakes. Based on previous survey work conducted by ERG, CSC expects that 60% of the recipients will respond to the survey. The survey will be performed using a web-based survey instrument. CSC has chosen a Web-based instrument for the first survey because the current outreach messages and information about beach conditions are provided to potential beach goers via the internet through weather focused web sites (i.e., www.weather.gov), beach web sites, mobile alerts and applications. CSC expects that the web-based survey will take 20 minutes to complete.

Beach Intercept Survey

For the intercept survey, a trained interviewer will sample potential respondents from beachgoers at selected beaches. ERG will select one to two heavily visited beaches on Lake Michigan and will recruit one to two interviewers to visit those beaches on expected busy days in the summer to collect survey responses.¹ ERG will train interviewers on the protocol for selecting respondents from the individuals encountered on the selected beach, with the goal of obtaining a random sample of eligible individuals on the beach. The statistical design that will be used to generalize the results to the population is discussed under Section B, Question 2 below. The sample size being employed will allow for a fair degree of accuracy for the types of questions being used in the survey (i.e., five point scales) within the budget constraints of the project, and ERG will employ weighting to accurately estimate population parameters.

CSC has arranged for the use of experienced field interviewers through contacts with academic institution in the region. The interviewers will provide potential interviewees with a fact sheet and will wear clothing that indicates they are with a research institution. Based on these factors, CSC expects that 70% of respondents invited to participate in the intercept survey will complete it. CSC expects that the intercept survey will take 15 minutes to complete.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

For the web-based survey, the data will be analyzed to identify trends among the respondents related to use of information and forecasts of beach conditions and perception of beach hazards and to assess whether statistically significant differences exist between demographic groupings, with particular attention paid to potentially high-risk groups, both in terms of understanding, risk perception, and behavior. The data gained from this survey will be used for the creation of improved messages about beach hazards. The purpose of the web-based survey is not to be able to extrapolate the information to the population of beach-goers.

The beach intercept survey data will be analyzed to assess respondent understanding of and response to new messages in terms of ability to assess beach conditions, risk perception, and intended behavior. Data will not be extrapolated to the population of Lake Michigan beach goers The

This data collection does not directly provide data to measure a GPRA performance measure, but indirectly supports NOAA in decreasing drowning incidents in the Great lakes by enhancing current and developing new education and outreach tools to inform swimmers on the risks of currents and escape methods. This support NOAA’s long term goal of a “Weather-Ready Nation” under the objective of “Reduced loss of life, property, and disruption from high-impact events.” The survey’s results will improve understanding of beachgoers’ attitudes, methods of assessing, and perceptions of beach-related hazards.

Part B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Web-based Survey

Target Population and Sampled Population

The target population for the survey is individuals who use NWS and other forecasts, educational materials, and outreach messages about beach conditions in the Great Lakes. The sampled population, however, will be individuals who reside in regions likely to visit the Great Lakes that have opted in to an email list to be surveyed about topics like this. The sampled population diverges from the target population because it is restricted to just those individuals who have provided an email address.

Estimated Universe, Sample Size and Expected Response Rates

Table 1 provides CSC’s estimates of the universe for this survey effort, along with the sample sizes and expected response rates. CSC has estimated needing a sample size of 200 individuals to meet the statistical criteria described under Question B.2 below. We have increased the target sample size to 336, reflecting nonresponse.

Table 1. Estimated Universe, Sample Size, and Response Rates.

States Likely to visit Great Lakes	Sampled Population (Number of Email Addresses in List)	Allocated Sample Size	Expected Response Rate	Sample Size Adjusted for Nonresponse
Illinois	98,743	25	60%	42
Indiana	98,073	25	60%	42
Michigan	98,527	25	60%	42
Minnesota	97,476	25	60%	42
Ohio	98,818	25	60%	42
Wisconsin	97,525	25	60%	42
Pennsylvania [a]	2,241	25	60%	42
New York [a]	15,003	25	60%	42
Total	606,406	200	60%	336
[a] These number reflect only Great Lakes coastal counties within the state.

Beach Intercept Survey

Target Population and Sampled Population

The target population of the intercept survey is individuals and families visiting Lake Michigan beaches. The sampled population will be eligible individuals visiting one to two specific Lake Michigan beaches during a set period during the summer. The sampled population diverges from the target population because it is restricted to individuals who are 18 years old and older, and captures only a snapshot of the visitors to Lake Michigan beaches during the summer.

Estimated Universe, Sample Size and Expected Response Rates

Table 2 provides CSC’s estimates of the universe for this survey effort, along with the sample size and expected response rates. CSC has little information on the total number of visitors to Lake Michigan beaches; however, we expect that the total number of annual beach visitors may exceed one million.² Our total targeted sample size is 90 responses; however, we are also interested in obtaining at least 30 responses from men 18-22 and at least 30 from parents with children under 13. These two groups have been singled out since young adult males (aged 18-22) may exhibit more risky behavior and attitudes and parents of young children may exercise additional restraint. The remaining 30 responses can come from other groups (e.g., women, older men, etc.).

Table 2. Summary of estimated universe, target sample size, expected response rate and number of contacts needed to hit the target.

Respondent Type	Estimated Number of Lake Michigan Beach Visitors	Target Sample Size [a]	Expected Response Rate	Contacts needed to Hit Target
Total	Unknown	90 [a]	70%	129
Groups with specific targets [c]
Men, 18-22	Unknown	30 [b]	70%	43
Parents with children under 13	Unknown	30 [b]	70%	43
[a] This assumes the use of 2 interviewers working 2 days each for 6 hours per day and collecting an average of 2-3 interviews (each) per hour. [b] These reflect minimum numbers for each group and count as part of the 90 total respondents. [c] CSC is particularly interested in the responses from these groups. The other 30 responses can be drawn from other groups.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Web-Based Survey

Statistical Method for Stratification and Sample Selection

CSC selected a sample size for this survey based on resource constraints and obtaining sufficient responses from states whose residents visit the Great Lakes. This sample size was 200, adjusted to 336 to reflect potential nonresponse. The sample will be randomly selected from the list of email addresses that we have.

Estimation Procedure and Accuracy

As noted above, CSC does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates is not meaningful to calculate.

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

Beach Intercept Survey

Statistical Method for Stratification and Sample Selection

The sample size for this project was determined based on available budget for the project. CSC has selected a sample of 90 responses on the beach intercept survey, adjusted to a sample of 129 to reflect potential nonresponse.

Potential respondents will be selected using a systematic sampling approach on the beach. Interviewers will start at one end of the beach (or at a pre-designated place) and select someone from every 5^th-10^th group/family on the beach to interview. If the beach is sparsely populated on the sampling day, the interviewer will select every 5^th group; if the beach is crowded, the interview will select every 10^th group. If the beach appears somewhere between sparse and crowded, the interview will have discretion to select an interval between 5 and 10 to select respondents. If the interviewer reaches the “end” of the beach, he/she will turn around and continue back using the same interval being to it does not result in selection of the same groups as before.

The survey design is not imposing any specific stratification on the sample as a whole. However, there is need to collect this information from young adult males (aged 18-22), who may exhibit more risky behavior and attitudes, and from parents of young children, who may exercise additional restraint. Thus, we will specifically target samples of at least 30 each from these two groups. There are no additional sub-group targets set for the this survey.

Estimation Procedure and Accuracy

As noted above, CSC does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates is not meaningful to calculate.

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Web-based Survey

CSC expects that a high response rate is achievable for the web-based survey; however, CSC and ERG will continue to follow good survey practices to ensure high participation, including the following:

• ERG will send the potential respondents a pre-notification email to inform them of the upcoming survey.

• ERG will send the email with the survey link 3-4 days after the pre-notification email.

• ERG will send 2 reminders to non-responders, one and two weeks following the email with the survey link.

ERG will also assess non-response using the available demographics from the email list file. This includes gender, income, and, when available, age.³ ERG will also assess nonresponse based on geographic location.

Beach-intercept Survey

CSC, through its subcontractor ERG, will use local academic researchers that have experienced field interviewers to perform these interviews. During the data collection, the interviewers will collect observable demographics regarding those that refuse to participate. This will allow us to compare the demographics of those that did and did not participate. Relevant demographics will include: gender, race/ethnicity, number of children, and general age range.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

CSC consulted with Eastern Research Group, Inc. (ERG) on the development of the survey instrument. ERG has significant experience assessing technical assistance provided by Federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer satisfaction with services. The survey development process was informed by reviewing relevant background materials on the demographics of drowning incidents, current beach hazard outreach campaigns, expert interviews, and a literature review. The development of the survey questions was also informed by feedback from staff of Michigan Sea Grant working on related efforts.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

CSC has contracted with Eastern Research Group, Inc. (ERG) of Lexington, MA to design the survey instruments and implement the surveys. ERG’s project manager for this work is Jenny Helmick (781-674-7342; Jenny.Helmick@erg.com); ERG’s task lead for survey development is Dr. Lou Nadeau (781-674-7316; lou.nadeau@erg.com).

1 ERG has defined a “busy day” as one in which the forecast is for very warm temperatures and mostly sunny skies.

2 http://www.crh.noaa.gov/mqt/?n=lakemichiganripcurrenthistory.

3 The email list has a gender and a value for income for more than 90 percent of the email addresses, but only has ages for less than 20 percent of the email addresses.