Supplemental and Part B Questions

NOAA National Weather Service

Hurricane Arthur Social Science Research Project

Part A and B DOC/NOAA Customer Survey Clearance

OMB Control No. 0648-0342

Expiration Date: 04/30/2015

Hurricane Arthur Social Science Research Project

January 23, 2015

1. Supplemental Questions for DOC/NOAA Customer Survey Clearance
   (OMB Control Number 0648-0342)

1. Explain who will be conducting this survey. What program office will be
   conducting the survey? What services does this program provide? Who are the customers? How are these services provided to the customer?

NOAA’s National Weather Service (NWS) and National Hurricane Center (NHC) predict tropical cyclone storm movement and impacts. The NHC issues tropical cyclone warnings, watches, and other information products through websites, through NOAA Weather Radio, and through interactions with partners such as the media, government officials, emergency managers, and community groups.

In 2014, the NHC issued a new, experimental storm surge graphic (the “Potential Storm Surge Flooding Map") for the first time—during Hurricane Arthur, the first named storm of the 2014 hurricane season. The graphic depicts areas along the Gulf and Atlantic coasts of the United States that have a significant risk of life-threatening inundation by storm surge from a tropical depression, storm, or hurricane. It urges those in the marked areas to follow evacuation and other instructions from local officials. During Hurricane Arthur, the NHC issued the map at right, which outlined coastal areas of North Carolina that had the potential to see storm surges of greater than 3 feet above ground level during the storm.

T

Potential Storm Surge Flooding Map for Hurricane Arthur

he NHC is looking for input on how the map was used by its partners (primarily decision-makers, broadcast meteorologists, emergency managers, and community members) during Hurricane Arthur. This input will be useful in helping the NHC address any confusion or concerns identified by partners in using or interpreting the map, so as to ensure the continued development and implementation of this experimental product.

To accomplish this, NWS seeks to conduct one-on-one interviews with a small group of decision-makers, emergency personnel, and broadcast meteorologists who used the map during Hurricane Arthur.

2. Explain how this survey was developed. With whom did you consult regarding content during the development of this survey? Statistics? What suggestions did you get about improving the survey?

NWS contracted with Eastern Research Group, Inc. (ERG) on the development of the interview guide and questions. ERG has significant experience in conducting detailed interviews, focus groups, and surveys for federal agencies that focus on customer satisfaction and outcome attainment. To develop the interview guide and discussion questions, ERG worked with Dr. Betty Morrow, a well-known and respected social scientist, who has conducted many research projects with NWS stakeholder groups related to communicating and interpreting tropical cyclone hazards, as well as evacuation studies and risk behavior/communication studies. ERG and Dr. Morrow also worked closely with Rick Knabb, the Director of the NHC, Jamie Rhome, the NHC Storm Surge Team lead, and Robert Berg, an experienced NHC forecaster.

The interview questions focus on how organizations or individuals accessed the map, how they used the map, and whether the map had any effects on decision-making. Questions also probe for whether responders think the categories (and corresponding colors) of storm surge levels depicted are appropriate: blue (less than 3 feet), yellow (greater than 3 feet), orange (greater than 6 feet), and red (greater than 9 feet).

The map reflects potential, not expected, storm surge levels. There’s a 1 in 10 chance that the storm surge will be higher than what is shown on the map at individual locations. Suggestions for improving the interview guide and discussion group questions included adding a question to gather input on whether the 1 in 10 chance is a reasonable threshold to depict potential storm surge flooding.

3. Explain how the survey will be conducted. How will the customers be sampled (if fewer than all customers will be surveyed)? What percentage of customers asked to take the survey will respond? What actions are planned to increase the response rate? (Web-based surveys are not an acceptable method of sampling a broad population. Web-based surveys must be limited to services provided by Web.)

The survey will be in the form of one-on-one telephone interviews in a webinar format. Interviewees will be able to view static screen shots of the map (as it was depicted during the storm) on their computer through a Go-To-Meeting type of format. ERG and Dr. Morrow will schedule and conduct an interview with a total of up to 8 emergency managers, 8 government officials, 6 broadcast meteorologists, and 4 community group representatives.

The NWS/NHC will provide an initial list of 20 individuals to interview based on its knowledge of the partner community and how the map was used during Arthur. The list will be derived from conversations that the NWS/NHC has had with these individuals during or after the storm; these interviewees include 6 emergency managers, 6 broadcast meteorologists, and 8 government officials. The interviewees with broadcast meteorologists will include individuals from private entities, such as The Weather Channel, that used the map on air during the hurricane.

However, NWS/NHC may not have full knowledge of how the map was used on a local level. Therefore, the interview guide will also ask the initial interviewees if they shared the map in their community, and if so, do they have recommendations for additional interviewees for ERG and Dr. Morrow to interview. ERG anticipates that the recommendations for additional interviewees would be at the local emergency management or community group level. ERG and Dr. Morrow would be looking for recommendations for additional interviewees based on whether those individuals used the map for community planning and response or community groups with responsibilities for emergency response, such as the Red Cross.

ERG will contact all of interviewees to introduce the project, our role in the project, and request that an interview be scheduled at a convenient time. NWS expects that 100 percent of the interviewees will schedule an interview with ERG, given the importance and level of interest in this product.

In scheduling each interview, ERG will introduce ourselves, explain our affiliation, describe what information we are seeking, and how long the interview will take:

My name is XX. I am with Eastern Research Group (ERG), and I am calling to schedule an interview with you on behalf of NOAA’s National Weather Service.  I am hoping to ask you some questions about an experimental storm surge map that was used for the first time during Hurricane Arthur in North Carolina in 2014. The interview should be 40 to 60 minutes long.  Do you have any availability over the next couple of weeks to schedule this interview? We will not release your name or information that could identify you as part of this interview process or in our subsequent reports to the National Weather Service.

ERG will generate detailed notes from the interviews.

4. Describe how the results of this survey will be analyzed and used. If the customer population is sampled, what statistical techniques will be used to generalize the results to the entire customer population? Is this survey intended to measure a GPRA performance measure? (If so, please include an excerpt from the appropriate document.)

NWS/NHC will use the information resulting from this data collection to help guide refinements (e.g., confidence levels, flooding threshold levels) for the “Potential Storm Surge Flooding” map while the graphic is still in experimental use. This information will also help NWS/NHC determine if additional education, marketing, training, or partner coordination is warranted to ensure that organizations and individuals are aware of the map, know how to access it, know how to use it, and are being consistent and accurate in their communications about the map.

NWS/NHC will have ERG perform thematic analyses on the detailed interview notes to identify key findings and recommendations.

The data do not directly contribute to a GPRA measure.

NWS/NHC is not using any statistical methods to select participants from the population and will select all population members in the interviews.

2. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe includes an initial list of 20 individuals from the NWS/NHC (6 emergency managers, 8 government officials, and 6 broadcast meteorologists) and 6 additional potential interviewees (2 more emergency managers and 4 community members). The additional 6 interviewees would be based on recommendations provided from the initial interviewees.

Therefore, the total number of potential interviewees is 26 individuals (8 emergency managers, 8 government officials, 6 broadcast meteorologists, and 4 community members).

No statistical methods are being used in the initial participant selection or in the selection of potential additional interviewees based on recommendations from the initial set of interviewees.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical Method for Stratification and Sample Selection

The NWS is not using statistical methods for collecting these data.

Estimation Procedure and Accuracy

The NWS does not need to extrapolate the results to the population and will therefore not need to estimate population parameters from the collected data. This also means that the accuracy of the estimates in not meaningful to calculate

Unusual Problems Requiring Specialized Sampling Procedures

None are required.

Periodic Data Collection Cycles

This request is for a one-time data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

ERG will contact all of interviewees to introduce the project, our role in the project, and request that an interview be scheduled at a convenient time. NWS expects that 100 percent of the interviewees will schedule an interview with ERG and Dr. Morrow, given the importance and level of interest in this product and that these interviewees were in prior contact with NWS/NHC during or after the storm. ERG will also ask these interviewees if they have any other recommendations for individuals to interview based on their knowledge of the individuals and groups involved in community emergency planning/response with whom they may have shared the map.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

The NWS consulted with Eastern Research Group, Inc. (ERG) on the development of the interview guide and questions. ERG has significant experience assessing technical assistance provided by federal agencies through detailed interviews, focus groups, stakeholder engagement, and surveys that focus on customer satisfaction with services. The interview guide and question development process was informed through discussions with NHC staff .

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The NWS has contracted with Eastern Research Group, Inc (ERG) of Lexington, MA to design the interview guide and discussion questions and implement the data collections. ERG’s project manager for this work is Linda Girardi (703-841-0501; linda.girardi@erg.com).

Hurricane Arthur Social Science Research Project 4