0648-0797 Supporting Statement Part B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Tornado Watch/Warning Post-Event Evaluation

OMB Control No. 0648-0797

SUPPORTING STATEMENT PART B - (Questions and guidance for Responses)

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Potential respondents will include any member of the U.S. public that was recently (within the previous 30 days) in or near a tornado.

No records exist regarding the exact number of individuals affected by tornadoes each year in the U.S. Thus, we generate an estimate calculated on the following known information.

Based on FEMA reports¹ and covering 11 tornadoes declared as major disasters from 2017 to 2020 (covering 58 counties in 10 states), the average number of impacted households in violent tornadoes is approximately 1000. Path length for violent tornadoes averages 10x higher than path lengths for strong tornadoes, and 100x higher than path lengths for weak tornadoes. There are approximately 1200 tornadoes per year, with 1% violent tornadoes, 19% strong tornadoes, and 80% weak tornadoes.

The average number of tornadoes by category ranges per year 12 violent, 228 strong and 960 weak tornadoes.

Furthermore, the number of households in the path of tornadoes in the US per year can be calculated using these figures. Therefore, 1000 households exposed to violent tornadoes multiplied by the average of 12 per year, gives 12,000 households. Similarly, for strong tornadoes the figures are 100 households multiplied by 228, giving a total of 22,800 households, whilst 10 households exposed to weak tornadoes can be multiplied by the 960 weak tornadoes that occur, with a total of 9600 households. When adding these figures together, a total of 44,400 households exposed yearly to tornadoes

With an average of approximately 2.5 individuals per household, the total number of people exposed to tornadoes each year is estimated to be 111,000. This is likely to be an upper-bound estimate since disasters are a function of both hazard and population exposure, and thus, more dense populations are more likely to incur major damage from tornadoes.

We are using convenience sampling approaches to collect data. A citizen science web tool developed by the National Severe Storms Laboratory (NSSL) is the main sampling tool to collect data. After tornado events, participants may access the tool and respond to the questions as a web survey. We aim to publicize this tool as well as have a QR code that will allow members of the public with a smart device to quickly access the landing page for the survey. Some NWS forecasters may be able to point participants to this tool while on damage surveys if someone shows interest in sharing their tornado experience. By publicizing through social media as well as through the NSSL website, we aim to draw people to take part in the survey. Although the survey doesn’t currently exclude anyone, a computer’s/smart device’s IP address is used to log into the server (the IP address is only held temporarily for the allotted time before erasure), so that repeated reports and surveys cannot be submitted within a certain predetermined time-frame. After some initial advertising over the last two years, we estimate that we will reach a little over 1% of the tornado-affected population, or around 1,200 people, through this method. However, this is conservative and on the low side of our estimation.

Information Collection	Population or Potential Respondents Universe (a)	Number of Respondents Selected (b)	Expected Response Rate (c)	Expected Number of Respondents (d) = (b) x (c)
NSSL web tool	111,000	1,200	100%	1,200
Total				1,200

2. Describe the procedures for the collection of information including:

   • Statistical methodology for stratification and sample selection. No statistical sampling methods will be employed at this stage. We use a convenience sampling approach because this is an exploratory data collection exercise designed to offer new insights into post-event responses that will inform a later, more systematic investigation.

• Estimation procedure

We will generate descriptive statistics for each survey item, and eventually conduct more sophisticated regression analysis to link behaviors to demographic characteristics.

• Degree of accuracy needed for the purpose described in the justification,

There is no specific degree of accuracy required for this collection.

• Unusual problems requiring specialized sampling procedures

The collection of data after disaster events requires a special set of procedures to reach populations under difficult conditions, in particular, when they may be displaced.

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Data collection is anticipated to go on continuously throughout the year as tornado events occur.

An exploratory pilot study is appropriate at this stage, as little is known about the wide array of event types that take place, nor have measures like these been deployed before in any attempt to collect this data more systematically (Eldridge et al. 2016). Basic descriptives, even with a biased sample, will offer insight into latter stages of program development. This pilot study could also provide the basis needed to obtain funding to do more systematic research. We estimate that with 5-10 years of longitudinal data, we may be able to assess trends and identify gaps in knowledge that will lead to more research and development. Having said this, the survey instrument is designed to be iterative and responsive to changing knowledge, with questions able to be augmented to improve how questions are asked, or to respond to feedback or comments that the public might improve clarity, or to address potential future knowledge gaps.

Reference cited:

Sandra M Eldridge, Gillian A Lancaster, Michael J Campbell, Lehana Thabane, Sally Hopewell, Claire L Coleman, Christine M Bond. Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. PLoS ONE. 2016; 11(3)

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Participants approached by NWS forecasters and directed to the NSSL web tool will be told their participation is completely voluntary, and the NWS forecaster will appear in their official capacity, increasing trust in the data collection process. If problems are experienced with data collection, our team will evaluate the nature of the issue and generate a plan for improvement. All data collected will be used to understand the ways different groups, across different events, receive, understand, and respond to NWS watch/warning information.

Participants accessing the NSSL web tool will self-select into the study, and the protocol is very short, promoting successful completion and reduced burden (estimated time to complete the web survey is five to ten minutes, with a median of seven minutes). The NSSL media team is involved in crafting messaging and media engagement to promote the NSSL web tool. The more the public that engages with the web tool following tornado events, the broader the sampling base and the more representative of impacted populations. The web tool does not require specific software or the downloading of an App, this was a purposeful decision to be as inclusive as possible and minimizing cost (through use of cell-phone data, for instance). Participants will only require web access through computers, tablets or smartphones. Links to the tool will be disseminated through social media, including outreach to broadcasters who can disseminate the link to much wider populations. If successful, this web tool will remain in place, with the public able to continue to use it when tornadoes occur. This will also allow researchers to gain insight into what safety messaging is effective in initiating behavior change when tornado watches and warnings are given.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Periodically, the responses will be evaluated to understand the geographic scope of events receiving replies, the scope of different kinds of tornado events for which responses are obtained, and the population characteristics of responses. These measures will be compared to the total potential sample of tornado events and census tract-level demographic data to identify any biases in data collection. For example, it could be the case that respondents that volunteer this kind of information might skew older, female, and white, based on outcomes typical of mail surveys. This information will be used to inform a subsequent, more systematic (funded) investigation.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Consultants on statistical aspects of the design:

Dr. Makenzie Krocak, NSSL, 405-325-6273, makenzie.krocak@noaa.gov

Dr. Kodi Berry, NSSL, 405-325-6270, kodi.berry@noaa.gov

Agency unit collecting/analyzing information:

NOAA National Severe Storms Laboratory

1 These reports were furnished by the National Weather Service.  They reported the following procedure for retrieving them:

FEMA declared disasters

https://www.fema.gov/disasters/disaster-declarations?field_dv2_state_territory_tribal_value=All&field_year_value=2010&field_dv2_declaration_type_value=All&field_dv2_incident_type_target_id_selective=49120

Steps:

·         Filter search by year and incident type (tornado)

·         Click on each search result to see the details

·         Click on Reports and Notices

·         Scroll down and click on the Preliminary Damage Assessment Report link