0648-NSSL SUPPORTING STATEMENT Part A - 72621

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Tornado Watch/Warning Post-Event Evaluation

OMB Control No. 0648-XXXX

SUPPORTING STATEMENT PART A

Abstract

This is a request for a new collection of information.

Each year over 1000 tornadoes affect communities across the United States, yet very little is known about how individuals receive, interpret, and respond to information from the National Oceanic & Atmospheric Administration (NOAA) (including weather alerts such as tornado watches and tornado warnings delivered via the National Weather Service (NWS)) relating to this hazard. In fact, only a small sample of tornadoes ever receive study, and most often those are only the largest tornadoes. No generalizable information on tornado warning response after real-world events exists. The National Weather Service and National Severe Storms Laboratory (NSSL) have designed this data collection instrument to allow for more routine collection of this information. Potential respondents will include any member of the U.S. public that was recently (within the previous 30 days) in or near a tornado. The information would be collected by NWS forecasters using their Damage Assessment Tool (DAT), and by members of the public who voluntarily access a web tool developed by the National Severe Storms Laboratory.

Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
   
   Congress established Social Behavioral and Economic Sciences (SBES) policies for weather forecasting in the Weather Research and Forecasting Innovation Act (WRFIA) of 2017. The legislation specified, at a high level, that the purpose of the NOAA watch/warning system is to inform action to prevent loss of life and property (Sec. 406), and that any changes to the system must preserve the benefits of the current system and be guided by the findings of social scientific research.  NOAA responded to this guidance in its 2020-2026 R&D Vision, calling out reductions in societal impacts from hazardous weather in Vision Area #1.  Key Question 1.4 asks, “How can NOAA enhance communications, products, and services to enable informed decision-making?”  The vision document prescribes that this should be done by first understanding the risk communication process that supports decision-making by the public.  Optimizing NOAA’s communication outcomes requires that messages are widely disseminated, understood, and actionable.
   
   Another important policy set forth in WRFIA (Sec. 103) is the Tornado Warning Improvement and Extension Program (TWIEP).  The goal for this program is “to reduce the loss of life and economic losses from tornadoes through the development and extension of accurate, effective, and timely tornado forecasts, predictions, and warnings, including the prediction of tornadoes beyond 1 hour in advance.”  The TWIEP Implementation Plan submitted to Congress by NOAA (2017) includes several key areas of SBES research and development, including research into the ways gap-filling probabilistic information could be generated, disseminated, used by core partners, and ultimately applied to inform decision-making on the part of the U.S. public.  To evaluate the success of this new technology and assure that any changes maintain the benefits of the current watch/warning system (WRFIA Sec. 406), we need baseline data regarding the performance of the current system.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The data to be collected includes a small set of questions relating to tornado watch/warning reception, understanding, and response. The survey is estimated to take six minutes to complete, within a range of five to seven minutes. This was based on testing with NSSL staff invited to use the web tool. The participants will be any member of the U.S. public that has been in or near a recent tornado event.

The survey will be administered through the following means:

(1) The National Weather Service Damage Assessment Tool (DAT). This is a storm damage data collection instrument that NWS forecasters complete when evaluating structural damage after tornado events. NWS will add the tornado watch/warning survey items onto the DAT as an optional menu for forecasters if they encounter tornado survivors during their damage assessment.

(2) A citizen science web tool developed by the National Severe Storms Laboratory (NSSL). After tornado events, participants may access the tool and respond to the questions as a web survey.

Ultimately, we envision all the data being compiled into a centralized database. Responses will be anonymous/no identifiable information will be collected or retained. The dataset may then be used by NWS, NSSL, and their academic partners to conduct analyses of system effectiveness.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The survey instrument will be delivered through two mechanisms: an NSSL web tool and the NWS Damage Assessment Tool. This web tool will rely on voluntary submissions when tornadoes are experienced. Being web-based means that costs for getting to and from the field are reduced. Currently much of this information is only recorded in large impact events, meaning we are not able to understand the broader pattern of response (or not) when tornado watches and warnings are given.

The information collection also involves the use of the Damage Assessment Tool, which is a handheld information technology tool that forecasters use in the field for inputting data as they conduct damage assessments. The use of this technology for this data collection is thus efficient, as forecasters are already using the technology in their damage assessment task, and they can choose to collect this data as opportunities arise.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Question 2

Currently, the only systematic data collected about severe weather warning effectiveness comes from annual customer satisfaction surveys conducted by the National Weather Service. Those surveys do not take a post-event perspective, only capturing general thoughts on how the system performs as a whole. The questions also do not measure comprehension and real-world response. Thus, we do not have the data necessary to evaluate performance in real events as needed to satisfy the requirements of the WRFIA.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

This collection is directed at individuals or households only. There is no impact to small businesses or other small entities.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

If data collection is not conducted, NOAA will be hampered in its efforts to evaluate the societal performance of its tornado watch/warning program. .

7. Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with OMB guidelines.

This collection will be conducted in a manner consistent with OMB guidelines.

8. If applicable, provide a copy and identify the date and page number of publications in the Federal Register of the agency's notice, required by 5 CFR 1320.8 (d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

The notice in the Federal Register was published on Friday January 15, 2021. (–86 FR 3998) No comments were received during the 60-day period

NWS and NSSL scientists consulted with University of Delaware faculty, University of Oklahoma staff students, and faculty, and University of Alabama faculty to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported. The average survey time of six minutes was confirmed, and discussions were had about wording and clarity of language used. This was used to iterate on initial questions and to generate the question set that we currently have.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gifts will be provided for participation.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy. If the collection requires a system of records notice (SORN) or privacy impact assessment (PIA), those should be cited and described here.

We will not collect PII. The web survey tool is entirely anonymous, with no PII requested beyond the survey respondent’s location. The location (once verified) is shared as coordinates only with no personal attributable data. A notice of confidentiality is included, alongside a Frequently Asked Questions (FAQs) button. These are clickable and clearly displayed above the survey in a box titled ‘Privacy’ that includes information about privacy. An example is included in the following text: “This tool relies fundamentally on the good will of internet users like you, and we appreciate and respect the valuable data you provide. We keep all information private and secure. Any personal information (such as address for coordinate look-up) is automatically processed and summarized (devoid of any personal information) in our final products such as the zip code map, for example, which only shows approximate locations of tornado touchdowns.

Any other ‘personal information’, we are referring to, includes voluntarily provided (without requesting) in the final question that asks respondents the following: ‘What other information would you like us to know about this event? (This can be anything you want to share about your experience with this tornado)’. We are not asking for PII, however, a person may offer it anyway. Data from this is included in a separate data base, with the intention of stripping out any PII before usage in further research.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

We will not collect sensitive data.

12. Provide estimates of the hour burden of the collection of information.

The number of respondents and frequency of response are hard to quantify. This is because the data collection is based on voluntary contribution of participants after a tornado has occurred. As the frequency and magnitude of tornadoes vary, the number of people impacted also fluctuate. The average wage rate (in this case taken to be the total private non-farm payroll) amount of $30.17 (data taken from the US Bureau of Labor Statistics - https://www.bls.gov/news.release/empsit.t19.htm ) was used to calculate the hourly wage rate for respondents.

Information Collection	Type of Respondent (e.g., Occupational Title)	# of Respondents/year (a)	Annual # of Responses / Respondent (b)	Total # of Annual Responses (c) = (a) x (b)	Burden Hrs / Response (d)	Total Annual Burden Hrs (e) = (c) x (d)	Hourly Wage Rate (for Type of Respondent) (f)	Total Annual Wage Burden Costs (g) = (e) x (f)
Tornado Watch/Warning Post Event Evaluation		1500	1	1500	6 minutes	150	30.17	4525.50
1. NSSL web tool respondent costs	Any individual over the age of 18 that has experienced a tornado event	1100	1	1100	hrs (6 mins)	110	For private nonfarm payrolls= $30.17.	$3318. 70
2. NWS Damage Assessment Tool respondent costs	Any individual over the age of 18 who has experienced a tornado event	400	1	400	hrs (6 mins)	40	$30.17	$1206.80
Totals				1500		150		$4525.50

*Cite official source of hourly wage rate - The estimate of average burden per response is based on expert review of proposed questions. BLS’s Occupational Outlook Handbook - a good wage source. https://www.bls.gov/bls/blswage.htm

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

There are no costs to the respondents as this is an online or in person survey.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

Cost Descriptions	Grade/Step	Loaded Salary /Cost	% of Effort	Fringe (if Applicable)	Total Cost to Government
Federal Oversight
Web Development costs	Web Developer	$47,330	30		$14,199
Extraction and analysis of data sets and report writing, covered by CIMMS salary as the VORTEX-SE social sciences coordinator.	Research Scientist	$63,788	10		$6,378.80
NWS Forecaster Survey Delivery Time	GS 14/15	$226,720	2.9		$6,574.88
Contractor Cost		N/A	N/A	N/A	N/A
Travel					$0
Other Costs:					$0
TOTAL					$27, 152. 68

15. Explain the reasons for any program changes or adjustments reported in ROCIS.

This is a new collection of information.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Findings from the Tornado Touchdown App and NWS DAT interviews will be collated and reported annually. The report will include basic summary statistics of each response item, with some simple analysis for different socio-demographic characteristics (when data available) and geographic area, including NWS County Warning Areas and zip codes. Data collection is estimated to begin in July 2021 and continue indefinitely, with annual reporting in July of each year beginning July 2022. The reports will be made available to external audiences with data, in de-identified form, available by request.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency plans to display the expiration date for OMB approval of the collection on all instruments.

18. Explain each exception to the certification statement identified in “Certification for Paperwork Reduction Act Submissions."

The agency certifies compliance with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).

3