0648-HEAT SUPPORTING STATEMENT Part B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

NWS Extreme Heat Social and Behavioral Sciences Research

OMB Control No. 0648-XXXX

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Focus groups:

The respondent universe for the focus groups will be screened and recruited to include professionals who have a role in messaging and assisting with behavior change to improve extreme heat outcomes. They will be recruited from a mix of public, nonprofit, corporate, and academic organizations. A diversity of respondents will be recruited to address potential variation in region of the country and population served. Six 90-minute focus groups will be conducted with six participants each for a total of 48 participants. The focus group is not a sampling of respondents from the respondent. It does not entail snowball sampling. The Focus Group is a non-random, purposive sampling.

The known respondent universe is varied and numbers in the thousands. Some examples include e.g., approximately six cities with heat officers, approximately four large public relations firms that work on public service messaging, thousands of nonprofits and community-based organizations that support frontline community members and approximately six universities with academics significantly involved in extreme heat research.

The focus groups are a qualitative methodology and therefore will not be used to produce statistical descriptions or make representative statements. There has not been a prior collection of this nature.

Survey:

The respondent universe for this collection of survey information will be members of the Dynata panel, an online non-probability panel with over 2 million members that offers representative sampling. Within the sampling universe, respondents will be randomly selected to collect 1,000 nationally distributed completed surveys with representation of those most affected by extreme heat in urban, suburban, and rural areas. We expect to screen 1,300 respondents for a total of 1,000 completes.

Dynata panel members are recruited to participate in the survey by invitation. Panel members are assigned a unique ID which is used to maintain profile information, administer screeners, and track participation in surveys. To monitor data quality, Dynata performs quality checks such as participation limits, screening questions, digital fingerprinting, random and illogical response checks, and removing “flatliners” and “speeders.”

The survey data will be weighted to reflect national Census statistics for age, gender, race and ethnicity, and any other pertinent demographic variables.

2. Describe the procedures for the collection of information including:

Focus Group

1. Statistical methodology for stratification and sample selection:

No statistical sampling methods are used for selection of the focus groups. Participants will be screened and recruited from networks of practitioners who support historically underserved populations disproportionately impacted by heat, government officials who have led extreme heat projects, academics who study extreme heat messaging and public messaging experts, based on a database maintained by contractor Climate Resilience Consulting.

As described above, respondents will be recruited who represent the regional variety in heat events and population served. Six 90-minute focus groups will be conducted with six to eight participants each for up to a total of 48 participants.

Prior to group discussion, consent forms will be obtained from all participants. Prospective participants will receive an initial email from the National Weather Service, followed by an invitation and scheduling emails from the contractor.

2. Estimation procedure:

The remote focus groups will last 90-minutes and will be conducted via Microsoft Teams. The groups will be led by trained moderators following the focus group moderator guide, and will be audio and video recorded to ensure a verbatim record is captured. A note taker will also take detailed notes in the event the recorder fails.

3. Degree of accuracy needed for the purpose described in the justification:

The focus groups are a qualitative methodology and therefore will not be used to produce statistical descriptions or make representative statements. The questions asked will be the same across all focus groups to help analyze the qualitative information, using the focus group moderator guide.

4. Unusual problems requiring specialized sampling procedures:

No specialized sampling procedures are required.

5. Any use of periodic (less frequent than annual) data collection cycles to reduce burden:

Each focus group will meet only once. There has not been a prior collection of this nature.

Survey

1. Statistical methodology for stratification and sample selection,

The survey will be conducted by a contractor using an online non-probability panel. The Dynata panel is a national opt-in panel consisting of a general U.S. population sample with geographic and demographic characteristics targeted to match those of Census estimates of the general population. While the panel itself is representative of the U.S. population, weighting will be used in the event that the final sample does not match Census estimates. There will be no stratification of the sample for design and analysis purposes. Use of the non-probability panel is designed to be a low-burden fit-for-purpose methodology to get quick turnaround feedback on weather messaging from the general population.. Panel members will be invited to participate in the survey by the panel provider to achieve a Census representative sample. The panel provider will be given broad criteria for who to sample based on race and ethnicity, age, and urban/suburban/rural status. There will be no formal stratification of the sample. If needed, the final dataset will be weighted to match Census estimates of the general population. To participate in the survey, respondents will click on a participation link that will be provided either through a direct email from Dynata or on their panel dashboard.

2. Estimation procedure,

As described above, we estimate that we will need to contact 1,300 panel members to achieve 1,000 completed surveys, for an estimated survey response rate of 77%.

3. Degree of accuracy needed for the purpose described in the justification,

A sample size of 1,000 will allow for large enough samples of subgroups by key demographics including race, age, and urban/rural status. With a sample size of 1,000, using a simple random sample, we expect a maximum margin of error of 3.1% with 95% confidence.

4. Unusual problems requiring specialized sampling procedures, and

We do not anticipate any problems that would require specialized sampling procedures. Additional sample will be contacted, if needed, in order to reach 1,000 completes.

5. Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The survey will be conducted only once and there will be no periodic data collection. There has not been a prior collection of this nature.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Focus Groups:

Since focus groups are a qualitative methodology, response rate and non-response are not relevant issues for this data collection. To ensure adequate response, we will invite up to 40% more focus group participants than we expect to participate. Participants will receive $75 for their time.

Survey:

We plan to take the following measures to maximize response to the survey:

• The respondent universe for this collection of survey information will be members of the Dynata panel, an online non-probability panel with over 2 million members that offers representative sampling. Dynata panel members are recruited to participate in the survey by invitation.

• Provide respondents with a small token of gratitude of $2 for their participation in keeping with the panel provider’s program, which is designed to create a fair value exchange for the time and effort members spend providing their opinions;

• Optimize the web survey for completion by mobile device in order to reduce respondent burden;

• Keep the length of the survey to an average of 15 minutes;

• Extend the fielding period past seven days if adequate response has not been achieved.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Review of all instruments (focus group protocol and survey questionnaire) will be conducted by internal staff at NWS and the contractors, as well as external advisors to the project who are academic experts in heat resilience messaging. The survey has been cognitive tested with nine participants to ensure it does not exceed the expected duration and that respondents can answer questions in the way they are intended.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Kimberly McMahon, kimberly.mcmahon@noaa.gov, 301-427-9692
Danielle Nagele, danielle.nagele@noaa.gov, 301-427-6919
Mary Mullusky, mary.mullusky@noaa.gov, 301-427-9372

CRC/Abt Team

Joyce Coffee, joyce@climateresilienceconsulting.com
Karen Stein, karen_stein@abtassoc.com
Alexis St. Juliana, alexis_stjuliana@abtassoc.com
Allison Ackermann, allison_ackermann@abtassoc.com
Kate Glabo, kate@kimlundgrenassociates.com
T. Jonathan Lee, t.jonathan@climateresilienceconsulting.com

3