1028-0000 SS-A Water Resources_20220503-dcdhR20220509

Supporting Statement A

Water Resources Management – Institutional Resiliency, Hazards Planning, and Data Delivery Needs

OMB Control Number 1028-NEW

Terms of Clearance: None

Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection.

Water information is fundamental to national and local economic well-being, protection of life and property, and effective management of the Nation’s water resources. The USGS works with partners to monitor, assess, conduct targeted research, and deliver information on a wide range of water resources and conditions including streamflow, groundwater, water quality, and water use and availability. This information collection will provide information to the USGS Water Resources Mission Area that will allow for understanding the resiliency of water management institutions and how best to deliver data to water data users. The Organic Act of March 3, 1879, authorizes the USGS to conduct this research and Section 9 of the Secure Water Act directs the Secretary of Interior to consult with the USGS and ensure that strategies are developed to address potential water shortages, conflicts, and other impacts to water users and the environment of each service area – this information collection supports that work.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Be specific. If this collection is a form or a questionnaire, every question needs to be justified.

The information will be used by the USGS – Water Resources Mission Area to,

1) develop metrics of institutional resiliency in water resource management institutions; and 2) develop an understanding of data delivery needs of users of water data and information to support the development of data delivery and decision support tools.

Four questionnaires will be used to collect information in support of the two goals listed above.

1. Institutional Resiliency pre-interview questionnaire:

The institutional resiliency pre-interview survey will allow us to establish the type of decisions the respondent makes in order to identify candidates for interviews. For each institution type, we plan to interview a total of at least three people: one individual whose primary decision-making responsibilities can be described as operational, one whose primary decision-making can be described as strategic, and one whose primary decision-making can be described as regulatory. By stratifying our sample pool in this manner, we believe we will be able to draw more meaningful conclusions because we expect that answers to the interview questions will vary more between different decision types than between different institutions. This pre-interview questionnaire will allow us to identify the type of decision-maker the survey respondent is.

Water Resource Management Decision-Making – Pre-Interview Survey

Objective of Q1 and Q2: These responses will provide information about the organization the respondent works for and their role in that organization. This will help us identify if the respondent is a good candidate for an interview.

Q1 Please tell us a little about yourself.

• Name ________________________________________________

• Job title ________________________________________________

• Name of organization ________________________________________________

• Email address ________________________________________________

Q2 In your own words, briefly describe your roles and responsibilities within your organization. 

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Objective Q3: The responses to question 3 will allow us to categorize the respondent as an operational, strategic, or regulatory decision-maker.

Q3 Indicate your level of agreement with the following statements. 

The decisions I make most frequently in the normal course of my job...

	Strongly disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Strongly agree
Have outcomes that can be observed immediately
Are in response to data collected for the purpose of understanding the current state of the hydrologic system
Consider long-term trends or forecasts
Impact an individual or group's legal or negotiated access to water
Rely on metrics or thresholds that have been pre-determined
Require rapid communication to the public to inform them or changes to current water quality or quantity

Involve responding to acute hazard events impacting water supply or urgent threats to public safety
Can be described as proactive evaluations of system processes over the long-term to ensure long-term sustainability
Can be described as actions based law or treaty
Can be described as adjustments or actions due to short-term observed system changes (i.e., normal system fluctuations or unexpected events).

Objective Q4: This question allows the respondent to provide us with any information about their decision-making process that they feel was not communicated thus far from their responses to previous questions.

Q4 Are there other decisions or factors that influence your water resource management decisions that you would like to share?

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

Objective Q5: Question five allows the survey respondent to self-identify as someone that would like to participate in an interview.

Q5 Would you be willing to participate in a 60-minute interview on the topic of water resource management?

• No

• Maybe

• Yes

2. Institutional Resiliency Interview Question Guide

We will interview individuals within a range of water management institutions (federal, state, local, and Tribal) to better understand: 1) how individuals responsible for these different decision types characterize resilience for their immediate institution and their larger hydrologic system, 2) how manager definitions of resilience confirm or diverge from our a priori resilience framework of responding, monitoring, anticipating, and learning, and 3) how constraints and flexibilities within each of these decision categories reduce or contribute to institutional resilience as both we and the interviewee have defined it. The data gathered from the interviews will be used to develop metrics of institutional resiliency by which water resource institutions can be assessed.

Set 1 – Background and Context

Objective: Gain a better understanding of interviewee background and perspective on system in which they operate, in order to uncover tacit knowledge they may bring to their decision-making. Explore immediate (organizational/brick-and-mortar) and broader (ecological/socio-cultural) water management contexts through conceptual modeling exercise.

1.1 Tell me a little about your position with [organization] and how you came to work here.

Probe: Have you worked in other water management contexts (different sectors, regions, or roles) prior to your job here?

1.2 In your own words, can you describe [organization’s] essential functions when it comes to water management?

Probe: Who are your primary constituents?

Probe: What is your organization’s jurisdiction (geographic or sectoral)?

1.3. Tell me a bit about how things run in [organization] on a daily basis, throughout the year.

Probe: Are there cyclical/seasonal changes in the system (ecological, climatological, political, etc.) that are known and expected to impact how [organization] operates?

Probe: How do those daily operations fit into the organization’s essential functions?

Probe: How are daily operations designed to achieve organization’s long-term goals?

For question 1.4 an online whiteboard will be used (see figure on following page) to walk participant through an exercise visualizing the different layers of the technical, socio-cultural and ecological system in which they and their organization work. After completing exercise, participant’s preferred language can be used throughout remainder of interview to refer to different scales or sectors of the comprehensive system they develop.

1.4 What other water management actors, institutions or socio-cultural structures (state or federal regulations, collaborative decision-making bodies, private utilities, etc.) play a role in managing water resources for the system in which you work?

Probe: How do those other actors impact the organizational mission of [organization] and your work as a water manager?

Probe: Are there any water management actors, institutions, or socio-cultural structures that support your water management decisions?

Figure 1: Organizational template to be used in question 1.4

Set 2 – Resilience

Objective: Explore how interviewee defines and operationalizes resilience for different levels of the system in which they operate (both organizational/brick-and-mortar and ecological/socio-cultural as described in diagramming exercise).

Brief intro for interviewer to provide to interviewee: We have now talked a little bit about both the specific function of [organization] and the larger ecological and socio-cultural systems in which [organization] operates. I am now going to ask you some questions related to the idea of resiliency and would like you to answer each (if possible) in relation to both scales. Does that make sense?

2.1 What does resilience mean to you, when thinking about both X [organization] and the ecological/socio-cultural system in which [organization] operates?

2.2 Describe the range of conditions you would call “normal or baseline” for the purposes of your work and/or your organization’s essential functions.

Probe: Is there a specific equilibrium point (i.e., supply=demand, input=withdraw, etc.) you are aimed at achieving, or is there a range of acceptable fluctuation?

Probe: How do you know when conditions have shifted outside that range or beyond that point?

2.3 Based on your definition, what are some of the ways you think both [organization] and the larger ecological/socio-cultural system successfully achieve resilience?

Probe: Is there anything missing (i.e., processes, relationships, information, modifications to physical infrastructure, etc.) that you believe would increase resilience?

2.4 From your perspective, are there aspects of either [organization] or the larger ecological/socio-cultural system currently hindering achievement of resilience?

Probe: <Gauge interviewee comfort level for this topic before asking follow-up probe.> What would it take, from your perspective, to change those?

Set 3 – Decision Making

Objective: Understand the constraints and flexibilities for decision-making experienced or observed by the interviewee, and how those factors influence the achievement of resilience at organizational and system levels.

Brief intro for interviewer to provide to interviewee: You recently took a survey that asked you to indicate how much you agree with different statements about the kinds of decisions you make in your work. For this study, we interested in three categories of water management decisions; operational, strategic, and regulatory [short description of each – paraphrased from information in intro]. We recognize that most people make decisions in the course of their work that could fall into any of those categories, or a combination of several, but will primarily make one type of decision on a regular basis. Based on your responses we understand most of your decisions to be [operational, strategic, regulatory]. Do you agree?*

For this final set of questions, I am going to ask you about the decisions you make for your job. We would like you to think about your answers primarily in relation to your [operational, strategic, regulatory] decisions, but if there is information you would like to provide that would be more relevant to other types of decisions you make we can discuss that as well.

*If they disagree with how we characterized their decision-making, we will ask them to self-identify among the three options [operational, strategic, or regulatory] and use that identity for the remaining questions. At the end, we will ask them if there is someone else in their organization that we could talk to who might better represent the category of decision-maker we assigned them to originally.

3.1 Can you give me some examples of the kinds of decisions you typically make in the course of your work?

Probe: What are the easiest decisions you typically make? Can you explain why they are easy?

Probe: Can you tell me about a particularly difficult decision you had to make and what made it difficult?

3.2 How would you describe your typical process for making these different decisions when conditions are “normal” [remind them of how they defined “normal” earlier in interview]?

Probe: What is the role of: external input from others, monitoring data, policy or procedure, etc.

Probe: How do you overcome the challenges you face when making difficult decisions [i.e., the difficulties mentioned in previous question, or other impediments such as uncertainty, competing objectives, risk perception, etc.]

3.3 How does your decision-making process change when conditions shift to “abnormal?”

Probe: Do the factors listed above still play a role, and if so, what is it?

Probe: Do your approaches to resolving uncertainty or balancing tradeoffs change?

3.4 In what ways do the [operational, strategic, regulatory] decisions that you make contribute (or are intended to contribute) to resilience – as you defined it – for both your organization and the larger ecological/socio-cultural system?

Probe: What are the aspects of your organization or the larger ecological/socio-cultural system that enable you to make those decisions? [i.e., organizational structure, institutional culture, readily available and reliable data, supportive stakeholders, critical partnerships, etc.]

3.5 Are there [operational, strategic, regulatory] decisions that you wish you could make that you believe would contribute to resilience?

Probe: What prevents you from making those decisions [i.e., are there flip-sides to the enabling factors mentioned above that, instead, constrain decision space]?

Wrap-up

Objective: Allows the interview participant to tell us anything they have not yet had the opportunity to discuss and identify others we should speak to.

Is there anything else you would like to share about your role or decision-making as a water manager that we have not already discussed?

Is there anyone else in your organization you think we should talk to? <especially if they identified with a different decision type than we assigned to them>

3. Data Delivery Needs Survey

This survey is being conducted by the National Water Census team at the U. S. Geological Survey (USGS) [LINK TO EMAIL ADDRESS: gs-w_nwc@usgs.gov].

The National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

The purpose of this survey is to give the NWC team a better understanding of how we can best serve future users’ needs. The results from this survey will be used to guide further development of the NWC and help determine how NWC outputs should be integrated with existing USGS water data.

This survey should take 10-15 minutes to complete. Your responses will not be shared with anyone outside of the USGS NWC team, but may be anonymized, aggregated, and published in a future report or journal article. We will share the survey results with you and will be sure to get permission if we would like to include direct quotes in any way. We thank you for your time!

Section 1: Basic demographic information

Objective: Responses to the questions in section 1 will allow us to coordinate logistics for an interview or for contact info. * denotes required questions

1. *First name: __________

2. *Last name: __________

3. *Email address: _______

4. Phone number: _______

5. (Dropdown menu) *State in which you live, work, and/or are most interested in water issues

6. *Would you intend to use the National Water Census in a professional or personal capacity?

• Personal capacity (Go to Section 2)

• Professional capacity (Go to Section 1a)

• Both personal and professional capacities (Go to Section 1a)

Section 1a: Professional affiliation details

7. Organization name: ___________

8. Position: ____________________

9. Select the type of organization you are professionally affiliated with.

• Congress

• Federal agency

• State agency

• City, town, or township agency

• County agency

• Tribe

• University

• NGO

• Private / industry

• Other (Includes political subdivisions not otherwise categorized, including water authorities; irrigation, flood control, drainage, and other districts; etc.): _________

10. Select the type of professional role that mostly closely match yours.

• Scientist/researcher

• Natural resource manager

• Policy maker/advisor

• Regulator

• Advocate

• Consultant

• Lobbyist

• Landowner

• Business owner

• Press/reporter

• Other: ________________

11. Select the sector that you are professionally associated with.

• Water resources

• Water utility

• Manufacturing

• Energy/minerals

• Agriculture

• Forestry

• Fishing

• Disaster/emergency management

• Land development/construction

• Environment/conservation

• Climate change adaptation

• Transportation

• Health

• Tourism/recreation

• Other: _________

12. Are you employed by USGS?

• Yes

  • Select the area(s) you are part of.

    • Internal – Mission Area

      • [TYPE THE Mission Area]: __________

    • Internal – Science Center

      • [TYPE THE SCIENCE CENTER]: __________

    • Internal – Region

      • [TYPE THE REGION]: _________

• No

Section 2: Current USGS water data usage

Objective Q1: Responses to the questions in this section will allow us to better understand data and information delivery mechanisms that have worked well for users in the past and to catalog favorite and least favorite examples of data delivery mechanisms.

1. (Multiple answer) Select all the way(s) in which you have accessed or viewed USGS water data in the past year.

• National Water Dashboard

• Monitoring Location Pages (example)

• NWIS mapper

• Water Quality Watch

• Water Watch

• Groundwater Watch

• Water Quality Portal

• National Groundwater Monitoring Network

• WaterAlert

• USGS Water Services

• ulmo (using Python)

• dataRetrieval (using R)

• Email a specific USGS team for data extraction

• Other (type out separated by commas): _______

• I did not access or view USGS water data in this past year but have before. (Skip to #3)

• I have never accessed or viewed USGS water data but would like to in the future. (Skip #3)

Objective Q2: To better understand user workflows and the necessary characteristics of useful datasets.

2. Approximately how often have you accessed or viewed USGS water data in the past year?

• Everyday

• A few times a week

• About once a week

• A few times a month

• About once a month

• A few times a year

• About once a year

Objective Q3: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

3. (Multiple answer) Select the typical purpose(s) you have for using or wanting to use USGS water data.

• Policymaking

• Managing environmental resources

• Environmental advocacy

• Emergency management

• Long-term planning

• Real-time decision-making

• Exploratory science

• Research (any organizational category – e.g., academic, government, private consulting, non-profit, etc.)

• Teaching – K-12

• Teaching – Higher Education

• Communicating science to the general public

• For personal or family safety

• For recreational activities / sports (boating, fishing, swimming, etc.)

• To understand local water issues (drinking water, flooding, pollution, etc.)

• For local environmental resource use (lake associations, hunt clubs, home-owners' associations, etc.)

• For fun / out of personal curiosity

• Other: _______

Section 3: How you would use NWC

Objective Q1: To better understand user workflows and the necessary characteristics of useful datasets.

1. *Select the ways in which you would anticipate interacting with modeled water supply, demand, and availability outputs that are part of the National Water Census.

• Obtaining and analyzing raw model output data (Go to Section 3a)

• Viewing web-based summary or status visualizations, fact sheets, and/or reports based on model outputs (Go to Section 3b)

• Both of the above (Go to Section 3a)

Section 3a: Which NWC outputs are of interest to operationalized pull users, if any

Objective Q1: To better understand user workflows and the necessary characteristics of useful datasets.

1. * (Multiple answer) The National Water Census is slated to serve the following new modeled (as opposed to measured) variables and computed trends. Select all that would be useful to you in either your work or for personal use.

Variables related to water quantity and flow:

• Modeled evapotranspiration, or ET (canopy evaporation, surface evaporation, and transpiration combined)

• Modeled runoff

• Modeled streamflow

• Modeled canopy water storage (liquid and frozen)

• Modeled groundwater recharge

• Modeled groundwater storage depth (from bedrock up)

• Modeled long-term average baseflow

• Modeled long-term average depth to water table

• Modeled snow depth, snow-water-equivalent, snow-pack, albedo, and temperature

• Modeled snowmelt

• Modeled soil frozen water content

• Modeled soil moisture

• Modeled soil temperature

• Calculated historical trends in measured streamflow (from USGS stream gages)

Variables related to water quality:

• Modeled groundwater quality concentration

  • Nitrogen

  • Phosphorus

  • Salinity

  • Other:_______

• Modeled surface water quality concentration or loads

  • Nitrogen

  • Phosphorus

  • Salinity

  • Wastewater percentage

  • Concentration of emerging contaminants of concern (consumer products, pharmaceuticals)

  • Sediment

  • Temperature

  • Other: ______

• Calculated surface water quality concentration, loads, and/or historical trends (based on monitoring locations)

  • Nitrogen

  • Phosphorus

  • Salinity

  • Wastewater percentage

  • Concentration of emerging contaminants of concern (consumer products, pharmaceuticals)

  • Sediment

  • Temperature

  • Other: ______

Variables related to water use, supply, and demand:

• Modeled water withdrawal (irrigation)

• Modeled water withdrawal (thermoelectric)

• Modeled water withdrawal (public supply)

• Modeled water consumption (irrigation)

• Modeled water consumption (thermoelectric)

• Predicted modeled reservoir storage

• Calculated historical trends in measured water storage

• Modeled total water demand

• Modeled total water supply

• Modeled water availability (calculated as supply minus demand)

Variables related to aquatic ecosystems:

• Modeled likelihood of hydrologic modification exceeding ecological threshold for fish (ecoflows)

• Ecoflows statistics (selected statistics relevant to the ecological threshold analysis)

None of the above:

• None of the above

If respondent selects “None of the above,” go to Section 4b. If respondent selects anything other than “None of the above,” go to Section 4a.

Section 3b: Which NWC outputs are of interest to check status users, if any

Objective Q1: To better understand user workflows and the necessary characteristics of useful datasets.

1. *(Multiple answer) The National Water Census is slated to serve the following new modeled (as opposed to measured) variables and computed trends. Select all that would be useful to you in either your work or for personal use.

• Modeled variables and computed trends related to water quantity and flow. (Examples include modeled streamflow, evapotranspiration, precipitation, soil moisture, snow-water-equivalent, and groundwater recharge, as well as historical trends in measured streamflow.)

• Modeled variables and computed trends related to water quality. (Examples include modeled groundwater quality and stream temperature, as well as historical trends in surface water quality load.)

• Modeled variables and computed trends related to water use, supply, and demand. (Examples include modeled water consumption for irrigation / thermoelectric purposes, withdrawal for public supply / irrigation / thermoelectric purposes, and total supply / demand, as well as historical trends in measured water storage.)

• Modeled variables and computed trends related to aquatic ecosystems. (Examples include modeled likelihood of hydrologic modification exceeding ecological thresholds for fish and the statistics relevant to this analysis.)

• None of the above

If respondent selects “None of the above,” go to Section 4b. If respondent selects anything other than “None of the above,” go to Section 4a.

Section 4a: Details on how respondent would use NWC

There are 4 possible chosen outputs: Water quantity and flow, Water quality, Water use / supply / demand, Aquatic ecosystems. Thus, the respondent will go through this section a max of 4 times.

Objective Q1: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

1. (Multiple answer) For model output related to [EACH CHOSEN OUTPUT], select all the ways in which you would use this information.

• Policy making

• Managing environmental resources

• Environmental advocacy

• Emergency management

• Long-term planning

• Real-time decision-making (using data from the last weeks, days, or hours)

• Exploratory science

• Research (any organizational category – e.g., academic, government, private consulting, non-profit, etc.)

• Teaching – K-12

• Teaching – Higher Education

• Communicating science to the general public

• For personal or family safety

• For recreational activities / sports (boating, fishing, swimming, etc.)

• To understand local water issues (drinking water, flooding, pollution, etc.)

• For local environmental resource use (lake associations, hunt clubs, home-owners' associations, etc.)

• For fun / out of personal curiosity

• Other: _______

Objective Q2-Q7: To better understand user workflows and the necessary characteristics of useful datasets.

2. (Multiple answer) For model output related to [EACH CHOSEN OUTPUT], what model scenarios are you interested in? Select all that apply.

• Historical (previous ~40 years)

• Recent time period (last month, last week, or yesterday)

• Real-time or current conditions

• Short-term forecasts (next few hours to ~2 days out)

• Medium-term forecasts (~2-10 days out)

• Subseasonal forecasts (~10 days to 1 month out)

• Seasonal forecasts (~1-3 months out)

• Long-term climate change-driven projections (50-100 years out)

• Testing what would happen if something changed or some action were applied

3. For model output related to [EACH OUTPUT CHOSEN], what would be the ideal update frequency?

• Sub-daily

• Daily

• Weekly

• Monthly

• Annually

• Other: _______

4. For model output related to [EACH OUTPUT CHOSEN], with what regularity would you visit the delivery system or use code / web services to obtain these data?

• Daily

• Weekly

• Monthly

• Annually

• As needed when there’s an emergency

• Other: ____

5. (Multiple answer) For model output related to [EACH OUTPUT CHOSEN], what spatial scale(s) or resolution(s) would be most appropriate for your analyses and workflows?

• CONUS-wide / national

• HUC6

• HUC8

• HUC12

• NHD+ medium resolution catchments

• NHD+ medium resolution river reaches

• Utility service areas

• Water districts

• Major aquifer extents

• Ecoregions

• Physiographic provinces

• County

• City

• State

• Other: _____

6. (Multiple answer) For model output related to [EACH CHOSEN OUTPUT], how would you prefer to access, visualize, and/or download the data? Select all that apply.

• Plotted on a static map

• Plotted on an interactive (clickable and zoomable) map

• Important features of the data interpreted and summarized in a factsheet

• Important features of the data interpreted and summarized in a StoryMap

• Important features of the data interpreted and summarized in a longer report

• Download via dataRetrieval in R

• Download via ulmo in Python

• Download via USGS Water Services

• Download via standalone NWC-specific API (not USGS Water Services)

• Download by clicking through an interactive map

• Download by clicking through a standalone NWC data portal

• Download by clicking through another existing USGS data portal (e.g., Model Catalog, ScienceBase)

• Other: ________

7. If the output were available on the appropriate spatial and temporal resolutions, over the needed spatial and temporal extents, and downloadable or viewable via your preferred method, how likely would it be for you to incorporate model output related to [EACH CHOSEN OUTPUT] into your current workflows?

• Highly unlikely (no chance)

• Unlikely

• Neither likely nor unlikely

• Likely

• Highly likely

Section 4b: Understanding why the respondent would not use any listed NWC outputs and what they would want instead

Objective Q1-Q3: To better understand user workflows and the necessary characteristics of useful datasets.

1. Are you interested in modeled outputs other than those listed above as part of the National Water Census?

• Yes

• No

2. If yes: List the modeled outputs and / or variables you are interested in (separated by commas).

3. If no: What would make you more interested in modeled outputs?

Section 5: Conclusion

Objective: To coordinate logistics for an interview or for contact info.

1. Would you be willing to participate in a virtual interview to share more information and ideas? The interview will last ~1 hour and help us learn more about your thoughts on the NWC and potential needs of yours that the NWC could fulfill. If yes, we will follow up with you via email to schedule.

• Yes

• No

2. Is there anything else you would like us to know?

Thank you so much for your participation! If you have any questions for us, please email us at gs-w_nwc@usgs.gov. We will be following up with you via email shortly if you signed up for a virtual workshop. Otherwise, we will let you know when the results of this survey are being published or presented.

4. Data Delivery Needs Interview

Interviews will be conducted with samples of two different user populations. Those that fall into the “operationalized pull” category and those that fall into the “check status” category. Operationalized pull refers to data users who pull data from multiple sources, including USGS Application Programming Interface (API) services, to use via custom dashboards and tools optimized for their location and needs. Operationalized Pull users, on average, use the most USGS water information, returning repeatedly for refreshed data to pull into their own systems. Operationalized pull users include individuals that support decision-makers as opposed to those making decisions as well as water users. Check status refers to users that perform routine checks of a few parameters for specific sites, primarily using the hydrographs. Check Status users are our largest user type by number of unique users. Each user generally looks at a handful of sites for the latest water conditions. Check status users include decision-makers and non-profits. 

Operationalized pull interview questions

Italicized text is intended to be read to the interview participants. Questions are bullet pointed, with specific probes or follow up questions further bulleted. Additional instructions are [bracketed]. The objective of each set of questions is italicized and underlined and provided before the bulleted questions.

Thank you for agreeing to participate in this interview. I’d like to start by learning more about your typical day and workflows. 

Objective: To better understand user workflows and the necessary characteristics of useful datasets and how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Can you give me a quick overview of a typical day at your job?

  • What tasks do you do?

  • What decisions do you make?

  • Who do you interact with?

  • Does this vary from season to season, month to month, or year to year?

  • What kind of outputs / deliverables do you generate in your work?

    • Who do you have to show or deliver these to?

  • What parameters or variables are most important to you in your work?

  

[The following are water quantity user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• If you develop projections or plans for water supply, how far into the future do you forecast?

• What are the primary water quality concerns that are incorporated into your water supply planning?

• How would you rate your ability to predict the future requirements for water supply in the short term (over the next year)?

  • How important is this ability?

• How would you rate your ability to predict the future requirements for water supply in the long term (over the next 20 years)?

  • How important is this ability?

[The following are water quality user specific questions]

• Share the top 3-5 water quality parameters you believe can be modeled well.

• What are the primary water quality concerns that are incorporated into your water supply planning?

• What is the best way to present data for those in your organization who work with watershed stakeholders like landowners who implement BMPs?

[The following are water use/demand user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the community/ agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• How do you define and use the terms water supply, demand, and availability?

• How do law and compacts restrict the type of work you do or the solutions that you propose for certain problems?

• What is your preferred way(s) to have water demand information organized for queries?

  • Sum of all the use in a certain area, separate surface water and groundwater total use in an area, each water use category separate [irrigation, hydro-electric, public supply, etc.]?

[The following are aquatic ecosystems user specific questions]

• Please describe the primary ways you or your organization seeks to manage for ecoflow [environmental flow] needs of the aquatic community.

• What is your process for calculating a set of key accepted ecoflow statistics that are used in regulations or general planning?

• What would the benefit be to having nationally consistent ecoflow statics available on a regular basis?

• When determining ecoflow thresholds for decision-making related to waterways with various species, how do you synthesize all the various biologic needs?

• What are the key aquatic communities that are used within your regulation/permitting/management processes?

• Would you use ecoflows data on a national or regional scale, or will you always be seeking local information?

We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.

Objective: To better understand 1) user workflows and the necessary characteristics of useful datasets; 2) how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job; 3) data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples; 4) To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• Tell me more about your computational or analytical workflow doing <data-related task described in answers to the above section>.

  • What are all the variables, datasets, and types of data needed to do this work?

  • Do you use measured data in this work?

    • If yes: If you had modeled data for the same variable, how would that impact your work?

  • Do you already use modeled data in this work?

    • If yes: If you had measured data for the same variable, how would that impact your work?

  • Do you pair or combine measured data (sensor, gage, sample, observation) with modeled information in this work?

    • If yes: Tell me about that. What would make this process easier?

    • If yes: Does it concern you if or when measured and modeled data conflict with each other? How would you deal with that if so?

  • What are (or would be) the trade-offs between 1.) having modeled data available at a low spatial resolution throughout your study area and 2.) having measured data at points scattered throughout your study area?

  • What are other limitations of these datasets you have used that we haven’t yet touched on?

  • Are you interested in model predictions of yesterday's or last week's hydrological conditions at locations where we are not collecting observational data?

    • If yes: Why?

• Describe the uncertainty information associated with the measured or modeled data you currently access and incorporate into your workflows.

  • How do you use that uncertainty info?

  • If no answer: What kind of uncertainty information do you wish was available? How would you use it if it was?

• Describe how you currently go between different websites, data portals, or APIs to download all the necessary datasets to accomplish your analytical workflow.

  • What are some of the challenges you face when gathering and integrating these disparate datasets which may have different spatial and temporal scales?

  • What would make this process easier?

  • What are your favorite and least favorite websites / data portals / APIs and why?

  • In general, do you prefer to directly query an API, use a clickable data portal, or go through a software package to download data for your work?

  • [For water quantity user] Would you be interested in accessing a water budget model in the Cloud which you could run for a selected area?

We would now like to show you a list of the new modeled datasets that the National Water Census will serve, which you already saw once in the online survey. As a reminder, the National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Looking at this list of model outputs [show them the list they’ve already seen on the online survey via screenshare], which would you incorporate into your workflows? As a reminder, these were the outputs you selected when you took the online survey [show them their survey answers via screenshare]. Are these outputs still of primary importance to your work?

[For water quantity user]

• Would or how would you use subcomponents of more integrated variables from water budget process models? For example: multiple flow components going into runoff, six different storage components that are part of soil moisture, or multiple components representing snow and ice. [Show this doc for all possible components via screenshare.]

-----------------------------------------------------------

Now we are going to show maps of some of the model outputs you said you would be interested in to demonstrate data types, extent, resolution, and patterns of these outputs across large areas of the country. We will ask the same questions for each set of model output maps we will show, which we are sharing in the chat. [Share simplified version of questions in this section (without prompts) in the chat. Show model output maps via screenshare. For all the questions in this section, refer to the maps shown via screen share, corresponding to one of the four following data types: water quantity-related, water quality-related, water use-related, aquatic ecosystems-related. Iterate through the following section for each data type the respondent is interested in.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Data display:

• Would you rather this [INSERT one of the four data types above] information be presented as raw values, in the context of historical values / as historical quantiles, or in relation to a relevant threshold?

  • If a threshold, which threshold?

    • [For water quality user] Exceedance of a maximum contaminant level?

• [For water quality user] What is your preferred way to have water quality levels reported?

  • Concentration, load, or benchmark exceedance?

• [For salinity] How do you typically measure salinity or prefer to have salinity reported?

  • Specific conductivity, total dissolved solids, chloride concentration, actual salinity?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

Spatial and temporal resolutions and extents:

• What would be an ideal time period over which to have this [INSERT one of the four data types above] information?

• What would be an ideal temporal resolution for this [INSERT one of the four data types above] information?

• What would be an ideal spatial scale or extent over which to have this [INSERT one of the four data types above] information?

• What would be an ideal spatial resolution for this [INSERT one of the four data types above] information?

  • Examples include public utility service areas, NHDPlus Catchments, HUC12, HUC8, HUC6, states, major regional watersheds, raster dataset.

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

Data format:

• What format would you want downloaded data for this [INSERT one of the four data types above] information to be in?

  • Tabular, csv, netcdf, json via API, other?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

Role of data in workflows:

• If the calculations and inputs were clear and transparent, and available at favorable temporal and spatial scales, how would you incorporate this [INSERT one of the four data types above] information into your workflows?

• What are the impacts that these data would have on your job?

• What decisions or deliverables would you be able to make using this data?

• How accurate do these model outputs have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need hydrologic drought category projection or do you need streamflow in cfs within minimal confidence interval to apply your own interpretations?

• What other modeled outputs not on the above list would be useful to you?

  • How would you use those?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Supplementary information and uncertainty:

• What documentation, summary information, or visuals about this model output would you like to see before you download and use the data?

  • Examples include: Size, format, spatial resolution, temporal resolution, spatial extent, temporal extent

• If presented with model diagnostic or performance plots, how would you use that information? [Show example model diagnostic and performance plots associated with this particular model output, as needed or requested, via screenshare.]

• If presented with uncertainties associated with this particular model output, how would you use that information?

• Would it be useful for us to also serve model input datasets? [Show list of model inputs associated with this particular model output via screenshare.]

  • How would you use these?

• How would you use or interpret ensemble output from multiple models serving the same variable?

Visuals:

• What would be the value of a web-based visualization of the model output like the one we’re showing here to you or your colleagues?

  • If they find a web-based visualization valuable:

    • What other information or mapped GIS layers would you want to see along with or on top of this modeled output? [Show examples corresponding to those suggested by modeling teams via screenshare.]

    • What are the key features that would make you want to return to the visualization?

    • What key features would policymakers / managers in your organization or others you communicate information with most appreciate?

    • What would you change about this visualization or way of presenting the information?

Those are all of the questions that I have for you. Is there anything else you’d like to tell me about how you use USGS water data or how you might use the National Water Census that I didn’t ask about?

Extra Questions below may be substituted in based on the respondent background or used in a follow up interview the following year.

-----------------------------------------------------------

[The following questions below are chosen to be discussed depending on how the respondent answered the question above: “In general, do you prefer to directly query an API, use a clickable data portal, or go through a software package to download data for your work?” within the question section starting “We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.”]

[If the respondent answered that they prefer a clickable data portal, go through the following]

Next, we will show example data portals that may be used to serve National Water Census data or that may be similar to what a National Water Census data portal will look like. We will ask the following set of questions for each example data portal. [Send the below questions in the chat. Walk through example websites via screenshare.]

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• If NWC data were delivered on this data portal, would that be convenient / easy for you to use?

• What makes it easy or difficult to use?

Data portal #1:

NOMADS or NWC NOMADS-like prototype

Data portal #2:

Model Catalog (perhaps with an NWC badge)

Data portal #3:

ScienceBase (perhaps with an NWC badge)

[If the respondent answered that they prefer an API, go through the following]

• Is there an existing API that you already use that you think NWC data could imitate or even slot into / become part of?

[If the respondent answered that they prefer a software package, go through the following]

• What software would you prefer to use for downloading NWC data?

  • Python, R, other?

• Does it make sense for an NWC data download function to be added to dataRetrieval / ulmo / other existing packages or libraries you already use to download USGS water? Or would you prefer a standalone, NWC-specific data download package?

-----------------------------------------------------------

Next, we will show you some potential ways in which we may represent, calculate, and display water supply, demand, and water availability as part of the National Water Census. We are still figuring out exactly how we will do this, so we are interested in getting feedback from you on what would make sense and be useful. We will ask the following set of questions for each example calculation or visualization. [Send the below questions in the chat. Show example visualizations via screenshare.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them and to better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Do these calculations of water availability make sense to you?

• How would you use information on water availability calculated or presented in this way in your work?

• Are you interested in separate calculations of availability based on surface water and groundwater sources?

• How do you define and use the terms water supply, demand, and availability? How are those definitions the same or different from what we are showing?

Visualization #1:

Visualization #2:

Map of water budget showing places that are under and overbudget

• Please describe how you would interpret this representation of supply and demand.

Visualization #3:

NASA GLDAS output/water balance ESRI app: Water Balance App (arcgis.com)

Visualization #4:

Late fall 2019 water availability data viz (IWAAs concept map) | U.S. Geological Survey (usgs.gov) by Vizlab

-----------------------------------------------------------

[This is for later interviews with people we’ve already talked to before. For each of the below tasks, we will ask respondents to share their screen, which they will have been informed of ahead of time.]

We would love for you to walk us through in more detail on how you currently incorporate USGS and other data into your technical workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Could you share your screen and walk us through a typical workflow when you’re downloading and / or using USGS data?

• Could you share your screen and walk us through how you use R / Python / ArcGIS / others to compile / analyze / download USGS data?

• Could you share your screen and walk us through how you currently use USGS Water Services?

• Could you share your screen and walk us through how you currently download data via USGS data portals / websites?

-----------------------------------------------------------

[This is for later interviews regarding later version of NWC.]

As we consider the important elements to provide for users, we wonder about the utility of being able to run models yourself or having access to model code that you could work with directly.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Do you already integrate USGS modeling code or runs into your workflows?

  • If yes: Tell me about how you do that.

    • Do you run models on your local machine, in the cloud, or on a cluster / supercomputer?

Objective: To better understand data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples.

• What would make this easier to do?

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• If no: Would you want to run USGS models on your own if they were readily accessible?

  • If yes: Where would you want to do that? On your local machine, in the cloud, or on a cluster / supercomputer?

Check Status user interview questions

Italicized text is intended to be read to the interview participants. Questions are bullet pointed, with specific probes or follow up questions further bulleted. Additional instructions are [bracketed]. The objective of each set of questions is italicized and underlined and provided before the bulleted questions.

Thank you for agreeing to participate in this interview. I’d like to start by learning more about your typical day and workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets and how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Can you give me a quick overview of a typical day at your job including the things you have to do, the decisions you have to make, and the people you interact with?

  • Who do you rely on to keep you up to date on all the information you need to know?

    • What is their title?

  • Does your workflow vary from season to season, month to month, or year to year?

  • What outcomes or deliverables do you generate in your work?

    • Who do you serve by creating these outcomes?

[The following are water quantity user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• If you develop projections or plans for water supply, how far into the future do you forecast?

• What are the primary water quality concerns that are incorporated into your water supply planning?

• How would you rate your ability to predict the future requirements for water supply in the short term (over the next year)?

  • How important is this ability?

• How would you rate your ability to predict the future requirements for water supply in the long term (over the next 20 years)?

  • How important is this ability?

[The following are water quality user specific questions]

• Share the top 3-5 water quality parameters you believe can be modeled well.

• What are the primary water quality concerns that are incorporated into your water supply planning?

• What benefit would it be to have nitrogen, phosphorus, and salinity information available for all surface waters and groundwater in your region?

• What benefit would it be to know the wastewater percentage in streams in your region?

• How important is it to have stream temperature and sediment information for streams in your region?

• What is the best way to present data for those in your organization who work with watershed stakeholders like landowners who implement BMPs?

[The following are water use/demand user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the community/ agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• How do you define and use the terms water supply, demand, and availability?

• How do law and compacts restrict the type of work you do or the solutions that you propose for certain problems?

• What is your preferred way(s) to organize information about water use for decision making?

  • Sum of all the use in a certain area, separate surface water and groundwater total use in an area, each water use category separate [irrigation, hydroelectric, public supply, etc.]?

[The following are aquatic ecosystems user specific questions]

• Please describe the primary ways you or your organization seeks to manage for ecoflow [environmental flow] needs of the aquatic community.

• What is your process for calculating a set of key accepted ecoflow statistics that are used in regulations or general planning?

• What would the benefit be to having nationally consistent ecoflow statics available on a regular basis?

• When determining ecoflow thresholds for decision-making related to waterways with various species, how do you synthesize all the various biologic needs?

• What are the key aquatic communities that are used within your regulation/permitting/management processes?

• Would you use ecoflows data on a national or regional scale, or will you always be seeking local information?

We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Tell me more about your workflow <making decisions related to ___ / doing advocacy related to ___ / communicating ___information to your stakeholders>.

  • What are all the datasets and information sources (measured or modeled) needed to do this work?

  • Do you use combined measured data (sensor, gage, sample, observation) with modeled information in this work?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• If yes: Does it concern you if or when measured and modeled data conflict with each other? How would you deal with that if so?

• What are (or would be) the trade-offs between 1.) having modeled data available at a low spatial resolution throughout your study area and 2.) having measured data at points scattered throughout your study area?

• What are other limitations of these datasets you have used that we haven’t yet touched on?

• Are you interested in model predictions of yesterday's or last week's hydrological conditions at locations where we are not collecting observational data?

  • If yes: Why?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Describe how you currently use different websites, visualizations, and other resources to make decisions, advocate, or communicate about important aspects of your work.

Objective: To better understand data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples.

• What are some of the challenges you face in understanding or interpreting information from sources which may have different spatial or temporal scales?

• What would make this process easier?

• Out of the ones you named, what are your favorite and least favorite websites / visualizations / other resources and why?

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

• In general, what types of information sources do you prefer and find most useful?

  • Reports, factsheets, static maps, websites, interactive maps, StoryMaps, others? [Show examples of these as needed via screenshare.]

We would now like to show you a list of the new modeled datasets that the National Water Census will serve, which you already saw once in the online survey. As a reminder, the National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Looking at this list of model outputs [show them the list they’ve already seen on the online survey via screenshare], which would you incorporate into your workflows? As a reminder, these were the outputs you selected when you took the online survey [show them their survey answers via screenshare]. Are these outputs still of primary importance to your work?

-----------------------------------------------------------

Now we are going to show maps of some of the model outputs you said you would be interested in to demonstrate data types, extent, resolution, and patterns of these outputs across large areas of the country. We will ask the same questions for each set of model output maps we will show, which we are sharing in the chat. [Share simplified version of questions in this section (without prompts) in the chat. Show model output maps via screenshare. For all the questions in this section, refer to the maps shown via screen share, corresponding to one of the four following data types: water quantity-related, water quality-related, water use-related, aquatic ecosystems-related. Iterate through the following section for each data type the respondent is interested in.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Data display:

• Would you rather this [INSERT one of the four data types above] information be presented as raw values, in the context of historical values / as historical quantiles, or in relation to a relevant threshold?

  • If a threshold, which threshold?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• What would be an ideal temporal and spatial resolution for this information?

• What would be an ideal time period over which to have this information?

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

Data download:

• Would you be interested in downloading this data?

  • If yes: What format would you want downloaded data to be in?

    • Visualization / image, tabular, csv, other?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

Role of data in workflows:

• If the calculations and inputs were clear and transparent, how would you incorporate this information in your regular workflows?

• What decisions or deliverables would you be able to make using this data?

• How accurate does this model output have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need drought category projection or do you need streamflow in cfs within a minimal confidence interval to apply your own interpretations?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Supplementary information and uncertainty:

• What documentation, summary information, or visuals about the model output would you like to see before you use or explain the data?

  • Examples include: Size, format, spatial resolution, temporal resolution, spatial extent, temporal extent

• How would you use or interpret ensemble output or multiple models serving this same variable?

Visuals:

• What other information or mapped GIS layers would you want to see along with or on top of this modeled output? [Show examples corresponding to those suggested by modeling teams via screenshare.]

• What are the key features that would make you want to return to this visualization?

• What key features would policymakers / managers in your organization or others you communicate information with most appreciate?

• What would you change about this visualization or way of presenting the information?

-----------------------------------------------------------

Next, we will show you some potential ways in which we may represent, calculate, and display water supply, demand, and water availability as part of the National Water Census. This can be a combination of data from all four themes or map sets we just looked at. We are still figuring out exactly how we will do this, so we are interested in getting feedback from you on what would make sense and be useful. [Show example visualizations via screenshare.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

• How do you define and use the terms water supply, demand, and availability?

  • How are those definitions the same or different from what we are showing?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• How would you use information on water availability presented or calculated in this way in your work?

• What decisions or deliverables would you be able to make using this data?

• How accurate does this model output have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need drought category projection or do you need streamflow in cfs within a minimal confidence interval to apply your own interpretations?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

• Are you interested in separate calculations of availability based on surface water and groundwater sources?

• What other information or mapped GIS layers would you want to see along with or on top of this water supply, demand, and availability info? [Show examples corresponding to those suggested by modeling teams via screenshare.]

• What are the key features that would make you want to return to this visualization?

• What key features of this visualization would policymakers / managers in your organization or others you communicate information with most appreciate?

• What would you change about this visualization or way of presenting the information?

[The following sections below are chosen to be discussed depending on how the respondent answered the question above: “In general, what types of information sources do you prefer and find most useful? Reports, factsheets, static maps, interactive maps, StoryMaps, others?”]

Since you have seen some of the output data and one draft way we are considering presenting supply and demand information, we would like to know about your preferences for how this information is delivered. Next, we will show an example <report, factsheet, static map, website, interactive map, or StoryMap> that may be similar to what delivering National Water Census data would look like. [Walk through a report, factsheet, static map, interactive map, or StoryMap via screenshare.]

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

• [For report, factsheet, StoryMap] What information, maps, tables, charts, and other visualizations would you most want to see in a NWC <report, factsheet, or StoryMap>?

• [For static map, website, interactive map] What information would you most want to see in the <static map, website, or interactive map>?

• How regularly would you like to see information products updated or refreshed?

Those are all of the questions that I have for you. Is there anything else you’d like to tell me about how you use USGS water data or how you might use the National Water Census that I didn’t ask about?

Extra questions below may be substituted in based on the respondent background or used in a follow up interview the following year.

-----------------------------------------------------------

[This is for later interviews with people we’ve already talked to before. For each of the below tasks, we will ask respondents to share their screen, which they will have been informed of ahead of time.]

We would love for you to walk us through in more detail on how you currently incorporate USGS and other information into your workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Could you share your screen and walk us through a typical workflow when you’re making a decision about _____?

• Could you share your screen and walk us through a typical workflow when you’re preparing a presentation for _____ or to communicate _____?

• Could you share your screen and walk us through a typical workflow when you’re first learning about _____?

• Could you share your screen and walk us through a typical workflow when you’re searching for information about ______?

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden and specifically how this collection meets GPEA requirements.

The Institutional Resiliency pre-interview survey and Data Delivery survey will be administered electronically using the survey software Qualtrics. This will facilitate ease of administration on the part of the research team and reduce the burden of response for respondents. The Institutional Resiliency and Data Delivery interviews will be conducted using an online video-conferencing platform, over the phone, or in person depending on the interview participants access to technology and comfort level with technology. We will provide an option to conduct interviews over the phone if the interviewee is more comfortable participating over the phone or in person if the interviewee is comfortable meeting in person and this is feasible for the research team.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

There is no duplication. A thorough literature review on the topic of institutional resiliency and water resource management has been conducted to provide necessary background information as well as ensure that our research is novel. Additionally, outreach across the USGS-Water Resources Mission Area has been conducted to ensure duplicative efforts have not been planned elsewhere.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

The collection of information does not impact small businesses.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

This information collection is required to fulfill the objectives and technical requirements of two Water Resources Mission Area Programs – the Social & Economic Drivers Program and the Integrated Water Availability Assessments Program – part of the Congressionally sub-allocated budget program the Water Availability and Use Science Program. If the collection is not conducted, or is conducted less frequently, we will be unable to fulfill the objectives of the projects within these programs and will not meet the program specific technical requirements.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

* requiring respondents to report information to the agency more often than quarterly;

* requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

* requiring respondents to submit more than an original and two copies of any document;

* requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;

* in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study;

* requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

* that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

* requiring respondents to submit proprietary trade secrets, or other confidential information, unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no circumstances that require us to collect the information in a manner inconsistent with OMB guidelines.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and in response to the PRA statement associated with the collection over the past three years, and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every three years — even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

We published a 60-Day Federal Register notice 86 FR 68510 on December 2,2021.  We did not receive any comments in response to that notice.

We consulted with a wide variety of subject matter experts across the US Geological Survey to ensure the completeness, understandability, and conciseness of all 4 information collection instruments. The titles of those individuals and summary of feedback received is provided in the table below.

Table 1 Commenters on the survey or announcement

U.S. Geological Survey Water Mission Area Program Manager and Research Hydrologist Denver, Colorado Reviewed data delivery survey and interview questions. Provided feedback on the specifics of questions asked in the data delivery survey and interview question guide. Suggested changes to wording of some questions, which were accepted.	U.S. Geological Survey Water Mission Area Research Social Scientist Denver, Colorado Reviewed data delivery interview questions. Provided comments related to the flow of the interviews. Specifically, recommended changes to wording to make the interview more conversational, which were accepted.
U.S. Geological Survey Rocky Mountain Region Senior Scientist Denver, Colorado Provided additional questions for pre-interview survey (institutional resiliency) and changed wording of some questions to make them more accurate. These wording suggestions were accepted.	U.S. Geological Survey Northeast Climate Adaptation Science Center Acting Deputy Director Amherst, Massachusetts Reviewed institutional resiliency pre-interview survey and interview questions. Reviewed for content as well as policy …and scientific validity. Expressed that they did not have any policy or science concerns. Suggested a slight wording change to the survey that was accepted.
U.S. Geological Survey Rocky Mountain Region Research Social Scientist Reviewed institutional resiliency interview questions. Provided suggestions to clarify the wording on some questions. These clarifications were accepted.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

We will not provide payments or gifts to respondents.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The USGS does not provide an assurance of confidentiality. However, respondents will remain anonymous beyond the research team. Assurance will be provided in the form of an informed consent document presented to respondents before information is collected as well as display of the Privacy Act Statement on all written materials (questionnaire and informed consent document) and stated verbally as part of interviews.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Respondents will not be asked questions of a sensitive nature.

12. Provide estimates of the hour burden of the collection of information. The statement should:

* Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

* If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.

* Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here.

We are using the Bureau of Labor Statistics Employer Costs for Employee Compensation, USDL-22-049, published on 03/18/2022, to determine our dollar value for burden hours. The value used is $38.07 per hour for public respondents (private industry) and $54.96 for State, local and Tribal government respondents.

Table 2 Respondent burden

Participant / Activity	Number of Responses	Minute per response	Burden Hours	Dollar Value for Burden Hr
Public individual reads announcement or instructions for survey and completes survey	15	75	18.75	$713.81
Subtotal	15	75	19.00	$713.81
State, Local, Tribal govt reads announcement or instructions and completes survey and interview	135	75	168.75	$9,274.5
Total Subtotal	135	75	169	9,274.5
Total	150		188.00	$9,989.31

13. Provide an estimate of the total annual non-hour cost burden to respondents or recordkeepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected in item 12.)

* The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life) and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information (including filing fees paid for form processing). Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

* If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

* Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

We have not identified any non-hour cost burden associated with this collection. 

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

We used the Office of Personnel Management Salary Table 2022-GS to determine the hourly wage rate for all personnel that will be involved in administrating surveys, conducting interviews, and analyzing and interpretating the resulting data. To calculate benefits, we multiplied the hourly rate by 1.6 to account for benefits.

Table 3 Federal Government Expenses

Position	Grade/ Step	Hourly Rate	Annual Hrs	Fully Loaded Hr Rate	Total Labor Value
Research Social Scientist (Denver locality pay)	13/1	$49.85	40	$79.79	$3,91.60
Biologist/Decision Analyst (Rest of US)	12/2	$39.30	120	$62.88	$7,545.60
Student Trainee Geography (Rest of US)	7/2	$22.15	20	$35.44	$708.80
Physical Scientist (Rest of US)	11/5	$35.96	40	$57.54	$2,301.60
Geographer (Rest of US)	9/1	$26.22	40	$41.95	$1,099.93
Physical scientist (Rest of US0	12/1	$38.03	40	$60.85	$2,434.00

Table 4 Other Federal Government Expenses

Journal publication costs	$1,000
Conference Registration	$1,000
Transcription Services	$6,720

The total cost to the government for this information collection is $ 26,001.53

15. Explain the reasons for any program changes or adjustments in hour or cost burden.

This is a new information collection.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

We anticipate publishing the findings of this information collection as a scientific journal article – with accompanying U. S. Geological Survey data release via the Science base online platform, a summary report for participating stakeholders, and presentations at scientific conferences (e.g., American Geophysical Union [AGU]).

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

We will display the OMB Control Number and expiration date on appropriate materials. 

18. Explain each exception to the topics of the certification statement identified in "Certification for Paperwork Reduction Act Submissions."

There are no exceptions to the certification statement.