Data Delivery - Interview

Data Delivery Needs Interview

OMB Control Number: xxxx

Expiration Date: xxxx

This interview is being conducted by the National Water Census team at the U. S. Geological Survey (USGS) [LINK TO EMAIL ADDRESS: gs-w_nwc@usgs.gov].

The National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

The purpose of this interview is to give the NWC team a better understanding of how we can best serve future users’ needs. The results from this survey will be used to guide further development of the NWC and help determine how NWC outputs should be integrated with existing USGS water data.

We estimate that it will take an hour to complete the interview. Your responses will not be shared with anyone outside of the USGS NWC team, but may be anonymized, aggregated, and published in a future report or journal article. We will share the survey results with you and will be sure to get permission if we would like to include direct quotes in any way. We thank you for your time!

Privacy Act Statement: You are not required to provide your contact information in order to submit your survey response. However, if you do not provide contact information, we may not be able to contact you for additional information to verify your responses. If you do provide contact information, this information will not be shared with any other organization and will only be used to initiate follow-up communication about this project if needed. The records for this collection will be maintained in the appropriate Privacy Act System of Records identified as [DOI Social Networks (Interior/USGS-8) published at 76 FR 44033, 7/22/2011]. Paperwork Reduction Act statements: [16 U.S.C. 1a7] authorized collection of this information. This information will be used by the U.S. Geological Survey to better serve the public. Response to this request is voluntary. We estimate that it will take 15 minutes to prepare and respond to this collection. We will not distribute responses associated with you as an individual. We ask you for some basic contact information to help us interpret the results and, if needed, to contact you for clarification. 

The Paperwork Reduction Act of 1995 (44 U.S.C. 3501), states that an agency may not conduct or sponsor, and you are not required to respond to a collection of information unless it displays a currently valid OMB control number and expiration date. You may submit comments on any aspect of this collection. Please note that comments submitted in response to this collection are public record. Comments on this collection should be sent to the Clearance Office at gs-info_collections@usgs.gov.

Interviews will be conducted with samples of two different user populations. Those that fall into the “operationalized pull” category and those that fall into the “check status” category. Operationalized pull refers to data users who pull data from multiple sources, including USGS Application Programming Interface (API) services, to use via custom dashboards and tools optimized for their location and needs. Operationalized Pull users, on average, use the most USGS water information, returning repeatedly for refreshed data to pull into their own systems. Operationalized pull users include individuals that support decision-makers as opposed to those making decisions as well as water users. Check status refers to users that perform routine checks of a few parameters for specific sites, primarily using the hydrographs. Check Status users are our largest user type by number of unique users. Each user generally looks at a handful of sites for the latest water conditions. Check status users include decision-makers and non-profits. 

Operationalized pull interview questions

Italicized text is intended to be read to the interview participants. Questions are bullet pointed, with specific probes or follow up questions further bulleted. Additional instructions are [bracketed]. The objective of each set of questions is italicized and underlined and provided before the bulleted questions.

Thank you for agreeing to participate in this interview. I’d like to start by learning more about your typical day and workflows. 

Objective: To better understand user workflows and the necessary characteristics of useful datasets and how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Can you give me a quick overview of a typical day at your job?

  • What tasks do you do?

  • What decisions do you make?

  • Who do you interact with?

  • Does this vary from season to season, month to month, or year to year?

  • What kind of outputs / deliverables do you generate in your work?

    • Who do you have to show or deliver these to?

  • What parameters or variables are most important to you in your work?

  

[The following are water quantity user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• If you develop projections or plans for water supply, how far into the future do you forecast?

• What are the primary water quality concerns that are incorporated into your water supply planning?

• How would you rate your ability to predict the future requirements for water supply in the short term (over the next year)?

  • How important is this ability?

• How would you rate your ability to predict the future requirements for water supply in the long term (over the next 20 years)?

  • How important is this ability?

[The following are water quality user specific questions]

• Share the top 3-5 water quality parameters you believe can be modeled well.

• What are the primary water quality concerns that are incorporated into your water supply planning?

• What is the best way to present data for those in your organization who work with watershed stakeholders like landowners who implement BMPs?

[The following are water use/demand user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the community/ agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• How do you define and use the terms water supply, demand, and availability?

• How do law and compacts restrict the type of work you do or the solutions that you propose for certain problems?

• What is your preferred way(s) to have water demand information organized for queries?

  • Sum of all the use in a certain area, separate surface water and groundwater total use in an area, each water use category separate [irrigation, hydro electric, public supply, etc.]?

[The following are aquatic ecosystems user specific questions]

• Please describe the primary ways you or your organization seeks to manage for ecoflow [environmental flow] needs of the aquatic community.

• What is your process for calculating a set of key accepted ecoflow statistics that are used in regulations or general planning?

• What would the benefit be to having nationally consistent ecoflow statics available on a regular basis?

• When determining ecoflow thresholds for decision-making related to waterways with various species, how do you synthesize all the various biologic needs?

• What are the key aquatic communities that are used within your regulation/permitting/management processes?

• Would you use ecoflows data on a national or regional scale, or will you always be seeking local information?

We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.

Objective: To better understand 1) user workflows and the necessary characteristics of useful datasets; 2) how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job; 3) data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples; 4) To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• Tell me more about your computational or analytical workflow doing <data-related task described in answers to the above section>.

  • What are all the variables, datasets, and types of data needed to do this work?

  • Do you use measured data in this work?

    • If yes: If you had modeled data for the same variable, how would that impact your work?

  • Do you already use modeled data in this work?

    • If yes: If you had measured data for the same variable, how would that impact your work?

  • Do you pair or combine measured data (sensor, gage, sample, observation) with modeled information in this work?

    • If yes: Tell me about that. What would make this process easier?

    • If yes: Does it concern you if or when measured and modeled data conflict with each other? How would you deal with that if so?

  • What are (or would be) the trade-offs between 1.) having modeled data available at a low spatial resolution throughout your study area and 2.) having measured data at points scattered throughout your study area?

  • What are other limitations of these datasets you have used that we haven’t yet touched on?

  • Are you interested in model predictions of yesterday's or last week's hydrological conditions at locations where we are not collecting observational data?

    • If yes: Why?

• Describe the uncertainty information associated with the measured or modeled data you currently access and incorporate into your workflows.

  • How do you use that uncertainty info?

  • If no answer: What kind of uncertainty information do you wish was available? How would you use it if it was?

• Describe how you currently go between different websites, data portals, or APIs to download all the necessary datasets to accomplish your analytical workflow.

  • What are some of the challenges you face when gathering and integrating these disparate datasets which may have different spatial and temporal scales?

  • What would make this process easier?

  • What are your favorite and least favorite websites / data portals / APIs and why?

  • In general, do you prefer to directly query an API, use a clickable data portal, or go through a software package to download data for your work?

  • [For water quantity user] Would you be interested in accessing a water budget model in the Cloud which you could run for a selected area?

We would now like to show you a list of the new modeled datasets that the National Water Census will serve, which you already saw once in the online survey. As a reminder, the National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Looking at this list of model outputs [show them the list they’ve already seen on the online survey via screenshare], which would you incorporate into your workflows? As a reminder, these were the outputs you selected when you took the online survey [show them their survey answers via screenshare]. Are these outputs still of primary importance to your work?

[For water quantity user]

• Would or how would you use subcomponents of more integrated variables from water budget process models? For example: multiple flow components going into runoff, six different storage components that are part of soil moisture, or multiple components representing snow and ice. [Show this doc for all possible components via screenshare.]

-----------------------------------------------------------

Now we are going to show maps of some of the model outputs you said you would be interested in to demonstrate data types, extent, resolution, and patterns of these outputs across large areas of the country. We will ask the same questions for each set of model output maps we will show, which we are sharing in the chat. [Share simplified version of questions in this section (without prompts) in the chat. Show model output maps via screenshare. For all the questions in this section, refer to the maps shown via screen share, corresponding to one of the four following data types: water quantity-related, water quality-related, water use-related, aquatic ecosystems-related. Iterate through the following section for each data type the respondent is interested in.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Data display:

• Would you rather this [INSERT one of the four data types above] information be presented as raw values, in the context of historical values / as historical quantiles, or in relation to a relevant threshold?

  • If a threshold, which threshold?

    • [For water quality user] Exceedance of a maximum contaminant level?

• [For water quality user] What is your preferred way to have water quality levels reported?

  • Concentration, load, or benchmark exceedance?

• [For salinity] How do you typically measure salinity or prefer to have salinity reported?

  • Specific conductivity, total dissolved solids, chloride concentration, actual salinity?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

Spatial and temporal resolutions and extents:

• What would be an ideal time period over which to have this [INSERT one of the four data types above] information?

• What would be an ideal temporal resolution for this [INSERT one of the four data types above] information?

• What would be an ideal spatial scale or extent over which to have this [INSERT one of the four data types above] information?

• What would be an ideal spatial resolution for this [INSERT one of the four data types above] information?

  • Examples include public utility service areas, NHDPlus Catchments, HUC12, HUC8, HUC6, states, major regional watersheds, raster dataset.

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

Data format:

• What format would you want downloaded data for this [INSERT one of the four data types above] information to be in?

  • Tabular, csv, netcdf, json via API, other?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

Role of data in workflows:

• If the calculations and inputs were clear and transparent, and available at favorable temporal and spatial scales, how would you incorporate this [INSERT one of the four data types above] information into your workflows?

• What are the impacts that these data would have on your job?

• What decisions or deliverables would you be able to make using this data?

• How accurate do these model outputs have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need hydrologic drought category projection or do you need streamflow in cfs within minimal confidence interval to apply your own interpretations?

• What other modeled outputs not on the above list would be useful to you?

  • How would you use those?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Supplementary information and uncertainty:

• What documentation, summary information, or visuals about this model output would you like to see before you download and use the data?

  • Examples include: Size, format, spatial resolution, temporal resolution, spatial extent, temporal extent

• If presented with model diagnostic or performance plots, how would you use that information? [Show example model diagnostic and performance plots associated with this particular model output, as needed or requested, via screenshare.]

• If presented with uncertainties associated with this particular model output, how would you use that information?

• Would it be useful for us to also serve model input datasets? [Show list of model inputs associated with this particular model output via screenshare.]

  • How would you use these?

• How would you use or interpret ensemble output from multiple models serving the same variable?

Visuals:

• What would be the value of a web-based visualization of the model output like the one we’re showing here to you or your colleagues?

  • If they find a web-based visualization valuable:

    • What other information or mapped GIS layers would you want to see along with or on top of this modeled output? [Show examples corresponding to those suggested by modeling teams via screenshare.]

    • What are the key features that would make you want to return to the visualization?

    • What key features would policymakers / managers in your organization or others you communicate information with most appreciate?

    • What would you change about this visualization or way of presenting the information?

Those are all of the questions that I have for you. Is there anything else you’d like to tell me about how you use USGS water data or how you might use the National Water Census that I didn’t ask about?

Extra Questions below may be substituted in based on the respondent background or used in a follow up interview the following year.

-----------------------------------------------------------

[The following questions below are chosen to be discussed depending on how the respondent answered the question above: “In general, do you prefer to directly query an API, use a clickable data portal, or go through a software package to download data for your work?” within the question section starting “We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.”]

[If the respondent answered that they prefer a clickable data portal, go through the following]

Next, we will show example data portals that may be used to serve National Water Census data or that may be similar to what a National Water Census data portal will look like. We will ask the following set of questions for each example data portal. [Send the below questions in the chat. Walk through example websites via screenshare.]

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• If NWC data were delivered on this data portal, would that be convenient / easy for you to use?

• What makes it easy or difficult to use?

Data portal #1:

NOMADS or NWC NOMADS-like prototype

Data portal #2:

Model Catalog (perhaps with an NWC badge)

Data portal #3:

ScienceBase (perhaps with an NWC badge)

[If the respondent answered that they prefer an API, go through the following]

• Is there an existing API that you already use that you think NWC data could imitate or even slot into / become part of?

[If the respondent answered that they prefer a software package, go through the following]

• What software would you prefer to use for downloading NWC data?

  • Python, R, other?

• Does it make sense for an NWC data download function to be added to dataRetrieval / ulmo / other existing packages or libraries you already use to download USGS water? Or would you prefer a standalone, NWC-specific data download package?

-----------------------------------------------------------

Next, we will show you some potential ways in which we may represent, calculate, and display water supply, demand, and water availability as part of the National Water Census. We are still figuring out exactly how we will do this, so we are interested in getting feedback from you on what would make sense and be useful. We will ask the following set of questions for each example calculation or visualization. [Send the below questions in the chat. Show example visualizations via screenshare.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them and to better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Do these calculations of water availability make sense to you?

• How would you use information on water availability calculated or presented in this way in your work?

• Are you interested in separate calculations of availability based on surface water and groundwater sources?

• How do you define and use the terms water supply, demand, and availability? How are those definitions the same or different from what we are showing?

Visualization #1:

Visualization #2:

Map of water budget showing places that are under and overbudget

• Please describe how you would interpret this representation of supply and demand.

Visualization #3:

NASA GLDAS output/water balance ESRI app: Water Balance App (arcgis.com)

Visualization #4:

Late fall 2019 water availability data viz (IWAAs concept map) | U.S. Geological Survey (usgs.gov) by Vizlab

-----------------------------------------------------------

[This is for later interviews with people we’ve already talked to before. For each of the below tasks, we will ask respondents to share their screen, which they will have been informed of ahead of time.]

We would love for you to walk us through in more detail on how you currently incorporate USGS and other data into your technical workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Could you share your screen and walk us through a typical workflow when you’re downloading and / or using USGS data?

• Could you share your screen and walk us through how you use R / Python / ArcGIS / others to compile / analyze / download USGS data?

• Could you share your screen and walk us through how you currently use USGS Water Services?

• Could you share your screen and walk us through how you currently download data via USGS data portals / websites?

-----------------------------------------------------------

[This is for later interviews regarding later version of NWC.]

As we consider the important elements to provide for users, we wonder about the utility of being able to run models yourself or having access to model code that you could work with directly.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Do you already integrate USGS modeling code or runs into your workflows?

  • If yes: Tell me about how you do that.

    • Do you run models on your local machine, in the cloud, or on a cluster / supercomputer?

Objective: To better understand data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples.

• What would make this easier to do?

Objective: To better understand how users want to be able to download NWC output, whether that be via API, data portal, or software package.

• If no: Would you want to run USGS models on your own if they were readily accessible?

  • If yes: Where would you want to do that? On your local machine, in the cloud, or on a cluster / supercomputer?

Check Status user interview questions

Italicized text is intended to be read to the interview participants. Questions are bullet pointed, with specific probes or follow up questions further bulleted. Additional instructions are [bracketed]. The objective of each set of questions is italicized and underlined and provided before the bulleted questions.

Thank you for agreeing to participate in this interview. I’d like to start by learning more about your typical day and workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets and how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Can you give me a quick overview of a typical day at your job including the things you have to do, the decisions you have to make, and the people you interact with?

  • Who do you rely on to keep you up to date on all the information you need to know?

    • What is their title?

  • Does your workflow vary from season to season, month to month, or year to year?

  • What outcomes or deliverables do you generate in your work?

    • Who do you serve by creating these outcomes?

[The following are water quantity user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• If you develop projections or plans for water supply, how far into the future do you forecast?

• What are the primary water quality concerns that are incorporated into your water supply planning?

• How would you rate your ability to predict the future requirements for water supply in the short term (over the next year)?

  • How important is this ability?

• How would you rate your ability to predict the future requirements for water supply in the long term (over the next 20 years)?

  • How important is this ability?

[The following are water quality user specific questions]

• Share the top 3-5 water quality parameters you believe can be modeled well.

• What are the primary water quality concerns that are incorporated into your water supply planning?

• What benefit would it be to have nitrogen, phosphorus, and salinity information available for all surface waters and groundwater in your region?

• What benefit would it be to know the wastewater percentage in streams in your region?

• How important is it to have stream temperature and sediment information for streams in your region?

• What is the best way to present data for those in your organization who work with watershed stakeholders like landowners who implement BMPs?

[The following are water use/demand user specific questions]

• What are the main sources of water you need information about in order to ensure there is enough supply for the community/ agency/department/organization you serve? [Can leave open-ended or give potential choices: reservoirs, lakes, rivers, wetlands, groundwater (deep or shallow), re-use of water from one industry applied to another]

• How do you define and use the terms water supply, demand, and availability?

• How do law and compacts restrict the type of work you do or the solutions that you propose for certain problems?

• What is your preferred way(s) to organize information about water use for decision making?

  • Sum of all the use in a certain area, separate surface water and groundwater total use in an area, each water use category separate [irrigation, hydro electric, public supply, etc]?

[The following are aquatic ecosystems user specific questions]

• Please describe the primary ways you or your organization seeks to manage for ecoflow [environmental flow] needs of the aquatic community.

• What is your process for calculating a set of key accepted ecoflow statistics that are used in regulations or general planning?

• What would the benefit be to having nationally consistent ecoflow statics available on a regular basis?

• When determining ecoflow thresholds for decision-making related to waterways with various species, how do you synthesize all the various biologic needs?

• What are the key aquatic communities that are used within your regulation/permitting/management processes?

• Would you use ecoflows data on a national or regional scale, or will you always be seeking local information?

We wish to learn about ways in which you utilize and gather water data from USGS and other sources in your workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Tell me more about your workflow <making decisions related to ___ / doing advocacy related to ___ / communicating ___information to your stakeholders>.

  • What are all the datasets and information sources (measured or modeled) needed to do this work?

  • Do you use combined measured data (sensor, gage, sample, observation) with modeled information in this work?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• If yes: Does it concern you if or when measured and modeled data conflict with each other? How would you deal with that if so?

• What are (or would be) the trade-offs between 1.) having modeled data available at a low spatial resolution throughout your study area and 2.) having measured data at points scattered throughout your study area?

• What are other limitations of these datasets you have used that we haven’t yet touched on?

• Are you interested in model predictions of yesterday's or last week's hydrological conditions at locations where we are not collecting observational data?

  • If yes: Why?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Describe how you currently use different websites, visualizations, and other resources to make decisions, advocate, or communicate about important aspects of your work.

Objective: To better understand data and information delivery mechanisms that have worked well for users in the past and to catalog favorite/least favorite examples.

• What are some of the challenges you face in understanding or interpreting information from sources which may have different spatial or temporal scales?

• What would make this process easier?

• Out of the ones you named, what are your favorite and least favorite websites / visualizations / other resources and why?

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

• In general, what types of information sources do you prefer and find most useful?

  • Reports, factsheets, static maps, websites, interactive maps, StoryMaps, others? [Show examples of these as needed via screenshare.]

We would now like to show you a list of the new modeled datasets that the National Water Census will serve, which you already saw once in the online survey. As a reminder, the National Water Census (NWC) is a new USGS data delivery system, currently under development, that will serve modeled water supply, demand, and availability information across the US. It will greatly extend the existing set of USGS water data outputs by providing three new types of information: 1.) Modeled water availability and hydrologic subcomponents with complete spatial coverage across the nation; 2.) Quantitative trends in water availability and quality calculated at measured sites; and 3.) The status of streamflow conditions relative to their ability to support healthy aquatic ecosystems.

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• Looking at this list of model outputs [show them the list they’ve already seen on the online survey via screenshare], which would you incorporate into your workflows? As a reminder, these were the outputs you selected when you took the online survey [show them their survey answers via screenshare]. Are these outputs still of primary importance to your work?

-----------------------------------------------------------

Now we are going to show maps of some of the model outputs you said you would be interested in to demonstrate data types, extent, resolution, and patterns of these outputs across large areas of the country. We will ask the same questions for each set of model output maps we will show, which we are sharing in the chat. [Share simplified version of questions in this section (without prompts) in the chat. Show model output maps via screenshare. For all the questions in this section, refer to the maps shown via screen share, corresponding to one of the four following data types: water quantity-related, water quality-related, water use-related, aquatic ecosystems-related. Iterate through the following section for each data type the respondent is interested in.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Data display:

• Would you rather this [INSERT one of the four data types above] information be presented as raw values, in the context of historical values / as historical quantiles, or in relation to a relevant threshold?

  • If a threshold, which threshold?

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• What would be an ideal temporal and spatial resolution for this information?

• What would be an ideal time period over which to have this information?

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

Data download:

• Would you be interested in downloading this data?

  • If yes: What format would you want downloaded data to be in?

    • Visualization / image, tabular, csv, other?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

Role of data in workflows:

• If the calculations and inputs were clear and transparent, how would you incorporate this information in your regular workflows?

• What decisions or deliverables would you be able to make using this data?

• How accurate does this model output have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need drought category projection or do you need streamflow in cfs within a minimal confidence interval to apply your own interpretations?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

Supplementary information and uncertainty:

• What documentation, summary information, or visuals about the model output would you like to see before you use or explain the data?

  • Examples include: Size, format, spatial resolution, temporal resolution, spatial extent, temporal extent

• How would you use or interpret ensemble output or multiple models serving this same variable?

Visuals:

• What other information or mapped GIS layers would you want to see along with or on top of this modeled output? [Show examples corresponding to those suggested by modeling teams via screenshare.]

• What are the key features that would make you want to return to this visualization?

• What key features would policymakers / managers in your organization or others you communicate information with most appreciate?

• What would you change about this visualization or way of presenting the information?

-----------------------------------------------------------

Next, we will show you some potential ways in which we may represent, calculate, and display water supply, demand, and water availability as part of the National Water Census. This can be a combination of data from all four themes or map sets we just looked at. We are still figuring out exactly how we will do this, so we are interested in getting feedback from you on what would make sense and be useful. [Show example visualizations via screenshare.]

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

• How do you define and use the terms water supply, demand, and availability?

  • How are those definitions the same or different from what we are showing?

Objective: To better understand how users would use NWC model output to make decisions and whether these model outputs would improve their ability to do their job.

• How would you use information on water availability presented or calculated in this way in your work?

• What decisions or deliverables would you be able to make using this data?

• How accurate does this model output have to be to be useful in your decisions or workflows?

  • [For water quantity user] Do you just need drought category projection or do you need streamflow in cfs within a minimal confidence interval to apply your own interpretations?

Objective: To better understand how we can help users better communicate their work and how we can better contextualize and/or interpret our data for them.

• Are you interested in separate calculations of availability based on surface water and groundwater sources?

• What other information or mapped GIS layers would you want to see along with or on top of this water supply, demand, and availability info? [Show examples corresponding to those suggested by modeling teams via screenshare.]

• What are the key features that would make you want to return to this visualization?

• What key features of this visualization would policymakers / managers in your organization or others you communicate information with most appreciate?

• What would you change about this visualization or way of presenting the information?

[The following sections below are chosen to be discussed depending on how the respondent answered the question above: “In general, what types of information sources do you prefer and find most useful? Reports, factsheets, static maps, interactive maps, StoryMaps, others?”]

Since you have seen some of the output data and one draft way we are considering presenting supply and demand information, we would like to know about your preferences for how this information is delivered. Next, we will show an example <report, factsheet, static map, website, interactive map, or StoryMap> that may be similar to what delivering National Water Census data would look like. [Walk through a report, factsheet, static map, interactive map, or StoryMap via screenshare.]

Objective: To better understand how users want to have NWC output presented to them, whether that be via reports, factsheets, static maps, interactive maps, StoryMaps, or other methods.

• [For report, factsheet, StoryMap] What information, maps, tables, charts, and other visualizations would you most want to see in a NWC <report, factsheet, or StoryMap>?

• [For static map, website, interactive map] What information would you most want to see in the <static map, website, or interactive map>?

• How regularly would you like to see information products updated or refreshed?

Those are all of the questions that I have for you. Is there anything else you’d like to tell me about how you use USGS water data or how you might use the National Water Census that I didn’t ask about?

Extra questions below may be substituted in based on the respondent background or used in a follow up interview the following year.

-----------------------------------------------------------

[This is for later interviews with people we’ve already talked to before. For each of the below tasks, we will ask respondents to share their screen, which they will have been informed of ahead of time.]

We would love for you to walk us through in more detail on how you currently incorporate USGS and other information into your workflows.

Objective: To better understand user workflows and the necessary characteristics of useful datasets.

• Could you share your screen and walk us through a typical workflow when you’re making a decision about _____?

• Could you share your screen and walk us through a typical workflow when you’re preparing a presentation for _____ or to communicate _____?

• Could you share your screen and walk us through a typical workflow when you’re first learning about _____?

• Could you share your screen and walk us through a typical workflow when you’re searching for information about ______?