2020 DWINSA ICR for OMB_APPENDIX C

Appendix C – Comments and Response to Comments Received on the 2020 DWINSA Peer Review

Introduction to the 2020 DWINSA Peer Review Comment and Response

EPA has assessed the drinking water infrastructure needs of public water supplies through a national survey of drinking water systems every four years since 1995. Using a stratified random statistical sample of water systems, the Agency assesses need by evaluating project-level data from each system sampled including information about the type of project, its size or capacity, and its cost. A new sample was selected for each Assessment and the same basic approach (with some notable exceptions) was used in 1995, 1999, 2003, 2007 and 2011.

Section A.3.c of the ICR supporting statement describes the changes to each Assessment’s data collection instrument, statistical methodology, policies, and added questions. Section A.3.c also describes the pre-tests and formal peer reviews that were completed for each change. For the previous DWINSAs, when the data collection instrument, statistical methodology, or policies remained unchanged from the most recent survey for which a peer review addressed the change, a peer review was not implemented. When new questions and codes have been added, such as for the 2011 DWINSA’s “green” and climate readiness questions, EPA conducted a limited peer review of these new questions.

EPA proposes to use the modified panel approach with the 25 percent refresh of the sample for the 2020 State DWINSA and for the 2020 Native American DWINSA. This statistical methodology change was peer reviewed and implemented for the 2015 State DWINSA (the Native American DWINSA was not conducted in 2015). Because the data collection instrument, statistical methodology, and policies that were peer reviewed under previous Assessments have not been changed, these elements of the 2020 DWINSA were not peer reviewed.

For the 2020 DWINSA, three categories of questions are proposed to be added that were not in previous Assessments: Lead Service Line (LSL) questions, Operator Workforce (OpW) questions, and Iron and Steel (I&S) questions. Data obtained from responses to the LSL and OpW questions may be used to inform EPA policy and were peer reviewed. Data obtained from responses to the I&S questions will aid EPA in management of the American Iron and Steel (AIS) requirements of the Safe Drinking Water Act section 1452(a)(4). Therefore, the I&S questions were not formally peer reviewed. Instead, these questions were reviewed and revised by EPA AIS technical experts.

The effort to capture data related to water system lead service line inventory is in response to the following mandate included in America’s Water Infrastructure Act of 2018 (AWIA):

AWIA Section 2015 State Revolving Loan Funds, (e)(2)

“Any assessment conducted… after the date of enactment of America’s Water Infrastructure Act of 2018 shall include an assessment of costs to replace all lead service lines (as defined in section 1459B(a)(4)) of all eligible public water systems in the United States, and such assessment shall describe separately the costs associated with replacing the portions of such lead service lines that are owned by an eligible public water system and the costs associated with replacing any remaining portions of such lead service lines, to the extent practicable.’’

For the LSL category of questions, the peer review charge was to assess whether the request for information will capture known service line inventory information accurately and meet the assessment information collection requirement as mandated by AWIA. EPA asked technical experts on drinking water distribution systems and lead service line issues to peer review the LSL background information and questions. Ms. Kathy Martel, P.E., and Ms. Anne Jaffe Murray were asked to perform this peer review. Ms. Martel is a registered professional engineer with more than 30 years of consulting engineering and drinking water utility experience. She has delivered technical presentations on managing distribution system water quality and co-authored numerous technical guidance documents and research reports on distribution system water quality and infrastructure condition assessment topics published by EPA and the Water Research Foundation. Ms. Martel is also a Grade 4 certified operator for both water treatment and water distribution systems. In the last year, Ms. Martel taught each of the operator certification classes for the State of Maine from very small systems to Grades 3 and 4 water treatment and water distribution. Ms. Anne Jaffe Murray is a technical expert on the Lead and Copper Rule (LCR) and has worked on various aspects of the rule and its revisions since its inception. She is also a technical expert on the Lead-Free Rule. Ms. Jaffe Murray holds a Masters degree in Public Health and a Masters degree in Business Administration in Environmental Health Management. Ms. Jaffe Murray has over 35 years of experience in the environmental field.

The drinking water industry has predicted that in the near future there will be large-scale retirements of drinking water treatment and distribution system operators with insufficient workforce replenishment. These conditions create a potential for a drinking water system operator workforce shortage. The effort to capture additional information on drinking water system operator workforce vacancy and hiring concerns through the OpW questions is prompted by these predictions. The DWINSA is designed to collect statistically valid data. It therefore provides a unique opportunity to collect important information on workforce availability over the next 5 years and next 10 years. For water systems responding to the 2020 DWINSA, EPA will collect information related to the drinking water system operator workforce in a series of 10 questions. This peer review charge was to assess whether the request for information will collect data accurately and inform the discussion of workforce shortage concerns. The technical experts asked to perform the peer review on the OpW questions are Ms. Kathy Martel, P.E., and Ms. Ashley Arayas. Ms. Martel’s qualifications were described for the LSL peer review. Ms. Arayas holds a Masters degree in Environmental Science and Policy and has over seven years’ experience as a Small System Capacity Trainer. Ms. Arayas’ technical focus has been on drinking water system sustainability and capacity development, including water sector workforce issues.

The peer review materials for the LSL and OpW categories of questions included background information and instructions that would be provided to the survey participants as well as the questions as they would be presented in the data collection instrument. A separate document of the peer review charge and specific charge questions was also provided.

Summary tables of the charge questions for which the 2020 DWINSA peer reviewers provided comment and EPA’s responses are provided below. The comments are presented without attribution by name to either reviewer. For the summary tables, in some cases the questions and/or comments have been condensed and consolidated. If the peer reviewer did not provide actionable comments for a particular charge question that question is omitted from the summary table.

In addition to the comments included in the table, the peer reviewers suggested a number of edits to the questions and to the background information and instructions that will be provided to the systems. These suggestions were related to clarity and consistency and did not significantly change the meaning of the questions or the information collected. Revisions recommended by the peer reviewers have been incorporated, as appropriate, into the revised 2020 DWINSA questionnaire Lead Service Line Inventory tab and Workforce tab included in Appendix B.

Summary of Peer Review Comments on the DWINSA 2020 Lead Service Line Questionnaire (State and AI/ANV Survey Tools)

Reviewer 1

Charge Question	Reviewer 1 Feedback (State Survey Tool)	Reviewer 1 Feedback (AI/ANV Survey Tool)
Please provide any comments on the descriptions of Rows 1-4b of the lead service line questionnaire.	Regarding Row 1 - for service lines that contain lead pipe, what about plastic pipe that contains lead plasticizers.	Same comment as State Survey.
EPA Response: EPA expanded language in the Background text of the lead service line questionnaire to clarify that this questionnaire is not intended to be an assessment of all potential sources of lead in drinking water. The text in the background section of the questionnaire now reads as follows: “This inventory is a baseline effort to collect information on lead service lines in public water systems, it is not a comprehensive assessment of all potential sources of lead in drinking water. Other potential sources of lead, such as brass or bronze valves or fittings that contain lead, may be included in future data collection efforts.”
Charge Question	Reviewer 1 Feedback (State Survey Tool)	Reviewer 1 Feedback (AI/ANV Survey Tool)
Does the Background section in the LSL Questionnaire adequately describe the purpose and intent of the LSL inventory data collection.	Yes.	Yes.
EPA Response: No action needed.
Does the lead service line inventory data collection approach in the questionnaire meet the stated AWIA mandate	There are no questions on the number of partial LSLs. That is, the number of LSLs where the utility or customer replaced their SL but the other lead section remains. The proposed LCR includes estimated costs for the public water systems and households to replace their LSLs.	Same comment as state survey
EPA Response: EPA will use the cost models developed as part of the DWINSA data collection effort and data collected as part of the lead service line questionnaire to estimate costs for both the public water system and homeowners to replace the portion of lead service lines for which they are responsible.

Charge Question	Reviewer 1 Feedback (State Survey Tool)	Reviewer 1 Feedback (AI/ANV Survey Tool)
EPA estimates that state utilities will require 23 minutes to complete the LSL Questions tab in the survey instrument and AI/ANV utilities will require 10 minutes to complete the LSL questions tab, do you think this is an accurate estimate of the time required to complete these questions?	Reviewer 1 estimated will take state utilities a total of 40-60 minutes to complete the survey questions depending on whether they provide cost documentation for lead service line replacements. The time estimate includes 10 minutes for a first read through of the instructions and questions, 30 minutes to find and compile the data and complete the responses, and 20 minutes to provide cost data.	Reviewer 1 estimated it will take AI and ANV a total of 25 minutes to complete the survey questions including 5 minutes for a first read through of the instructions and questions, and 20 minutes to find and compile the data, and complete the responses.
EPA Response: The DWINSA seeks to collect existing information on service lines that is readily available to the utility. For CWSs serving over 3,300 persons and NPNCWS serving more than 10,000 persons, the burden estimate includes 10 minutes for the utility and the state to discuss the supplemental questions. The majority of this time is expected to be spent discussing the lead service line questions. This is in addition to the 23 minutes estimated for the utility to complete the lead service line questionnaire. An additional 20-minute phone-call is anticipated to further discuss the LSL questions with the state, and another one-third of the systems are anticipated to need yet another 15-minute call. In total, two-thirds of the systems are anticipated to take 53 minutes to research and complete the lead service line questionnaire with one third of systems needing 68 minutes to complete the questionnaire.

Reviewer 2

Charge Question	Reviewer 2 Feedback (State Survey Tool)	Reviewer 2 Feedback (AI/ANV Survey Tool)
Please provide any comments on the descriptions of Rows 1-4b of the lead service line questionnaire.	Reviewer 2 recommended that EPA add language to clarify that the services lines counted in Row 1 should include any service lines which are believed to be all or in part comprised of lead pipe. Reviewer 2 recommended that question 3b be broken into two questions in order count separately the number of galvanized pipe previously downstream from lead pipe and previously downstream from lead connectors.	Same as comment state survey.
EPA Response: EPA clarified the language in Row 1. Per the recommendation of Reviewer 2, Row 3b is now broken into two rows (3b and 3c) to discern galvanized pipe previously downstream from lead pipe from galvanized pipe previously downstream from lead connectors. EPA also added row 3d, for galvanized pipe that was previously downstream from an unknown source of lead.
Charge Question	Reviewer 2 Feedback (State Survey Tool)	Reviewer 2 Feedback (AI/ANV Survey Tool)
Does the Background section in the LSL Questionnaire adequately describe the purpose and intent of the LSL inventory data collection.	It is confusing that EPA says no “Service line replacement costs will be estimated by EPA and do not need to be provided by public water systems.”, but then goes on to ask for cost documentation for lead service line replacements.	N/A (this text not included in AI/ANV survey).
EPA Response: EPA has specified in the text of the background section of the state lead service line questionnaire that it is not a requirement for systems to provide cost documentation. However, EPA requests that systems volunteer such documentation as part of the survey response, if available.
Charge Question	Reviewer 2 Feedback (State Survey Tool)	Reviewer 2 Feedback (AI/ANV Survey Tool)
Does the lead service line inventory data collection approach in the questionnaire meet the stated AWIA mandate	Yes.	Yes.
EPA Response: No action needed.
Charge Question	Reviewer 2 Feedback (State Survey Tool)	Reviewer 2 Feedback (AI/ANV Survey Tool)
EPA estimates that state utilities will require 23 minutes to complete the LSL Questions tab in the survey instrument and AI/ANV utilities will require 10 minutes to complete the LSL questions tab, do you think this is an accurate estimate of the time required to complete these questions?	The estimate seems low assuming the system must read and understand the questions, find the needed information, complete the survey, and review and it will be additional burden to provide lead service line replacement cost information and supporting documentation. However, there will be a reduction in burden for systems in those states which already require a lead service line inventory as they should already have most of the information required for the questionnaire readily available.	Is there a separate burden to read and understand the instrument? If so, then estimate is more reasonable but still seems low. These systems are not providing ownership or cost information so I agree the estimated burden should be lower than the state utilities. However, the system must still consult their records, complete the survey, and review the information. A minimum of 1 hour to 2 hours seems more reasonable.
EPA Response: The DWINSA seeks to collect existing information on service lines that is readily available to the utility. For state CWSs serving over 3,300 persons and NPNCWS serving more than 10,000 persons, the burden estimate includes 10 minutes for the utility and the state to discuss the supplemental questions. The majority of this time is expected to be spent discussing the lead service line questions. This is in addition to the 23 minutes estimated for the utility to complete the lead service line questionnaire. An additional 20-minute phone-call is anticipated to further discuss the LSL questions with the state, and another one-third of the systems are anticipated to need yet another 15-minute call. In total, two-thirds of the systems are anticipated to take 53 minutes to research and complete the lead service line questionnaire with one third of systems needing 68 minutes to complete the questionnaire. Similarly, additional time beyond the 10 minutes allocated to complete the questionnaire is included for AI/ANV utilities to consult with EPA Regions.

Summary of Peer Review Comments on the DWINSA 2020 Workforce Questionnaire

Reviewer 1

Charge Question	Reviewer 1 Feedback (State & AI/ANV Survey Tool)
Are questions 1-10 of the Workforce Questionnaire easy to understand and answer?	It would be helpful if Question 1 included additional detail about which types of employees should be counted in order to ensure consistency between responses. Question 2b. should further clarify what is meant by “on-site”. Question 10 should be revised to allow the respondent to check multiple options.
EPA Response: Additional language was added by EPA to clarify that water system staff (either directly employed by the water system or contracted employees) should be counted if they are responsible for day-to-day operations at the water system. EPA added language to clarify that “on-site” implies time spent at the water system for physical operations and maintenance. Question 10 was revised to all the respondent to check up to 3 options.
Charge Question	Reviewer 1 Feedback (State & AI/ANV Survey Tool)
Are there any additional questions that should be included to better capture information on future drinking water system workforce needs?	Suggest trying to collect additional information on challenges associated with inability of operators to meet certification requirements. This may occur because of difficulty passing the exam and/or changes to the certification required to operate the system as the system expands or becomes more sophisticated. The questionnaire could also collect additional information about what sort of support would be helpful from industry to address difficulties in hiring operators.
EPA Response: This initial Workforce data collection effort will allow EPA and industry associations to determine specific topics on which further data collection is most valuable. For example, if results from the initial questionnaire indicate a substantial number of systems are having trouble getting operators certified, industry associations can identify this as an issue for further attention. Additionally, by allowing the respondent to select up to 3 choices in Question 10 it is anticipated that certification challenges will be captured if applicable.

Charge Question	Reviewer 1 Feedback (State & AI/ANV Survey Tool)
Does the workforce questionnaire capture information that is relevant to EPA’s stated goal of determining drinking water system workforce shortfalls over the next 5 and 10 years?	Yes, if EPA’s goal is limited to operator workforce shortfalls.
EPA Response: No action needed.
Charge Question	Reviewer 1 Feedback (State & AI/ANV Survey Tool)
EPA estimates that it will take survey respondents (State, AI and ANV) 5 minutes to complete the supplemental Workforce questionnaire, do you think this is an accurate estimate of the time required to complete these questions?	I estimate it will take 30 minutes in total. I think it will take 10 minutes for a first read through of the instructions and ten questions, and an additional 5 minutes to respond to most questions. Question 1, 2, and 5 will require an additional 15 minutes to check human resources records and then complete the responses.
EPA Response: EPA intends that the survey respondent will use their best professional judgement to answer questions about staffing and workforce issues at their system. EPA does not expect respondents to conduct additional research of human resource records as part of their questionnaire response. In many cases the system will have staff from multiple departments contribute the survey response, including staff familiar with system staffing and workforce details. EPA anticipates that if a survey respondent does not have the information required to answer a particular question on hand, they will skip the question or select “Don’t Know” as their response.

Reviewer 2

Charge Question	Reviewer 2 Feedback (State & AI/ANV Survey Tool)
Are questions 1-10 of the Workforce Questionnaire easy to understand and answer?	Question 1 should clarify what personnel should be included (i.e. FTEs, full-time personnel, types of contractors etc.) Questions 6 should also include 10 year needs and the flow would be improved if Question 6 was moved to before Question 5.
EPA Response: Additional language was added by EPA to clarify that water system staff (either directly employed by the water system or contracted employees) should be counted if they are responsible for day-to-day operations at the water system. EPA added language to clarify that “on-site” implies time spent at the water system for physical operations and maintenance. Question 6 was expanded to include 10-year needs and was moved to before Question 5.
Charge Question	Reviewer 2 Feedback (State & AI/ANV Survey Tool)
Are there any additional questions that should be included to better capture information on future drinking water system workforce needs?	Suggest trying to collect additional information potential challenges related to hiring full time operators versus contracted operators.
EPA Response: This initial Workforce data collection effort will allow EPA and industry associations to determine specific topics on which further data collection is most valuable. Full time system operators versus contracted operators is a nuanced topic that impacts systems differently based on system size and geographic location. If identified as a priority topic, EPA or other industry associations can explore this issue more in future data collection efforts.
Charge Question	Reviewer 2 Feedback (State & AI/ANV Survey Tool)
Does the workforce questionnaire capture information that is relevant to EPA’s stated goal of determining drinking water system workforce shortfalls over the next 5 and 10 years?	Yes, although it may be helpful to try and further identify specific challenges to recruitment and retention, such as retirement, stress/burn-out, and training/apprenticeship opportunities.
EPA Response: By changing Question 10 to allow the respondent to select up to 3 choices, EPA expects to collect additional information to help further discern the specific challenges facing systems with recruitment and retention.
Charge Question	Reviewer 2 Feedback (State & AI/ANV Survey Tool)
EPA estimates that it will take survey respondents (State, AI and ANV) 5 minutes to complete the supplemental Workforce questionnaire, do you think this is an accurate estimate of the time required to complete these questions?	If they have the questions in advance and know they need to have personnel files and company staffing/HR information available maybe but the first few questions might take 5 minutes a question to find, verify, and document the answers (non-multiple choice/perception questions).
EPA Response: EPA intends that the survey respondent will use their best professional judgement to answer questions about staffing and workforce issues at their system. EPA does not expect respondents to conduct additional research of human resource records as part of their questionnaire response. In many cases the system will have staff from multiple departments contribute the survey response, including staff familiar with system staffing and workforce details. EPA anticipates that if a survey respondent does not have the information required to answer a particular question on hand, they will skip the question or select “Don’t Know” as their response.