Attachment H

EPA ICR No. 2195.06 OMB Control No. 2070-0169

ATTACHMENT H

2021 Consultation for ICR Renewal for Submission of Protocols and Study Reports for Research Involving Human Subjects

ACC Antimicrobial Exposure Assessment Task Force Feedback October 12, 2021

1. Publicly Available Data

1. Are the data that the Agency seeks available from any public source, or already collected by another office at EPA or by another agency?

In limited cases, publicly available exposure data exist. Any existing public data are reviewed by AEATF II to determine if they meet the technical needs and the current quality standards prior to generation of new data.

2. If yes, where can you find the data? Is the available data truly duplicative, or are only certain data elements available which may not address our data requirements very well?

Published literature and publicly available databases are possible sources of publicly available exposure monitoring data. Usually, the available data are not duplicative as many critical elements are missing, they are not representative of the scenario being investigated. they are lacking quality control aspects, or they do not lend themselves to use in a generic database. Another potential issue is that any exposure monitoring data generated with human volunteers must meet current ethics standards, and typically the information needed to determine whether those standards are met is missing or unavailable in which case the data cannot be used to support a pesticide registration.

2. Frequency of Collection

1. Can the Agency collect the information less frequently and still produce the same outcome?

The AEATF II does not believe that Agency is seeking the human exposure data too frequently.

3. Clarity of Instructions

1. The rule is intended to require respondents to provide certain data for the Agency’s use. Is it clear from the regulations and other Agency guidance what you are required to submit and how to submit it? If not, what suggestions do you have to clarify the information?

The rule gives a general explanation of the process, but does not specify what needs to be submitted, how it needs to be submitted, and the steps leading up to the submission. In addition, the required changes to human subject recruitment and consenting processes since 2006 are not clearly documented. As was

pointed out in our 2016 response, these new procedures and requirements need to be incorporated into a revision to the OCSPP Series 875 Test Guidelines or other formal EPA-generated guidance document.

2. Do you understand that you are required to maintain records?

Yes, keeping detailed records is standard practice for the AEATF II as part of Good Laboratory Practice regulations. However, the volume of records that are generated and that must be retained has increased significantly. The number of pages in study protocols and final reports has increased 10 to 15-fold as a result of the EPA’s 2006 rule, Protections for Subjects in Human Research (2006 Rule).

3. Is it difficult to format the information for submission so that it is clear, logical and easy to understand?

The issue with formatting the submission is that it is very time-consuming due to the large number of documents required for each study submission, including multiple versions of the documents (e.g., protocol, informed consent form, survey reports, detailed sampling plans, SOPs, IRB correspondence, IRB review reports, recruitment flyers, researcher CVs, Spanish translations, etc.). Prior to the 2006 Rule, protocol submissions consisted of about 40 pages. After the 2006 Rule became effective, that submission increased to over 500 pages, which are arranged in volumes to make the review of the submission more manageable. Just the process of arranging and checking the final documents for a protocol submission involves a minimum of two people, one technical and one clerical, for approximately two to four days.

4. Are there forms associated with this process? If so, do you use them? Are they clear, logical, and easy to complete?

The only form provided by EPA is a short checklist of items specified under 40 CFR 26.1125 that must be included in each protocol submission. The form is derived from the regulation and is not particularly detailed or time-consuming to complete once all the documents have been formatted and paginated into the submission volumes. There is some duplication of information requested on the form. This form has not been updated since it appeared in 2006, and the form could be improved as it does not ask for sufficient detail to clearly capture all of the key information that a reviewer might need especially if the protocol has been reviewed by the IRB multiple times.

4. Electronic Reporting and Record Keeping

The Government Paperwork Elimination Act requires that agencies make available electronic reporting alternatives to paper-based submissions. Entities that submit study protocols and/or reports in response to EPA’s 2006 final rule may elect to submit the information either on paper, or electronically, via email, CD, or DVD.

1. What do you think of electronic alternatives to hard-copy data submissions?

The AEATF II is almost paperless in its documentations, so electronic submissions are preferred.

2. Are you keeping your records electronically? If yes, in what format?

Yes, AEATF II records are kept in several forms including MS Word, Excel, Adobe Acrobat PDF, JPEG, and e-mail files. The hard copies of the study- specific raw data are archived along with the electronic file (PDF) of the final reports. Printed hard copies of the final reports that are still being required for submission to EPA.

3. Does electronic submission benefit you by reducing your burden or permitting greater efficiency in compiling the information?

Since all documents associated with the protocol or study report are generated and stored electronically, an electronic submission is easier. The AEATF II submits final reports “electronically”, although three hard copies are also required for submission to the EPA Document Processing Desk. These final report documents can be over 1,000 pages making this a time-consuming and environmentally wasteful requirement. As we stated in our 2016 ICR Renewal comments, electronic submissions would be even more beneficial if the need for paper hard copies was entirely eliminated. Other EPA departments, including the Antimicrobials Division (AD), no longer require paper documents; the AEATF strongly urges the EPA Document Processing Desk to adopt a policy for electronic submissions without also requiring duplicative paper copy submissions.

5. Burden and Costs

1. The labor rates EPA will use to estimate costs for regulated entities are taken from the May 2014 National Industry-Specific Occupational Employment and Wage Estimates for NAICS code 541710 (Research and Development in the Physical, Engineering, and Life Sciences), published by the Bureau of Labor Statistics. The BLS fully-loaded hourly rates for this industry are $195/hour for management,

$99/hour for technical staff, and $59/hour for clerical staff.

Do you think these labor rates are appropriate? Can you suggest another NAICS code that would be more appropriate?

The labor rates used by EPA are lower than that incurred by the AEATF II, especially for the technical staff. The professional technical and management work of the AEATF II is done by highly specialized research scientists with MS or PhD degrees who work for the AEATF II on a consulting basis. The more appropriate rates are $320, $250, and $70 per hour for the management, technical, and clerical classifications, respectively.

2. EPA will estimate annual costs by multiplying the estimated average cost of burden hours associated with each of several classes of activities by the estimated number of times each year that class of activity is expected to be performed. Here are EPA’s estimates from the draft ICR:

Table 1. Weighted average burden and cost estimates for respondents for research involving intentional exposure of human subjects

Activities	Average Burden Hours Per Response			Total Per Response
	Management $195/hr	Technical $99/hr	Clerical $59/hr	Hours	Cost ($)
Rule familiarization and training	3	4	3	10	$1,156
Prepare and submit protocol for IRB review	11	83	13	107	$11,103
Prepare and submit protocol for review by the EPA and the HSRB	35	168	16	219	$24,353
Document ethical conduct of a completed study for which the EPA and the HSRB have reviewed the protocol; prepare and submit completed study for review by the IRB, the EPA, and the HSRB	43	987	51	1,081	$108,822
Store, file, and maintain records	6	15	8	29	$3,121
Total per response	98	1,257	91	1,446	$148,555

Annual Burden: 1,446 hours per response * 5 responses per year = 10,122 hours

Annual Costs: $148,555 per response * 5 responses per year = $1,039,886

Please note that the 5 “responses” per year include 5 protocols plus 5 studies. This approach to defining responses is consistent with that used and approved for this ICR in previous years.

Please enter your estimates of the incremental paperwork burden in hours by management, technical, and clerical staff associated with each occurrence of each activity listed into the table “Respondent Estimates”. Base your estimates on your experience since the rule became effective in 2006, and on your projections for the paperwork and recordkeeping burden of each activity over the period covered by the ICR renewal—i.e., between May 1, 2022 and April 30, 2023.

Please explain how you arrived at your estimates, and please estimate only the incremental burden imposed by the paperwork requirements associated with the rule, not the costs of conducting the research or costs you would have incurred if the rule were not in effect.

Respondent Estimates

Weighted average burden and cost estimates for respondents for research involving intentional exposure of human subjects

Activities	Average Burden Hours Per Response			Total Per Response
	Management $195/hr1	Techni cal $99/hr	Clerical $59/hr	Hours	EPA Cost ($)	AEATF II Cost ($)
Rule familiarization and training2	8	10	7	25	2,963	5,550
Prepare and submit protocol for IRB review3	25	225	30	280	28,920	66,350
Prepare and submit protocol for review by the EPA and the HSRB4	100	500	40	640	71,360	159,800
Document ethical conduct of a completed study for which the EPA and the HSRB have reviewed the protocol; prepare and submit completed study for review by the IRB, the EPA, and the HSRB5	60	1,000	60	1,120	114,240	273,400
Store, file, and maintain records6	10	40	10	60	6,500	13,900
Total per response	203	1775	147	2,125	223,983	519,000

Please note that the 5 “responses” per year include 5 protocols plus 5 studies. This approach to defining responses is consistent with that used and approved for this ICR in previous years.

1. Rates are from the May 2014 National Industry-Specific Occupational Employment and Wage Estimates for NAICS code 541710 (Research and Development in the Physical, Engineering, and Life Sciences), published by the Bureau of Labor Statistics.

If you wish to add typical industry labor costs which are different from the ones published by BLS, you are welcome to do so. For that reason, just in case, we have added a column with the title “Based on industry cost.”

Note, there was no such column; we added a “based on Industry cost” column. The estimated industry costs are based on typical industry labor costs ($320, $250, and

$70 per hour for the management, technical, and clerical classifications, respectively)

2. Consider this a one-time activity. Enter your estimate of what your total burden will be for rule familiarization and training during 2022-2025. Since you are already familiar with the rule, you may have little additional burden for this activity.

3. Estimate your average paperwork burden of preparing for a single IRB review which would not have occurred but for the requirements of the human studies rule. Consider IRB reviews both before and after EPA/HSRB review.

4. Estimate your average paperwork burden of preparing a single submission to EPA of a protocol proposing research involving intentional exposure of human subjects. Treat each repellent testing protocol as a single protocol, however many test materials may be involved.

5. Estimate your average paperwork burden to document the ethical conduct of a single study for submission

to EPA when the protocol has already been reviewed by EPA and the HSRB. Treat all reports reflecting a single execution of one protocol as a single activity, however many test materials may be involved.

6. Estimate your average paperwork burden for managing and archiving records of each submitted protocol or study report.

There is additional work associated with managing, storing and archiving documents as records containing confidential subject information (ICF, comprehension forms, subject information forms).

3. Please estimate in Table 2 below the frequency with which you expect to incur the paperwork burden associated with each class of activity described in Table 1. Your responses will be combined with those from others in EPA’s revised burden estimate. Please explain any assumptions underlying your estimates.

The AEATF II is approaching the last few years of its research program, so the paperwork burden will subside significantly in 2022 because there will likely be no new studies conducted.

Table 2

Respondent Burden Estimates: Estimated Frequency of Activities

Activities	Projected Number of Occurrences by Year
	May 2022- April 2023	May 2023- April 2024	May 2023- April 2024
Prepare and submit protocol for IRB review1	0	0	0
Prepare and submit protocol for EPA and HSRB review2	0	0	0
Document ethical conduct of a completed study for which EPA and the HSRB have reviewed the protocol3	1	0	0
Store, file, and maintain records	1	0	0

Notes for Table 2:

1. Count IRB submissions that would not have occurred but for the requirements of the human studies rule, including those both before and after EPA/HSRB review.

2. Count each repellent testing protocol as a single occurrence, however many test materials it may involve.

3. Count each executed repellent protocol only once, however many test materials or physical study volumes it may involve.

4. The Agency assumes there are no capital costs within the scope of this Information Collection Request. Do you agree?

The AEATF II agrees.

5. Are there other activities or incremental costs associated with the paperwork burden imposed by the human studies rule, not listed in the tables but which should be accounted for?

New SOPs and revisions to SOPs have been required to address the changes imposed by the human studies rule. Although a number of new and updated SOPs now exist, continual revisions are still needed based on feedback from EPA and the HSRB. There are management, technical, and clerical costs associated with this activity.

2022/2025 Consultation for OPP ICR Submission of Protocols and Study Reports for Environmental Research Involving Human Subjects

Response by: Agricultural Handler Exposure Task Force, LLC (AHETF)

1. Publicly Available Data

1. Are the data that the Agency seeks available from any public source, or already collected by another office at EPA or by another agency?

No. Existing public data and existing data submitted to regulatory agencies in the U.S. and other countries by AHETF members were reviewed by AHETF for applicability to its needs prior to the generation of new data.

2. If yes, where can you find the data? Is the available data truly duplicative, or are only certain data elements available which may not address our data requirements very well?

This is not applicable to the AHETF.

2. Frequency of Collection

1. Can the Agency collect the information less frequently and still produce the same outcome?

This is not applicable to the AHETF.

3. Clarity of Instructions

1. The rule is intended to require respondents to provide certain data for the Agency’s use. Is it clear from the regulations and other Agency guidance what you are required to submit and how to submit it? If not, what suggestions do you have to clarify the information?

The rule gives a general overall explanation of the process but does not cover exactly what and how it needs to be submitted. However, after considerable interaction with EPA since 2006, the AHETF now knows what and how to submit data successfully. Extending beyond AHETF, other registrants who have not had this interaction will likely have difficulty knowing how to make submission. EPA should consider developing guidelines.

2. Do you understand that you are required to maintain records?

Yes, keeping detailed records was a standard practice for the AHETF as part of the Good Laboratory Practice (GLP) regulations. However, the volume of records that needed to be kept increased significantly. The number of pages in protocols and final reports increased 10 to 15- fold as a result of the final rule.

Page 1 of 5

3. Is it difficult to format the information for submission so that it is clear, logical, and easy to understand?

The format utilized by AHETF, developed over time through discussions with EPA, is now clear and standardized but it was still time-consuming for the AHETF to format the submission materials due to the large number of documents required for each study (e.g., protocol, informed consent form, survey reports, detailed sampling plans, SOPs, IRB correspondence, flyers, letters to qualified study participants, Spanish translations, etc.). Prior to the final rule, protocols contained about 40 pages. After the final rule became effective, that number increased to over 2000 pages. The AHETF and EPA then agreed to some efficiency that lowered the number of pages to 400 to 550 (still more than 10 times what it was before the final rule).

4. Are there forms associated with this process? If so, do you use them? Are they clear, logical, and easy to complete?

The institutional review board (IRB) had several forms that needed to be completed. The only form provided by EPA was a checklist of items from the rule that had to be covered in every protocol. The form was taken directly from the rule and was not especially difficult to complete but did take a significant amount of time. In order to improve the clarity and efficiency of the protocol and report submissions, AHETF created new formats and tables to convey the information required.

4. Electronic Reporting and Record Keeping

The Government Paperwork Elimination Act requires that agencies make available electronic reporting alternatives to paper-based submissions. Entities that submit study protocols and/or reports in response to EPA’s 2006 final rule may elect to submit the information either on paper, or electronically, via email, CD, or DVD.

1. What do you think of electronic alternatives to hard-copy data submissions?

The AHETF was almost paperless in its documentations, so it definitely preferred electronic submissions.

2. Are you keeping your records electronically? If yes, in what format?

Yes, records were kept in several forms including MS Word, Excel, Adobe Acrobat, E-mail files, and CDs. Key documents were also stored on a task force server for easy access by AHETF members and EPA. The only hard copy documents handled by AHETF were study raw data that were archived and hard copy reports that were required for submissions to EPA. All the hard copy raw data and other GLP-required documents in the AHETF archives were eventually loaded on to USB flash drives that were sent to each Task Force member company for permanent archiving. The original hard copy data and reports in the archives were then destroyed as allowed by EPA Advisory 83.

Page 2 of 5

3. Does electronic submission benefit you by reducing your burden or permitting greater efficiency in compiling the information?

Most of the information was generated electronically, so converting this to hard copy for the submission was an additional burden. The AHETF submitted final reports “electronically”, although two hard copies of each report were required to be submitted prior to the electronic sending. It was with the hard copy submission that the MRID number was assigned.

Reducing the effort to only the electronic submission (and obtaining the necessary MRID number prior to submission) was helpful to the efficiency of the overall submission process.

5. Burden and Costs

1. The labor rates EPA will use to estimate costs for regulated entities are taken from the May 2014 National Industry-Specific Occupational Employment and Wage Estimates for NAICS code 541710 (Research and Development in the Physical, Engineering, and Life Sciences), published by the Bureau of Labor Statistics. The BLS fully-loaded hourly rates for this industry are $168/hour for management, $87/hour for technical staff, and

$50/hour for clerical staff. Do you think these labor rates are appropriate? Can you suggest another NAICS code that would be more appropriate?

The rates used by EPA were less than that incurred by the AHETF, especially for the technical people. The professional technical and management work of the AHETF was done by highly specialized research scientists who work for the AHETF on a consulting basis, so a classification for researchers with MS or Ph.D. degree requirements were be more appropriate. The more applicable rates for the next three years are $300, $250, and $75 per hour for the management, technical, and clerical classifications, respectively. This does not account for the sweat equity that goes into these programs by representatives of the member companies whose time is not charged to the Task Force.

2. EPA will estimate annual costs by multiplying the estimated average cost of burden hours associated with each of several classes of activities by the estimated number of times each year that class of activity is expected to be performed.

Please enter in Table 1 on the next page your estimates of the incremental paperwork burden in hours by management, technical, and clerical staff associated with each occurrence of each activity listed. Base your estimates on your experience since the rule became effective in 2006, and on your projections for the paperwork and recordkeeping burden of each activity over the period covered by the ICR renewal—i.e., between September 1, 2016 and August 31, 2019.

Page 3 of 5

Please explain how you arrived at your estimates, and please estimate only the incremental burden imposed by the paperwork requirements associated with the rule, not the costs of conducting the research or costs you would have incurred if the rule were not in effect.

The estimates are based on AHETF records on hours spent by consultants and actual costs.

Table 1 Respondent Burden Estimates:

Unit Costs of Discrete Activities Required by the New Rule

	Average Burden Hours Per Occurrence			Total Per Response
					Cost ($) Estimated by EPA	Actual1a Cost ($) to AHETF
	Mgt	Technical	Clerical
	$1681	$87	$50	Total Hour
Activities
Rule familiarization and training (per protocol)2	0	0	0	0	0	0
Prepare and submit protocol for IRB review3	0	0	0	0	0	0
Prepare and submit protocol for EPA and HSRB review4	0	0	0	0	0	0
Document ethical conduct of a completed study for which EPA and the HSRB have reviewed the protocol5	65	2000	60	2125	187,920	524,000
Store, file, and maintain records6	5	5	5	15	1,525	3,125
TOTALS	70	2005	65	2,140	189,445	527,125

Notes for Table 1:

¹ Rates are from the May 2014 National Industry-Specific Occupational Employment and Wage Estimates for NAICS code 541710 (Research and Development in the Physical, Engineering, and Life Sciences), published by the Bureau of Labor Statistics.

^1a Rates utilized form the those quoted from AHETF in Section 5, Burden and Cost of this response.

Please note: In case you wish to add it, a column has been added that reflects the actual costs to the AHETF.

As stated above, the AHETF used highly qualified technical consultants for doing the research, including preparation of protocols and reports. Their rates were higher than those specified by the Department of Labor.

² Consider this a one-time activity. Enter your estimate of what your total burden will be for rule familiarization and training during 2022-2025. Since you are already familiar with the rule, you may have little additional burden for this activity.

Page 4 of 5

³ Estimate your average paperwork burden of preparing for a single IRB review which would not have occurred but for the requirements of the human studies rule. Consider IRB reviews both before and after EPA/HSRB review.

This is not applicable to the AHETF since it will not be submitting any protocols in 2022-2025

⁴ Estimate your average paperwork burden of preparing a single submission to EPA of a protocol proposing research involving intentional exposure of human subjects. Treat each repellent testing protocol as a single protocol, however many test materials may be involved.

This is not applicable to the AHETF since it will not be submitting any protocols in 2022-2025

⁵ Estimate your average paperwork burden to document the ethical conduct of a single study for submission to EPA when the protocol has already been reviewed by EPA and the HSRB. Treat all reports reflecting a single execution of one protocol as a single activity, however many test materials may be involved.

This cost increased significantly due to difficulties in recruiting study participants under the ethics rules established by the Agency and the HSRB. The recruitment process requires very extensive documentation and record keeping by those who assemble lists of names, direct initial phone calls, make more detailed follow up phone calls, and visit with potential cooperators at their locations.

⁶ Estimate your average paperwork burden for managing and archiving records of each submitted protocol or study report.

3. Please estimate in Table 2 below the frequency with which you expect to incur the paperwork burden associated with each class of activity described in Table 1. Your responses will be combined with those from others in EPA’s revised burden estimate. Please explain any assumptions underlying your estimates.

The AHETF completed its research program, so this is no longer applicable.

Page 5 of 5