| Comment Source | Comment Type | General Comment | Comment Response | Notes |
| Association of American Medical Colleges (AAMC) | Other | In general, respondents supported, as does the AAMC, the goal of standardizing a format for interim research progress reports across federal agencies for the purposes of simplifying or streamlining reporting procedures. No concerns were expressed about the categories or elements for the mandatory and optional information sought in the new format. | Thank you for your comment. | |
| AAMC | Other | We have insufficient response to estimate the extent of the additional administrative burden imposed by the RPPR or to provide informed proposals for reducing this burden, but we need to caution that respondents uniformly believed that the requested information in total would “substantially” add to the already prodigious paperwork burden under which our investigators labor. The AAMC thus shares COGR’s overriding concern for the extent to which agencies may mandate “optional” reporting or create further requirements. | This comment has been noted. The work group will do everything possible to consider the burden in development of the RPPR format. | |
| Arizona State University (ASU) | Other | It is considered a strength that the proposed format generally follows NSF reporting categories since this would result in less change for researchers currently involved with NSF funded projects. Additionally, standardizing the report format in a uniform manner would be beneficial to investigators and their staff since they would no longer need to become familiar with the various formats of individual agencies. However, since the proposed report format is very comprehensive, it will result in an increased reporting requirement for those investigators working with agencies not currently requiring as much detail in their interim reports. | The RPPR collects more extensive data than is currently collected, but the work group thinks both agencies and grantees will receive better information. There may be additional burden on at least the first report for the project, but we expect to repopulate the next form with any information that carries over. The work group thinks this improved report will help agencies collect better information. | NOAA is ok with this. This puts more structure around pieces that used to be narrative. We could live with this. At ED, we have a general performance report and that format is not useful for our research programs. This report really fits better for those programs. |
| ASU | Other | It was suggested to trim down the interim progress report to just a summary of the year's progress in research, a brief summary of future plans, lists of publications/patents, and a broader impact/outreach statement…eliminating any other subcategories. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | While we understand the desire to keep this to the absolute minimum, that will not suffice for the kind of information NSF needs. Let's discuss mandatory versus optional. Agencies can pick and choose what they need to perform their reviews for a specific program. This could vary from agency to agency within program to program. One mandatory while others optional allow the maximum flexibility to collect the information required without overburdening others. |
| ASU | Other | It is felt that the single mandatory category of "Accomplishments" is the key element to be reported in the progress report and that the others should be optional. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | This is how the format has been designed, so we are comfortable with this. |
| ASU | Instructions | Generally the instructions are considered straightforward and easy to understand. | Thank you for your comment. | No response required. |
| ASU | Instructions | Improvement in the electronic flow for completing sections of the report without having to refer to pages of instructional materials is desired in order to make the process of completing the report more intuitive. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | This comment will need to be deferred to those developing the electronic systems. Our requirement was to create the report, not the system. |
| ASU | Other | The lack of standardization of agency progress reports is an administrative burden for faculty investigators and electronic submission of the progress reports will lend itself to a more efficient process. However, not having a common solution for collecting the information electronically will be confusing to the investigators and will add to that burden. We support initially having federal agencies test and implement this with universities participating in the Federal Demonstration Partnership and then when a common agency solution is determined to implement it with all universities. | The workgroup agrees that testing with the impacted community is worthwhile. This is an electronic system implementation issue. The workgroup will forward your comment to those who develop the electronic solution. | We have to get the FDP to agree to this. We also need to know which agency will host this because FDP would not be able to host. This will definitely be on the agency side, so who would volunteer? Absolutely a good idea to make sure all of the kinks are out and I hope that any electronic development would do testing. This goes back to idea that Grants.gov was supposed to be find, apply, and report, which is not going to happen. Would FDP consider piloting the non-electronic version? There would be some time where we will need to use both paper and electronic. Some people will continue to need to use paper. |
| ASU | Other | Access to view the PI submitted electronic progress reports is desired by institutional officials that have accepted the federal funds and provide oversight in order to ensure that deliverables such as progress reports are provided in a timely manner. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | Electronic issue. Ties into who should submit - PI or AOR. This should be kept in mind for any agency doing electronic implementation. Is there a question about organizations wanting access to their reports. They did not want to submit, they wanted access. |
| ASU | Other | Faculty investigator responses indicated that downloadable fillable forms are preferable to web-based forms based on their experiences with NSF FastLane which they indicated still has several awkward problems with annual reports and drop-down menus which are not very flexible. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | This comment not consistent overall with other comments we have received. We will pass this on, but we get very few requests for downloadable forms. The faculty who sat on the webcast said they wanted web based forms. |
| ASU | Other | Administrators indicated that web-based forms would be preferable because it can provide a more user friendly format without the need for access to multiple software programs or upgraded programs. A caveat was that having the ability to create a printable copy from a web-based form would be necessary in order to retain a copy of the form for institutional and investigator record keeping purposes. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | This comment not consistent overall with other comments we have received. We will pass this on, but we get very few requests for downloadable forms. The faculty who sat on the webcast said they wanted web based forms. |
| Columbia University Lamont-Doherty Earth Observatory CU/LDEO | Other | In principle, the categories for which information is requested are all reasonable, in that they represent beneficial activities supported by federal funding agencies. In practice, reporting has become part of a giant vice that is squeezing researchers from all directions. On the one hand, funding is becoming more difficult to secure, as agency budgets fail to keep up with inflation, or worse, decline in actual dollars. At the same time, the number of researchers competing for funding is increasing at a rapid pace. Consequently, agencies expect researchers to "do more for less", which leads to a real situation where funding is insufficient to complete the proposed research. On the other hand, reporting requirements and other administrative obligations are constantly increasing. The combined effect is that it is becoming more and more difficult to complete research projects given the funds that are awarded by agencies. | The work group will keep this in mind as it develops the final version of the RPPR. Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. | We will keep this in mind as we develop the final version of the RPPR. We have flexibility of mandatory versus optional in order to minimize the burden. |
| My general recommendation is to minimize the reporting to include only elements that are absolutely essential. The real impact of federally funded research, and the value to the nation, comes from completing the activities (e.g., publishing papers, creating new education initiatives, etc.), not from reporting on them. My plea is to minimize the amount of administrative paperwork so as to maximize the amount of time that researchers are able to devote to the primary activity. | ||||
| To illustrate, the time needed to prepare an annual report is estimated to vary between 5 and 16 hours (see Section IV "Paperwork Reduction Act" in the text extracted from the Federal Register and poster at http://www.nsf.gov/bfa/dias/policy/rppr/fedreg.pdf). I have many research projects where I receive only 1/2 month per year of salary, and some where I receive only 1/4 month per year of salary. When I receive only one week of salary per year, I can hardly afford to devote a day or more to preparing an annual report. Reporting is a valuable and important part of federally funded research, but it should not become so burdensome that it diminishes significantly the quality and quantity of research that can be performed under federal research grants. | ||||
| U.S. Department of Commerce--National Institute of Standards and Technology DoC/NIST | Context: NIST has grant-making authority with respect to a range of research and science programs focused predominantly on measurement science and standards. In addition, NIST has two statutory extramural programs. Comments from the NIST laboratories will begin with "Lab:" and comments from the extramural programs will begin with "Extramural:" | Thank you for your comment. | ||
| Other | Lab: We applaud the new standardized progress report because we believe its concise format will result in saved time for the grant recipients, and also allow the federal government to more effectively collect grant data across various agencies. | Thank you for your comment. | ||
| DoC/NIST | Other | Lab: We note that much of the information on the report is actually optional. So that raises two questions for us: 1. Who determines if it is optional information or not--the Federal agency or the recipient?; 2. and when would it be determined? At the time of the award? | The awarding Federal agency determines which categories are required or optional. This should be determined as early as possible, preferably at the time the funding opportunity is issued. |
The Federal agency determines whether categories are optional or required. This should be determined as early as possible, preferably at the time the funding opportunity is issued. This should also be addressed in the agency instructions for implementing the form. We should revisit this language and strengthen it. Make clear that agencies are the ones to make this mandatory or optional. |
| DoC/NIST | Other | Lab: The listing of information requirements looks complete, yet the agency has plenty of flexibility to streamline the reports it will require depending on the project and eliminate many of those requirements. All but one of the major categories of information are considered to be optional, and the only mandatory category, "Accomplishments," has several subcategories, which are also essentially optional. Therefore, we do not think that the reporting format will add significantly to the administrative burdens either of agencies or awardees, particularly if the format is standardized and is used for different types of projects. If it has not already been done, before publication, the format should be reviewed by a panel of people in the grant recipients community, not only by government agencies. | Thank you for your comment. This part of the clearance process gives the recipient community an opportunity to provide its feedback on the form. | |
| DoC/NIST | Other | Extramural: The following reporting categories, currently suggested as optional, should be considered as mandatory reporting elements: products/outcomes, participants, impacts. | The awarding agency can make it required if they desire, but the work group would like to maintain the flexibility for those agencies who do not deem this necessary or appropriate. | |
| DoC/NIST | Instructions | Extramural: If the awardees pay attention to the instructions provided within the format it would eliminate the PMT interpretations of percentage of progress evaluated against the tasks and milestones on a quarterly basis. This will help in the consistency between the PMTs interpretation of the amount of progress and the actual | Thank you for your comment. | I think they are saying they like it. |
| DoC/NIST | Other | Extramural: All electronic format is an advantage. Downloadable or web based is not all that important so long as confidentiality can be guaranteed. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | Thank you. We will be providing these comments to the people developing the system. |
| DoC/NIST | Other | Extramural: Information provided will make it easier to track diffusion on an on-going basis. | No response required. | |
| DoC/NIST | Other | Extramural: The first report will take a lot of time to complete, but subsequent quarterlies should be far easier and less time consuming. | The workgroup agrees. There may be additional burden on at least the first report for the project, but the workgroup expects to repopulate the next form with any information that carries over. It thinks this improved report will help agencies collect better information and will be less burdensome. The RPPR collects more extensive data than agencies currently collect, but our grantees feel like both agencies and grantees will receive better information. | We agree. We believe the advantages as we go through the process is worth the burden being imposed. |
| DoC/NIST | Other | Extramural: Much of the superfluous information is "up front," which implies that it is the MOST important information. Actually most of the important information, is part of the optional section and typically at the end, which implies even of lower importance. So it might be very difficult to teach our program priorities because the format of the reporting will appear to subconsciously undermine it each and every time it is used. | There is no prescribed order. Each agency may determine the order. If an agency determines that an optional section is mandatory for its recipients, the order may be influenced. | There is no prescribed order. Agencies will determine the order. This highest category is mandatory which makes the question confusing. If an agency determines something is mandatory, that influences the order. We tried to make it clear that the format is going to be difficult to provide in paper form. |
| DoC/NIST | Other | Extramural: We believe our typical award recipients would find this template very tedious and burdensome because of having to wade through so much that isn't very relevant. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. | If agencies don't want to ask for something, agencies don't have to ask for this. It can be a very short report. |
| DoC/NIST | Other | Extramural: We support the idea and would be interested in participating in the Beta testing. | Thank you. The work group will provide this comment to the agencies developing electronic systems so they know that NIST would like to participate. | |
| DoC/NIST | Other | Extramural: If it is possible for the template to be more of a collection of "modules" that each program selects and puts in an order that makes sense for the program's mission and priorities, this could be more easily tailored to reflect the program priorities relevant to us...this template is clearly focused toward a basic research program with a major program priority of education impact from the research, not impacts on society from the research, and the actual structure of the template implies it heavily. | This is meant to be broad to allow maximum flexibility so agencies can use it for all research and research related programs. | As long as they use the categories we provide, they can put it in whatever order they want. Reality is that they are correct. This has been focused on a basic research program. Not just research, but research and related activities. This is the broad stroke to put in maximum flexibility so agencies can use it for all research and research related programs. |
| DoC/NIST | Other | Extramural: This template doesn't really cover the reporting requirements as stated in OMB Circular A-110. Outline of the template and some comments by section compared to the reporting requirements in the DOC regulations 15 CFR Part 14, which implement OMB Circular A-110, for programs that use cooperative agreements or fund research that is beyond basic curiosity driven education oriented research. See regulations as copied below with bolded italics of things that are not adequately addressed in the template, more specific comments follow to address these concerns: 14.51 Monitoring and reporting program performance. (d) When required, performance reports shall generally contain, for each award, brief information on each of the following: (1) A comparison of actual accomplishments with the goals and objectives established for the period, the findings of the investigator, or both. Whenever appropriate and the output of programs or projects can be readily quantified, such quantitative data should be related to cost data for computation of unit costs. | There are two reporting forms, RPPR and PPR. Agencies should use the PPR for its programs that require this kind of cost data. Agencies can select which report meets their reporting needs. | Isn't this something we are trying to get away from? If they want this, they should be using the PPR, not RPPR. We have kept away from cost issue. They have to say generically in the changes section (if an agency opts to use this section). We have stayed away from costs because in some agencies this is a PI submission versus and institution submission. Some agencies have not implemented that their quantitative data be related to their cost data. Can agencies address this specifically? All we are allowing agencies to do is use the instructions to clarify a requirement by using additional program specific requirements. I recommend they use the PPR for their programs that require this kind of cost data. There are two reporting forms. Agencies can select which report meets their reporting needs. |
| (2) Reasons why established goals were not met, if appropriate. (3) Other pertinent information including, when appropriate, analysis and explanation of cost overruns or high unit costs. (e) Recipients shall not be required to submit more than the original and two copies of performance reports. (f) Recipients shall immediately notify the DOC operating unit of developments that have a significant impact on the award-supported activities. Also, notification shall be given in the case of problems, delays, or adverse conditions which materially impair the ability to meet the objectives of the award. This notification shall include a statement of the action taken or contemplated, and any assistance needed to resolve the situation. | There are two reporting forms, RPPR and PPR. Agencies should use the PPR for its programs that require this kind of cost data. Agencies can select which report meets their reporting needs. | As above. At ED, some components have asked for overall expenditure for a grant. Is it possible for agencies to add something around Federal expenditures? To get cost data, can get this from the PPR. NIH has recommended building an optional section for the cost component. We would have to have something in the instructions for using this that institutions would provide that information. Something like, "use of this section must be submitted by an organizational representative. This is addressed in PPR. Institutions are concerned with providing that information. Some agencies do not do it this way at all. |
||
| DoC--National Oceanic and Atmospheric Administration (DoC/NOAA) | Other | This standardized progress report format contains most of the information necessary for internal reporting as well as provides a good basis for the reporting of accomplishments. | Thank you for your comment. | |
| DoC/NOAA | Other | The flexibility to add program specific questions in the Optional Categories is appreciated and will allow the Program Office to retrieve specific data necessary for internal reporting and program monitoring. | Thank you for your comment. | |
| DoC/NOAA | Other | The standardized research report format provides detailed guidance and may be used as a template for progress reports provided by research grant recipients. The report format may also serve as broad guidance for education grant recipients who conduct research to educate and train students. | Thank you for your comment. | |
| DoC/NOAA | Other | There would need to be a period of educating the grantees as to what the report is looking for, as grantees might not be currently collecting or quantifying data such as training and professional development under their present system (RPPR., Page 3.) | The work group has noted your comment and it will be considered in the implementation of the RPPR format. | |
| Council On Government Relations (COGR) | Other | The single mandatory category – Accomplishments – focuses the progress report on its key element – progress in the science – and we urge agencies to give careful consideration before requiring any of the optional or agency-specific categories. | The work group has noted your comment. | |
| COGR | Other | Our over-riding concern remains the provision allowing agencies to create agency or program specific mandatory categories, report formats or instructions. We understand the need for flexibility to accommodate unique statutory or program-specific reporting but we urge agencies to remember the goal of the Federal Financial Assistance Management Act to streamline and simplify financial assistance procedures. We recognize that the GPC can only propose and recommend report formats and we will look to the Office of Management and Budget (OMB) in its review and approval of information collections to assist in achieving maximum uniformity. Our principal question to OMB will be what criteria it will use to assess and, if appropriate, permit agencies to impose non-standard requirements? We endorse the caution to the agencies to minimize the degree to which they supplement the proposed standard categories. The goal of achieving efficiencies and benefit to recipients through the standardization of reporting is lost with a broad proliferation of agency-specific forms. | Different agency needs will require report flexibility and agencies can use the RPPR to address any agency specific needs. | We don't know what OMB's criteria are. This will be the same for our competing application systems. We have to get approval from OMB on these forms and if OMB has forms, then different agency needs will require report flexibility and agencies can use the RPPR to address any agency specific needs. |
| COGR | Other | We would include an additional caution to avoid “re-purposing” sections of the mandatory and optional categories as well. This process of re-purposing fields has plagued the electronic application submission process through Grants.gov and we would hope to avoid this as we move forward with the reporting formats. | The work group has noted your comment. | |
| COGR | Other | The real lasting benefit of the standard Research Performance Progress Report will only be realized when it is available for completion and submission electronically. The lack of standardization in progress reporting is the principal burden identified by investigators participating in the survey of administrative burdens conducted by the Federal Demonstration Partnership. Despite differences both in institutional and individual work environments, the faculty respondents reported a similar set of top administrative burdens with progress reporting topping the list. Our experience is that the creation and implementation of electronic solutions at the Federal level is a time-consuming process. Deferring implementation of a standard format for progress reporting until there is a common solution for collecting the information electronically would be disheartening to our investigators. We would urge the consideration of a rapid deployment of an electronic solution as a demonstration project through the Federal Demonstration Partnership. Tapping into the expertise on university campuses may speed the deployment of the solution. | The work group agrees that testing with the impacted community is worthwhile. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | Use the response already provided to FDP. |
| U.S. Department of Defense (DOD) | Other | The proposed format for interim performance reports [72 FR 63629, November 9, 2007] generally strikes a good balance. It contains sufficiently detailed guidance on content to achieve greater standardization across agencies and programs. At the same time, it provides needed flexibility to be usable across those diverse agencies and programs. | Thank you for your comment. | |
| DOD | Other | Address the format as a research policy matter, rather than an issue for a type of funding instrument. Given that the Research Business Models Subcommittee is an arm of a research policy rather than grants policy body, it seems curious that we developed a format for reporting on research program performance and then proposed it for comment only for grants. It seems that we might increase the benefit of the format if we viewed it more broadly as being related to the program, rather than to grants, contracts, or other legal instruments used to transfer program funds. DoD and some other agencies carry out research programs using both assistance instruments and procurement contracts (selecting for each project the appropriate legal vehicle). | If DOD would like to use this for contracts, it may do so. That flexibility is available. |
|
| Similarly, universities and other organizations receive procurement contracts, as well as assistance instruments, for research performance. They therefore could benefit more from the reporting format if agencies’ use of it were not limited to their assistance awards. We believe the format appropriately can be used for basic research, at least some applied research, and perhaps some advanced research, irrespective of the kind of agreement used. Although an individual agency can in principle expand the format’s use for its own programs, RBM will better serve the community by identifying as a matter of Government‑wide research program policy—not grants policy—all of the programmatic types for which we think the format would be appropriate. | If DOD would like to use this for contracts, it may do so. That flexibility is available. |
|||
| RECOMMENDATION I.1: Specify the format’s use for all basic research and encourage its use, to the extent practicable, for applied and advanced research (also known as pre‑competitive technology development), especially for projects carried out by universities and other non‑profit organizations. | We deliberately did not define for an agency what research and research related is. They already have the flexibility to do this. This should be clarified in the agency specific instructions. DOE would not accept this for contracts. If we try to specify what type of research, it takes away the flexibility of using the RPPR or PPR. |
|||
| DOD | Other | Provide more data, rather than text, fields. It is costly and burdensome to import text to populate agency data systems. RECOMMENDATION I.2: The format should include data fields for information that any agency wishes to have as data. To collect data on journal articles, DoD awarding offices request data fields for article title and journal name, volume, date, and page number. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| DOD | Other | Provide for attachments, such as graphics, that can not be transmitted as text or data. RECOMMENDATION I.3: Explicitly state in the instructions that the format permits attachments of supplementary material not suitable for text or data fields | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | This is an electronic system implementation issue that will be forwarded at the appropriate time. Except where agencies need to submit data in the form, an attachment is ok. This is an implementation issue. |
| Disgruntled in Houston | Other | It would be very useful for a Congressional Oversight to take a look at the practice of release of findings from investigations that were sponsored by Federal grants. Some Federal agencies (ATDSR, to be precise) demands that researchers not publish results until the agency gave them a clearance. Meanwhile agencies have their own agendas and their own turf war. Research investigators and the scientific discovery are lost in this war somewhere. | These issues should be addressed with the Agency for Toxic Substances and Disease Registry (ATSDR). The work group will refer your specific concerns to ATSDR. | |
| To exemplify, for reason I cannot understand, ATSDR seeks the so-called “courtesy” approval from CDC (???) for release results of research on the grants they sponsored (there is no “courtesy” reciprocation on the side of CDC). Meanwhile CDC has “chip on their shoulders,” as the agency considers some areas of research (ex, birth defects) their “turf” and therefore, blocks requests for release of investigations sponsored by ATSDR. | These issues should be addressed with the Agency for Toxic Substances and Disease Registry (ATSDR). The work group will refer your specific concerns to ATSDR. | We have noted your comment. | ||
| Meanwhile, the public has no opportunity to even glance results and does not receive much needed information. Please disallow this outrageous practice of demanding release rights to the intellectual discovery in exchange for Federal grant money. This is not the purpose of research, and scientists should not be pushed around by bureaucracy in exchange for much needed federal dollar. | These issues should be addressed with the Agency for Toxic Substances and Disease Registry (ATSDR). The work group will refer your specific concerns to ATSDR. | We have noted your comment. | ||
| U.S. Department of Education (ED) | Instructions | The proposed Research Performance Progress Report (RPPR) intermingles general background information with instructions. We recommend having one “background information” section and then a separate “instructions” section for respondents. Even though the current draft includes sections that say “instructions,” the reader has to search for them and it remains unclear what information is provided for reader edification and what information requires a response. This could lead to inconsistent responses. Separate sections would resolve the problem. | The work group has noted your comment. | This is associated with trying to provide a format via this medium. Until you can see how an electronic system can do this, it would be difficult to see how this really works. People were really thinking about this like a form. It is a challenge for those who are viewing this for the first time and don't see the benefits or this format. |
| ED | Other | The second sentence of the second paragraph on page 6360 of the Federal Register notice states: “Award recipients would be required to report on the mandatory category and may be required to report on the mandatory category and may be required to report on any of the optional categories...” (italics added.) Further, page 1 of the draft RPPR states, “Federal agencies will direct recipients to report on the mandatory category and may direct them also to report on the optional categories, as appropriate.” Based on this, it appears that the words “may be required to report on the mandatory category,” above, can be deleted. In any case, the language needs to be clarified. | Thank you for your comment. There was a typo in the notice, which has now been addressed. | There was a typo in the notice and after working with OMB, they printed a correction. |
| ED | Instructions | We recommend that the form clarify how respondents with multiple projects within one award will input their data. For example, many National Institute on Disability and Rehabilitation Research (NIDRR) grantees conduct multiple projects in multiple categories (e.g., research, training, technical assistance, dissemination). There should be a clear and established way to enter data for each project in the award | Different agency needs will require report flexibility and agencies can use the RPPR to address any agency specific needs. | Agency specific requirement. |
| ED | Other | While this document was published in the Federal Register, we cannot determine whether this is one of the notices required under the Paperwork Reduction Act of 1995 (PRA) or, if it was published under the PRA, whether it was the first or second required PRA notice. We note that the comment period is 60 days, which suggests this is the first PRA notice. If this document was published pursuant to the PRA, we recommend that the next notice clarify that it is published under that Act. | This is the first 60 day PRA notice. | This is the first 60 day PRA notice. OMB cleared this as a PRA notice. |
| ED | Other | The notice is not clear regarding the source of the burden hour estimate of 5 to 16 hours. Is this estimate based only on NSF grantee reports or reports submitted to all agencies? If the estimates are based only on NSF experience, we strongly encourage NSF to provide estimated burden hours for each agency or provide one estimate of average burden hours based on all agency experience with reporting. This effort is essential so a single form, with a single OMB control number can be approved for use by all agencies. | This estimate is based on all agencies' input on what the burden may be. | This estimate is based on all agencies input on what the burden may be. |
| ED | Other | The notice does not discuss record keeping burden for the proposed RPPR. Given the fact that the reports are designed to collect data only regarding the most recent reporting period, we believe that grantees would have to maintain past reports, so there will be at least some record keeping burden associated with the RPPR. We strongly urge NSF to include an estimate of the record keeping burden in the final PRA notice. | There will be a burden estimate in the final PRA notice. | We will still have to have a burden estimate in the final PRA notice. This is all over the map on what agencies expect. |
| ED | Other | The format for the form was not clear from the file that was made available for comment. The draft RPPR describes each category and provides instructions on each category, but it does not give notice of how much space will be available for each category. Persons responsible for reporting need to know the maximum size of each category. In a paper form, the space allocation is clear through visual inspection and instructions that inform reporters of whether they may provide additional information on a separate sheet of paper. In the electronic context, a filer needs to know how many alpha and numeric character spaces are available for each category. Please specify the size of each reporting field. We recommend that the next Federal Register notice for the RPPR make available to the public, both a paper version of the form and a document that describes how many character spaces would be available for each field of the expected electronic format. | This is a format, not a form. Agencies can define page limits when appropriate. |
They keep thinking this is a form. Make clear that if agencies require it in a paper format, how would that work? Would there be a character limitation if a system is used? |
| ED | Other | The proposed RPPR does not explain the implications under the PRA for agency inclusions of added categories or additional instructions. Will additional instructions require PRA approval by OMB? We assume that added categories of reporting will require PRA approval by OMB. However, will agencies be required to vet additions and instructions through the NSTB RBM, the Grants Policy Committee for PL 106-107, or some other process before going to OMB or can an agency go directly to OMB? | As specified in the Federal Register notice, agencies may develop additional agency- or program-specific reporting categories and instructions, however, to maintain maximum uniformity, agencies will be instructed to minimize the degree to which they supplement the standard categories. Additional agency- or program-specific reporting categories are subject to review and clearance by OMB via the PRA process. Prior to submission to OMB, however, agencies will be requested to consult with the RBM subcommittee, to ensure non-duplication of like data by the research agencies. | It says in one place that this will need to be cleared by OMB. |
| ED | Other | The draft format does not contain the required language regarding effect of not displaying a currently valid OMB control number (5 CFR 1320.5(b)(2)(i) &(ii)) nor does it describe how the form will make that information available on or with the form “in a manner that is reasonably calculated to inform the public” of that information. In other words, how and where on a paper version or in the display of the electronic version would the RPPR display the OMB control number and the required request for persons using the form to provide comment to OMB regarding the burden imposed by the form. | The OMB clearance number will be on the notice. | We have a way to show OMB clearance number, but it will be different by electronic system. |
| ED | Other | It is not clear from the notice when the drafters of the proposed RPPR anticipate that the form would be implemented by agencies, once approved by OMB. Current ED research programs either use OMB approved program-specific performance reports or use the standard ED 524B, Grant Performance Report (http://www.ed.gov/fund/grant/apply/osep/continuationpackage2008/324.doc). Some programs have electronic data collection systems or databases that have been designed to collect and house data for a specific program. It would be helpful to know the implementation timeframe for the RPPR and whether agencies are expected to implement the RPPR form for current grantees mid-project period or whether the form will only apply to new research grants awarded after a certain date. As noted above, it would be useful to know the expected procedures for obtaining approval for agency or program specific categories or instructions for the RPPR in order for agencies to better gauge when the form can be fully implemented for their research programs. | These are implementation issues that are yet to be determined. Once the work group goes through the development of the format, implementation concerns will be easier to address. |
These are implementation issues that are yet to be determined. Once we go through the development of the format, implementation concerns will be easier to address. No agency will be prepared to implement right away. |
| ED | Other | We also recommend that further information be provided regarding whether “specialized research grant” programs that use other OMB approved reporting formats instead of the RPPR will need to follow a certain procedure to become exempt from using the RPPR. | It is up to the agencies to determine which programs are research or research related programs. | We have not determined this for agencies. Agencies should decide. |
| ED | Other | Additionally, the data requested in the proposed RPPR should allow for improved monitoring, management, and reporting about the research activities. When designing the electronic system, we suggest that the developers explore organizing the input screens around these different uses for the data. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | Thank you for your comment. We will forward your comment to those who develop the electronic solution. |
| Emory University | Other | By in large I am happy with the draft format. It is critical that this form be simplified to its bare bones, and most of the optional sections can be dropped. | Thank you for your comment. | Given how this format is created, agencies will only be using the sections they feel they need to determine appropriate progress. Some agencies will only use the required section. The optional sections are ones we want to retain. |
| U.S. Department of Energy--National Energy Technology Laboratory (DOE/NETL) | Other | The stated objective of the initiative is “to establish a uniform format for reporting performance on Federally-funded research projects” administered through research grants and cooperative agreements. Based on this objective, it is most appropriate that the reporting format accommodate the fundamental principles of sound project management, which all award recipients are expected to follow. These periodic reports must provide sufficient information for Government program officials to assess progress toward project objectives within the context of the planned work scope, schedule, and budget. Without timely and accurate progress reporting on research and development (R&D) projects, it is not possible to satisfy requirements of the Government Performance and Results Act for evaluating progress toward R&D program goals. | The work group agrees. This is consistent with what the work group has attempted to do. | |
| DOE/NETL | Other | The draft RPPR does not include sufficiently detailed content that would satisfy DOE project management policy and OMB requirements to report, monitor and validate cost, schedule, and technical progress relative to the performance baseline. This information is vital to DOE decision-makers. | If the RPPR is not suffcient, the PPR may be used, or an agency may develop its own agency specific format, subject to OMB approval. | They want monthly reporting, they want to see every reciept. Much more detail than they need. |
| DOE/NETL | Other | The standard approach to research progress reporting should also consider the frequency of reporting needed to satisfy project management and OMB requirements for monitoring and validating progress toward program goals and objectives. Standard project management practice is to report and assess progress at least monthly to ensure effective communications among project participants. Current financial assistance regulations specify that progress reporting shall not be required more frequently than quarterly or less frequently than annually. Quarterly reporting may be appropriate for basic science grants. Applied RD&D projects have time-critical milestones relative to goals and objectives, they tend to be complex, and they are likely to have significant scope, cost, and schedule risks. The award recipients must communicate progress among project participants at least monthly to effectively manage the work in accordance with the established baseline. DOE must monitor the progress monthly to independently assess performance trends, respond to risk events before they become critical, and provide timely status reports needed for management decisions. | The work group is keeping with the reporting requirements currently specified in A1-10. If an agency wants additional requirements it may request clearance from OMB. | The RPPR was designed for basic research projects. |
| DOE/NETL | Instructions | In general, the draft RPPR does not provide a clear and definitive format that would be instructive for individuals managing R&D projects. The directions could be simplified to clearly state the project information that must be reported. | The work group believes they have accomplished this. | Wouldn't the agency provide the reporting requirements |
| DOE/NETL | Other | Final implementation should be deferred until a common approach is established for reporting electronically, preferably through a web-based application. It is not clear that the intent of standardized reporting is to “collect information” but rather is intended to provide a means to effectively report progress toward project objectives. This progress is reported relative to the planned scope, cost, and schedule. If the recipients of financial assistance awards are actively managing the R&D project, which includes effectively communicating among project participants and their management, there should be no burden to the recipient in reporting progress monthly. It is the exception for R&D organizations not to be proficient in information technology for management and communications purposes. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | We shouldn’t do anything until we have it electronically. |
| U. S. Department of Health & Human Services (HHS) | Context: To assist in the development of this response, the Office of Grants Policy, Oversight and Evaluation, OS/ASRT/OG of the U.S. Department of Health and Human Services solicited input from the HHS grant awarding agencies, which in turn solicited input from their grants and program management staff. Substantive feedback was received from the National Institutes for Health, the Food and Drug Administration [comments not received through the formal Federal Register process--we are following up with HHS to try to locate the FDA comments], the Agency for Healthcare Research and Quality, the Centers for Disease Control, the Health Resources and Services Administration, the Centers for Medicare and Medicaid Services, and the Administration on Aging. While OGPOE had anticipated to submit a single response on behalf of all HHS grant awarding agencies, it is clear that this is not possible as there is little consensus among our awarding agencies and programs regarding the proposed RPPR elements and implementation. | No response required | ||
| Alternatively, we have encouraged HHS awarding agencies to submit their comments directly to you. The comments provided in this response attempt to highlight some of the issues identified by multiple agencies which OGPOE feels warrant significant attention. | ||||
| HHS | Other | There was no consensus among reporting HHS awarding agencies regarding the appropriateness of the categories and elements. More than half of commenters expressed support for the categories and elements, however some questioned the utility of the elements to agencies and cited an excessive burden on grantees. Another comment noted that although appropriate, some of the categories and elements would need some revision to include currently required program-specific reporting information in the interim progress reports as well as in final reports. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS | Other | Should other categories be mandatory? Several HHS awarding agencies requested that multiple additional elements be mandatory, some of which are listed above in response to Question 2 [change in personnel, unobligated balance, IRB/IACUC approval, clinical trials]. One agency requested that Product/Outcomes be mandatory, and another agency requested Impact be mandatory. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS | Other | However, these awarding agency and/or program-specific needs might be satisfied if awarding agencies retain the authority to require their grantees to respond to the questions/elements within an optional category. Please note that this requirement (i.e., that a grantee must respond to an optional category where required to do so by the awarding agency) should be more clearly stated in the RPPR—comments received by OGPOE suggest that some HHS awarding agencies’ staff understood that it might be up to the grantee to choose whether it would respond to an optional category. | The work group will keep this in mind as it develops the final version of the RPPR. Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. |
This is not the first time this issue has come up. Use of optional and mandatory is not clear enough. |
| HHS | Instructions | In general, HHS awarding agency commenters found the instructions straightforward and easy to understand, although some noted that agency-specific and program-specific instructions will be necessary to capture required reporting elements (e.g., PART), to ensure the responses can be compiled in aggregate for agency’s entire program, and to otherwise meet agency-specific or program-specific needs. | Thank you. The agency may consider using the PPR. It has specific section for reporting on PART. The agency may also pursue developing agency specific requirements through OMB. | |
| HHS | Other | A clear majority of awarding agencies recommended delaying agency implementation until a common electronic solution is operational. Agencies cited excessive burden to both agencies and grantees, as well as the applicability of the Government Paperwork Elimination Act, as the primary reasons for this position. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS | Other | There was no consensus regarding whether to use a downloadable fillable form or a web-based form. | No response required | |
| HHS | Other | The provision of a clear definition of “research” and additional guidance regarding the applicability of this format to agencies’ research programs would also be helpful. Some HHS staff commented that they were unsure as to the applicability of the RPPR to their programs and/or whether it would be used in addition to or in lieu of their programs’ current program-specific progress reporting forms. | Agencies should determine if this format is appropriate for their needs. The RPPR attempts to give flexibility to determine what is right for their program. | There will be research programs, in which the community asks why arent they using the RPPR? We have this same issue on the development of research terms. |
| HHS | Other | OGPOE applauds the efforts taken to develop a standard research performance progress report, however, the conflicting reactions to the proposed interim Research Performance Progress Report provided by a number of programs within the seven responding HHS agencies and their array of programs leads OGPOE to have serious concerns about adopting the RPPR without additional review and revisions. Given the potential for disruption to the annual flow of research award funding across government that is dependent upon such report information for continued funding and considerations for additional new funding, OGPOE recommends the National Science Foundation and the Research Business Models subcommittee continue to work with agencies and grantees to further refine the format. | So noted. The work group will continue to work to make sure that everyone's comments are carefully considered when developing the next version of the RPPR. | |
| HHS- Administration on Aging (HHS/AoA) | Other | Are the categories and elements appropriate? Yes | No response required | |
| HHS/AoA | Other | Should other categories be mandatory? No. | No response required | |
| HHS/AoA | Instructions | Are the instructions straightforward and easy to understand? Yes. | No response required | |
| HHS/AoA | Other | Should agencies defer final implementation until there is a common solution for collecting the information electronically? No. | No response required | |
| HHS/AoA | Other | Should agencies use a downloadable fillable form or web based form to report progress? Yes. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS--Agency for Healthcare Research and Quality (HHS/AHRQ) | Other | This format appears to be appropriate for patient safety grants that I have had in the past. This may be somewhat overwhelming for my dissertation grantees. If it is clear that N/A is okay and not all fields are mandatory, this should work for this mechanism (vehicle –new word learned yesterday during the Project Management Training). | No response required | |
| HHS/AHRQ | Other | This draft report looks very similar to a final report format. It asks for the impact made by the project which could be problematic because it may force grantees to make draw conclusions or make assumptions about their projects before they are really ready to do so. | If the requirement is a burden, the agency need not request it from grantees. Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/AHRQ | Other | The categories and elements appear appropriate. | Thank you. | |
| HHS/AHRQ | Other | Abstracts are nice to have for Agency Highlights (250 word maximum is satisfactory). | Thank you for the comment, the work group will consider it when developing the final format. |
|
| HHS/AHRQ | Other | The general structure of the report seems fine. One question would be how much tailoring would be allowed if the report ‘format’ is approved. If we would like to use the progress reports to monitor results related to our PART goals—would we be able to add those fields? As well as other more specific fields/questions we currently have. | The agency may consider using the PPR. It has a specific section for reporting on PART. The agency may also pursue developing agency specific requirements through OMB. | |
| HHS/AHRQ | Other | There will be demand to make heavy use of Agency and Program specific supplemental formats. The organization and response categories from the draft format make sense as does the idea that there would be a single mandatory format with optional formats for specific types of grants. It may be unrealistic to expect agencies to minimize the number of agency or program specific formats, however, given that each Agency and program will have it’s own PART objectives, they will see their information needs differently and so some flexibility around assuring that Agencies and programs have the ability to collect data on their PART metrics seems appropriate. | The agency may consider using the PPR. It has a specific section for reporting on PART. The agency may also pursue developing agency specific requirements through OMB. | |
| HHS/AHRQ | Instructions | Structured vs. unstructured questions – may want to allow Agencies to define instructions differently. There is an open ended nature to the draft format. What agencies and their contractors often want or need to do is aggregate and summarize findings from the program as a whole and compare reporting outcomes across grantees. Unambiguous, closed ended questions are more conducive to this objective. Close ended questions are challenging to incorporate into a format that’s meant to work across federal grants, however, there may be opportunity for different agencies or programs to use/provide different types of instructions associated with each of the sections of the report that allow the responses to be analyzed in a closed ended manner – for example, AHRQ’s definition of what they want grantees to report in the “accomplishments” section may be different and more closed ended than another Agency and if the format can accommodate variations in the instructions to grantees, this may be a way to have a consistent format but also allow each Agency to get responses structured in a manner that makes sense for them. | The standardized instructions were developed to ensure consistency across agencies wherever possible. The work group will forward your comment about the programattic information required to those who develop the electronic solution. | Agencies have been allowed to add agency specific requirements on past forms and there is no consistency. To do this gives the illusion of standardization. This group sounds like they need the PPR more than the RPPR. Though the work group will send this to the electronic folks to see what can be done about the programatic info needed. |
| HHS/AHRQ | Other | Should other categories be mandatory? No, unless we have different formats for different mechanisms (grants) and this would defeat the purpose of streamlining. | No response required. | |
| HHS/AHRQ | Instructions | Are the instructions straightforward and easy to understand? 1. Yes. 2. It is long and wordy. Is it possible to have the report format and a guide? This is just a thought. 3. Yes, but I think the report requirements are a bit lengthy. | The electronic format will make the RPPR easy to use. Tabs will allow the grantee to view the essential instructions only. | In an electronic system, much of the instructions will be pared down, the justification for reporting can be hidden on a different tab. |
| HHS/AHRQ | Other | Agencies should be able to use a fillable form that is savable to their own PCs. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS--Centers for Disease Control & Prevention (HHS/CDC) | Other | The categories and the elements appear to be appropriate; however, the formatting appears to be difficult to delineate the individual categories and the corresponding instructions and examples for each. | The form is designed as an electronic format. The format will be clarified when in electronic format. | The work group was designing this as an electronic format. Is there a better way to show the format on paper? |
| HHS/CDC | Other | Should other categories be mandatory? The sponsoring agency/program component may wish to make some of the optional categories mandatory (program-specific decision). This comment in no way recommends changing the elements and instructions within the category. | Thank you. | |
| HHS/CDC | Other | Should other categories be mandatory? Budget and table of key personnel. Current copies of all IRBs if human subjects are involved. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/CDC | Other | Should other categories be mandatory? IACUC, Clinical Trials information should also be addressed when applicable. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/CDC | Other | Should other categories be mandatory? Yes, the Products/Outcomes Category. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/CDC | Other | Should other categories be mandatory? The one piece that is missing that I think is important as a mandatory element is a description of challenges (e.g., problems, barriers, delays) that they experienced and how they were resolved as well as any challenges that they anticipate and how they propose to prevent them. This information can help our staff to focus in on the issues of greatest concern to the PI's and also allow us to learn whether challenges that we are concerned about are recognized by the PI. | Many of these requirements are available in the optional sections of the format. The awarding agency can make the sections required if they desire, but the work group would like to maintain the flexibility for those agencies who do not deem this necessary or appropriate. | |
| HHS/CDC | Instructions | Are the instructions straightforward and easy to understand? Overall, instructions appear to be easy to read. In the Mandatory category, sub-topic "Accomplishments" and "Instructions-Accomplishments", it seems the words "project" and "activity" are used interchangeably. Interchangeable use of these terms is confusing. See the first bullet under "Accomplishments" and the first sentence under instructions. It seems the agency approves the project and the activities therein. | Thank you. The language on the format will be updated. |
Not sure why it would be problematic. Activites are not the project. |
| HHS/CDC | Instructions | The instructions are relatively straightforward; however, consistency with wording needs to be applied such as "none," "nothing to report," and "no change" are listed as possible answers to various questions. | Thank you, the work group will work to standardize the language. |
Prefer nothing to report. Except there are certain blocks where it would not be appropriate. "None" refers to a number or specific response. |
| HHS/CDC | Other | Should agencies defer final implementation until there is a common solution for collecting the information electronically? No | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS/CDC | Other | Should agencies use a downloadable fillable form or web based form to report progress? A web-based or fillable form for the report has value. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS/CDC | Other | Should agencies use a downloadable fillable form or web based form to report progress? We suggest piloting the format using the downloadable fellable form. The pilot will inform the identification of a common solution for collecting the information electronically and the appropriateness of the elements and its usability. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS/CDC | Other | We applaud the efforts being undertaken to establish a uniform format for reporting performance on Federally-funded Research projects. It will certainly improve our ability to compare research projects across the government. | Thank you. | |
| HHS/CDC | Other | How will this replace the current 2590 form that we use for interim progress reports (http://grants2.nih.gov/grants/funding/2590/2590_forms.pdf)? | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. For agencies currently using the PHS2590, the bulk of the material collected in the PHS 2590 will be replaced by the RPPR. However some material currently collected is not part of the RPPR. The work group envisions the PHS2590 will eventually be amended to reflect only those agency-specific requirements not covered in the RPPR. | |
| HHS/CDC | Other | Will there be page limitation for each section or for the overall report? | This is a format, not a form. Agencies can define page limits when appropriate. | |
| HHS/CDC | Other | The required section length is reasonable and the rest is optional. I think there would be value in a standard report, no surprises from PGO each year and it allows comparability. | Thank you. | |
| HHS/CDC | Other | I am not clear if it is in addition to the 2590, so we need clarification on that point. If it is then I think it is an unnecessary burden to the PI and against the paper reduction act's intent. | It is not in addition to the 2590. The work group agrees that both formats should not be used. For agencies currently using the PHS2590, the bulk of the material collected in the PHS 2590 will be replaced by the RPPR. However some material currently collected is not part of the RPPR. The work group envisions the PHS2590 will eventually be amended to reflect only those agency-specific requirements not covered in the RPPR. | |
| HHS--Health Resources and Services Administration (HHS/HRSA) | Other | Please note that owing to the diversity of HRSA grant programs, there were a wide array of comments, some of which were mutually exclusive depending on the question or issue being commented on. While some felt there might be utility in its implementation others expressed concern that it would be burdensome, and not achieve its intended results. To better understand the reactions generated by the proposed form, we have not attempted to develop a consensus response and have provided as many of the comments as we felt appropriate. | No response required | |
| HHS/HRSA | Other | Prior to the implementation of a standard progress reporting tool for research projects HRSA believes it is essential that a definition be provided for “research project”. Is research, for purposes of utilizing this form, limited to biomedical-type research or is this meant to incorporate social science type research as well? Further, if the definition is restricted to biomedical research, can agencies which award social science research be exempt from use of the form if it does not adequately capture reporting needs of the agency? | Different agency needs will require report flexibility and agencies can use the RPPR to address any agency specific needs. | |
| HHS/HRSA | Other | Is there an implementation timetable for the proposed reporting form? HRSA’s progress report and performance measurement is electronified. Therefore, any required changes in format/questions will require system changes. A timeframe for implementation will be necessary in order for HRSA to implement these changes in the enterprise–wide project management system which incorporates performance measurement and progress reporting. | While there is no timetable at this time, part of the overall implmentation, agencies are always prvoided implmentation instructions for how much time permitted to use this format. | Agencies are always given plenty of advance notice for when a form will be implemented. |
| HHS/HRSA | Other | This seems to be an extremely lengthy and burdensome progress report format. The Federal Register notice says one intent is to make it easier to administer Federal grant programs through standardization of the types of research information required in performance reports. It is doubtful this will be the result of this format. Rather, it is likely to add additional burden to the reporting system. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/HRSA | Other | In addition, the focus is on interim, or progress reports. It is questionable whether the additional burden imposed by the new interim reporting requirements is worthwhile. Much of the information asked for could be usefully included in a final report, and appears to add little value when reported on an interim basis. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/HRSA | Other | However, another office noted that the overall reporting changes proposed by the National Science Foundation will significantly enhance awardee accountability and promote a better understanding of the impact of public investments in research. | Thank you for your comment. It should be noted that this proposed format was developed by an interagency group. | |
| HHS/HRSA | Other | One HRSA Bureau noted that while the proposed reporting form is quite comprehensive, it would be cumbersome for some of its research grants to Centers because these Centers complete or work on anywhere from 4 to 30 research projects a year. It was further noted, however, that grants to Centers would presumably remain exempt from using this form. | Thank you for your comment. | |
| HHS/HRSA | Other | One respondent noted that in general, because research projects vary so much, the proposed reporting form should be more open ended, with more general questions but fewer of them. While another respondent noted, the proposed form appeared to cover the broad elements that its office was asking its grantees to report on. | Different agency needs will require report flexibility and agencies can use the RPPR to address any agency specific needs. | |
| HHS/HRSA | Other | One commenter noted, research projects are more "step-dependent" than other kinds of projects like demonstrations or especially training grants. That is, success of the project is very dependent on each step or stage being carried out successfully. While research projects vary greatly, most follow a pattern of steps similar to the following: hypothesis development; methodological development; instrument development; survey administration; data collection; data analysis; report writing; and report dissemination. | No response required. | |
| HHS/HRSA | Other | The draft is surprisingly general (page 2) in pinpointing where the grantee stands with regard to progress on the project - in terms of the above steps. Also, it appears there are at least three dimensions the supporting agency would be interested in regarding a project's status: (1) the salience or policy-importance of the research, (2) threats to success of the research, and (3) the locus on the time scale for completion. It does not seem that the report form addresses those clearly. Further, beginning around page 5, the questions appear burdensome and possibly unnecessary. These are generally included in the grantee's approved proposal, and what the agency needs to know is if there has been any variance from the original plan rather than a reiteration of what's already been stated. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | The report can cover what they are asking for. |
| HHS/HRSA | Other | One Bureau noted two cross-cutting concerns about the RPPR. The first is reporting level. The instructions should clarify the level of reporting, either at the overall level or at the individual project level. The Rural Health Research Centers (RHRCs) each have one cooperative agreement that covers four years, with three individual research projects added each year, i.e., a total of twelve individual projects by year 4. It was not clear whether “Accomplishments” and “Products” would be reported once at the overall level, for each year of the cooperative agreement, or for each individual project that is in progress. The Bureau noted the latter approach would result in a much larger reporting burden and was concerned about the RPPR meeting their need to report on the progress at the overall level of an entire research center (as opposed to simply reporting on a stand-alone research study). | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| In short, the RPPR seems reasonably well-suited to the collection of information about the progress of individual research studies. However, it does not provide an adequate mechanism for reporting on the progress of the full scope of what our research centers do, such as providing technical expertise to public and private stakeholders and disseminating policy-relevant observations to diverse rural, local, state and federal stakeholders. Because the RHRCs frequently respond to requests for information that go well beyond the funded research studies, it is essential to ORHP to document progress on this important service to our constituents. Consequently, the Bureau recommends that the RPPR collect research progress on an overall level. | ||||
| HHS/HRSA | Other | In addition, some of these categories, for example those pertaining to development of human resources and infrastructure, are particularly relevant to overall reporting at the level of a research center as opposed to an individual rural health research project. For example, a center provides “opportunities for research and teaching” and can improve the representation of underrepresented groups in the sciences. With the emphasis in the cooperative agreement on minority student/faculty recruitment and development, significant outcomes in human resources would be expected at the overall level, rather than individual project-level outcomes. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/HRSA | Other | The second crosscutting concern is reporting burden and the method for electronic collection of the data. Preliminarily it seems that estimates of the burden for completing this form (5 to 16 hours) are understated. Information necessary to complete this form is significantly more extensive than that which is currently reported in interim reports to ORHP, such as project-specific details on individuals who have worked at least one person-month per year on a study. These, as well as other provisions in the RPPR, pose an undue burden for awardees that may more than double the estimated reporting time. We estimate that 32 to 40 hours seems to be more reasonable. Of course, if optional categories are made mandatory, burden will be higher. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | Optional is at the discretion of the agency. If don’t need it, don’t ask for it. |
| HHS/HRSA | Other | The optional sections pose substantial burden on the respondents and the purpose and utility of the information is not evident. If a form is going to be standardized, optional sections have very limited utility. Information obtained from optional sections should never be used to make generalizations about the universe of respondents, as only a portion of respondents will provide the information. | The required use of the optional forms will be at the discretion of the agencies, not the grantees. Additionally, the optional format has been succesfully used before to produce informative reports. | |
| HHS/HRSA | Other | Another commenter noted that although appropriate, some of the categories and elements would need some revision to include currently required program specific information reporting to be included in the interim progress reports as well as in final reports. | Agencies may pursue developing agency specific requirements through OMB. | |
| HHS/HRSA | Instructions | There was general agreement that the instructions are straightforward, but one commenter found it questionable to have so many optional categories in the form. | The required use of the optional forms will be at the discretion of the agencies, not the grantees. | Again confusion about who decides if it is optional. |
| HHS/HRSA | Other | Regarding the question of deferring implementation until there is a common situation for collecting this information electronically: OMB requires the option of submitting information to the Federal Government electronically, except under special circumstances (one time surveyor for special populations (migrant, homeless, etc.)) The Government Paperwork Elimination Act (GPEA) builds on the requirements and scope of the Paperwork Reduction Act of 1995 (PRA). All transactions that involve Federal information collections covered under the PRA are also covered under GPEA. GPEA required Federal agencies, by October 21, 2003, to allow individuals or entities the option to submit information or transact with the agencies electronically and to maintain records electronically, when practicable. The electronic submission, maintenance, and/or disclosure of information is regarded as a means of decreasing the burden and/or increasing the practical utility of the collection. | Thank you, your comment has been noted. | |
| HHS/HRSA | Other | That said, there is general agreement in HRSA that agencies should defer until there is a common solution. | Thank you, your comment has been noted. | |
| HHS/HRSA | Other | This information, especially to truly make a “technology transfer” and share products and tools developed through Federally funded research should be captured electronically in a searchable format and downloadable in a fillable form. Suggest the use of a web-based form that is backed by a relational database that will actually be useful/capable of searching. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS/HRSA | Other | The issue of electronic reporting is a critical one for reporting burden. If the form is one that can be downloaded and filed electronically so that previous year’s documents could be used as a base and edited/updated, this would be user-friendly and fairly quick. If information has to be entered into a website it will mean that every year awardees are starting over from scratch, which is much more time consuming. Also, HRSA awardees have experienced situations with Grants.gov in which the systems are down and additional staff time is needed due to lost work, multiple tries at getting in, etc. Therefore, ORHP recommends that electronic reporting be on an electronic form that is easily downloaded and that implementation of the RPPR be deferred until a common electronically downloadable form exists and has been tested. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS/HRSA | Other | Is OMB approval required for the narrative questions under “accomplishments?” Agencies have been able to obtain this narrative information without asking specific, standardized questions by providing brief instructions for respondents to describe project goals, accomplishments, problems, solutions, etc. | Yes, OMB will approve will the entire final format. The intent of the format was to standardize the reporting requirements across agencies. | |
| HHS/HRSA | Other | The material notes that standardized reporting will have benefits -- however, it is notable that much of the proposed form asks for information in open ended, free form text. Thus, extracting and compiling results in order to make statements about national impacts etc. will not be automated. | The nature of research reporting requires that the information requests remain open ended to allow for accurate reporting of accomplishments. If an agency requires more specific questions, it may pursue developing agency specific requirements through OMB. | You can use search engines to search text, there are electronic tools to help with the research. |
| HHS/HRSA | Other | As the sponsor of this proposed format, has NSF reviewed similar forms from other Federal agencies? The Department of Agriculture has several forms under a currently valid OMB number (0524-0042) for research funds awarded for basic and applied research. The forms are brief and include a progress report, research funds (expenditures, disbursements, and unliquidated obligations), and a final progress/termination report. The forms are up on the web under the Dept of Agriculture’s Current Research Information System (CRIS). [http://www.ocio.usda.gov/forms/doc/AD0421-N-04-99.pdf] | The work group started with a process of looking at existing research progress reports, and the collective decision of the group was to start with the NSF format. It has been extensively modified to meet the agencies needs. | |
| HHS/HRSA | Other | Concern has been raised over existing performance measures and data collection and the impact of the new form on such program specific data collections. For example, MCH grantees are currently required to complete program specific information on Performance measures and Data collection forms through HRSA EHBs, as a part of their non-competing continuation/progress report. Will this RPPR replace that reporting requirement? | This format has been developed for submission of progress reports for research and research related awards. If program specific performance data is necessary the agency may also pursue developing agency specific requirements and clearing them through the PRA process with OMB. | |
| HHS/HRSA | Other | Will there be a maximum page limit for the RPPR progress report? | There is not a page limit, but an agency may set one. | |
| HHS--National Institutes of Health (HHS/NIH) | Other | NIH is supportive of this format and these categories are appropriate for most NIH research project grants and cooperative agreements. | Thank you for your support. | |
| HHS/NIH | Other | NIH plans to use the option of augmenting the RPPR with agency-specific requirements for other programs that use an annual progress report (i.e., training and career development programs), as well as a streamlined progress report option that we use for a large majority of our research programs. | Thank you for your comment. | |
| HHS/NIH | Instructions | NIH supports the optional fields; however we suggest clarifying that the “option” is that of the agency, not the institution. While this is mentioned in the beginning, it might be helpful to include a note under the “Optional Categories” section that these are to be used at the discretion of the agency, not the institution. | Thank you, the instructions and format will be updated to clarify this point. | |
| HHS/NIH | Instructions | Are the instructions straightforward and easy to understand? Yes | Thank you for the comment. | |
| HHS/NIH | Other | Agencies should wait until there is a common electronic solution. Downloadable fillable forms would be appropriate as an interim solution. However, if there is a web-based form that would tie to agencies’ current systems (e.g. NIH uses the eRA Commons system), then the web-based forms may be useful. [NIH currently supports both electronic and paper submission with the electronic submission provided through eSNAP, and the downloadable forms being provided on our forms Web site.] | Thank you for the comment. | |
| HHS/NIH | Other | In the Federal Register notice on page 4, last paragraph and in the RPPR in page 2, second paragraph, the statement is made that “Agencies also may develop additional agency- or program-specific reporting categories and instructions (e.g., the National Institutes of Health may need to collect additional information on clinical trial awards); however, to maintain maximum uniformity, agencies will be instructed to minimize the degree to which they supplement the standard categories.” We believe this statement is confusing in light of the fact that an earlier statement is made in both documents that “agencies may use other OMB-approved reporting formats for specialized research grants, such as centers/institutes, clinical trials, or fellowship/training awards.” We believe the suggestion that clinical trials might be subject to a totally different reporting format or they might be asked limited additional questions on the standard form introduces unnecessary confusion into these documents about how such an important aspect of NIH research will be handled. | The workgroup feels that this paragraph provides necessary details about the format. | |
| Therefore, we strongly suggest that the paragraph “Agencies also may develop additional agency-or program-specific reporting categories and instructions (e.g., the National Institutes of Health may need to collect additional information on clinical trial awards); however, to maintain maximum uniformity, agencies will be instructed to minimize the degree to which they supplement the standard categories,” should be deleted from both documents. Each document has an example of how the EPA might modify a reporting category that precedes the NIH example, which is much clearer and should suffice as an example of how a category might be modified. | ||||
| National Association of State Universities and Land-Grant Colleges (NASULGC/ ESCOP) | Other | The Experiment Station Committee on Organization and Policy (ESCOP) strongly supports a single standard reporting format for recipients to report progress on activities supported by Federal agencies’ research grants. This would reduce the reporting burden on university research faculty who receive grants from multiple Federal agencies. However, for this standard format to be both efficient and effective it must capture all appropriate and relevant information regarding the research activity and the resulting database must be extractable or “mineable” for information useful to granting agencies and recipient institutions. The detailed comments below outline some activity information particularly important to agricultural research that the currently proposed RPPR format would not capture. ESCOP suggests that these new or modified data elements be included in the final RPPR format. | Thank you for the comment. | |
| NASULGC/ ESCOP | Other | ESCOP suggests that a similar web-based portal be established to access the RPPR to enter report information for all Federal-funding agencies. The design and programming of this portal should be developed by a joint group, which represents all Federal-funding agencies and stakeholders (grant recipients) to ensure it meets the needs of each individual agency and recipient institutions. In addition, the RPPR should appear through this portal with all appropriate fields pre-populated with previously entered core information to avoid re-entering data that have not changed from grant initiation or previous reporting periods. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| National Center for Atmospheric Research (NCAR) | Other | The National Center for Atmospheric Research (NCAR), which is a FFRDC, funded by the National Science Foundation and managed by the University Corporation for Atmospheric Research. We have five laboratories and several divisions within each laboratory. Each division and laboratory has many goals and objectives. NCAR receives their core funding from NSF through a large cooperative agreement. We also receive individual NASA, NOAA, FAA, DOD, etc awards as interagency agreements through NSF. These funds are added to the main cooperative agreement, but issued under a separate cooperative support agreement. Each year we are required to submit an annual progress report for the core funding under the main cooperative agreement and another progress report for the interagency agreement transfer awards (A decision is pending as to whether we will submit one report containing information for all the IA awards combined or submit one report for each award). It can be quite challenging to submit a progress report for a large cooperative agreement using a format that is geared more towards an individual investigator award. | This is an agency specific issue. | This will need to be worked out between NSF and the grantee. |
| National Science Foundation (NSF) | Other | There needs to be a place for reporting on the outcome of supplements. Many of these are for additional training and outreach opportunities beyond the original scope of the project, and this information should be captured and associated with the award. | Agencies may pursue developing agency specific requirements through OMB. | How are supplements handled now? They are under the same report, not separate. All lumped in to the annual report. |
| NSF | Other | Why are research centers/institutes, clinical trials, fellowship/training awards excluded from this project? (See last sentence in second paragraph, page 1) Throughout the introductory comments it is made clear that agencies may provide additional program-specific instructions and may develop additional agency- or program-specific reporting categories. Couldn't this, therefore, include specific language related to each of the excepted types of awards? | At an agency's discretion the format may be more widely used . The work group's intent was to give agencies flexibility for reporting on the noted examples. | |
| NSF | Other | Will use of this reporting instrument enable the automatic capture of data (i.e., for products/outcomes; training; participants; partner organizations; impact)? | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | Will the current work group be the ones developing the electronic solution? There may be a good chance that we will get involved in this process There are some folks who have made commitments, such as the GMLOBs, and agencies that received exceptions. We should define this ahead of time, where we think this will go. |
| NSF | Other | Will the submitted reports be searchable? It is extremely important to develop a system that has a dynamic and searchable data base. We presently store a lot of information in our electronic reporting system, but we do not have the ability to search that system (short of searching by each pdf file), or, most importantly, aggregate information from multiple reports across a variety of organizational levels. It would be wonderful to have an ability to identify and aggregate a variety of kinds of scientific impacts across reports at the cluster, division, and directorate levels for example. I hope this effort will result in an improvement in this sense so that we can make efficient use of the information we are collecting. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Other | As far as I can discern, there is only one mandatory reporting category (i.e., Accomplishments) based on the organization of this document. All other categories fall under Optional Categories, including Participants, and Impacts. It seems we would want other categories of information collected routinely in all granting agencies. | Only one section is mandatory, while the other sections are optional at the discretion of the agencies. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| NSF | Other | Whatever system is developed it should be platform neutral. I would recommend the Grants.gov model of downloadable forms be avoided as uploading and downloading files can cause corruption. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Other | What are the plans for linking progress reports for all years of the award? The system should make it possible for the PI to use a previously submitted progress report as a template for a current submission. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Make the report more quantitative. The proposed report appears to be LESS quantitative than the current NSF reporting format. Given that we are always being asked COV questions for which data do not exist, perhaps this is an opportunity to collect more pertinent information. | Thank you for your comment. | It is not less quantitative. There must be some other data type that they desire. | |
| NSF | Other | It is very close to the current NSF Progress Report - although it is somewhat more burdensome for our PIs, especially in the level of detail requested about ongoing activities. | Thank you for the comment. | |
| NSF | Other | Why not reorder as subject-verb-object: who did what with what effect, basically reversing the order of accomplishments and personnel? Doing so improves the flow, because the report goes from accomplishments to their impacts and significance. | Because agencies will not use all the sections, reorganizing it this way would not make sense. | |
| NSF | Other | If standardized categories were used throughout, similar to those used for the demographic items, the data would be more useful. | The work group developed a format that standardizes as much as possible. There are challenges involved in developing something uniform across the agencies. | Not sure how we could do this, but open to ideas. Would you perhaps put this in agency guidance for the PIs? Is it standardized enough across agencies that we should have icncluded this in the format. It is hard to develop something uniform across the agencies. We standardized as much as possible. |
| University of Arizona | Other | The basic thesis of this policy is that this will "save work" and "increase comparisons of research" across federal agencies. This policy will do nothing of the kind. It's another make-work project and unfunded mandate on already struggling researchers, who have seen the possibility of even getting federal grants reduced to success rates (even with excellent reviews) of <10%. | The development of the format was a legitimate attempt by an inter-agency workgroup, under OFTP, to develop a standard format. The intent of the format was never to "Increase comparisons of research across agencies". As with any standardization project, there may be a short term burden increase in order to produce a long term gain. | There may be a short term impact for a long term goal of streamlining…. |
| University of Arizona | Other | Different federal programs have different goals. What is a nice report for NASA might be useful for USDA or USGS. This is another mindless scheme cooked up in Washington. “Although different Federal agencies utilize a variety of formats for reporting progress on activities supported by research grants, similar information is usually collected. These variations increase administrative effort and costs for recipients of Federal awards and make it difficult to compare research programs across government." There is no evidence this is the case. | Quite the contrary, the work group has heard from the grants community that this is wanted and desired. | |
| University of Arizona | Other | "Development of standard reporting categories will facilitate the development of a common electronic solution for collecting the information in lieu of collecting it through numerous agency-unique reporting forms currently used." This is a make-work scheme for some federal subcontractor. | The development of the format was a legitimate attempt by an inter-agency workgroup, under OFTP, to develop a standard format. As with any standardization project, there may be a short term burden increase in order to produce a long term gain. | |
| University of Arizona | Other | "In furtherance of Public Law 106-107, this proposed format will directly benefit award recipients by making it easier for recipients to administer Federal grant programs through standardization of the types of research information required in performance reports. This proposed format is for interim progress reports only,..." but not final reports, so what's the point? | This is the first step, the work group will develop the final report after concluding the efforts on the progress report. | |
| University of Arizona | Other | " Agencies may develop an agency- or program-specific category, if necessary, to meet programmatic requirements, although agencies would be instructed to minimize the degree to which they supplement the standard categories." So, they won't be standard, anyway. More work for nothing. | The work group has worked hard to minimize the level of agency specific requirements. | |
| University of Arizona | Other | "Comment is requested on any aspect of the reporting components proposed, such as appropriateness, feasibility, completeness, or mandatory specification. The NSF also invites the general public and other Federal agencies to comment on estimates of burden as required by the Paperwork Reduction Act of 1995 (PRA)." Clearly, this increases paperwork and is not consistent with the above act. | As with any standardization project, there may be a short term burden icrease in order to produce a long term gain. | |
| University of New Orleans | Other | One of the things that we should be able to do with electronic tools we have nowadays is receive a form to complete that has all the boilerplate already added. So when an award is made it would be good to have the reporting sheet already set up with title, PI, dates, amounts etc. It should also include the goals and objectives from the proposal and the questions should be have they changed, not what are they? Similarly with the who is involved - all the info on people who were on the proposal could be on the form already and the question is then what have they done and who else is involved. | Thank you for your comment. This is an electronic system implementation issue. We will forward your comment to those who develop the electronic solution. | |
| University of New Orleans | Instructions | One of the things that takes too much time in reporting is the stuff that doesn't normally change - make the forms include that upfront and ask me to report if it has changed. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| University of New Orleans | Instructions | It would also be helpful for mandatory fields to be noted - if its OK to respond 'none' or NA to a question make that the default. If the form requires information make that clear on the form. I assume this will all be online and in a format where it doesn't accept the form if the mandatory fields are not completed. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| University of New Orleans | Other | It is good that the “mandatory” sections will be highlighted. | Thank you for the comment. | |
| University of New Orleans | Other | It will be good if the report can be submitted via email; this will make life easier. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| University of Vermont | Other | As stated in the very last sentence of the Supplementary Information, section I. Background: "Information, once reported, does not have to be provided again on subsequent reports." All of the information collected via grants.gov should be available to the RPPR. You should not implement the RPPR unless and until grants.gov can feed such information into it. The goals and objectives information collected via grants.gov would be displayed on the RPPR with an option to identify changes only if there were any during the reporting period. | The information collected on Grants.gov would not be appropriate for the RPPR, as the information often changes between application and award. | What is submitted through Grants.gov is usually too high, and then we budget it down. So Grants.gov is not the right place to get this information. There's also the case where there are multiple funders. The information may be available on agency systems. |
| University of Vermont | Other | Recipients need to know how their reports will be used in government and other information systems and who will be reading what they write. Will these reports be public information? If so, how will they be available and searchable? What search criteria and report outputs will be available? Researchers will want to be able to find peer-reviewed publications for a particular topic. I see nothing in grants.gov or the draft RPPR that would categorize these reports or provide useful criteria for searching by different topics or categories of research. All the outputs needed from a database should be identified before the input forms are designed. | Language will be added clarifying this issue. | There was nothing ever discussed by the group to make the whole report publicly available. The final reports are made available, but the interim reports are not. |
| University of Vermont | Other | When you allow the recipient to enter "none" or "nothing to report..." or similar phrase, it diminishes the importance of the information (it is effectively optional). When someone is searching the database of research reports they are going to see many reports that just say "nothing to report.” If you must have optional fields, leaving them blank is better than asking users to enter "nothing to report”. From experience, I know that many will not say exactly “nothing to report” as instructed but will instead say something like “I do not have anything to report at this time” or try to explain why they have nothing to report. You have no way for computer logic to determine whether an actual report has been entered unless it is left blank. | The work group agreed that using "none" or "nothing to report" is more accurate. While a blank could represent "nothing to report" it may also represent a spot that the grantee forgot to fill in. | Our experience has shown that if you don't ask, you cant assume that blank means nothing to report. Blank could be forgot to fill it in. There could be a check box in the electronic implementation. How big a difference is there between "nothing to report" and "I dont have anything to report at this time." We could add "at this time"? |
| Comment Source | Comment Type | Other Elements That Sould be Included Under the Reporting Categories | Comment Response | Notes |
| DoC/NOAA | Other | NOAA would suggest the following additions to the Mandatory Reporting Category: 1) Guidance is needed regarding the level of detail necessary to address the mandatory reporting category. Perhaps a character or word limit would be useful. | If desired, agencies may issue word limits within their guidance. | What is the concern? That someone will write too much? This is a decision that needs to be discussed with the program manager. |
| 2) A summary project description which also includes expected outcomes, milestones, location of activities, PIs and Co-PIs as well as the flexibility to request other project information the program office may need to be addressed. This would help the Program Managers to review each progress report in a timely manner. | A summary abstract would add to the burden. The reports will not be large enough to justify abstracts. | |||
| 3) Current budget information is also needed to help with project monitoring. The government standard forms do not supply enough detailed budget information to address the monitoring needs of the Program Manager. | A budget section will be created. |
This is a technical communication. If they need more budget information perhaps they should look for the PRA. Optional budget category? NIH will be needing it. Use the detailed budget in the RNR. Is the project report the right place to put this? NIH and HHS request this. Are there others outside of HHS? |
||
| DoC/NOAA | Instructions | The report format should include in its questions, one or two questions that address the quantifiable aspects of the research, performance measures and metrics. The questions should be designed to not only illicit a yes/no response but a brief description as to how something was conducted...for example, "How have collaborators or contacts been involved"? The questions should include some management questions that may provide some helpful insight into the infrastructure that led to the significant accomplishments/outcomes/deliverables. | Agencies may pursue developing agency specific requirements through OMB. | Are there specific examples of where it should go? Do we want that level of detail? |
| ED | Other | As noted above, some ED research programs use the standard ED 524B, Grant Performance Report, which not only includes reporting on grant performance, but also financial reporting on items such as indirect costs, budget expenditures and carryover, thus eliminating the need for grantees to submit separate financial reports, such as the SF-269, Financial Status Report. ED has found that having both financial and performance information combined in the same report has facilitated the review of both annual and final reports from grantees. For example, if a grantee reports in the ED 524B that it has expended all of its grant funds for a given budget period, however, it has not conducted planned activities and achieved project objectives, ED program staff would closely review potential problems with the grant prior to awarding continuation funding for the grant. ED recommends the addition of an optional Financial Data Category to the RPPR for agencies that would also like to collect financial and budgetary information with the performance information in the RPPR. | A budget section will be created. |
This is different than adding a budget section. NIH might like this. Even in the special reporting requirements we took away anything about report balances. So have a second financial category? What would we get on this part of the report? Wouldnt this be agency specific? |
| DOE/NETL | Other | The DOE is committed to financial assistance award and administration practices that are based on sound project management principles. It is expected that each financial assistance award will reflect the following principles: project objectives are defined up front and used to judge project success; project performance risks (technical, financial, and otherwise) are identified and mitigated in the implementation strategy; scope, schedule, and budget are established for each project; and projects are managed and reported against established scope, schedule, and budget. Implicit in these expectations is that each project has an associated management plan commensurate with the perceived risk of implementing the project. The project or research management plan is developed and maintained by the recipient of the award. The plan delineates the scope, schedule, budget, and risks for the project. Award recipients must report progress relative to what they planned to accomplish within the allotted schedule and budget. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. | This approach offers agencies a lot less flexibility.They may use the PPR instead of the RPPR. |
| It is suggested that the RPPR format be simplified to include the following categories: *[a. Title page; b. Executive Summary - What is the research intended to accomplish?; c. Technical Progress - What was done and what does it mean?; d. Issues/Problems - What identified or unidentified risk events occurred, and how are they being handled?; e. Schedule Status - Are milestones being met?; f. Cost Status - Are funds being expended as anticipated?; g. Optional Reporting - Optional special information to support program mission]. Only category “g” is optional. While the proposed RPPR format makes reference to the above categories and does suggest tailoring of the optional elements, this simplified format clearly delineates the progress reporting expectations for the project. This structure is not dependent on basic, applied, developmental or demonstration activities; the level of detail will vary with the complexity of the project and associated risks. | ||||
| DOE/NETL | Other | A significant portion of DOE program and project activities, which constitute annual expenditures in excess of $2 billion, are applied research and development and technology demonstration (RD&D) programs and projects that have the primary objective of introducing a concept or product into the marketplace. The proposed format[*] focuses on basic science grants that do not have the time-critical goals and objectives that are characteristic of programs developing technologies that address industry issues, such as those in DOE’s Office of Energy Efficiency and Renewable Energy, Office of Fossil Energy, and Office of Electricity Delivery and Energy Reliability. The Office of Management and Budget (OMB) has recognized differences in performance monitoring expectations between the basic science and the applied RD&D programs. The format delineated above[*] provides the fundamental project status data needed to monitor performance relative to program goals and objectives. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. | |
| DOE/NETL | Other | With the above approach, the award recipients report what they are doing using their established management systems, and the reports clearly document their accountability to the expectations established by the financial assistance agreement. Standard project management practice is to report schedule and cost status in tabular or graphical form to allow assessment of variance from the planned values; it would be beneficial for purposes of standardization to provide an example of each. | Most agencies don’t require this level of detail. | |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? IRB approval date, for human subjects. | The format will be updated to include a special reporting requirements section in the "Changes" portion of the report. |
Could put it in the special reporting section. But even there it is framed differently. The last optional section needs to be agency specific requirements. We could reformat a bit. Changes, Problems section is essential.Keep the Changes, Problems section and then have a new special reporting requirements section. What will happen when new requirements are developed, will the form have to be recreated? This is a living document. If something new comes out, we will need to build it into the format. Cant the dates be included in the agency specific instructions? For NIH, they are captured as distinct data elements, so having them as text is not useful. If we dont put these distinct data elements in the RPPR, NIH will need to have them on agency specific forms. |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? IACUC approval | The format will be updated to include a special reporting requirements section in the "Changes" portion of the report. | |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? Clinical Trials | The format will be updated to include a special reporting requirements section in the "Changes" portion of the report. | |
| HHS/AoA | Other | Other elements that should be included under the reporting categories: Institutional Review Board approval dates; Institutional Animal Care and Use Committee approval dates; Clinical trial – Phases I, II, III; Data & Safety Monitoring Plan; FWA number; Other key personnel; Authorized Official Representative | The format will be updated to include a special reporting requirements section in the "Changes" portion of the report. | |
| HHS/CDC | Other | Other elements that should be included: A budget for the next budget period must be included in table form followed by a narrative justification for each budget item. | A budget section will be created. |
|
| HHS/CDC | Other | Other elements that should be included: A table of key personnel, their percent effort and their salary must be included followed by a narrative contribution to the project by each individual. | This is covered under the participant section. | |
| HHS/CDC | Other | Other elements that should be included: Budget information should be mandatory. This was not included. Also, options for budget format such as modular should also be considered. | A budget section will be created. |
|
| HHS/CDC | Other | Other elements that should be included: We suggest additional mandatory categories: IRB status (performance sites and expiration dates); anticipated unobligated dollars; major challenges and solutions. | These elements are covered under the accomplishments and the proposed Budgetary sections. | |
| HHS/CDC | Other | Other elements that should be included: I believe that an expenditure report should be provided as part of the progress reporting protocol so the Project Officer can better determine if the project is on track. I noticed the report asks for financial information regarding any partner contributions, therefore I believe this expenditure tracking reporting by the coag recipient is appropriate. | A budget section will be created. |
|
| HHS/HRSA | Other | Given the delays often encountered by Institutional Review Boards reviews, especially in projects undertaken in multiple institutions, it may be instructive to have grantees elaborate on such delays, the reasons for them, and plans to overcome unintended consequences. | There is a section provided for this purpose. | |
| HHS/HRSA | Other | If the project were visited by an agency directed TA provider or other evaluative team, the outcomes of that review and its impact on the work of the project in the next project period would be helpful to know. | The evaluation team should provide this, not the recipient. However, agencies may pursue developing agency specific requirements through OMB. | |
| HHS/HRSA | Other | A commenter noted that for MCH funded research, there would need to be developed additional instructions under “Accomplishments” and “Changes/Problems/Special Reporting Requirements” for grantees to provide current information on recruitment and would require the completion of an Inclusion Enrollment Report Form. | Agencies may pursue developing agency specific requirements through OMB. | |
| HHS/HRSA | Other | Perhaps other program specific categories should be programmatically determined. For example, content related to funding factors, faith-based and community based partnerships and national objectives related to Healthy People 2010 and health literacy should be included. | Agencies may pursue developing agency specific requirements through OMB. | |
| HHS/HRSA | Other | A suggestion was made to require that annual IRB review documents and materials associated with “conditions of award” be attached to the progress report if not provided previously. | The format will be updated to include a special reporting requirements section in the "Changes" portion of the report. | |
| HHS/NIH | Other | Budget Category: Suggest consideration of an optional budget category. For those agencies where this progress report is also used to conduct the fiscal/administrative review prior to the next award, a detailed budget is appropriate. For consistency, suggest using the detailed budget from the SF424 (R&R). Instructions could include a request for information on any significant unobligated balances. | A budget section will be created. |
|
| NASULGC/ ESCOP | Instructions | ESCOP also suggests that consideration be given to including a reporting field that would allow grant recipients to reference related projects and other sources of funding (federal, state, private, etc.) that support the overall research program that includes the specific grant’s activity. The intent would be to also report the institutional support for the program. This recommendation would allow funding agencies and recipient institutions to more accurately determine “leveraging” across funding sources and to identify interaction among research support entities. | This is beyond the scope of the format. The focus of this format is on single Federal grants. |
| Comment Source | Comment Type | Cover Page Data Elements Comment | Comment Response | Notes |
| ASU | Data Element | Most respondents felt the categories and elements included are appropriate. | Thank you for the comment. | |
| ED | Data Element | The Cover Page for the hard copy version of the RPPR needs to include a signature line for the authorized representative. | A signature block will be added. The work group will notify those designing the electronic solution that there will need to be the capability to both submit with signature and and without a signature as required by the agency. |
|
| Emory University | Data Element | It would be very helpful if the form were prepopulated with as much information as possible, especially all the COVER PAGE DATA ELEMENTS, and if possible Specific Aims and Goals, timelines, etc. from the grant proposal. These data elements should be readily available metadata from the electronically submitted grant proposal, which I am assuming is stored at Grants.gov in an XML format following some sort of DTD or Schema. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? No capture of the Administration Official data (name, title, address) signing for the institution. | The format will be updated accordingly. |
|
| HHS--Centers for Medicare and Medicaid HHS/CMS | Data Element | Add additional elements: grant amount, due date of report, signature and title of project director, and date of report. | A block for capturing submission date will be added to the format. |
We don’t have a due date currently. Submission date would be helpful for paper. |
| HHS/CDC | Data Element | The following elements should be included on the cover page: email address and telephone number of the PI and business official; Human Subjects and/or Vertebrate Subjects involvement; and direct, indirect and total cost for the next budget period. | The form will be updated based on the comment. |
|
| HHS/NIH | Data Element | Add data elements would capture any changes in Project/Performance Site locations. This will be critical in view of the new FFATA requirements. Specific data elements would be those found in the SF424 (R&R) Project/Performance Site Location. | The work group will gauge the government wide need for this data element. |
We should wait until we determine if there are others that would want to capture this. Add it into the area where we are talking about other changes. Could add "Change in performance site" as a discrete data element. |
| HHS/NIH | Data Element | Add Administrative Official data (name, title, address, phone, fax, e-mail) signing for the institution/ organization. This can be optional data for those agencies where this is not a requirement of the annual progress report; however adding this to the Federal wide form would eliminate the need for NIH to accommodate this as an agency-specific requirement. This is somewhat tied to the lingering question of submission. If electronic submission is the end result, these data elements may become obsolete. However, anticipating a paper usage of this for some time, we would require these data elements. | The format has been updated, in response to a previous comment, to capture similar information. |
| Comment Source | Comment Type | Accomplishments Comment | Comment Response | Notes |
| ASU | Instructions | In the mandatory section it asks the investigator to report changes to the agency-approved application or plan. If these are significant changes that would require prior agency approval, the instructions need to indicate that the investigator needs to work with the appropriate institutional administrator to request agency approval of the change. | The work group will update this section based on the comment. |
We tried to address this earlier in a different way. We may want to do something to ensure things are not being reported twice. How do we address this? |
| DoC/NIST | Instructions | Extramural: Template really does not emphasize comparison of actual to planned as specified in the DOC regs, it stresses only the "findings of the investigator" as preeminent, which is fine for basic research and not for research beyond research and for cooperative agreements. "What wasn't accomplished that was planned?" is not part of the instructions. See 15 CFR 14.41 on reporting to see all the options. "Both" requirements are needed. | The work group will update this section based on the comment. |
These are the sort of things a program needs to know. Add a statement to identify what was not accomplished. |
| DoC/NIST | Instructions | Extramural: Reasons why goals were not met and why appears to be mandatory and not optional in the regulations, however, the proposed template makes it optional. But very few basic research grants have "established goals," which is not pointed out, but they could describe the baseline of the technology/science in the knowledge base at the start of the grant/cooperative agreement that is in academia and/or the private sector. | This concern has been addressed by the comment above. | |
| DoC/NIST | Instructions | Extramural: Strongly suggest that accomplishments include three key sections: 1. Baseline of the technology/science; 2. Accomplishments against goals; and 3. Changes, Problems, Opportunities, Special Requirements. | Sections 2 and 3 are addressed in other sections on the format. The work group feels that proposed section 1 requests a level of detail more in depth than desired for a progress report. | |
| DoC/NIST | Instructions | Extramural: The question on dissemination (page 3) really needs to be moved to the IMPACTS section, as it appears to be redundant in the Accomplishments section. | The apparent redundancy is intentional to ensure agencies using only a select few of the optional sections capture this data. | If you are not using anything but accomplishments, then you would not be getting this information. Dissemination is very important. Does not need to be moved. |
| DoC/NOAA | Instructions | The fifth bullet under accomplishments states: "What do you plan to do during the next reporting period to accomplish the goals and objectives?" The accompanying instructions go on to state: "Describe briefly what you plan to do during the next reporting period to accomplish the goals and objectives and explain significant changes in approach or methods from the agency approved application or plan." It would be helpful to specifically request grantees to address whether or not a no cost extension will be needed. Grantees should be requested to address a caption of this nature: Grantees should state that there are either no foreseeable circumstances that will require an extension or that they are expecting that there may be a need based on upcoming events that will require an extension to be filed before the next progress report | Agencies may pursue developing agency specific requirements through OMB. | A PI might be confused and think that this counts as a request for a no cost extension. |
| ED | Instructions | Unit of analysis: It is not clear in the instructions for the mandatory Accomplishments category whether the unit of analysis for reporting on accomplishments is at the “activity” (see page 2) or the “project” (see page 3) levels. The seemingly interchangeable use of the terms “activity” and “project” may cause problems in implementation as many larger research grants have multiple projects and each project typically involves multiple activities. We recommend clarification on what the unit of analysis is for reporting accomplishments. | This has been changed to the project level based on another comment. | |
| ED | Instructions | While the proposed RPPR, includes a mandatory category entitled Accomplishments that covers accomplishments made during the reporting period under the approved goals and objectives of the project, this category does not specifically address the grantee’s progress towards meeting performance measures that 1) the Federal agency has established for the research grant program, if any or 2) the grantee has established for its project in the approved grant application. While agencies may use different terminology to describe similar concepts, we recommend that the term “performance measures” be specifically included under the Accomplishments category to ensure proper reporting on performance measures for the project or program, where applicable. | This is accomodated by the new category "Special Reporting Requirements." | |
| ED | Instructions | Further, in addition to reporting on progress towards meeting their performance measures, it would be useful if grantees would be required to provide a discussion of performance measurement trends, if applicable. This could include process, as well as outcome measures, for example, % of scheduled dates met, % of customers who value the product, % of funded research reports in peer reviewed journals. | Agencies may pursue developing agency specific requirements through OMB or may consider using the PPR for their reporting requirements. | |
| ED | Instructions | Some agencies use different terms to cover milestones and target dates for a project. This agency uses terms that reflect those used in the Government Performance and Results Act of 1993 (GPRA). Thus, we talk about performance targets and interim performance targets. We think language consistent with GPRA terminology would make the form more easily applicable to research programs managed by more agencies. We would support a change that added to the list of bullets under the accomplishments heading a new bullet entitled: Interim and final performance targets of the approved application. | This would potentially confuse the grantees. | |
| ED | Instructions | There appears to be some redundancy in requesting that grantees describe any changes in the goals, objectives, approach, methodology or scope originally approved in the application. For instance, under major goals of the activity under the Accomplishments category, the Project Director (PD) or Principal Investigator (PI) is asked to list any revised goals and objectives. Later, under plans for the next reporting period, they are asked to explain significant changes in approach or methods. Finally, under changes/problems, the form asks that the PD/PI report any changes in approach during the reporting period and reasons for these changes. We suggest that reporting on any changes in goals, objectives, approach, methodology or scope be reported once in the beginning of the form under major goals of the activity. | This section will be updated to reduce redundancy. |
|
| ED | Data Element | The mandatory question regarding training opportunities in the Accomplishments category anticipates two bases for not providing information: training was not intended; or, there were no significant training developments during the reporting period. However, the form offers only one response for both situations, “none.” We believe that the report would be more useful to agencies if each possibility were addressed separately. If no training opportunities were planned under the grant, the report could include a response of “None planned.” If there were no significant developments under planned training opportunities, the report could include a response of “Nothing significant.” These responses would be easy to include in paper versions of the form with check-off boxes and an on-line version of the form could similarly display check-off boxes. | "Nothing to report" accurately describes both of these reponses. | |
| ED | Instructions | The wording of the “training and professional development” question is vague and could be construed to be limited to individuals working or involved with the project (e.g., undergraduate or graduate students, post-doctoral fellows, college faculty, K-12 teachers). This definition seems overly restrictive and could have the effect of excluding a substantial amount of training and professional development activities provided to others. For example, many large research grants target outside audiences (e.g., practitioners in the field or ultimate beneficiaries of the research/consumers) in order to increased knowledge of recent research findings and build capacity to utilize findings. | This section will be updated based on the the comment. |
|
| ED | Instructions | Under the question “How have the results been disseminated to communities of interest?,” we recommend also asking what evidence there is, if any, of the effectiveness of these dissemination efforts or strategies. | Gathering this evidence would increase the reporting burden of agencies. | |
| ED | Instructions | Tracking Performance Over Time: In both the Accomplishments and Impact categories, there is no provision for identifying accomplishments and/or impacts that have been reported in more than one reporting period (e.g., budget period or project year), but at different stages of development. Most noteworthy accomplishments of research and development (R&D) activities are not discrete events that occur within a one year time frame and then disappear; rather, both accomplishments and impacts typically mature over time from smaller, incremental accomplishments or milestones. For example, in one year a grantee may report the successful development of a product (e.g., a database) or technology that is reported as a peer-reviewed publication the following year. Agencies need to have a way to link these accomplishments and track their evolution. We recommend that RPPR form ask grantees: (a) whether accomplishments and/or impacts reported in the current reporting period (e.g., budget period or project year) are an outgrowth of accomplishments and/or impacts reported in a previous period; and (b) if so, describe the relationship. [Comment repeated on Impact tab.] | There is no change necessary. Reporting periods are artificial endpoints. The grant life cycle is comprised of all the reporting periods from award to close out. | |
| ED | Instructions | Linking Accomplishments and Impacts to Goals: We are concerned that the current version of the form provides no mechanism by which grantees can link accomplishments and impacts to goals and/or performance measures. This is particularly important for grants with multiple projects and goals – such a linkage would help agencies determine grantee progress in different areas. We recommend that the RPPR form require grantees to link their accomplishments to their goals and/or performance measures. One way to do this would be for the form to use an internal numbering system identifying and linking project goals and activities with the goals and accomplishments to which they apply. [Comment repeated on Impact tab.] | The work group feels this linking will naturally occur. | Wouldn’t the PI automatically be linking the accomplishments to the approved application. We do not not need to elaborate on it. |
| DOE/NETL | Instructions | Only the Accomplishments section of the draft RPPR focuses on actual progress in any appreciable way, and the nature of the information described is focused on basic science grants. The information is not adequate to support DOE’s substantial involvement and due diligence for advanced R&D and major demonstration projects administered through cooperative agreements. These projects are not administered through grants and many are unique, cost-shared partnerships between DOE and private sector organizations. As such, these projects are subject to all of the requirements associated with financial assistance rather than those associated with acquisition. However, the nature of the projects is such that the participants must explicitly define expectations in terms of scope, cost, and schedule that are more typical of a contract funding mechanism. The private sector participants must adhere to accepted project management standards to effectively execute these complex projects that have significant private sector and Government investment. | No response required. | |
| For NETL to be able to exercise due diligence as regards project management, it is imperative that reporting mechanisms--particularly content and frequency--be consistent with those used in the private sector for projects of this nature. Specifically, we need to be informed, on a monthly basis, of the project status categories delineated above. See [*] on the Other Elements tab. | ||||
| HHS/AoA | Instructions | The following element could be added under Mandatory Reporting Elements (Accomplishments): On pace with reviewed and approved workplan? | This issue has been resolved by another comment. | |
| HHS/CMS | Instructions | Under ACCOMPLISHMENTS: What was learned? Add the following: What are the approved major goals and objectives of the activity. | The act of the agency funding it signifies approval. | |
| HHS/CMS | Instructions | Under ACCOMPLISHMENTS: What was learned? Add the following: What is the status of each goal to be achieved during the reporting period? | This is addressed by reporting on progress. | |
| HHS/CMS | Instructions | Under ACCOMPLISHMENTS: What was learned? If appropriate, what corrective action is planned to resolve implementation problems. Is technical assistance required from grantor agency? | Agencies may pursue developing agency specific requirements through OMB. | Corrective action has a very negative connotation. This would be responded to under the Changes/problems section. |
| HHS/CDC | Instructions | Reasons goals were not met should be stated. | This is addressed in the format. | |
| HHS/CDC | Instructions | "No change" is unacceptable for plans for next reporting period. The Aims should be stated regardless of whether or not they have changed. | The aim of the form is to alleviate the reporting burden on the grantee. Agencies should have this information. | Could remove the paranthetical if that would help. So would they need to repeat their application? Focus on goals for the next reporting period. Change the parenthetical? If it has been provided before we are not going to keep asking for them to say it again. Leave as is. |
| HHS/HRSA | Instructions | Under what was accomplished (top of page 3), it is suggested that the form also identify any barriers or challenges the grantees encountered. | This issue is addressed in the Problems and Changes section. | |
| HHS/NIH | Instructions | Accomplishments: In the instructions for “What was accomplished under these goals” suggest including “positive and negative” as part of the item 3) significant results. For this reporting period describe: 1) major activities; 2) specific objectives; 3) significant results, including major findings, developments, or conclusions (both positive and negative); and 4) key outcomes or other achievements. As the project progresses, the emphasis in reporting in this section should shift from reporting activities to reporting accomplishments. | The section will be updated based on the comment. |
|
| NASULGC/ ESCOP | Instructions | Add a data element on the need or problem that is being addressed by the research activity. In other words, what is the potential usefulness to society of the ultimate application of these research results? | No action needed. | |
| NASULGC/ ESCOP | Instructions | Add a data element to indicate other sources of funding (federal, state, private, etc.) that support the overall research program that includes the specific grant’s activity. | Agencies may pursue developing agency specific requirements through OMB. | |
| NASULGC/ ESCOP | Data Element | In the major goals and objectives data element, if they have not changed from the original proposal, this field should be pre-populated. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NASULGC/ ESCOP | Instructions | In the results dissemination data element, the report should indicate if the dissemination was through a formal outreach organization, such as the agricultural cooperative extension or industrial extension services, or an independent activity. | This information should already be provided. | |
| NSF | Instructions | What was accomplished under these goals? The different between significant results and key outcomes was not clear. Although the document already contains detailed instructions about the optional sections, it could include distinctions between the two to avoid confusion. | No change needed. | |
| NSF | Instructions | How have the results been disseminated to communities of interest? Typically PIs considered research publications and abstracts of poster/oral presentations in meetings as modes of disseminating results to the communities of interest. Are they supposed to be included here? Products/outcome optional category will also include this information. Will this information also be presented in the key outcomes under the accomplishments section? | This will be collected in the Products/Outcomes section and under the Accomplishments. | This section is designed to get outreach materials. Cant have a specific example for every type of award done. |
| NSF | Instructions | On p. 3 the following appears near the bottom: "Describe briefly what you plan to do during the next reporting period to accomplish the goals and objectives and explain significant changes in approach or methods from the agency approved application or plan." It seems to me that describing significant changes in approach or methods is redundant, at least in part, with information requested at the bottom of p. 10 and on p. 11: CHANGES/PROBLEMS/SPECIAL REPORTING REQUIREMENTS. | This issue has been addressed by another comment. | |
| In this context, if this document represents the ordering of information that the PI will present in the progress report, then as a program manager reviewing the information I would want to see a description of significant scientific changes, or changes in plans about staff that could affect the scientific progress of the award, to be presented right after a listing of the activities and accomplishments, or imbedded within that section. Such changes can affect plans for achieving objectives in the next reporting period, that is, the section on p. 3 where information on such changes was first requested in the document. | ||||
| NSF | Instructions | We agree that this should be a mandatory category for the RPPR. This information documents the progress of the research during the reporting period, thus providing information to the funding agency and other stakeholders about the progress of the work and accountability by the researcher. | Thank you for your comment. | |
| University of Vermont | Instructions | The guidance for the mandatory accomplishments category states: "List the major goals of the activity as stated in the approved application or as approved by the agency." You should not be asking recipients to repeat the goals and objectives for each reporting period. This information should be collected once at the project's inception, and should already have been provided in the grants.gov electronic application process. If it is not requested in grants.gov as you need it for the RPPR, then you should modify the way it is collected in grants.gov so that the information is usable at reporting time. | Thank you for your comment. This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. |
| Comment Source | Comment Type | Products/Outcomes Comment | Comment Response | Notes |
| ASU | Instructions | Requesting invention, patent application, and license information is duplicative and should be eliminated from the interim report since it already should be collected through the use of iEdison (https://s-edison.info.nih.gov/iEdison/) by all agencies. Perhaps instructions should emphasize that this information is to be reported through iEdison. | The purpose of this section is to provide the program officer with a record of all that has occurred within the reporting period, including patents. | |
| ED | Instructions | In the heading: “Products/Outcomes: what has the project produced,” the term “outcomes” should be changed to “outputs”. “Outcomes” refers more to changes in policy, practice and behavior – this is covered in a later section. This particular section addresses accomplishments such as publications, rather than the results of those accomplishments. Therefore the correct term would be “outputs.” This should be changed throughout that section. | The term has been updated based on the comment. |
Outcomes is a more consistent language? No, Outputs are only the tangible things. |
| ED | Instructions | Regarding the Products/Outcomes category, the draft format does not cover copyrightable works, which are a significant possibility for a research project. For example, the researcher could have written code for a program that will help others in future research. Computer code is copyrightable as well as patentable, depending upon the context of the program. We recommend a category for copyrightable products be added to the proposed RPPR, in addition to the journal articles that might be copyrighted as well. | The list in the examples is not meant to be inclusive. However the format will be updated to clarify this. |
|
| ED | Instructions | The form asks respondents to identify any networks and/or collaborations fostered by the research activities. This information most likely will be in the grant application. What would be more useful would be information on the impact and effectiveness of the collaboration/s. | There is a collaboration after the work begins, so the information captured in the grant application may not be complete. | |
| ED | Instructions | Publications, Conference Papers, and Presentations: Through experience, some ED program offices have found that merely asking grantees to provide lists of publications, conference papers and presentations produces data that are not very useful for external accountability or for internal monitoring. This approach places too much emphasis on the number of outputs rather than on the quality and on how they contribute to advancing agency or grantee goals. This is particularly true for conference papers and presentations, where for large grants and/or well-established PIs the list can be very long, numbering upwards of 50 presentations, with many of them variations on each other with the same basic content and different titles. | Agencies may pursue developing agency specific requirements through OMB. | |
| To avoid the pitfalls of misplaced quantification and increase the utility of this type of performance information, we recommend an recommended alternative approach that involves the following steps: (1) ask grantees to provide full citations for all journal and/or book/chapter publications produced in the performance period, thus allowing journal articles to be checked against an independent source (e.g., Thomson Scientific) to determine their peer-review status; (2) from this list, ask grantees to identify the 1-3 “most important” publications in terms of advancing grant and/or agency goals, and for each of these to summarize the key findings and implications; and (3) ask grantees to identify a “limited number” (e.g., no more than 10) of other types of publications (e.g., technical reports, policy briefs, etc.) and conference presentations that they think are most important in terms of advancing grant/agency goals. | ||||
| Emory University | Other | My sole criticism of the draft format is in the collection of publications and research reports. It is absolutely critical that scientific peer-reviewed research publications not be viewed as Optional. This should without question be included in the Mandatory section. | The work group feels that publications naturally fits in the products section. | |
| Emory University | Other | A peer-reviewed scientific article is the final product, the final data set and formal interpretation of the meaning of the data. These articles and the research that they represent were bought by the US taxpayer with tax dollars, who paid for the research, the analysis of the data, and even the writing of the article. The US public should be granted full and free access to these articles as soon as they are published, and most certainly by the following Progress Report. It should be the duty of the Granting Agency to collect these articles and assure that the articles are deposited in easily accessible and freely accessible forms. An excellent example is the NIH-NLM PubMed Central database. The only flaw of the PubMed Central database is the collection of penultimate versions, not the final version, of the article directly from the authors. This version may have many errors in it. It is critical that the final published article is the version that is deposited, which will assure that the fully peer-reviewed and edited (in essence, the fully corrected and final) version is the one in the public domain. The government should settle for nothing less than the real McCoy. | Please see the response to the comment above. | |
| HHS/CMS | Instructions | Under PRODUCTS/OUTCOMES: What has the project produced? Add a final bullet: Program Income Generated. | The work group feels that this is captured in the budget section. | Already addressed? Program income is only a product in the most general sense. If it will be required, put it in the budget section. |
| HHS/HRSA | Instructions | A concern with the current iteration of the RPPR involves the requirement to list peer-reviewed publications. While there is general agreement that the documentation of these research outcomes is important, it is noted that the time required to have a manuscript accepted for publication tends to be considerably greater than the one- to two-year duration of most of one program’s funded research studies. In other words, in most cases publications would appear long after the corresponding progress report had been submitted. Consequently, the commenter for the program in question did not feel much weight by funding agencies should be placed on this measure and recommended that intermediate measures such as submitted and/or accepted manuscripts should be included. | This issue has already been addressed in the format. | |
| HHS/HRSA | Instructions | One category, dissemination to “communities of interest,” requires further clarification. The category appears to refer to dissemination to the lay public (“communities… not usually aware of these research activities…. [to enhance] public understanding”). However, the first section, “communities… not usually aware of these research activities” could apply to any cross disciplinary communication. | This issue is covered in the accomplishments section of the format. | |
| HHS/HRSA | Instructions | Page 5, Web Site(s) or other Internet Sites(s) requests a listing of the URL’s where research activities are disseminated. A short description of each site would enhance the report. | The format is being updated based on the comment. |
|
| HHS/NIH | Instructions | Suggest adding a specific category for Data & Research material (such as cell lines, DNS probes, animal models). | This type of information should be placed under "other". |
|
| HHS/NIH | Instructions | Suggest adding a clarifying instruction in the Invention section clarifying that this is not a substitute for completing the required invention reporting. | The format has been updated to clarify this point. |
|
| NASULGC/ ESCOP | Instructions | Replace “Outcomes” with Outputs in the title of this category. All the data elements being reported in this category are more accurately classified as outputs using the standard logic model for depiction of a program’s action sequence. See the following URL for references on the use of this model and definitions of terminology. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html | This issue has been resolved by a previous comment. | |
| NASULGC/ ESCOP | Instructions | The Publications, Conference Papers, and Presentations data element information should include any electronic references (URL) that may be available for the publications listed. These URLs would then not be included under the subsequent Web Sites or Other Internet Sites data element and the instructions would indicate that. | The instructions are being clarified based upon your comment. |
|
| NASULGC/ ESCOP | Instructions | It should be clear whether or not publications reported in the Publications, Conference Papers, and Presentations data element may include those submitted, under review, accepted, in press, etc. or only those that are publicly available at reporting time. | This issue has been addressed by a previous comment. |
|
| NASULGC/ ESCOP | Instructions | Add a data element to report number of people not defined as participants in the Participant category who were directly involved in this research activity, i.e. focus groups, survey respondents, clinical trial subjects, taste panels, etc. An option may be given to include relevant demographic information on these people. | Agencies may pursue developing agency specific requirements through OMB. | |
| NCAR | Other | Similar to items one and two, our organization publishes approximately 800 journal publications per year. As mentioned above, it would be very helpful if the system could accommodate an upload of this information instead of typing each publication separately into the system. | This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Instructions | Do the key outcomes in the mandatory section on accomplishments overlap with this section? | No. | |
| NSF | Other | Will there be a facility for uploading pdfs of publications? | This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Other | George Strawn has indicated that NSF has a new program for reviewers and PI's which allows people to pick up their publications from standardized ISI categories. Would it be feasible to use that program for this project? | This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Other | We do not agree that the proposed "Products and Outcomes" category should be an optional category of the RPPR, as proposed in the Federal Register notice. Instead, we believe this should be a mandatory category as it would contain references to journal articles, conference proceedings, web sites, and other products resulting from federally- funded research. This category is a primary means for informing various stakeholders and interested parties of the outcomes and products of all federally-funded research. To not require the submission of such information is to undercut the very cornerstone of the methods used to further scientific research and ensure accountability and transparency for federal research dollars. | Only one section of the form is mandatory for agencies to request, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. | |
| University of New Orleans | Instructions | I am happy also happy to see that “conference papers” can be included and lumped in with regular publications. This makes sense because rarely do we actually publish results while we are still writing progress reports for particular projects. | Thank you for your comment. | |
| University of Vermont | Instructions | The instruction for publications asks the recipient to indicate the "status of the publication (published, accepted, awaiting publication, submitted, under review, other)." Articles or other works that have only been submitted or are under review are not actual publications and they may never be published. If you are asking for "publications", they should be only works that are published (meaning that they are reviewed, edited, cited, and can be located and obtained whether paper or electronic). "In press" or "accepted for publication" are at least going to be published, even though the location information is not yet known (when you include these, the same publication needs to be reported again in order to include the location information once it is known). You should provide a correct definition of what a "publication" is in the guidance so that only actual publications are included, otherwise this important category becomes diluted with inaccessible or unsubstantiated material. If information about submitted and under-review works is of value, it should be included in a different category. | The work group disagrees. It is easier for applicants and publication people to find it all in one category. |
| Comment Source | Comment Type | Participants Comment | Comment Response | Notes |
| DoC/NIST | Instructions | Extramural: Organization information should LEAD this section, and put all requests for optional individual demographics at the very end only. We are opposed to collecting demographic information of individual key personnel, etc. Grantor agencies should be able to clarify what they want and these instructions do not indicate how demographic information on individuals will be protected and how each individual gets to give approval to what information is provided. This could lead to potential lawsuits. An individual could potentially be maligned and the information may end up in a grant database that could really cause the person ongoing harm. If it remains optional, stronger language on how information will be protected is imperative! | The instructions are being clarified based on this comment. |
|
| DoC/NIST | Instructions | Extramural: Making the organizational information on participants a separate section would strengthen and raise the importance of this to the IMPACTS section, and help with tracking impacts later. Right now it comes across as too intertwined with the individual demographics and the value of the information comes across as less important. | The instructions have been updated to clarify the distinction between individuals and organizations. |
Right now, the descriptions section doesn’t really address the three sections. A collaborator is a person. It was lumped into one category because at the end of the day, the individual is the important level. Demographic data always makes people think in terms of individuals. |
| DoC/NOAA | Instructions | It is not relevant or appropriate to require demographic information (under the 'Participant' section) in these reports. Reports are required so we can monitor progress on grant activities and ensure appropriate progress is being made. We should not be using these reports as a means to collect data for some other purpose. (...Also, too late to do anything about this, but the proposed report format should have been described in the FRN itself to make it simpler for people to review!) | This is an optional section. If the agency does not want to request demographics then it does not need to request the information from the grantee. The instructions have been clarified to reflect that point. | |
| DoC/NOAA | Instructions | Demographic information. Please clarify if the demographic information as described on Page 7 regarding Gender, Ethnicity, Race, and Disability Status is a mandatory or optional field. Although I do understand the value of this information, I would expect that it would be optional, or at least include an entry under each entry that includes a declined to designate option. | This information request is optional. The instructions have been clarified to reflect that point. |
|
| DoC/NOAA | Instructions | On a separate comment, the level of information that colleges have about Disability Status of students is not often reflective of the student population, and that there is a choice of disclosure of such information by students, and others, so that data often collected in this area is not representative. (American Association of Advancement of Science, AAAS has some references I can look up if useful.). Do you know if before such a report would be implemented, information about which type of report that would be required would be included in the FFO or the awarding document? | The demographics being requested only pertains to the people who have directly worked on the award. | |
| COGR | Other | We appreciate the elimination of the request for the last four digits of the participants’ social security number in the Participants optional category. | Thank you for your comment. | |
| COGR | Other | We continue to be concerned with the demographic information requested on the research staff and potential conflicts with official institutional records and violations of state and Federal privacy laws. State laws and university policies often require this information to be redacted from any archived materials – electronic or paper. Providing this information within the RPPR could be viewed as a compliance risk for some institutions. Providing gender, ethnicity, racial and disability information is a private, personal decision and, we believe, not the prerogative of the principal investigator(s). Publicly-funded schools and those that receive federal funding must comply with the Family Educational Rights and Privacy Act (FERPA) provisions requiring written consent before the release of educational records or personally identifiable information, with some exceptions. The courts have stated that race/gender information, fall within this provision. We understand this is an optional category and the information is useful to Federal agencies in reaching government-wide education and training goals for under-represented populations. | This is optional and if another institution has regulations preventing the reporting on it, the grantee can choose not to provide this data. | |
| We share those goals and need to identify different ways to assist the Federal agencies in making those assessments but do not believe this is the appropriate mechanism for provision of that information. | ||||
| COGR | Other | We have similar concerns with the identification of participation of individuals by person month. The RPPR request for a quantification of time spent on the project may not reflect the institution’s official effort reporting and certification system. The appropriate mechanism for ascertaining the time spent on a project is through the institution’s effort reporting system. | This data can be generated from whatever database is most appropriate. | |
| ED | Instructions | With regard to privacy safeguards, we recommend that a sentence be inserted in the text on Page 5 under the “Participants” category, immediately after the words Privacy Act. Recommended text is in brackets below: “PARTICIPANTS: Who has been involved? … Confidentiality of demographic data will be in accordance with the agency's policy and practices for complying with the requirements of the Privacy Act. [This includes the requirement that agencies will ensure that System of Record Notices reflect any new, deleted or revised categories or use purposes of individually identifiable information collected.] Agencies use demographic data for statistical purposes, primarily to help: …” | The work group feels that this information would be potentially confusing for the grantees typically submitting the report. | |
| ED | Instructions | We believe that the disability status categories describing participants are too broad and should be re-categorized to better describe participants under our research grants, especially those funded by the National Institute on Disability and Rehabilitation Research in the Office of Special Education and Rehabilitative Services. The only disability categories listed in the proposed RPPR are: “Hearing Impairment, Visual Impairment, Mobility/Orthopedic Impairment, Other and None. Left out, for instance, are categories such as developmental and intellectual disabilities; psychiatric disabilities, and cognitive disabilities. These are large categories and should not be lumped into “other”. We recommend that the drafters of the RPPR format consider the thirteen categories identified under the Individuals with Disabilities Education Act (IDEA). The U. S. Census Bureau also constructed six categories of disability status for data collection. Those categories are: sensory disability, physical disability, mental disability, self-care disability, going outside the home disability, and employment disability. | The format is being updated based on the comment. |
These are the same data elements in the RNR, so we are keeping data elements consistent across reports. The are the same elements required at the time of application. |
| We recommend a more thorough discussion of disability and an expansion of the concept in the proposed RPPR data collection system. Finally, we recommend that next to the “other” category, a blank space be provided to allow applicants to write in the name of the other disability/disabilities. | ||||
| ED | Instructions | We recommend asking about participatory action research – how were stakeholders (including those with disabilities) included in the research process. | Agencies may pursue developing agency specific requirements through OMB. | |
| ED | Instructions | Page 5 of the Participants section states, “if a person has contributed significantly to the project during the reporting period, provide name, role...” This information likely would have been included in the application. It is unclear who, other than people/organizations named on the award and in the budget justification, this question is seeking to capture. We recommend clarifying this point. | The work group feels the instructions are clear. This section captures any individual or organization that has been directly involved with the project. | |
| ED | Instructions | We recommend that the drafters of the proposed RPPR format ensure that the race and ethnicity categories describing participants are consistent with the revised OMB guidelines regarding both the direct collection of racial and ethnic data from individuals and the manner that aggregate racial and ethnic data should be reported to agencies. When reporting aggregate data to agencies, individuals who identified as more than one race should be reported in the “two or more races” category. | So noted. | |
| ED | Instructions | On pages 5-6 there are a series of bullets describing how demographic data is used by agencies. Then there are a series of bullets that appear to be questions requiring a response. Again, it is not readily clear that the first set of bullets do not require a response. This is an example of an earlier general comment recommending separate sections for instructions and background. | The work group has clarified the section. |
|
| ED | Instructions | We recommend that the drafters provide definitions of the terms “one person month” and “whole person month” as used on page 6 of the proposed RPPR. | The report already does this. | |
| ED | Instructions | The example on page 6 asks for information about “funding support,” and the example provided as a response states “this award.” It would be more helpful if the respondent provided the name of the agency or agencies (or private funders) providing the support. | The report has been upgraded based on this comment. |
(Other than this award) "Sentence Funding is welcome but not required" is misleading. Pick a group that does Graduate fellowships. Ford Foundation? |
| DOE/NETL | Instructions | The labor-statistics-like information proposed in the draft RPPR, while necessary to support the National Science Foundation (NSF) mission, is inappropriate for inclusion in a progress report since it has no bearing on cost, schedule, or technical progress. This information is not likely to change on a quarterly basis and is more appropriate to be obtained as a separate “one-time” or “as required” report. | This information may change every year, requiring that agencies update it regularly. Many of the agencies only require annual and final reports. | |
| HHS | Instructions | The use of the term “participants” in the RPPR may be problematic. Some agencies equate this term with “participants of a study.” In the RPPR it refers to personnel on the grant. Suggest changing to “senior/key personnel” as this would be consistent with the SF424(R&R). Also, a few commenters felt grantees would provide unnecessary “participant” information on administrative personnel. | The people included in the "participants" section extend beyond the "senior/key personnel". | |
| HHS/HRSA | Data Element | The information requested on participants (individuals that have worked on the project) asking for name, gender, race/ethnicity, and disability status is subject to the Privacy Act and would require agencies to establish a system of records. The practical utility of this information is unclear. The data obtained will be unreliable and inaccurate, unless each respondent has in place a mechanism by which they collect this information on their participants in the format that is proposed. | The report has been upgraded based on this comment. |
|
| HHS/HRSA | Data Element | Even though this information is voluntary at present, there is serious concern regarding information on race, ethnicity, and disability about participants. In general, the proposed regulations fail to make a compelling case that the demographic data that will be reported “…benefit[s] everyone regardless of demographic category” or that reporting of this data will ensure that “under-represented groups will have the same knowledge of and access to programs…” (p. 5). By law, employees of one awardee are not required to divulge such characteristics. If researchers chose not to provide this information, we would not be able to comply with such an RPPR mandate should it be mandatory. | This section is not mandatory. Additionally, the work group has upgraded the report to clarify this point. | |
| HHS/HRSA | Instructions | For another awardee, disability status is considered health information and may not be disclosed at the individual level. It is possible that voluntary disclosure would be achievable. However, it is noted that obtaining “voluntary” disclosure would require the awardee to survey all participating faculty and staff, which might be perceived by them as intrusive. It is also possible providing individual-level data on disability status is a HIPAA violation. Certain disabilities may be present but not obvious (mental health impairments, physical conditions the individual prefers to keep private). Based on the rationale provided in the draft RPPR, what is really being sought is EEO information, which could be provided in aggregate with less violation of privacy. | This section is not mandatory. Additionally, the work group has upgraded the report to clarify this point. | |
| HHS/HRSA | Instructions | Re persons who have worked one person month or more: Clarification is needed regarding who are to be listed here. Include administrative staff? investigative staff? The reporting burden is substantial and duplicative. Identification of individuals and their roles was in the grant proposal. The detail included in the draft RPPR – e.g., the example of a change under-graduate to graduate status is given, “preferably explaining the change in involvement” – constitute an unreasonable reporting burden. Additional justification would be needed to explain the requirement for information at this level of detail. Few of our investigators would be identified as participants. E.g., the Principal Investigator devotes 30% FTE to the RHRC, but it is spread over three current projects and also Center-level administrative and policy/technical dissemination activities. Thus, the PI would not be included as a participant under the proposed definition. In some cases, reporting the number of people working on a project would pose a privacy risk. A definition on the order of 20% or more of all person hours spent on the project seems more reasonable. | The work group has clarified the instructions to address the concerns. Anyone who recieves compensentation from the project should be listed, no matter the role they play. | |
| HHS/HRSA | Instructions | Page 5 participants: Who has been Involved? states that demographic data submission for persons who have contributed significantly to the project is voluntary, but Page 6, Participants: What individuals have worked on the Project? has no note that the demographic data requested is voluntary. If the demographic data collection on page 6 is not voluntary it would appear to be a change in policy and it is suggested that this statement be reviewed by OGC. If the Page 6 demographic data are to be voluntary, then the note on Page 5 should be clear that it covers all demographic data and not just the data on Page 5 regarding significant contributors. | This section is not mandatory. Additionally, the work group has upgraded the report to clarify this point. | |
| HHS/NIH | Instructions | A bit of a disconnect exists in the current text. On page 5, where this category is first described, it indicates "If a person has contributed significantly to the project...". This wording could be construed to imply only senior/key persons since "contributed significantly" is how that category of participant is defined. However, later in the instructions the first question refers to "What individuals have worked on the project?" and includes each person (not just senior/key) who has worked at least one person month. | The report will be updated based on this comment. | |
| HHS/NIH | Other | NIH fully supports the concept of this being a full personnel report (not just senior/key persons). In fact we now have a legislative requirement to specifically report back to Congress on the workforce we support, in particular postdocs. So this level of detail will be even more critical for us now. | The report will be updated based on this comment. | |
| HHS/NIH | Instructions | Since the detailed instructions already cover what we need, our suggestion is to modify the first sentence in the 1st paragraph under the category description as follows: If a person has worked at least one person month on the project during the reporting period, provide name, role in project, and major activities performed. | The report will be updated based on this comment. |
|
| NASULGC/ ESCOP | Instructions | The definition of Participant should better explain that it does not include people who participated by generating research results through being part of activities such as focus groups, survey respondents, clinical trial subjects, taste panels, etc. | The work group will update the report to clarify that participants do not include those people who have participated in focus groups, trial subjects, etc. |
|
| NASULGC/ ESCOP | Instructions | In the individual participants data element, if the participant was an undergraduate or graduate student or in a post-doctorate position, the report should indicate that person’s current status if different than when participation occurred. This information will allow assessment of the educational impact of the activity. | The report currently indicates this. | |
| NASULGC/ ESCOP | Instructions | All three Participant data elements reported should provide additional information on international participants, particularly resources provided by the international participant or their institution. | Agencies may pursue developing agency specific requirements through OMB. | |
| NCAR | Instructions | At any given time we have around 550 employees working on the core funded research at or above the one person month per year. In the format proposed, we would need to list every participant, their role in the project, the # of months worked, their contribution and the funding support, in addition to their demographic information. We understand that NSF and other agencies need to have this information in their databases to track, however, as an FFRDC with a large cooperative agreement, it can be very time consuming to enter this information. It would be advantageous if there was an upload capability so that we did not need to type in each person and their corresponding information. In addition, it will be a very onerous task to acquire that specific information for each individual participant. | This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NCAR | Other | Last year we listed over 600 organization partners. To identify the partners, we either had to search in the database and select the partner or we had to add the organization within the Fastlane system. As mentioned above, it would be very helpful if the system allowed for an upload capability. | This is an electronic system implementation issue. The work group will forward your comment to those who develop the electronic solution. | |
| NSF | Instructions | This optional section requests information about collaborations. The same information is also requested products and outcomes category (networks and collaborations). | The work group will update the report based on this comment. |
|
| NSF | Instructions | While we appreciate the fact that participants involved now includes information on ethnicity, etc., we would also like to see information captured about the students trained under awards. These students are often not actually "participants" officially, but this is a part of the overall outreach provided by the award. We feel this information is important. Furthermore, it would be nice if the reporting could include advances achieved through the award. What did they start with and how many more students from diverse backgrounds are now heading out to careers in research? Where are the trainees going? Hopefully not only into medical fields....... | Agencies may pursue developing agency specific requirements through OMB. | |
| NSF | Instructions | There is a request for information on "non-formal collaborations or contacts with others outside the United States." This is made as part of a bulleted list of 'for instances' regarding collaborations. Given the increasing importance of international collaborations, shouldn't this request for information be raised above the level of one among several 'for instances'? | Agencies may pursue developing agency specific requirements through OMB. | |
| NSF | Instructions | Also, it is not clear what a 'non-formal collaboration' entails as opposed to a formal one - and why do we want information only on the non-formal ones? | The work group will update the report based on this comment. |
|
| NSF | Instructions | In the same vein as (3), given our interest in interdisciplinarity, mightn't information regarding this also be raised above its current status of one 'for instance'? In fact, more information may be needed about the disciplines involved and the character of the impacts (e.g., what, if anything, has been "transformed," and for whom?). | Agencies may pursue developing agency specific requirements through OMB. | |
| NSF | Instructions | On page 6 I think the phrase "funding information is not required, but welcome" actually applies to the demographic information being requested. That wording is confusing and allows the respondent to easily opt out. It might be better to phrase it along the lines of "the following demographic information is not mandatory, but would be extremely useful to NSF (or whichever agency is collecting)." | The work group feels that the current language is appropriate for the nature of the request. | |
| NSF | We agree that these two categories should be optional in the RPPR. | Thank you for your comment. | ||
| University of Vermont | Other | [You should not implement the RPPR unless and until grants.gov can feed such information into it.] The same applies to participant information which is collected in the grants.gov electronic application and is not likely to change during each reporting period. It should be collected at the project's inception via grants.gov, and displayed in the RPPR, with an option to identify changes only if there are any. | The work group plans to move forward with implementation. The data provided to Grants.gov is only for the pre-award phase. The work group will pass your comment along to those developing the electronic solution. There is the intention to develop a solution that will be capable of storing data and prepopulating the report where appropriate. | |
| University of Vermont | Other | Collecting gender, race, and ethnicity information about participants is not only time consuming but suggests that there may be more or less value associated with work done by people of one gender, race, or ethnic background than another. The use of the term "race" in reference to human beings is outdated. Scientists have established that genetically there is no such thing as "race" in the human population. The term “race” is nothing but subjectively assigned anatomical and social distinctions. Human genetic and cultural diffusion makes it senseless to attempt to divide human populations into these groups. Most Americans are of complex ancestry, are multi-ethnic and multi-“racial” and will select one of their ancestries over another at one time (for example on a census form), and a different one for another purpose (such as this one). This kind of data, especially when it is optional, would be highly misleading when used to develop statistics about populations being served, which you state as your reason for collecting this. | Agencies will not do any evaluation based on the information. The use of the report may have been misinitepreted. The demographics information being requested are government wide standard categories used consistently in several forms. |
| Comment Source | Comment Type | Impact Comment | Comment Response | Notes |
| ASU | Instructions | Some respondents indicated the need to expand further on the Optional Category for Impact on society. It was suggested that the Impact section ask for a description of potential policy implications or actual policy actions based on the scientific findings. This would provide an opportunity for investigators to describe the science in a more policy-relevant manner which would communicate the value of science in its ability to serve the values of society. | This goes beyond the scope of a grants specific progress report. However, agencies may pursue developing agency specific requirements through OMB. | |
| DoC/NIST | Other | Extramural: Impact section--this would be really great for the TIP emphasis. | Thank you for the comment. | |
| DoC/NIST | Instructions | Extramural: Very publication oriented…and very little quantitative information about the achievement of the science or technology relative to the knowledge-base to see how much "better" this actually is…you could really wave hands here and have no concrete impact to point to that is measureable in terms of improved science/technology achievement, so adding some quantitative comparison to a baseline of "what could be done at the start of the project" is needed or this is so much hand waving that it isn't funny. We would never get info like a "performance improvement of x over y for z dollars in this project" with these instructions it would all be anecdotal hand waving and counting of pubs, patents and presentations. Adding some questions to describe impacts relative to the previously defined baseline would greatly strengthen this and make it more concrete and less prone to hand waving. | No action Needed. | |
| ED | Instructions | Tracking Performance Over Time: In both the Accomplishments and Impact categories, there is no provision for identifying accomplishments and/or impacts that have been reported in more than one reporting period (e.g., budget period or project year), but at different stages of development. Most noteworthy accomplishments of research and development (R&D) activities are not discrete events that occur within a one year time frame and then disappear; rather, both accomplishments and impacts typically mature over time from smaller, incremental accomplishments or milestones. For example, in one year a grantee may report the successful development of a product (e.g., a database) or technology that is reported as a peer-reviewed publication the following year. Agencies need to have a way to link these accomplishments and track their evolution. We recommend that RPPR form ask grantees: (a) whether accomplishments and/or impacts reported in the current reporting period (e.g., budget period or project year) are an outgrowth of accomplishments and/or impacts reported in a previous period; and (b) if so, describe the relationship. | There is no change necessary. Reporting periods are artificial endpoints. The grant life cycle is comprised of all the reporting periods from award to close out. | [Comment was also under Accomplishments tab.] |
| ED | Instructions | Linking Accomplishments and Impacts to Goals: We are concerned that the current version of the form provides no mechanism by which grantees can link accomplishments and impacts to goals and/or performance measures. This is particularly important for grants with multiple projects and goals – such a linkage would help agencies determine grantee progress in different areas. We recommend that the RPPR form require grantees to link their accomplishments to their goals and/or performance measures. One way to do this would be for the form to use an internal numbering system identifying and linking project goals and activities with the goals and accomplishments to which they apply. | The work group feels this linking will naturally occur. However, if the program feels this capability is essential, it may pursue developing agency specific requirements through OMB. | Wouldn’t the PI automatically be linking the accomplishments to the approved application. We do not not need to elaborate on it. [Comment was also under Accomplishments tab.] |
| ED | Instructions | Regarding the optional Impact category, the instructions tell respondents to use “a ‘field’ or ‘discipline’ that corresponds to a single academic department.” We recommend adding the words, “if appropriate,” after “department.” All research may not be done in an academic setting. For example, this Department funds research on prosthetic devices, which are not generally covered by an academic discipline. | The work group will update the report based on the comment. |
|
| ED | Instructions | The second sentence on p. 8 should be amended to add the word “policy,” so the new sentence would read, “Describe distinctive contributions, major accomplishments, innovations, successes, or any change in policy, practice or behavior…” (italics added). | The work group feels the present instructions accurately captures the intent of the section. | |
| ED | Instructions | On page 8 it is unclear if the second set of bullets are background information or items requiring a response. If requiring a response, why are they separated from the instructions section on page 9. | The work group will update the report based on the comment. |
|
| ED | Instructions | Definition of Impacts: Defining impacts in part in terms of “major accomplishments” is likely to cause confusion among PD/PIs completing the RPPR form as it appears to be redundant with the previous Accomplishments category. Rather, the emphasis should be on describing the use or adoption of R&D findings and products produced under the grant award to advance knowledge, build capacity to generate research, change or improve policy, practice, and/or improve overall society. | The work group feels the language used in the report is appropriate. | |
| ED | Instructions | Impact on the Development of Human Resources: We are concerned that this type of impact described in the proposed RPPR is limited to the human resource development in science, engineering and technology instead of being extended to include all potential beneficiaries of R&D investments, including practitioners and end-users or consumers. We recommend that this definition be broadened to include impacts on the functioning of human subjects/participants in R&D studies. | The report currently addresses this concern. The specifics provided are examples, not a defininitive list of types of impact. | |
| HHS/HRSA | Instructions | A commenter from the Maternal Child Health Bureau noted the section on “Impact” is very broad and requests information that is beyond the scope of most Maternal Child Health(MCH) research projects. Currently MCH research grantees are required to report on “Evaluation/Impact” that describes the potential impact of the study on the health care of individuals being served and potential impact at the local, state, regional and/or national levels. | Agencies may pursue developing agency specific requirements through OMB. Additionally, the agency may consider using the PPR instead of the RPPR. | |
| HHS/HRSA | Other | A commenter noted on page 8 of the document “Draft Format” states that the information provided in the optional section “impact of the project” makes the case for Federal funding of research and education. If this is true, then this section should not be optional, it should be required. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. | |
| HHS/HRSA | Instructions | Bullets related to Impact and Changes in Practice may need to focus on educational evaluation outcomes, such as the impact of the intervention on targeted groups, especially targeted minority groups or be replaced by the questions under the category “What is the impact on society beyond science and technology?” | Agencies may pursue developing agency specific requirements through OMB. | |
| HHS/HRSA | Instructions | Bullets related to how the research benefits the nation may need to be worded to capture the future outcome, since some outcomes, such as organ donation often occurs at a later date. | The report currently addresses this concern. | |
| NSF | Page 9, under "Impact on the Development of the Principal Discipline(s) of the project," the respondent is told "How the field or discipline is defined is not as important as covering the impact the work has..." We STRONGLY object to that. A standardized taxonomy should be provided. SRS is working to harmonize our internal taxonomies and we are especially concerned about the development of new and emerging fields. If projects from the funding agencies which help foster both new and emerging projects as well as interdisciplinary projects are not coded, then the value of this data is severely limited. SRS could provide an interim taxonomy to be utilized and we will be revising our taxonomies in the future. | Keeping an updated list would be extremely time consuming and difficult. When a final taxonomy has been developed, the work group will consider including it as an update to the format. |
| Comment Source | Comment Type | Changes/Problems Comment | Comment Response | Notes |
| DoC/NIST | Instructions | Extramural: This section is critical to managing cooperative agreements and needs to be in the mandatory section. We strongly suggest expanding to include OPPORTUNITIES, that may not have been evident at the time the proposal was submitted, and capture them here. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. Agencies may choose which sections they require a grantee to complete. Agencies may also pursue developing agency specific requirements through OMB. | |
| DoC/NIST | Instructions | Extramural: Also make clear that this is not the process to use for asking for prior approval of something that requires advance approval or is a major problem, or addresses a special award condition. This is not at all clear in the instructions. | Based on the comment, the work group will update the section to clarify its purpose. |
|
| DoC/NIST | Instructions | Extramural: This is the section we mean: CHANGES/OPPORTUNITIES/PROBLEMS/SPECIAL REPORTING REQUIREMENTS If not previously reported in writing, provide the following additional information, if applicable: Changes in approach and reasons for change; Actual or anticipated problems or delays and actions or plans to resolve them; Changes that have a significant impact on expenditures; Significant changes in use or care of animals, human subjects, and/or bio hazards; Special reporting requirements specified in award terms and conditions; and As stated in the regulations, include any adverse conditions which materially impair the ability to meet the objectives of the award. | Agencies may pursue developing agency specific requirements through OMB. | |
| COGR | Other | We endorse the shift of the Changes/Problems/Special Reporting Requirements from mandatory to optional and appreciate the changes in the reporting of changes that have a significant impact on expenditures. As currently proposed, investigators are prompted to discuss events or situations that will affect expenditures without attempting to assign rates of expenditures | Thank you for your comment. | |
| ED | Instructions | Regarding changes to care of animals, human subjects, and/or biohazards, we recommend putting human subjects before animals in the list. Also, in matters relating to human subjects, the specific term of art for the approving entity is “institutional review board” or “IRB.” We do not believe that asking whether the change was approved by the “institution” is sufficiently precise. For example, a PI may desire to change some aspect of the research that involves human subjects. If the PI goes to the department chair and obtains approval, the researcher still must obtain approval from the IRB. We recommend changing the conditional statement “if required” to make it more specific. For example, the phrase might state, “If required by the IRB approval, 45 CFR part 46, or the human subjects assurance or policy of the institution, …” | The work group will update the section based on the comments concerns. |
|
| Also, not all changes have to be reported to the funding agency. However, if the change is the result of an IRB conclusion that there was a deficiency in compliance with protection of human subject requirements, that change must be reported to the agency. We recommend, at minimum, changing the final sentence to say “or” reported to the agency, instead of “and” reported to the agency. | ||||
| ED | Instructions | Generally, this Department does not do research involving animals, so we can’t speak to issues related to approvals regarding the use or care of animals, however, we recommend that the form address each of these areas of concern separately (i.e., human subjects, use or care of animals, and biohazards). By addressing each area as a separate subset of the category, the question won’t be so general that the instructions are not helpful or too confusing by mixing up the different requirements in the same question. | The work group feels no change is needed. | |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? Changes/Problems/Special Reporting Requirements: does not request information on changes to key personnel. | This concern has been addressed in response to a previous comment | |
| HHS/CMS | Instructions | Under Actual or anticipated problems or delays and actions or plans to resolve them--change sentence to read: Describe problems or delays encountered during the reporting period and corrective action planned to resolve implementation problems and state the effect these problems had on achieving the project goals during remaining schedule or plans to resolve them. | The work group feels the language currently used is appropriate. | |
| HHS/CMS | Instructions | Under Changes that have a significant impact on expenditures--add the following second sentence: Also indicate if carryover of unobligated funds is anticipated. | The work group feels this type of data is not appropriate for this section. | |
| HHS/HRSA | Instructions | It was further noted that the section on “Changes/Problems/Special Reporting requirements” does not request information on any changes in key personnel. MCH Research grantees are required to provide information on changes of key personnel and to provide biographical sketches for any new key personnel. | The work group feels this type of data is not appropriate for this section. | |
| NSF | Instructions | As with the current NSF version, there is a request for information regarding "Significant changes in use or care of animals, human subjects, and/or biohazards." The last sentence of this section (page 11) says; "If required, were these changes approved by the institution and reported to the agency?" Technically, this is a reasonable question - but PIs often do not realize that changes to protocols require IRB approval and therefore do not obtain the requisite approval. I assume that this is also the case for IACUC and biohazards So, it would not be amiss to insert a reminder to the PI here e.g. "Changes in approved protocols for human subjects research or research governed by an IACUC must be reported to the institutional governing body. Has this been done and approved?" | This concern has been addressed in response to a previous comment. | |
| NSF | Other | We do not agree that this category should be optional. Instead, we believe that this category should be mandatory because documenting the changes or challenges that occur during research work is an important way to communicate the actual work and progress of federally-funded research. This documentation provides agency staff with the information it needs to conduct proper oversight of research awards and address any problems. In addition, the information provides a historical "paper trail" of what was planned and what actually occurred. This is especially useful when agency staff is re-assigned to other projects or leave the agency, and historical knowledge about a project may be lost. | Only one section is mandatory, while the other sections are optional. This combination of mandatory and optional sections gives agencies the maximum flexibility to collect only the information required, without overburdening others with unnecessary requirements. Agencies can select the information needed to perform its reviews for a specific program. Information required can vary between agencies and programs. |
| Comment Source | Comment Type | Special Reporting Requirements Comment | Comment Response | Notes |
| HHS | Data Element | Are there other elements that should be included under the reporting categories? Changes/Problems/Special Reporting Requirements: does not request information on estimated unobligated balance. | This concern has been addressed in response to a previous comment. | |
| HHS/NIH | Instructions | Augment the question on “Significant changes in use or care of animals etc…” to also include specific fields for IACUC approval date & IRB approval date. (Or if all the structured data fields will be in the Cover, add these there), Regardless of location, agencies do have a requirement to monitor compliance of these approvals as part of the annual progress report. | The work group will update the form based on the comment. |
|
| NSF | Other | There is no place in the proposed document for the PI to indicate the percentage of funds expected to remain at the end of the reporting period. How will the Program Directors know if residual funds are larger than 20%? | This concern has been addressed in response to a previous comment. |
| Comment Source | Comment Type | Final Project Report Comment | Comment Response | Notes |
| ASU | Other | A similar format is desired for both interim and final reports. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| ASU | Other | Most responses indicated that this format with minor modifications would be appropriate for a final report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| ASU | Other | It was felt that using the same format for final reports will provide consistent information across the life of a project. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| ASU | Instructions | Some respondents indicated that as a matter of public policy, PIs should be required to report on the entire project in a final report, although from the perspective of a PI reporting for just the last year would be easier. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| ASU | Instructions | Others indicated that the PI should provide information for both the final period and a brief summary statement for the entire project period of the overall scientific and/or scholarly accomplishments. The other subcategories could just be completed for the last reporting period. Repeating information in these subcategories that was submitted in prior reports is redundant and should not be required in the final report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| DoC/NIST | Other | Lab: The proposed standardized format is for interim reporting only, not final reports. Why not standardize all reporting formats at the same time? | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| DoC/NIST | Other | Extramural: The basic information requested is not all that different from that we already require from our awardees in the quarterlies. What it will do is standardize the type of information supplied by all awardees regardless of PMT outlook. Having said that, I don't believe that the format being proposed is appropriate for a final report. We should continue to have a final quarterly and a final overall technical report at the conclusion. However, the final technical report should summarize the entire project where did it succeed; where did it fail; how does the failure impede the impact goals; company's future intentions as to goals of project. The company's final technical report should be only one arm of the projects closing documents. This must be accompanied by a report from the PMT covering the same points as the company's report and giving its own evaluation of the project and its future. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| DoC/NIST | Other | Extramural: It is also not a good template for a final report. We would not want to collect any of the diversity information for example, because that will really complicate things in terms of keeping personal identifiable information (PII) around. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| COGR | Instructions | With regard to the creation of a final reporting format, we believe this format with minor modifications could serve as a useful model. For final reporting, recipients can be expected to provide a final statement of scientific and/or scholarly accomplishments including if appropriate training and professional development activities. But other categories, particularly the optional categories of products/outcomes and participants should only add information for the last reporting period. Repeating publication information, the names of participants, etc., submitted in prior reports should not be required in the final report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| DoD | Other | An important distinction between an interim and final report is that for many agencies: The interim report is a relatively informal communication between a principal investigator and federal program manager; while the final report serves as an archival scientific or technical report that documents the purpose, methodology, results and conclusions of the research project and will be available, as appropriate, to other researchers, technologists interested in applying research results, and the public. Thus, the final report may be deposited in a repository such as the Defense Technical Information Center or National Technical Information Service. Some agencies require a grantee or contractor to submit that final report using a Standard Form 298 cover sheet that conforms with ANSI Standard 239.18, a requirement that often does not apply to interim reports. Notwithstanding the distinction between interim and final performance reports, it would be desirable for agencies and recipients to be able to use the same format for them. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| RECOMMENDATION II: The interagency team should seek to finalize the interim performance reporting format in a way that maintains the format’s usability for final reports also. Ideally, we should avoid unnecessary duplication by minimizing the information contained both in the cover sheet of the format and in the SF‑298 cover sheet for a final report. | ||||
| ED | Other | Regarding using this format for final reports, the developers of the format should consider using one system and format that collects data about the grant, about progress, about performance data, and narrative interim and final report information. For example, demographic data about the grantee and the discussion of interim progress provide needed background materials for interpreting the final outcomes. Separate formats for interim and final reports are likely to create duplication, both in the submission of data by grantees and in the review of data by Federal agencies. Grantees should not be required to provide data in the final report that they have provided previously in interim reports since this data should already be stored by the agency in an electronic format. It would, however, be useful for grantees to provide data in the final report on their final budget period, as well as any summary information and conclusions they have drawn regarding the overall success or impact of the project. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| ED | Other | We also note that if the draft format is used for final reports, these are permanent records for ED. If the final report is created and maintained in electronic format, then these records must meet the standards contained in the National Archives and Records Administration (NARA) guidance concerning the creation, maintenance, and transfer to NARA custody of permanent electronic records. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS | Other | A majority of HHS awarding agencies provided comments that with some modifications the proposed interim format could be appropriate for a final report. A minority of comments reflected an opinion that many of the categories would not be relevant to a final report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS | Other | If so, should recipients be directed to provide summary information for the entire project period, or just for the last period? There was no consensus among commenters regarding this question, however a majority of the responses received indicated that a reporting of the entire project period is necessary. It is important to note, however, some of these responses stated that information specific to the last period should also be included. Alternatively, some of the comments requested the final report address the last period but added that a summary of the entire project period would be helpful. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS | Other | If not, what information should be included in a final report? OGPOE received little feedback regarding this question. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/AoA | Other | Is the proposed format appropriate for a final report? Yes | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/AoA | Instructions | If so, should recipients be directed to provide summary information for the project period, or just the last period? Entire project period. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/AHRQ | Other | Is the proposed format appropriate for a final report? Modifications and editing would have to take place for the final report but could be used. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/AHRQ | Instructions | If so, should recipients be directed to provide summary information for the project period, or just the last period? The entire project period would be satisfactory. Everything would be in one report. I think a section should be dedicated to the last budget period then describe the project period as a whole. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/CDC | Other | It is stated that this document is used for the interim progress report only, and may be adopted for the final report. As such, an interim progress report will not be required during the last budget period of the project period. If assessment of progress has been evaluated for each continuing period within the project, the last and final report could provide accomplishments during the final budget period and a summary of the overall accomplishments and success of the project. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/CDC | Other | Is the proposed format appropriate for a final report? Yes. A complete summary of the entire project by year should be required. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/CDC | Other | Recipients should be directed to provide a summary for the last project period that should emphasize final outcomes, impacts, and products. A brief synopsis of the information presented in previous progress reports would be useful as well. If the NC/I/CO wants any additional information in the final report they should be directed to specify that in the FOA. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/CDC | Other | Appropriate for a final report? No, not in its present form. A final progress form needs to include an overall summary and accomplishments of the entire project period-more like an executive summary. Many of the listed categories are not relevant a final progress report. If so, should recipients be directed to provide summary information for the entire project period, or just for the last period? Just the last period. If not, what information should be included in a final report? A final progress form needs to include an overall summary and accomplishments of the entire project period-more like an executive summary. Many of the listed categories do not relate to the final progress report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/HRSA | Other | Format appropriate for a final report? One office asked how will agencies integrate existing OMB approved progress reports with agency and program specific elements into this proposed format? If agencies still have to maintain their separate OMB approved reports, there will be a duplication of information since respondents will have to provide the cover page elements for this proposed standard form and most of the same information when reporting using the agency form. Unless there is an integrated government-wide portal or system that collects the cover page information and then permit respondents access to the agency specific form, there will be duplication of effort and burden. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/HRSA | Other | Format appropriate for a final report? One commenter indicated “NO” noting further that for a final report, some of the categories in the RPPR would require too much detail. It appears that the RPPR would require too much detail on some information including personnel, changes/problems rather than focusing on the outcomes/findings of the research project. To minimize reporting burden for grantees submitting final reports, MCH Research grantees are required to submit a concise final report that addresses the following categories: Introduction (nature of research, purpose and methods, nature of findings); Literature review; Study design and methods; detailed findings; Discussion and Interpretation of Findings; and List of products. Also, a final report is submitted in narrative and no forms need to be completed. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/HRSA | Other | Format appropriate for a final report? Another commenter, indicated yes, and once the RPPR is final it should be used for both progress and final reporting to provide comparison information and minimize reporting burden. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/HRSA | Other | -If so, should recipients be directed to provide summary information for the entire project period, or just for the last period? Several commenters recommended the inclusion of a summary for the entire project period, being mindful of any reporting burden to the grantees. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/HRSA | Other | -If not, what information should be included in a final report? One commenter noted that although the format is appropriate, some additional information may be useful. The final report could include information about how the project builds on previously funded agency grants, lessons learned, suggestions for continued study; plans for dissemination locally and nationally; and sustainability of the program in the targeted region/population in the absence of federal funding. One (copy) of the products resulting from the project should be included with interim and final reports. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| HHS/NIH | Other | Is the proposed format appropriate for a final report? With the suggested edits noted throughout, yes, this would be adequate for a final report. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| NSF | Other | We believe that the RPPR provides an appropriate starting point for developing a format for final project reports. Except for the "Participants" category, all other categories proposed for the RPPR should be mandatory for the final project report format. These categories provide important information for documenting the work conducted and the results and impact of that work. | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. | |
| NSF | Other | In addition, the final project report format should include a requirement for some type of brief summary of the results of research that, in turn, could then be made available to the public. Recent federal audits and surveys indicate the public is interested in obtaining the results of federally-funded research, including publication citations and brief summaries. Requiring a brief summary in a final report would aid agencies in collecting information that they could, in turn, disseminate to the public | Thank you for your comment. The Working Group will consider this comment after development of the interim research performance progress report effort has been completed. |
| Comment Source | Comment Type | SRS Comment | Comment Response | Notes |
| SRS | Other | For SRS purposes, we suspect that the type of information requested would be more useful in project final reports than in a series of interim reports. What is the final progress report/how will it differ from the interim report (or what do you envision the final progress report to be?) | This format is intended only for use in submission of interim reports. The working group plans to look at the contents of the final progress report after development of the interim research performance progress report effort has been completed. This is clearly stated in both the federal register notice. | |
| SRS | Other | What is the frequency or timing or reporting period required of university PI reporting for interim progress reports (twice a year?; three times during the life cycle of an individual grant? etc). | For the minimum reporting frequency requirements, please refer to OMB Circular A-110. However, agencies may specify their own reporting frequencies, so long as it is in compliance with A-110. | |
| SRS | Other | For example, asking for what the project produced (publications, technologies, inventions, and patent applications) seems burdensome for an interim progress report especially given that these are the same sorts of reporting metrics required of universities to provide i-Edison at the completion of the research activity | The working group noted that not every agency reports to I-Edison. Additionally, the format allows agencies to choose which information they collect, which will reduce duplication of data requests. Finally, this format has been designed for use by Program Officials, which is a different audience than those who use I-Edison. | |
| SRS | Instructions | Suggest providing clarification that this report is only for grants and cooperative agreements not contracts. Research is done under contracts and these requirements do not apply to contracts. Uniform wording is used to clarify this project. Please see wording on page 1 first paragraph. | This distinction has been clarified in the most current version of the format. However, the working group notes that agencies who feel this is appropriate for use on certain contracts may do so. | |
| SRS | Other | To be consistent with the above point, suggest replacing “award” or “research grants” with “research award” throughout the document. | We believe this is accomodated by changes in the most recent version of the format. | |
| SRS | Instructions | SRS has a specific need for data related to postdocs. To ensure that postdocs are included when completing the form, we suggest providing an example that uses postdocs. Please see the example “Postdocs who started out as and was previously reports as a Graduate Student” on page 6 under the "What individuals have worked on the project?” | This is an agency specific issue. As necessary, agencies can provide specific instructions with the format. The examples provided in the report are not meant to be limiting. | |
| SRS | Instructions | Suggest collecting the demographic information immediately following the list of “What individuals have worked on the project?” | The working group discussed this issue in great detail and believes that it is best to have the information in separate components to maximize use of the format by the agencies supporting research and research-related activities. | |
| SRS | Other | Although above we have suggested that the demographic information on page 7 is reported with the list of individuals working on the project on page 6, we also want to point out that given that the voluntary nature of the demographic information and its sensitivity, requesting aggregate counts may be less burdensome and thus easier than the current request. However, if aggregate counts are requested, a separate form would facilitate standardization. | The working group fully understands the voluntary nature of the demographic information, and, its sensitivity. However, as included, the format can be implemented to permit an individual to respond. Using composite data would push the burden of collection on the institution, and, they have voiced strong oppostiion to this. Additionally, some agencies' mandates would not be satisfied by the collection of aggregate data. | |
| SRS | Instructions | Please clarify the following demographic information instruction on page 8, “Submission of such demographic information is voluntary and need not be provided if previously submitted.” | The working group has modified the language to clarify the intent of this sentence. | |
| SRS | Other | Please clarify – what’s the relationship between the PPR and the RPPR | The latest version of the format clarifies this point. | |
| SRS | Instructions | As with the draft report RPPR, we suggest providing clarification that this report is only for grants and cooperative agreements not contracts | The latest version of the format clarifies this point. | |
| SRS | Instructions | Please provide clarification for “full body of your comments in the text of the message and as an attachment.” Are submissions requested as text as well as electronic files? | The working group has clarified the instructions. | |
| SRS | Instructions | Please clarify the meaning of “person month” – is it intended to mean “FTE”? | The working group believes that the current language is clear. | |
| SRS | Instructions | How is “significantly” defined? Is it defined as the “nearest person month” as before (which needs clarification)? |
This is an agency specific issue. Agencies may provide clarification if they feel it is necessary. | |
| SRS | Other | Just to point out that none of the types disabilities originally listed include cognitive, mental or emotional disabilities. |
The types of disabilities in the current format are being kept as is to remain consistent with the R&R application forms. |
| # | Action Item for RPPR Finalization | Comment | Date Completed |
| 1 | Clarify concept of "optional" and "mandatory" sections | “General” tab # 92 | |
| 2 | Consider adding an abstract section to the final format | “General” tab #108 | |
| 3 | Consider how to clarify the RPPR format when in paper form | “General” tab #116 | |
| 4 | Change "activity" to "project" under first bullet of "Accomplishments and under the first sentence of the instructions. | “General” tab #122 | March 27, 2008 |
| 5 | Where appropriate, change “none” and “no change” to “nothing to report” | “General” tab #123 | March 27, 2008 |
| 6 | Check the document for language consistency | “General” tab #122 | |
| 7 | Create a "Budget" section. Use R and R budget section as a model. Add this section at the very end. | Other Elements tab # 5, 9 | May 7, 2008 |
| 8 | Add a "Special Reporting Requirements" section | Other Elements tab # 18 | May 7, 2008 |
| 9 | Add "Signature" blocks including name, and title | Data Elements Cover Page tab # 6, 12 | May 7, 2008 |
| 10 | Add a "Submission Date" block | Data Elements Cover Page tab # 14 | May 7, 2008 |
| 11 | Add a "Name of Submitter, if other than PDPI" block | Data Elements Cover Page tab # 16 | May 7, 2008 |
| 12 | Add blocks for "Submitter email and telephone #" | Data Elements Cover Page tab # 16 | May 7, 2008 |
| 13 | Consider adding a "Change in Primary Performance Site" data element | Data Elements Cover Page tab # 18 | May 7, 2008 |
| 14 | Develop language for mandatory accomplishments section re: significant changes | Accomplishments tab #3 | May 7, 2008 |
| 15 | Add "(Include a discussion of stated goals not accomplished during this reporting period.)" to the instructions. | Accomplishments tab #6 | May 7, 2008 |
| 16 | Move the last sentence of the second paragraph of the "major goals of the activity" section up to the first paragraph. | Accomplishments tab # 18 | May 7, 2008 |
| 17 | Delete parentheticial that starts "e.g., undergraduate…" from the "opportunities for training and professional development" section | Accomplishments tab # 20 | May 7, 2008 |
| 18 | Add"(both positive and negative)" after "conclusions" in item 3 of the "what accomplished" section. | Accomplishments tab # 41 | May 7, 2008 |
| 19 | Add the following language: "Agencies will determine if reports are available to the public, if they determine they are, agencies will be expected to develop specific search and report tools. | General tab # 205 | May 7, 2008 |
| 20 | Change the title of the "Products/Outcomes" section to "Products" | "ProductsOutcomes" tab # 6 | May 20, 2008 |
| 21 | Add "copyrightable works" as an example within the "Products" section | "ProductsOutcomes" tab # 7 | |
| 22 | Add "A short description of each site should be provided." to the instructions regarding websites | "ProductsOutcomes" tab # 21 | May 20, 2008 |
| 23 | Add an example list under the "Other" category. Use :cell lines, DNA probes, animal models as examples | "ProductsOutcomes" tab # 23 | May 20, 2008 |
| 24 | In the instructions add "This is not a substitute for any other invention reporting required under the terms and conditions of the award." | "ProductsOutcomes" tab # 24 | May 20, 2008 |
| 25 | In the "Publications" data element add language "Not necessary to include publications already referenced above." | "ProductsOutcomes" tab # 28 | May 20, 2008 |
| 26 | Add a data element requesting the status of the publication | "ProductsOutcomes" tab # 29 | May 20, 2008 |
| 27 | Separate the sentence beginning "Please provide optional demographic…" into a new paragraph to emphasize it. |
"Participants" tab # 3 | May 20, 2008 |
| 28 | Include the sentence notifying grantees that responses in this section are optional in instructions section. | "Participants" tab # 3 | May 20, 2008 |
| 29 | Change the title to "Participants and other collaborating organizations" | "Participants" tab # 4 | May 20, 2008 |
| 30 | Add "Do not Wish to Provide" as one of the data values. | "Participants" tab # 7 | May 20, 2008 |
| 31 | Add a blank space for users to list additional disabilities. | "Participants" tab # 18 | May 20, 2008 |
| 32 | Reorganize the text in "Demographics" section to clarify the purpose of the section | "Participants" tab # 23 | June 3, 2008 |
| 33 | Clarify that the data on funding support is for funding from awards other than the one being reported on | "Participants" tab # 25 | June 3, 2008 |
| 34 | Create a "Demographics" section | "Participants" tab # 33 | June 3, 2008 |
| 35 | Change 1st sentence in paragraph to read: If a person has worked at least one person month on the project during the reporting period, provide name, role in project, and major activities performed. | "Participants" tab # 41 | June 3, 2008 |
| 36 | Add clarification that that participants do not include those people who have participated in focus groups, trial subjects, etc. | "Participants" tab # 44 | |
| 37 | Move "Those who have worked on the project" sentence to the beginning of the paragraph | Participants tab # 44 | |
| 38 | Remove the request for collaboration data from the products section. | "Participants" tab # 53 | June 3, 2008 |
| 39 | Change "non-formal collaboration" to "Collaboration". | "Participants" tab # 56 | June 3, 2008 |
| 40 | Add "if appropriate" after "department" in the "Impact" section | "Impact" tab # 11 | June 3, 2008 |
| 41 | Reorganize the text in "Impact" section to clarify the purpose of the section | "Impact" tab # 13 | June 3, 2008 |
| 42 | Clarify the instructions to clarify the section's intended use ("prior approval" language). | "Changes" tab # 4 | June 16, 2008 |
| 43 | List human subjects before animals. | "Changes" tab # 11 | June 16, 2008 |
| 44 | Change language to read "Approval by the applicable institutional committee" | "Changes" tab # 11 | June 16, 2008 |
| 45 | Add "Also specify the IACUC and IRB approval dates" to the question beginning "Signicant changes to…" | "Special Reporting Requirements" tab # 5 | June 16, 2008 |
| File Type | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |