Performance Measures Project OMB passback response- 01-13-20 final

The Performance Measures Project: Improving Performance Measurement and Monitoring By CDC Programs

Part A: Justification

July 22, 2019

Contact: J. Stan Lehman

Telephone: (404) 639-2041

E-mail: syl5@cdc.gov

Program Performance and Evaluation Office

Office of the Associate Director for Policy and Strategy

Office of the Director

Centers for Disease Prevention and Control

Atlanta, Georgia

TABLE OF CONTENTS

A. Justification

1. Circumstances Making the Collection of Information Necessary

2. Purposes and Use of Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

14. Estimates of Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

List of Attachments

1. Authorizing Legislation

2. Sample Performance Measure Technical Specification Instrument

3. Sample Performance Measure Reporting Instrument

4. 60 Day Federal Register Notice

5. Federal Register Notice Comments

6. Research Determination (1 of 2)

7. Research Determination (2 of 2)

8. PPEO GenIC request template

• The Program Performance and Evaluation Office (PPEO) provides technical assistance to CDC programs and external partners to improve the impact of CDC-funded public health initiatives. An important PPEO activity is providing technical assistance to CDC programs as they develop Notice of Funding Opportunities (NOFOs) for cooperative agreements to award federal funds (e.g., to state and local recipients) to implement program activities. Cooperative agreements are the most common funding mechanism for implementing a specific CDC program (e.g., HIV testing, cancer screening, obesity prevention) across multiple jurisdictions. An important element of cooperative agreement implementation is development of program performance measures and the infrastructure for collecting and reporting these data by local recipients to CDC. The Performance Measures Project will work with selected CDC programs as they develop and implement new cooperative agreements to develop key performance measures for all funded recipients to report to CDC.

• Through participation in this Project, CDC programs and recipients of cooperative agreement funds will: 1) Develop strong performance measurement tools and practices; 2) define and operationalize priority performance measures tailored to a specific cooperative agreement; and 3) establish common data collection and reporting expectations across all recipients for a specific cooperative agreement. The Project focuses on addressing these issues during the early stages of cooperative agreement development and implementation.

• PPEO is requesting a generic clearance to allow participating CDC programs to develop information collection instruments specific to performance measures for a given cooperative agreement. Using standardized technical specifications these cooperative agreement-specific data will be used to assess program progress (at both the local recipient-specific and CDC program-wide level), modify programmatic practices as needed, and improve the accountability of federal funds provided through the cooperative agreement.

• Participating CDC programs will provide the cooperative agreement-specific information collection forms to each funding recipient to facilitate standardized performance data collection at the local level. Recipients will report these standardized performance data directly to CDC programs on a periodic basis to assess progress toward achieving desired program outcomes as defined by the cooperative agreement.

• The subpopulation impacted by this information collection is recipients of CDC cooperative agreement funding. The Project will include up to 25 different CDC cooperative agreements and could be scaled up at a later date through a revision. The number of recipients per cooperative agreement will vary dependent on program activity, and the expected average is approximately 35 recipients per cooperative agreement.

• Data will be analyzed by the participating CDC programs on a periodic basis to assess recipient progress toward the key outcomes defined by the cooperative agreement. The information collected is meant primarily for program improvement and accountability; it is not intended to be used as the principal basis for policy decisions.

1. Circumstances Making the Collection of Information Necessary

Each year, approximately 75% of the CDC’s congressionally appropriated funding goes to extramural organizations, including state and local partners, via contracts, grants, and, most commonly, cooperative agreements. A cooperative agreement is an award mechanism used when there will be substantial Federal programmatic involvement, meaning that the CDC program staff will collaborate or participate in project or program activities. The availability of funding for cooperative agreements is announced through a Notice of Funding Opportunity (NOFO). CDC awards approximately 65 new domestic, non-research NOFOs each year (each funded for 1 to 5 years). Cooperative agreements may have only a few funded recipients or more than 50 (such as when a CDC program provides funding to all states and territories). For the purposes of this generic, CDC programs developing new high profile, domestic, non-research NOFOs are eligible to participate. High profile NOFOs, as defined for this project, include those that fund all 50 states, have budgets higher than $10 million per year, and/or have significant stakeholder (partners, Congress, etc.) interest.

Monitoring and reporting of program performance is required of any non-federal entity receiving federal funds under 45 CFR 75.342 (Attachment 1); “The non-Federal entity must monitor its activities under Federal awards to assure compliance with applicable Federal requirements and performance expectations are being achieved”. Under this requested approval, CDC programs may customize the sample “Performance Measure Technical Specification Instrument ” (Attachment 2) and the sample “Performance Measure Reporting Instrument” (Attachment 3) to measure, at the local level, the desired public health outcomes of a particular cooperative agreement, in compliance with the Paperwork Reduction Act (PRA). Individual collection requests submitted under this generic approval will include the tailored forms and a supplementary template (Attachment 8).

The sample Performance Measure Technical Specification Instrument provides standardized technical specifications for operationalizing performance measures. The CDC program, in collaboration with the funded recipients will document these technical specifications for each performance measure developed for a particular cooperative agreement. The sample Performance Measure Reporting Instrument is what the CDC programs will adapt and use for all recipients to report the performance data in a standardized way.

This is the initial request for a generic clearance, thus CDC has no information from use by any CDC program offices. However, this request is based on the recent approval of a similar generic clearance (OMB approval number: 0970-0490, expiration date 1/31/2020) for the Administration for Children and Families (ACF).

The program-specific, generic ICs will be uncontroversial, low burden, and provide a significant benefit to recipients. The data collection is requested for three years.

2. Purpose and Use of the Information Collection

Information collection instruments that are customized to reflect program-specific performance measures will allow CDC programs to collect data for a particular cooperative agreement in a uniform and systematic manner across all recipients, will provide a single reporting format, and will reduce the burden on individual funded recipients. Standardized collection and reporting of performance measures will result in higher quality performance data and allow CDC to monitor how individual recipients are progressing toward implementing the activities and achieving their outcomes and allow CDC to more easily aggregate data across all funded organizations. These data will be used by CDC to monitor trends for each performance measure over the life of the cooperative agreement and show a program’s successes and challenges, make course corrections as needed, and identify technical assistance needs.

This generic clearance will also allow the funded recipients to efficiently report their performance data using an instrument customized to a particular cooperative agreement. Having a standardized instrument will make it easier for them to report data consistently and accurately across reporting periods. The staff from funded recipient organizations will be able to more easily enter data and review for completeness, enter basic summary data for reports, and save required data for use with other reporting systems.

3. Use of Improved Information Technology and Burden Reduction

CDC Programs and funded recipients will use some form of electronic data collection. Building on existing data infrastructure, this will be either a web page, email, or preferably, On-line Data Collection System (OLDC). OLDC is the most sophisticated approach based on web technology. Recipients may enter and retrieve information pertinent to their local program activity through electronic forms closely resembling the paper forms. OLDC reduces paperwork, allows for quicker processing, automatically completes required calculations, checks for potential errors, and provides security.

4. Efforts to Identify Duplication and Use of Similar Information

The information collected from funded recipients is not available from other sources in a consistent format.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection, unless contracted by local funding recipients.

6. Consequences of Collecting the Information Less Frequently

Performance measure data will be collected at least annually in accordance with program guidance and relevant cooperative agreement award terms and conditions. Less frequent reporting would undermine accountability efforts at all levels and negatively impact monitoring recipient progress. The periodic reporting schedule ensures that CDC responses to inquiries from HHS, Congress and other stakeholders are based on timely and up-to-date information.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A 60 Day Federal Register Notice (60-Day-19-19IJ; Docket No. CDC-2018-0118) was published in the Federal Register on 2/7/2019; volume 84, page 2521 (Attachment 4).

CDC received one non-substantive comment to the 60 day FRN (Attachment 5). No response was given.

CDC consulted with Molly Jones of ACF PRA leads Mary.Jones@acf.hhs.gov) in development of this request.

9. Explanation of Any Payment or Gift to Respondents

Respondents will not receive payments or gifts for providing information.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

Staff from the CDC Office of Science have reviewed this Information Collection Request and have determined that the Privacy Act is not applicable (Attachment 6, Attachment 7). The data collection does not involve collection of sensitive or identifiable personal information. Although contact information is obtained for each funded recipient (state or local jurisdiction), the contact person provides information about the organization, not personal information. No system of records will be created under the Privacy Act.

Recipients are required to provide data as a condition of cooperative agreement funding and will submit data via secure mechanism (eg, email, password protected website, Sharepoint).

While consent is not required to report aggregate data, recipient consent will be obtained if their specific data are used for publications, reports or other publicly disseminated information.

Aggregated information will be stored on an internal CDC SQL server subject to CDC’s information security guidelines. CDC staff, technical assistance, and training contractors will have varying levels of access to the system with role-appropriate security training, based on the requirements of their position(s).

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

The proposed generic clearance does not collect sensitive information.

12. Estimates of Annualized Burden Hours and Costs

A. Estimated Annualized Burden Hours

Up to 25 CDC cooperative agreements will be included in this project; phased in over the three year project period. There will be approximately 35 awards per participating cooperative agreement for a maximum of 875 recipients (across all participating cooperative agreements).

Table A.12-A. Estimated Annualized Burden to Respondents

Type of Respondents	Form Name	Number of respondents	Number of responses per respondent per year	Average burden per response (in hours)	Total burden (in hours)
CDC Cooperative Agreement Funding Recipients	Performance Measures Project Information Collection Tool	875	1	40	35,000
Total					35,000

B. Estimated Annualized Cost to Respondents

Estimates for the average hourly wage for respondents are based on the U.S. Department of Labor Bureau (DOL) of Labor Statistics May 2014 National Occupational Employment and Wage Estimates (http://www.bls.gov/oes/current/oes_nat.htm). Based on DOL data, the average hourly wage for an epidemiologist is estimated to be $37.37. The total estimated annualized cost is summarized in Table A.12-B.

Table A.12-B. Estimated Annualized Cost to Respondents

Type of Respondents	Total burden (in Hours)	Average Hourly Wage	Total Cost
CDC Award Recipients	35,000	$37.37	$1,307,950

13. Estimates of Other Total Annual Cost Burden to Respondents or Record

Keepers

No capital or maintenance costs are expected.

14. Estimates of Annualized Cost to the Federal Government

The cost to the Federal Government is approximately $43,800; 20% of a GS-14 step 5 epidemiologist and 25% of a GS-12 step 1 public health analyst).

15. Explanation for Program Changes or Adjustments

This is a new generic clearance.

16. Plans for Tabulation and Publication and Project Time Schedule

A. Time schedule for the entire project

OMB approval is requested for three years. CDC programs will begin participating by the end of 2019. Each CDC program developing a new cooperative agreement will submit a generic information collection for use by all funded recipients. Reports will be generated by the recipients per the cooperative agreement requirements. Data collection will begin when funding is awarded and will continue throughout the funding cycle.

B. Publication plan

Information collected by the funded organizations will be reported in internal CDC documents and shared with recipients. Summary data will be provided for inquiries from HHS, Congress and/or other stakeholders. Individual programs that are planning other types of publications will include that information in their specific request.

C. Analysis plan

CDC will not use complex statistical methods for analyzing information. All information will be aggregated for each cooperative agreement and reported with no personal identifying information in external documents. The information collected is meant primarily for program improvement and accountability; it is not intended to be used as the principal basis for policy decisions.

Most statistical analyses will be descriptive and will vary by individual cooperative agreement need.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

The Sample Performance Measure Instruments (Attachment 2, Attachment 3) will display the expiration date for OMB approval. All collections under this generic will also include the OMB expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

- 12 -