SS Part A_ADDM PME_OMB_12JUL2016

Performance Measurement and Program Monitoring

(Autism and Developmental Disabilities Monitoring (ADDM) Network

OMB # 0920-XXXX

Supporting Statement A

New

Project Officer:

Jon Baio, Ed.S.

Centers for Disease Control and Prevention

Phone: (404) 498-3873

Fax: (404) 498-0792

Email: jbaio@cdc.gov

November 20, 2015

Performance Measurement and Program Monitoring

• The goals of this project are to support performance measurement and program monitoring efforts; ensure awardee compliance with cooperative agreement requirements; maximize effectiveness and timeliness of technical assistance to awardees; and obtain information needed to demonstrate the program’s public health impact. These performance measures and program assessment activities are required of grantees who receive funding from CDC to enhance their capacity to conduct surveillance for autism spectrum disorder (ASD) and other developmental disabilities (DD) that co-occur with ASD among 8-year-olds and 4-year olds.

• The resulting data will be used to assess compliance with cooperative agreement requirements; assess capacity of ADDM Network sites and identify areas for improvement both within individual sites and as it pertains to the network as a whole; identify effective education/outreach activities; and demonstrate the impact of the ADDM Network.

• Data will be collected using a Microsoft Excel-based data collection tool. All ADDM Network sites will be required to submit data at six-month and two-year intervals.

• The population from which data will be collected is the 10 sites funded through the cooperative agreement plus the site administered by CDC.

• CDC will not use complex statistical methods for analyzing information. Most statistical analyses will be descriptive. Statistical modeling may be included to examine predictors of specified outcomes.

February 13, 2015

TABLE OF CONTENTS

Table of Contents

A.1. Circumstances Making the Collection of Information Necessary

A.2. Purpose and Use of Information Collection

1. Purposes and Use of Information Collection

2. Use of Improved Information Technology and Burden Reduction

3. Efforts to Identify Duplication and Use of Similar Information

4. Impact on Small Businesses or Other Small Entities

5. Consequences of Collecting the Information Less Frequently

6. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

7. Comments in Response to the Federal Register Notice

8. Explanation of Any payment or Gift to Respondents

9. Assurance of Confidentiality Provided to Respondents

10. Justification for Sensitive Questions

11. Estimates of Annualized Burden Hours and Costs

12. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

13. Estimates of Annualized Cost to the Federal Government

14. Explanation for Program Changes or Adjustments

15. Plans for Tabulation and Publication and Project Time Schedule

16. Reason(s) Display of OMB Expiration Date is Inappropriate

17. Exceptions to Certification for Paperwork Reduction Act Submissions

List of Attachments

1. a. Children’s Health Act of 2000 (CHA) 42 U.S.C. 247b-4b, Section 102.

b. Public Health Service Act (PHSA) 42 U.S.C. 241(a), Section 301(a)

2. a. List of Awardees

3. a. Federal Register Notice

b. Summary of Public Comments and CDC Response

4. a. Data Collection Tool

b. Interview Items

5. a. Funding Opportunity Announcement (FOA) DD15-1501

A. Justification

A.1. Circumstances Making the Collection of Information Necessary

The Information Collection Request is submitted under the classification of New. The Centers for Disease Control and Prevention (CDC) seeks Office of Management and Budget (OMB) approval to collect information from awardees funded under the “Enhancing Public Health Surveillance of Autism Spectrum Disorder (ASD) and Other Developmental Disabilities through the Autism and Developmental Disabilities Monitoring (ADDM) Network” cooperative agreement program, hereafter known as the ADDM Network, as well as the surveillance site administered by CDC. OMB approval is requested for the first three years of the four year funding period.

The funding opportunity announcement (FOA) accompanying this information collection request supports funding for sites to participate in the ADDM Network and will enhance surveillance activities at both prior and newly participating sites through two funding components. Component A funds surveillance of autism spectrum disorder (ASD) and developmental disabilities (DDs) (i.e. cerebral palsy (CP) and intellectual disability (ID)) among 8-year-olds. Component B funds surveillance of ASD among 4-year-olds. Component A is required for all applicants, while applying for Component B funding is optional. This information collection request includes activities

Although the ADDM Network is an ongoing surveillance system, this marks the first time that awardees are being asked to conduct rigorous performance measurement and program monitoring activities. CDC requests OMB approval to collect information from these awardees to monitor their progress and assist them in achieving their objectives. Funded grantees will monitor and report progress on certain objectives, activities, and performance measures every 6 months to CDC and every 2 years. A data collection tool has been developed to collect this information: Data Collection Tool (Attachment 4A). Information will be transmitted electronically to a designated contact person at CDC. Additional information will be collected from grantees through interviews conducted by CDC staff while on pre-established individual site calls (Attachment 4B). CDC plans to begin using the proposed performance measurement and program monitoring tools immediately upon receipt of OMB approval. CDC is authorized to collect information for public health purposes by Section 301(a) of the Public Health Service Act (Attachment 1B).

Background

The continued increase in autism spectrum disorder (ASD) prevalence over the past decade underscores the importance of continuing surveillance to monitor trends in the population. The Children’s Health Act of 2000 provides an important opportunity for state/local health departments and/or their designated representatives to conduct public health surveillance of ASD and other developmental disabilities (DDs). Section 102 of the Children’s Health Act of 2000, 42 U.S.C. 247b-4b (Attachment 1A) authorizes CDC to make grants and cooperative agreements for the collection, analysis, and reporting of data on ASD and other developmental disabilities. Specifically, it authorizes CDC to establish centers for the purpose of assisting with surveillance of ASD and other developmental disabilities. Based on this legislation, the National Center on Birth Defects and Developmental Disabilities (NCBDDD) began funding ADDM Network sites in 2000. The most recent phase of funding was launched in January 2015—“Enhancing Public Health Surveillance of Autism Spectrum Disorder and Other Developmental Disabilities through the Autism and Developmental Disabilities Monitoring (ADDM) Network” under the Funding Opportunity Announcement (FOA) DD15-1501. A list of awardees is provided in (Attachment 2A). The purpose of this FOA is to enhance the capacity of surveillance programs to implement or enhance a population-based, multiple-source surveillance program for ASD and other developmental disabilities that co-occur with ASD (cerebral palsy (CP) and intellectual disability (ID)).

Since the early 1990s, the number of children identified with autism spectrum disorder has risen markedly. The uncertainty regarding the cause of this increase and the pressing need for medical and educational services among this growing number of children has created a substantial level of concern among researchers, educators, policy makers, advocacy groups and the general public. Accurate and current prevalence estimates continue to be urgently needed, and CDC and its public health partners are in the best position to provide these prevalence estimates along with other critical information regarding the characteristics, co-occurring conditions, and functional level of children with ASD.

The CDC-funded ADDM Network is the only collaborative network to track the prevalence and characteristics of children with ASD and other developmental disabilities in communities across the United States. Since the start of the ADDM Network in 2000, CDC has funded 16 sites in Alabama, Arizona, Arkansas, Colorado, Florida, Maryland, Minnesota, Missouri, New Jersey, North Carolina, Pennsylvania, South Carolina, Tennessee, Utah, West Virginia, and Wisconsin. The ADDM Network sites are selected through a competitive award process. All sites collect data using the same surveillance method, which is modeled after CDC’s Metropolitan Atlanta Developmental Disabilities Surveillance Program (MADDSP). MADDSP is administered by CDC and represents the Georgia site in the ADDM Network.

A2. Purpose and Use of the Information Collection

9. How the Information will be used and for what purpose:

Monitoring and performance measures must be tracked monthly by sites and reported to CDC on a semi-annual basis (every six months) or biennial basis (every 2 years).

Information to be collected will provide crucial data for performance monitoring and program monitoring and will ultimately strengthen CDC’s ability to monitor awardee progress and provide data-driven technical assistance. Specifically, the information to be collected will help CDC ensure awardees’ compliance with FOA requirements (see attachment 5a, pg. 25-33), assess their capacity to conduct surveillance of ASD and other developmental disabilities, identify areas of strength and weakness among the awardees and across the ADDM Network as a whole, and identify effective community education and outreach strategies.

Information will be collected to document continuous progress toward intended outcomes of this funded program (as indicated in the FOA logic model) in line with the strategies, activities and outputs encompassed under the ADDM Network surveillance model. Such activities include partnering with representative and comprehensive data sources, thoroughly training staff to conduct project activities, efficient collection of study data using modernized information technology systems, performing community education and outreach activities, and producing quality datasets for analysis and publication. Outputs expected to result from these activities include improved understanding of ASD and other developmental disabilities (including knowledge of trends and disparities within the context of recent changes in diagnostic criteria for ASD), stronger relationships with partners and data sources, increased dissemination of ADDM data, and improved reliability and efficiency of ADDM data collection.

The Monitoring and performance measurement plan will outline the process for effective implementation of ADDM strategies and activities to successfully achieve work plan outputs and outcomes. Findings will be used by CDC and awardees to: 1) ensure program implementation and continuous system improvement, 2) demonstrate achievement of program outputs and short-term outcomes, 3) provide an evidence base for potential future changes to the ADDM Network surveillance methodology, and 4) assess the usefulness, scalability and effectiveness of community education and outreach strategies, including how well the strategies reach a diverse group of target populations. CDC’s monitoring and performance measurements will include both process and outcome assessments, and will be consistent with the logic model and approach specified in the ADDM Network funding announcement (DD15-1501, attachment 5a). Specifically, CDC’s measurements will assess: a) strategies and activities related to surveillance, analyses of surveillance data, and community education and outreach, and b) the resulting outputs and short-term outcomes as presented in the logic model. Findings will provide both the awardees and CDC with feedback to improve both current and future public health surveillance activities. Multiple measures address how sub-populations may be reached via ADDM Network activities and outputs. Additionally, measures of the “reach” of ADDM Network data via education and outreach activities will help to demonstrate the value of this federally-funded project. As appropriate, findings will be disseminated to specific sites, the whole ADDM Network, or in publications to improve upon or report on the ADDM Network’s performance in achieving strategic and scientific objectives. The information collected will also increase CDC’s capacity to respond in a timely and effective manner to requests for information on activities from Congress, the Department of Health and Human Services, national partner organizations, and other key stakeholders. As appropriate, CDC will also use the information collection to publish descriptive analyses and other key findings in peer reviewed journals.

The information collection will enable the accurate, reliable, standardized, and timely submission to CDC of each awardee’s Checklist, Worksheets, and Performance Measures. The information collection and reporting requirements have been carefully designed to align with and support the goals outlined in the ADDM Network cooperative agreement. The information collection plan will enable collection and reporting of the information in an efficient, standardized, and user-friendly manner that will generate a variety of routine and customizable reports. Local level reports will allow each awardee to summarize activities and progress towards meeting strategies and performance measure targets. CDC will have the capacity to generate reports that describe activities across multiple awardees. There are significant advantages to collecting information with this reporting tool:

• The requested performance measures are specific, measurable, achievable, relevant and time-framed (SMART). This formulation is intended to facilitate successful achievement of performance measures and is integral to CDC’s performance measurement and monitoring strategy for the program.

• The information being collected provides crucial information about each awardee’s activities and progress over the award period.

• Awardees will have the capacity to enter updates at six-month intervals, facilitating timely communications with and review by CDC, resulting in more effective technical assistance.

• Capturing the required information uniformly will allow CDC to formulate analyses and reports.

II. Intended Use of the Information:

CDC will use the information collected to monitor each awardee’s progress and to identify facilitators and challenges to program implementation and achievement of outcomes. Monitoring allows CDC to determine whether an awardee is meeting performance measures and to make adjustments in the type and level of technical assistance provided to them, as needed, to support attainment of their performance measures. Monitoring and assessment of activities also allows CDC to provide oversight of the use of federal funds, and to identify and disseminate information about successful education and outreach activities implemented by awardees. These functions are central to NCBDDD’s strategic priority around helping children live to the fullest by understanding developmental disabilities like ASD. Finally, the information collection will allow CDC to respond to routine inquiries regarding timeliness and quality of data reported by the ADDM Network.

Awardees will use the information collected to manage and coordinate their activities and to improve their efforts to conduct surveillance and carry out other requirements under the cooperative agreement, including site-initiated analyses and education and outreach activities. The data collection tool will allow awardees to fulfill their reporting obligations under the cooperative agreement in an efficient manner by employing a user-friendly instrument to collect necessary information for both six-month and two-year reporting. This approach, which enables awardees to save pertinent information from one reporting period to the next, will reduce the administrative burden of the submissions process. Awardee program staff will be able to review the completeness of data needed to generate required reports, enter basic summary data for reports at least every six months, and finalize and save required reports for upload into other reporting systems as required.

III. Impact on Respondent Privacy

No personal contact information will be collected. Because data is maintained in a secure, password protected system, there is no impact on respondent privacy.

A.3 Use of Improved Information Technology and Burden Reduction

The CDC contractor, in collaboration with CDC staff, has developed the Data Collection Tool using the Microsoft Excel platform. Since the use of Microsoft Excel is common, we think this user-friendly interface will be easier and more intuitive for awardees to use than special-purpose tools or software. Use of Microsoft Excel will require very little training, and awardees will use the data collection tool provided to record and update award information. Awardees will email completed Microsoft Excel spreadsheets to a designated CDC employee at six-month and two-year intervals. The contractor and/or CDC staff will transfer the data into an aggregate Microsoft Excel spreadsheet for analysis and reporting.

The data collection tool improves information quality by minimizing errors and redundancy. Having all of the information collected in the same place in the same manner will reduce the level of burden attributable to redundancy and reduce the workload to enter and maintain the data. Programs will be able to transfer data from one reporting period to another to minimize data re-entry. Although the exact activities for each site may vary, the use of a standard set of data elements, definitions, and specifications at all levels will help to improve the quality and comparability of performance information that is received by CDC for multiple awardees and multiple award types. Further, standardization will enhance the consistency of plans and reports, enable cross-program analysis, and will facilitate a higher degree of reliability by ensuring that the same information is collected on all strategies and performance measures depending on the awardee type (Component A, Component A & B).

A.4 Efforts to Identify Duplication and Use of Similar Information

The collection of this information is part of a federal reporting requirement for funds received by awardees. The information collected from awardees is not available from other sources.

A.5 Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A.6 Consequences of Collecting the Information Less Frequently

Reports will be collected at six-month and two-year intervals. A single instrument is furnished to sites for reporting, with specific items noted for submission within 6 months after the FOA is awarded, others reported on a biannual basis throughout the funding cycle, and the remaining topics to be addressed biennially. Less frequent reporting would undermine CDC’s ability to provide timely and data-driven technical assistance and to ensure that the awardees are in compliance with the cooperative agreement. The reporting schedule also ensures that CDC responses to inquiries from Congress, the Department of Health and Human Services, national partner organizations, and other key stakeholders are based on timely and up-to-date information.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

Federal Register Notice

A 60-day Federal Register Notice was published in the Federal Register on April 21, 2015, vol. 80, NO. 76, pp.22195-22196 (see Attachment 3a). CDC did not receive public comments related to this notice.

Other Consultations

The data collection instrument was designed collaboratively by CDC staff and the data collection contractor. Consultation will continue throughout the implementation process. There were no external consultations.

A.9 Explanation of Any Payment or Gift to Respondents

Respondents will not receive payments or gifts for providing information.

A.10 Assurance of Confidentiality Provided to Respondents

Respondents are cooperative agreement awardees. The data collection does not involve research with human subjects. The information collection does not require consent from individuals or IRB approval. While consent is not required to report aggregate data, awardee consent will be obtained if specific state data is used for publications, reports, or other publicly disseminated information.

10.1 Privacy Impact Assessment Information

1. The NCBDDD Information Systems Security Officer has reviewed this Information Collection Request and has determined that the Privacy Act is not applicable. The data collection does not involve collection of sensitive or identifiable personal information. Respondents are state/local governmental agencies or their designated representatives. Although contact information is obtained for each awardee, the contact person provides information about the organization, not personal information. No system of records will be created under the Privacy Act.

2. Submission of the completed data collection tool and access to awardee data will be controlled by limiting the number of CDC staff who receive and review the data. All data will be submitted to one specific CDC employee, who will combine data and save in aggregate Microsoft Excel spreadsheet. a CDC staff will have varying levels of access to the data with role-appropriate security training, based on the requirements of their position(s). Data will be stored on an internal CDC shared drive subject to CDC’s information security guidelines.

3. Awardees are required to provide data as a condition of cooperative agreement funding.
   
   

4. While consent is not required to report aggregate data, awardee consent will be obtained if specific state data is used for publications, reports, or other publicly disseminated information.

A.11. Justification for Sensitive Questions

The proposed data collection tool does not collect sensitive information.

A.12 Estimates of Annualized Burden Hours and Costs

Estimated Annualized Burden Hours

Current respondents are the 10 awardees of the ADDM Network cooperative agreement plus the ADDM Network site administered by CDC. There are two categories of respondents: 5 respondents who are completing Component A (includes CDC-administered site), and 6 respondents who are completing Component A&B. Awardees will report information to CDC about their activities and performance measures. One information collection instrument will be used: a Microsoft Excel-based Data Collection Tool (Attachment 4a).

This instrument will be used for all information collection and reporting. However, burden estimates for each information collection vary according to:

1. Whether the awardee is completing Component A or Component A&B.
   Burden is expected to be slightly greater for awardees completing Component A&B than awardees completing Component A only, since awardees completing Component A&B will report on additional activities and performance measures.

2. Reporting period (Initial six-month submission vs. subsequent submissions).
   Burden is expected to be slightly higher at the initial six-month reporting because the awardee will be entering data for the first time. However, the subsequent six-month reporting and the two-year reporting will be slightly lower given that most information has been previously populated and the awardees will only need to make updates and additions. Therefore, for each information collection, the burden table presents separate estimates for:

1. Awardees completing Component A and Component A&B.

2. Initial six-month reporting vs. subsequent six-month and two-year reporting

There are no costs to respondents other than their time. The total estimated burden hours are 122.

Table A.12-A. Estimated Annualized Burden to Respondents

Type of Respondents	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Component A Only (initial six-month submission)	Interview	5	1	3/60	.25
	Worksheets	5	1	90/60	7.5
	Performance Measures	5	1	30/60	2.5
Components A&B (initial six-month submission)	Interview	6	1	3/60	.30
	Worksheets	6	1	120/60	12
	Performance Measures	6	1	42/60	4.2
Component A only (subsequent six-month and two-year submissions)	Interview	5	5	3/60	1.25
	Worksheets	5	5	60/60	25
	Performance Measures	5	5	18/60	7.5
Components A&B (subsequent six-month and two-year submissions)	Interview	6	5	3/60	1.5
	Worksheets	6	5	90/60	45
	Performance Measures	6	5	30/60	15

B. Estimated Annualized Cost to Respondents

A project coordinator at each site will complete the data collection tool. The average hourly wage for a project coordinator is $25.00. The hourly wage rates for project coordinators are based on a rough average of the initial budgets requests submitted in the FOA applications by the awardees. The total estimated annualized cost is $3,050.00, as summarized in Table A.12-B.

Table A.12-B. Estimated Annualized Cost to Respondents

Type of Respondents	Form Name	No. of Respondents	Total Burden Hours	Average Hourly Wage	Total Cost
Component A only (initial six-month submission)	Interview	5	.25	$25	$6.25
	Worksheets	5	7.5	$25	$187.50
	Performance Measures	5	2.5	$25	$62.50
Component A&B (initial six-month submission)	Interview	6	.30	$25	$7.50
	Worksheets	6	12	$25	$300
	Performance Measures	6	4.2	$25	$105
Component A only (subsequent six-month and two-year submissions)	Interview	5	1.25	$25	$31.25
	Worksheets	5	25	$25	$625.00
	Performance Measures	5	7.5	$25	$187.50
Component A&B (subsequent six-month and two-year submissions)	Interview	6	1.5	$25	$37.50
	Worksheets	6	45	$25	$1,125.00
	Performance Measures	6	15	$25	$375
	Total				$3,050.00

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record

Keepers

No capital or maintenance costs are expected. Additionally, there are no start-up, hardware or software costs.

A.14 Estimates of Annualized Cost to the Federal Government

Development, Implementation, and Maintenance

The average annualized cost to the federal government is $160,320.00, as summarized in Table A.14-A. Major cost factors are related to data collection tool and maintenance. The contractor, who developed the data collection tool and is providing ongoing technical assistance, is Deloitte Consulting, LLP.

Table A.14-A. Annualized Cost to the Federal Government
Cost Category	Total
CDC Personnel 50% GS-13@$87,200/year = $43,600 20% GS-14 @ $103,000/year = $20,600 10% GS-11 @ $61,200/year = $6,120 Subtotal, CDC Personnel	$ 70,320
Contractor	$ 90,000
Total	$ 160,320

A.15 Explanation for Program Changes or Adjustments

This is a new collection.

A.16 Plans for Tabulation and Publication and Project Time Schedule

I. Time schedule for the entire project

The cooperative agreement cycle is four years. OMB approval is being requested for three years. Reports will be generated by the awardees per the FOA requirements at six-month and two-year intervals. Data collection began with the awarding of the cooperative agreements and will continue throughout the funding cycle.

II. Publication plan

Information collected by the awardees will be reported in internal and external CDC documents and on the website. CDC staff, in collaboration with awardees, may also submit descriptive analyses of the findings for publication in peer reviewed journals.

II. Analysis plan

CDC will not use complex statistical methods for analyzing information. All information will be aggregated and reported with no program identifiers present in external documents. Most statistical analyses will be descriptive. Statistical modeling may be included to examine predictors of specified outcomes.

A.16 - 1 Project Time Schedule

Activity Time Schedule
Notification of Tool Availability	Immediately upon OMB approval
User Training	Immediately upon OMB approval and ongoing through expiration date
Data Collection	1-36 months after OMB approval
Data Publication	12-36 months after OMB approval
Data Analysis	1-36 months after OMB approval

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

The ADDM Network will display the expiration date for OMB approval of the proposed data collection on the data collection tool and in other public forums as necessary.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

- 21 -