Ss-a_0920-15anc_09-02-2015

Formative and Summative Evaluation of the

National Diabetes Prevention Program

Existing Collection Without an OMB Control Number

Supporting Statement: Part A

Program Official/Project Officer: Stephanie M. Gruss, PhD, MSW

Centers for Disease Control and Prevention

National Diabetes Prevention Program

Telephone number: 770-488-8173

Fax number: 770-488-4639

Email address: Inf6@cdc.gov

September 2, 2015

Table of Contents

Part A: Justification

A1. Circumstances Making the Collection of Information Necessary

A2. Purpose and Use of Information Collection

A3. Use of Improved Information Technology and Burden Reduction

A4. Efforts to Identify Duplication and Use of Similar Information

A5. Impact on Small Businesses or Other Entities

A6. Consequences of Collecting the Information Less Frequently

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A9. Explanation of Any Payment or Gift to Respondents

A10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

A11. Institutional Review Board (IRB) and Justification for Sensitive Questions

A12. Estimate of Annualized Burden Hours and Costs

A12-1. Estimated Annualized Burden Hours

A12-2. Cost to Respondents

A13. Estimated Annualized of Other Total Annual Cost Burden to Respondents or Record Keepers

A14. Annualized Cost to the Federal Government

A15. Explanation for Program Changes or Adjustments

A16. Plans for Tabulation and Publication and Project Time Schedule

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

A18. Exceptions to Certification for Paperwork Reduction Act

Attachments

Attachment 1a Authorizing Legislation for the National Diabetes Prevention Program

Attachment 1b Authorizing Legislation for the Public Health Service Act

Attachment 2 National DPP RE-AIM Framework

Attachment 3a Spreadsheet for National DPP Intervention Sites

Attachment 3b Spreadsheet for National DPP Grantees

Attachment 3c Glossary of Terms for Completing National DPP Spreadsheets

Attachment 4 Introductory/Invitation E-Mail

Attachment 5 Reminder E-Mail

Attachment 6 Thank You E-Mail

Attachment 7 IRB Letter

Attachment 8a Federal Register Notice

Attachment 8b Public Comments and CDC Response

• Goal of the assessment: CDC will use the information gained from the assessment to discern lessons learned and effective strategies around 1) expanding the reach and sustainability of the National DPP lifestyle change programs, 2) improving recruitment and retention efforts, 3) increasing referrals, and 4) securing sustained commitment among insurance providers and employers to either reimburse organizations providing the program or providing an employee benefit option for the program so it is accessible to individuals most in need of this intervention.

• Intended use of the resulting data: This data collection effort is a key component of CDC’s assessment of implementation progress, need for technical assistance, and drivers of program success among 6 National DPP grantees. The objective is to provide high quality programmatic technical assistance to grantees and identify best practices for scaling and sustaining the National DPP.

• Methods to be used to collect: The National DPP Evaluator/Statistician will send an Excel data collection spreadsheet for completion by grantees and their respective intervention sites. The grantee will be responsible for completion of the spreadsheet as well as coordination of site responses. Through the spreadsheet, respondents at the grantee and site level will be asked to submit data describing the components of their intervention, highlighting recruitment strategies, identifying barriers and facilitators to implementation, and documenting necessary resources.

• Subpopulation to be assessed: The National DPP targets individuals at risk of developing type 2 diabetes. The data collection spreadsheets as an assessment tool will be provided to program directors and managers at all six grantee organizations, and program coordinators implementing the National DPP at intervention sites. The spreadsheets collect data including targeted participant demographics and lifestyle change coach credentials.

• How data will be analyzed: The data will be analyzed using descriptive statistics and bi-variate correlations for grantee-specific assessment reports and cross-site annual reports. Linear regression and hierarchical models will also be used in the cross-site annual reports.

OMB approval is requested no later than October 15, 2015.

Formative and Summative Evaluation of the National Diabetes Prevention Program

A. JUSTIFICATION

A.1 Circumstances Making the Collection of Information Necessary

This is a new Information Collection Request (ICR) supporting an assessment of the National Diabetes Prevention Program (National DPP), also known as formative and summative evaluation, which is funded by the Centers for Disease Control and Prevention (CDC). The programmatic activities are authorized by Section 399V-3 of the Public Law 111-148 (Attachment 1a). CDC’s general authority for research and inverstigations is provided by the Public Health Service Act (Attachment 1b).

Diabetes takes a significant toll on the public’s health and subsequently our nation’s health care system. In addition to 29.1 million people in the U.S. population diagnosed with diabetes, CDC estimates that 86 million adults aged 20 or older have prediabetes (CDC, 2014a). The good news is that findings from randomized controlled trials and translation studies have demonstrated that type 2 diabetes can be prevented or delayed in those at high risk, through a structured lifestyle intervention that can be delivered cost effectively in real-world settings (Ali et al., 2012; Albright & Gregg, 2013). The intervention focuses on achievement of modest weight loss (5-7%) and moderate increases in physical activity (Crandall et al., 2008; Diabetes Prevention Program Research Group, 2002; Hoerger, 2007; Lindstrom et al., 2006; Tuomilehto et al., 2001; Zhuo et al., 2012).

To achieve public health impact in reducing the burden of diabetes, the strategies that are effective for individuals must be adopted on a large scale. Toward this end, CDC established the National DPP to promote the large-scale implementation of evidence-based lifestyle change programs. The cornerstone of the National DPP is a 12-month, evidence-based lifestyle change curriculum aimed at increasing knowledge and awareness of healthy eating and physical activity among people at-risk for diabetes (Albright & Gregg, 2013). Any organization that successfully delivers the National DPP curriculum and achieves defined, participant-level outcomes is eligible to apply to CDC for recognition through the Diabetes Prevention Recognition Program (DPRP), which serves as a quality control lever for the National DPP. Criteria for obtaining recognition through the DPRP are documented in the Diabetes Prevention Recognition Program Standards and Operating Procedures (i.e., the Standards or DPRP Standards).

CDC currently funds six national organizations under Funding Opportunity Announcement (FOA): DP12-1212PPHF12 to establish and expand “a network of structured, evidence-based lifestyle change programs designed to prevent type 2 diabetes among people at high risk” (CDC, 2014b). The grantees are currently in Year 3 of a five-year cooperative agreement. The six National DPP grantees are working with approximately 110 intervention sites to deliver lifestyle change programs consistent with DPRP Standards. Grantee activities are designed to strengthen and sustain the National DPP by:

• increasing the number of delivery sites,

• developing delivery sites’ capacity to obtain and maintain DPRP recognition,

• gaining sustainable support for delivery sites from insurance companies in the form of reimbursement, and

• actively educating employers and insurance companies about the cost effectiveness of including the lifestyle change program as a covered health benefit and reimbursing delivery sites on a pay-for-performance basis.

CDC requests OMB approval for 3 years to collect the information needed to conduct a full formative and summative evaluation of National DPP implementation for cooperative agreement Years 3, 4, and 5. Respondents will include the six National DPP grantees and their associated implementation sites. Each respondent will complete an annual Excel spreadsheet. The spreadsheet is based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) evaluation framework (see Attachment 2) referenced in the FOA and lessons learned from technical assistance activities conducted in Years 1 and 2. The RE-AIM framework identifies pertinent questions around process and outcome measures for monitoring grantee’s activities and progress. In Year 1, the six grantees provided qualitative information to CDC about activites at their intervention sites. CDC and the grantees used the qualitative reports and the RE-AIM framework to define the data elements to be used for process evaluation. A prototype of the instrument was fielded in Year 2. The Year 2 responses allowed CDC and grantees to further refine the process indicators, data items, and response options needed to accurately characterize their program implementation activities. The information collection for Years 3, 4, and 5 (Attachments 3a and 3b) is based on these initial developmental activities.

A.2 Purpose and Use of Information Collection

The objective of the Formative and Summative Evaluation of the National DPP is to collect information needed to identify program-level factors that lead to successful implementation and best practices for achieving program sustainability and scalability at the community level. The spreadsheet to be completed by National DPP intervention sites is included as Attachment 3a. The spreadsheet to be completed by National DPP grantees is included as Attachment 3b. Each grantee will be responsible for completing a grantee-level data collection spreadsheet in addition to distributing site-level data collection spreadsheets to their intervention sites, and collecting and compiling the spreadsheets from their sites. The final spreadsheet to be submitted to CDC includes each grantee’s responses and one tab per intervention site’s responses. Questions across both versions are similar. Both grantees and intervention sites will report on program components, recruitment strategies, resource use, reimbursement systems, and implementation progress, including barriers and the strategies for overcoming barriers. In addition, the intervention sites will report additional information about participant incentives, lifestyle change class locations, average lifestyle coach and program coordinator salary information, and average cost per participant.

The data to be collected is a key component of CDC’s quality improvement process with current National DPP grantees. The target audience for the collection and sharing of information is the grantees and their lifestyle change program implementation sites. CDC will return an annual assessment report to each grantee and prepare an annual cross-grantee, de-identified, aggregate assessment report. These reports will form the basis for technical assistance to grantees as they scale up their lifestyle change programs, foster sustainable funding relationships, and assist their intervention sites through the DPRP recognition process. The National DPP grantees will be better equipped not only to guide program implementation across intervention sites, but also to support these sites towards achieving DPRP recognition.

The data collected will also be used to articulate the programs’ best practices to key stakeholders and to new sites that may be considering or planning to offer a diabetes lifestyle change program. Increasing the number of sites that offer evidence-based diabetes prevention programs is a key CDC objective, therefore, new sites may or may not be affiliated with one of the current National DPP grantees. For this reason, CDC requests but does not require respondents to report average cost information about program staff salaries and average program cost per participant. This information will be collected in de-identified, aggregate form. The main purpose of collecting the average cost information is to provide estimates that new sites could use for planning purposes.

Ultimately, the comprehensive formative and summative evaluation plan supports CDC’s ability to identify barriers and facilitators to program implementation across a wide variety of types of intervention sites, geographic locations, and client populations, and to share this information to strengthen program reach, success, and sustainability.

A.3 Use of Improved Information Technology and Burden Reduction

All (100%) of the data collected for this ICR will be through an electronic Excel spreadsheet. By using an electronic format for our assessment data collection, we will reduce the burden of respondents having to use a paper format and then mail their responses back to CDC. To further minimize burden on grantees and intervention sites, the data collection spreadsheet is due only once a year and includes comprehensive drop-down response options along with pre-populated information (i.e., grantee name, site name, site code, grant year, and fiscal year). In addition to drop-down response options, the spreadsheet includes free-response boxes to accommodate reporting unique program characteristics. Grantees will be offered technical assistance on the spreadsheet in the form of a Glossary of Terms (Attachment 3c), e-mail correspondence, webinars, and conference calls.

A.4 Efforts to Identify Duplication and Use of Similar Information

There are no similar data available that meet the needs of this proposed assessment. The proposed information collection is distinct from, but complementary to, other information collections that CDC conducts to monitor and evaluate National DPP efforts. The spreadsheets will provide information that is not included in the progress reports submitted to CDC by the six National DPP grantees. Similarly, the information requested on the spreadsheets is distinct from the information that intervention sites may provide to CDC through the Diabetes Prevention Recognition Program (OMB No. 0920-0909, exp. 12/31/2017). The Formative and Summative Evaluation of the National DPP looks at grantee and site-level data relating to implementation processes and capacity, not participant-level data. The DPRP process looks at de-identified data about individuals who are participating in a lifestyle change program. Although National DPP grantees are assisting some intervention sites to prepare for recognition through the DPRP, some of the organizations that apply to CDC for recognition through the DPRP are not affiliated with the six National DPP grantee organizations. The proposed assessment will collect information that is not available from other sources and provide unique insights about challenges and facilitators to large-scale implementation of effective, evidence-based lifestyle change programs for the prevention of type 2 diabetes.

A.5 Impact on Small Businesses or Other Entities

Some intervention sites may be small businesses. The impact on small businesses is anticipated to be minimal (average of 45 minutes per annual submission), and relates to their interaction with National DPP-funded organizations during the next three years. The information collection does not impose an ongoing record-keeping or reporting burden on intervention sites. The data collection burden on grantees is greater (i.e., up to 12 hours), but none are small businesses.

A.6 Consequences of Collecting the Information Less Frequently

Collecting this assessment data will allow CDC to increase access to the benefits of the lifestyle change programs of the National DPP for individuals at high risk of type 2 diabetes. Not collecting this data annually on program implementation from prior year would significantly reduce CDC capacity to support existing National DPP grantees through tailored technical assistance, which would negatively impact the capacity of grantees to support their intervention sites to successfully deliver the lifestyle change program for subsequent year of implementation. Failure to collect data for this assessment would also hinder identification of the organizational-level best practices for National DPP implementation. In summary, the main consequence of not collecting this information annually would be a disruption of CDC’s efforts to (a) aid current grantees in program implementation and sustainability for the remaining of the grant period, and (b) to aid other National DPP stakeholders in providing important lessons learned and effective strategies around program implementation.

Information will be collected once per year. There are no legal obstacles to reducing the burden.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A Notice was published in the Federal Register on June 9, 2015; Volume 80, No. 110, pp. 32562-32563 (Attachment 8a). Comments were received from one organization: America’s Health Insurance Plans (AHIP), in which CDC has responded on 08/20/2015 (Attachment 8b).

Based on the comments from AHIP, we have clarified the intent and scope of the Formative and Summative Evaluation of the National DPP grantees and made changes as follows:

• CDC clarified the relationship between the development and collection of information in Years 1 and 2, and the spreadsheets that are proposed for Years 3, 4, and 5 (Attachments 3a and 3b). The proposed information collection is an extension of previous efforts to provide specific actionable feedback to the grantees and sites for the purposes of program improvement.

• The estimated time burden for National DPP grantee has increased to up to 12 hours.

• The estimated time burden for National DPP intervention site has changed to the range of 30 to 60 minutes, with an average burden per response of 45 minutes.

• A question and corresponding response options on use of marketing materials has been updated.

• The wording of 3 questions about the cost of the intervention have been changed to request data only if the grantee is able to report the data.

• A glossary of terms for National DPP grantees and intervention sites has been attached to the Excel data collection spreadsheet for clarification of terms used within the spreadsheet itself (Attachment 3c).

A.9 Explanation of Any Payment or Gift to Respondents

Respondents will not receive any payment or gifts as a result of their completing and submitting the Excel data collection spreadsheet.

A.10 Protection of the Privacy and Confidentiality of Information Provided by Respondents

Respondents to the data collection spreadsheet will be National DPP grantee program-level directors and managers, and intervention site-level program coordinators. All information will be collected electronically.

Privacy Act Determination

CDC will not receive any personally identifiable information from spreadsheet respondents. Although CDC knows the names of the grantee organizations and program delivery sites, spreadsheet responses will not be directly linked to actual program participants. The spreadsheets do not collect participant-level data.

No system of records is being created under the Privacy Act for this data collection. The Privacy Act does not apply. Once the Excel data collection spreadsheets are submitted to CDC by the grantees, they will be shared via a secure file transfer site with the National DPP’s evaluation contractor. The contractor will analyze the data and provide feedback to CDC staff.

Access to the data collection spreadsheets will be limited to CDC authorized program staff. Project reports and manuscripts will contain aggregated de-identified (by grantee and grantee site) data only; results will not be associated with any individual respondent.

All spreadsheets will be in the form of electronic data files. Spreadsheets will be identified by the name of the grantee organization, the name of the program delivery site (if different from the grantee organization), and an assigned identification number for the site. Any file transfer will occur through a CDC Secure File Transfer Protocol (SFTP) site. CDC and its contractor will safeguard the responses and will not release any personally identifiable information.

A11. Institutional Review Board (IRB) and Justification for Sensitive Questions

The proposed information collection does not involve research with human subjects and IRB approval is not required (Attachment 7). Program directors and managers at the grantee level will be informed that submission of the grantee and intervention site-level data collection spreadsheet will occur in an introductory e-mail (Attachment 4) with which the spreadsheet will be sent. They will also be informed that the resulting data will be shared with them for program improvement purposes. Program directors and managers will also be informed that the data collected are part of National DP12-1212 mandatory data collection for evaluation; but, in an aggregated manner, only to be shared back with the grantees themselves or across grantees in a de-identifed, aggregate manner. Some data collected are voluntary (i.e., cost/salary data) and will be indicated as such on the spreadsheets.

It is possible that discussion of organizational barriers and average cost/salary data could be construed by a grantee or their delivery site as sensitive. Therefore, the response option of “not able to answer” is included within the spreadsheets or no response at all is permitted. No grantee is being required to answer sensitive questions.

A12. Estimate of Annualized Burden Hours and Costs

OMB approval is requested for annual information collection over a three-year period.

Each National DPP intervention site will complete an Excel spreadsheet (Attachment 3a). The annualized estimated number of respondents is 120, based on 110 sites in 2015, 120 sites in 2016, and an estimated 130 sites in 2017. The estimated burden per response is 45 minutes. Each site will submit a completed spreadsheet to its associated National DPP grantee.

Each National DPP grantee will also complete an Excel spreadsheet (Attachment 3b). The number of grantees is expected to remain constant (6) and the estimated burden per response is 12 hours. This estimate includes the time needed to answer grantee-specific questions, to coordinate the collection of site-specific spreadsheets, and to calculate summary information based on the site-specific spreadsheet reports.

A Glossary of Terms (Attachment 3c) will be provided to all respondents to ensure uniform reporting.

The total estimated annualized burden is 162 hours, as summarized in Table A.12-1 below.

Table A.12-1. Estimated Annualized Burden Hours

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in Hours)	Total Burden (in Hours)
National DPP Intervention Sites	Spreadsheet for National DPP Intervention Sites	120	1	45/60	90
National DPP Grantees	Spreadsheet for National DPP Grantees	6	1	12	72
Total					162

Table A12-2 presents the calculations for cost of annualized burden hours. Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey Estimate for a Social and Community Service Manager. Actual wages are unknown and may vary significantly depending on respondent employment status and the state, tribe, or territory in which they reside.

Table A.12-2. Estimated Annualized Cost to Respondents

Type of Respondent	Form Name	Number of Respondents	Total Burden (in Hours)	Average Hourly Wage*	Total Cost
National DPP Intervention Sites	Spreadsheet for National DPP Intervention Sites	120	90	$34.50	$3,105.00
National DPP Grantees	Spreadsheet for National DPP Grantees	6	72	$24.50	$1,764.00
Total					$4,869.00

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no costs to respondents other than their time.

A14. Annualized Cost to the Federal Government

Government personnel – Governmental costs for this project include personnel costs for federal staff involved in planning and designing the original National DPP assessment, the data collection spreadsheet and OMB materials, collecting and analyzing the data, and reporting, which includes approximately 15% of one GS-14/Step 2 lead public health advisor assuming $106,000 annual salary, 40% of one GS-12 evaluator/statistician assuming $85,000 annual salary, and 5% of one GS-13/Step 10 public health advisor assuming $112,000 annual salary.

Contracted assessment –The data analysis, assessment reporting, assessment technical assistance, and data collection spreadsheet refinements are being conducted under a contract with CDC’s assessment contractor. The annualized cost of the contract is estimated at $260,964 and includes costs for data management, programming, reporting, and technical assistance. The entirety of this amount is dedicated to the assessment implementation and reporting.

Table A.14-1. Estimated Annualized Cost to the Federal Government
Labor:
15% of one GS-14/Step 2 lead public health advisor time for project planning, management, OMB review, analysis of findings, and report writing	$15,900
40% of one GS-12 evaluator/statistician time for project planning, management, OMB review, coordination of data collection, data analysis, analysis of findings, and report writing and dissemination	$34,000
5% of one GS-13/Step 10 public health advisor time for project planning and coordination	$5,600
Contractor	$260,964
Total estimated cost	$316,464

A15. Explanation for Program Changes or Adjustments

This is a new data collection effort. The data collection spreadsheet was designed by CDC and refined based on grantee feedback.

A16. Plans for Tabulation and Publication and Project Time Schedule

Once CDC receives all data collection spreadsheets from grantees, the spreadsheets will be securely transferred to the CDC evaluation contractor and stored on a secure SFTP server, which is password protected and only accessible to CDC-authorized staff. Quantitative and qualitative program-level data abstracted from the grantee and intervention site Excel data collection spreadsheets will be analyzed using descriptive statistics and bi-variate correlations for grantee-specific assessment reports and the cross-site annual reports. Some correlations will utilize existing DPRP outcome data in a de-identified, aggregated manner by site. Linear regression and hierarchical models will also be used in the cross-site annual reports.

This data collection effort will result in several dissemination products including annual grantee assessment reports, annual cross-grantee assessment reports, and at least one manuscript.

CDC’s preferred timeline is outlined below. In order to complete the grantee assessment by December 29, 2015, OMB approval is requested no later than October 15, 2015 (sooner would be preferable).

Table A.16-1. Data Collection Time Schedule

Activity	Time Schedule
Distribute introductory E-mails and spreadsheet templates	October/November
Spreadsheets due	December
Data analysis	January - March
Feedback reports to respondents	April
Report on cross-site findings	September

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

The display of the OMB expiration date is not inappropriate.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

References

Ackermann, R. T. (2013). Working with the YMCA to implement the Diabetes Prevention Program. American Journal of Preventive Medicine, 44(4S4), S352–S356.

Ali MK, Echouffo-Tcheugui JB, Williamson DF. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Affairs 2012;31(1):67–75.

Albright AL, Gregg EW. Preventing type 2 diabetes in communities across the U.S. The National Diabetes Prevention Program. Am J Prev Med 2013;44(4S4):S346–S351.

Centers for Disease Control and Prevention (CDC). National diabetes statistics report: Estimates of diabetes and its burden in the United States, 2014. Atlanta, GA: U.S. Department of Health and Human Services; 2014a.

Centers for Disease Control and Prevention (CDC). National diabetes prevention program, about the program [online]. 2014b. Available at http://www.cdc.gov/diabetes/prevention/about.htm.

Crandall JP, Knowler WC, Kahn SE, Marrero D, Florez JC, Bray GA, et al. The prevention of type 2 diabetes. Nat Clin Pract Endoc 2008;4(7):382–393.

Diabetes Prevention Program Research Group. The Diabetes Prevention Program: Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. New Engl J Med 2002; 346:393–403.

Hoerger, T. J., Hicks, K. A., Sorensen, S. W., Herman, W. H., Ratner, R. E., Ackermann, R. T., et al. (2007). Cost-effectiveness of screening for pre-diabetes among overweight and obese U.S. adults. Diabetes Care, 30(11), 2874–2879. doi:10.2337/dc07-0885.

Lindstrom J, Llanne-Parikka P, Peltonen M., Aunola S, Eriksson JG, Hemiö K, et al. (2006). Sustained reduction in the incidence of type 2 diabetes by lifestyle intervention; follow-up of the Finnish Diabetes Prevention Study. Lancet 2006;368(9548):1673–1679.

Tuomilehto J, Lindstorm J, Eriksson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. New Engl J Med 2001;344:1343–1350.

Venditti, E., Kramer, K. (2012). Necessary Component for Lifestyle Modification Interventions to Reduce Diabetes Risk. Current Diabetes Reports, 12, 138-146. DOI 10.1007/s11892-012-0256-9.

Zhuo, X., Zhang, P., Gregg, E., Barker, L., Hoerger, T. J., Pearson-Clarke, T., & Albright, A. (2012, January). A nationwide community-based lifestyle program could delay or prevent type 2 diabetes cases and save $5.7 billion in 25 years. Health Affairs, 31(1), 50–60.