SSA_9084Delta_Clean_11-22-2016

SUPPORTING STATEMENT: PART A

OMB# 0920-0984

DELTA FOCUS Program Evaluation

May 11, 2016

Contact: Megan Kearns, PhD

Telephone: 770-488-1230

E-mail: wti8@cdc.gov

National Center for Injury Prevention and Control

Division of Violence Prevention

Centers for Disease Prevention and Control

Atlanta, Georgia

CONTENTS

Section Page

A. Summary table

Justification

A.1. Circumstances Making the Collection of Information Necessary

A.2. Purpose and Use of Information Collection

A.3. Use of Improved Information Technology and Burden Reduction

A.4. Efforts to Identify Duplication and Use of Similar Information

A.5. Impact on Small Businesses or Other Small Entities

A.6. Consequences of Collecting the Information Less Frequently

A.7. Special Circumstances Relating to the Guidelines of

5 CFR 1320.5(d)2

A.8. Comments in Response to the Federal Register Notice and

Efforts to Consult Outside the Agency

A.9. Explanation of Any Payment or Gift to Respondents

A.10. Protection of the Privacy and Confidentiality of Information

Provided by Respondents

A.11. Institutional Review Board (IRB) and Justification for Sensitive

Questions

A.12. Estimates of Annualized Burden Hours and Costs

A.13. Estimates of Other Total Annual Cost Burden to Respondents

or Record Keepers

A.14. Annualized Cost to the Government

A.15. Explanation for Program Changes or Adjustments

A.16. Plans for Tabulation and Publication and Project Time Schedule

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

A.18. Exceptions to Certification for Paperwork Reducation Act

Submissions

Attachments

A Authorizing Legislation: Family Violence and Prevention Services Act

B Authorizing Legislation: Domestic Violence Prevention Enhancement and Leadership Through Alliances (DELTA)

C1 Published 60-Day Federal Register Notice

C2 Public Comment

D Institutional Review Board (IRB) Determination Form

E Instrument - DELTA FOCUS Program Evaluation Survey

F List of New Survey Items

SUMMARY TABLE

• Goal of the study: Information will be collected from awardees funded under FOA-CE13-1302, the DELTA FOCUS (Domestic Violence Prevention Enhancement and Leadership Through Alliances, Focusing on Outcomes for Communities United with States) cooperative agreement program, to assess: a) satisfaction with each bi-directional communication and support channel for the DELTA FOCUS project, b) program and strategy implementation factors that affect awardees’ ability to meet the requirements of the cooperative agreement, c) awardee prevention knowledge and use of the public health approach, and d) sustainability of awardee prevention activities and successes. This collection will assess information about the overall program and its implementation and does not focus on program monitoring.

• Intended use of the resulting data: Information collected will be used to improve the national DELTA FOCUS program, program implementation at local and state levels, and provide CDC with information to respond to requests from within CDC, HHS, White House, Congress, and others about the program.

• Methods to be used to collect: Information will be collected electronically from awardees funded under FOA-CE13-1302 using a web-based survey, the DF Survey.
  
  

• The subpopulation to be studied: 100% of population (DELTA FOCUS awardees, sub-recipients, and empowerment evaluators), no sampling.
  
  

• How data will be analyzed: The data will be analyzed using descriptive and summary statistics.

A. JUSTIFICATION

CDC is requesting a Reinstatement with Change for three (3) years for the previously approved Information Collection Request OMB# 0920-0984 (Discontinuation date: Sept/2014).

Information will be collected electronically from awardees funded under FOA-CE13-1302, the DELTA FOCUS (Domestic Violence Prevention Enhancement and Leadership Through Alliances, Focusing on Outcomes for Communities United with States) cooperative agreement program. The DELTA FOCUS program is an initiative authorized by 42 U.S.C. 10418; and Sections 317(k)(2) and 393 of the Public Health Service Act (42 U.S.C. Sections 247b(k)(2) and 280b-1a, as amended (see Attachment A and B).

Minor changes have been made to the discontinued survey to clarify the intent of certain items and specify the timeframe respondents should consider when answering survey items. A limited number of new items have also been added to the survey (see Attachment F for new items). These changes will improve the quality of the data collected. In addition, the number of respondents has increased to allow all coalition staff affiliated with the DELTA FOCUS project to participate in the data collection. This will provide a more accurate and comprehensive perspective on areas in need of program improvement for the DELTA FOCUS project.

A.1. Circumstances Making the Collection of Information Necessary

Background

Intimate Partner Violence (IPV) is a serious, preventable public health problem that affects millions of Americans and results in serious consequences for victims, families, and communities. IPV occurs between two people in a close relationship. The term “intimate partner” describes physical, sexual, or psychological harm by a current or former partner or spouse. IPV can impact health in many ways, including long-term health problems, emotional impacts, and links to negative health behaviors. IPV exists along a continuum from a single episode of violence to ongoing battering; many victims do not report IPV to police, friends, or family.

Primary prevention means stopping IPV before it occurs. In 2002, authorized by the Family Violence Prevention Services Act (FVPSA), CDC developed the Domestic Violence Prevention Enhancements and Leadership Through Alliances (DELTA) Program, with a focus on the primary prevention of IPV. Since that time, DELTA has funded state domestic violence coalitions (SDVCs) to engage in statewide primary prevention efforts and to provide training, technical assistance, and financial support to local communities for local primary prevention efforts. DELTA FOCUS builds on that history by providing focused funding to states and communities for intensive implementation and evaluation of IPV primary prevention strategies that address the structural determinants of health at the societal and community levels of the social-ecological model (SEM).

The purpose of the DELTA FOCUS program is to promote the prevention of IPV through the implementation and evaluation of strategies that create a foundation for the development of practice-based evidence. By emphasizing primary prevention, this program will support comprehensive and coordinated approaches to IPV prevention. On March 2, 2013, CDC awarded 10 cooperative agreements to state domestic violence coalitions (SDVCs).

Each SDVC is required to identify and fund one to two well-organized, broad-based, active local organizations (referred to as coordinated community responses or CCRs) that are already engaging in, or are at capacity to engage in, IPV primary prevention strategies affecting the structural determinants of health at the societal and/or community levels of the SEM. SDVCs must facilitate and support local-level implementation and hire empowerment evaluators (EEs) to support the evaluation of IPV prevention strategies by the CCRs. SDVCs must also implement and with their empowerment evaluators, evaluate state-level IPV prevention strategies.

Additionally the cooperative agreement requires SDVCs to:

1. integrate, institutionalize, and sustain prevention principles, concepts, and practices within the grantee organization beyond DELTA FOCUS-funded personnel,

2. work collaboratively with other DELTA FOCUS SDVC awardees and CDC and/or CDC designees to create and sustain national-level dialogue on IPV prevention that emphasizes the importance of primary prevention, and

3. participate in and facilitate CCRs’ participation in the program-wide collaborative learning environment facilitated by CDC and/or CDC designees to support networking and learning opportunities.

In support of the SDVCs, CDC will provide technical assistance (TA) and subject matter expertise to awardees, in order to support a) awardees’ statewide efforts, b) awardees’ provision of TA, training and monitoring to the CCRs on local efforts, and c) awardees’ evaluation efforts. CDC will also collaborate with SDVCs on facilitating and sustaining a national dialogue to promote IPV prevention, supporting a collaborative learning environment among awardees, and developing a centralized evaluation of the national DELTA FOCUS program that is aligned with awardee evaluation plans.

The DF Survey will assess: a) satisfaction with each bi-directional communication and support channel (CDC-SDVC, SDVC-CCR, & SDVC-EE), b) program and strategy implementation factors that affect their ability to meet the requirements of the cooperative agreement, c) awardee prevention knowledge and use of the public health approach, and d) sustainability of awardee prevention activities and successes. CDC plans to administer the DF Survey immediately upon receipt of OMB approval.

A.2. Purpose and Use of the Information Collection

The proposed information collection will use a web-based survey to collect program evaluation data. Each respondent will be sent a direct link where they can fill out the survey. The information collection will be used to:

1. improve the national DELTA FOCUS program,

2. improve program implementation at local and state levels,

3. provide CDC with information to respond to requests from within CDC, HHS, White House, Congress, and other sources about the program, and

4. disseminate findings and lessons learned to the prevention field.

The information collection is designed to address specific objectives outlined in the DELTA FOCUS cooperative agreement. Thus, results of this program evaluation are not generalizable in the statistical sense. Instead, evaluation results can be used to modify existing practices when areas for improvement are discovered. Specifically, the information will be used to guide program improvements by CDC in the national DELTA FOCUS program implementation and program improvements by SDVCs in implementation of the program within their state. Not collecting this data could result in inappropriate implementation, resulting in ineffective use of tax payer resources. Thus, this data collection is an essential program evaluation activity and the results will not be generalizable to the universe of study.

The practical utility of this evaluation to the federal government is to assess opportunities and barriers to implementing the DELTA FOCUS program at the state and local levels, benefits and challenges of focusing on prevention strategies at the societal and community levels, and what data informed program improvements are needed. Thus, the information will allow CDC to provide more extensive oversight of the use of federal funds and to support the National Center for Injury Prevention and Control’s mission to reduce the burden of injury and violence (caused by intimate partner violence).

DELTA FOCUS awardees, SDVCs, can use the information collected to improve their implementations of prevention programs and strategies at the state level and to improve CCRs implementations at the local level. Like CDC’s efforts at continuous quality improvement on the national level, SDVCs are required in the cooperative agreement to engage in their own continuous quality improvement. The evaluation contractor will present the results of the survey to SDVCs and they can make data-informed corrections as needed. In this way the use of federal funds can be maximized.

CDC will also use the results of the information collection to generate reports, peer-reviewed publications, and translation materials in order to describe program performance, opportunities, and successes for the prevention field, CDC, and federal partners. Specifically the reports can be used to respond to inquiries from the HHS, the White House, Congress and other stakeholder inquiries about the DELTA FOCUS program at national, state, and local levels. The reports may influence future cooperative agreements in terms of supports needed for implementation and evaluation, level of funding; and may identify future data collection needs.

The DF Survey was previously implemented in year one of the DELTA FOCUS project. Data collected was useful in identifying areas requiring additional technical assistance from CDC. Data collected during the next years will allow for improved understanding of current barriers and opportunities the grantees have experienced during their implementation of DELTA FOCUS activities and what data-informed program improvement is needed. It will also help answer the larger question of how well the national DELTA FOCUS project is being implemented.

A.3. Use of Improved Information Technology and Burden Reduction

The DELTA FOCUS Survey will be a web-based questionnaire via Survey Monkey (Attachment E), taking advantage of electronic technology to improve information quality and reduce burden. 100% of responses involve electronic data collection to ensure better data quality due to reductions in data entry errors, as well as greater efficiency in administration.

A.4. Efforts to Identify Duplication and Use of Similar Information

The DELTA FOCUS program is supported by two different data collection processes, including the current DF Survey and a separate electronic Program Management Information System (PMIS; Monitoring and Reporting System for DELTA FOCUS awardees, OMB ID# 0920-0968). The DF survey differs from the PMIS because it will assess information about the overall program and its implementation and does not focus on program monitoring of ongoing activities to meet specific annual goals. The PMIS is utilized by grantees to report their progress toward cooperative agreement objectives for the purpose of generating their interim and annual progress reports. The DELTA FOCUS Survey information collection occurs twice in five years and is thus less frequent than the PMIS, which collects data twice each year for five years.

Since CDC is the only federal agency providing funding for state domestic violence coalitions to do prevention work by emphasizing prevention of intimate partner violence before it occurs, the information collected from DELTA FOCUS awardees is not available from other sources. The U.S. Department of Justice Office of Violence Against Women (OVW) does make funds available to territorial domestic and sexual violence coalitions primarily for providing services to victims of sexual and intimate partner violence. OVW also funds state coalitions with a focus on sexual violence to provide victim services and collaborate with federal, state, and local entities engaged in violence against women activities. DELTA FOCUS funding, directed to prevention activities that aim to have a population impact, does not duplicate the OVW funding focused on service provision for individuals.

A.5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A.6. Consequences of Collecting the Information Less Frequently

Data collection using the DF Survey occurred in the first year of the project period (2013) of the DELTA FOCUS program. It is currently scheduled to reoccur once a year for the project. The frequency of this data collection is needed to inform program improvement and capture changes that result from the program implementation. To collect this data less frequently or not conduct this evaluation at all would result in CDC failing to effectively demonstrate improvements in the program or adequately account for federal dollars spent on this public program.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A.8.a) A 60-day Federal Register Notice was published in the Federal Register on March 7, 2016 Vol. 81, No. 44, pp. 11800 (Attachment C1). CDC received one anonymous non-substantive comment (Attachment C2). Follow up information was not provided, so there was no reply from CDC to the non-substantive comment.

A.8.b) The data collection instruments were designed by CDC, there were no external consultations. Seven representatives of the DELTA FOCUS awardees reviewed the survey prior to the initial data collection in 2013 (1 SDVC project coordinator, 1 SDVC executive director, 3 CCR representatives, and 2 empowerment evaluators).

This information collection request is associated with the DELTA FOCUS cooperative agreement therefore the following program staff were actively involved in the original conceptualization and development of the DF Survey:

• Theresa Armstead, Behavioral Scientist, science lead, (770) 488-3904 tarmstead@cdc.gov

• Kirsten Rambo, Senior Service Fellow, project lead, (770) 488-0544 krambo@cdc.gov

• Jennifer Dills, Health Scientist, project officer, (770) 488-4273 jdills1@cdc.gov

• Pam Brown, Associate Service Fellow, project officer, (770) 488-1345 pbrown8@cdc.gov

• Rosalyn Lee, Behavioral Scientist, science officer, (770) 488-1530, rdl3@cdc.gov

The following members of the Research and Evaluation branch were consulted:

• Linda Anne Valle, Lead Behavioral Scientist, (770) 488- 4297 adv2@cdc.gov

• Kendell Cephas Childers, Public Health Program Specialist, (770) 488-1463 koc9@cdc.gov

• Greta Massetti, Lead Behavior Scientist, (770) 488-3943 ghz6@cdc.gov

A CDC staff person outside of the division of violence prevention was consulted:

• Chris Jones, Health Scientist, (770) 488-4993, vey2@cdc.gov

A.9. Explanation of Any Payment or Gift to Respondents

Respondents will not receive payments or gifts for providing information.

A.10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

The CDC Office of the Chief Information Officer has determined that the Privacy Act does not apply to this information collection request. No sensitive information or personal contact information (e.g. names, addresses, or phone numbers) will be collected. Only names of the organizations for whom the DELTA FOCUS awardees provide sub-awards will be collected.

No system of records will be created under the Privacy Act. The information collection does not require consent from individuals. While consent is not required to report aggregate data, awardee approval will be obtained if specific data is used for publications, reports, or other publicly disseminated information. Submission and access to data will be controlled by a password-protected login to the secure site. Because data are maintained in a secure, password protected system, and information will be reported in aggregate form, there is no impact on respondent privacy.

A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

The data collection is under a non-research contract and it has been determined by the National Center for Injury Prevention and Control’s IRB coordinator that IRB approval is not required. Please see the attached Research determination form (Attachment D).

Sensitive Questions

The DF Survey does not collect sensitive information or personal contact information (e.g. names, addresses, or phone numbers) will be collected. Only names of the organizations for whom the DELTA FOCUS awardees provide sub-awards will be collected (i.e. survey questions 1 and 2). The recipient will only provide information about implementation activities under the cooperative agreement. While the information collected will not be used as a means of reducing or canceling funding, awardees might view the information as sensitive. For example the coalition might fear repercussions if information entered is not perceived to favorably represent the coalition.

A.12. Estimates of Annualized Burden Hours and Costs

A) This is a request for a three-year approval. Table A-12-A details the annualized number of recipients, the average response burden per survey, and the total response burden for the DF Survey (Attachment E). Estimates of burden for the survey are based on previous experience with evaluation data collections by the evaluation staff and review by representatives of the DELTA FOCUS awardees, as well as a previous data collection using this survey in year one of the DELTA FOCUS project period. The DF Survey will be completed by 10 SDVC executive directors, 20 SDVC project coordinators, 19 SDVC-funded CCR project coordinators (9 SDVCs fund 2 CCRs and 1 SDVC funds 1 CCR), and 10 SDVC empowerment evaluators. The DF survey will take an average of 1 hour to complete for each respondent. The total estimated annualized burden is 59 hours.

Table A.12-A. Estimated Annualized Burden to Respondents

Type of respondents	Form Name	Number of respondents	Number of responses per respondent	Average burden per response (in hours)	Total burden (in hours)
DELTA FOCUS Awardees (SDVC executive directors, SDVC project coordinators, SDVC empowerment evaluators, and SDVC-funded CCR project coordinators)	DELTA FOCUS Survey - Att. E	59	1	1	59
Total					59

B) The average hourly wage for an executive director is $54.00. The average hourly rate for the SDVC project coordinator/prevention manager is $26.00 and for the CCR project coordinator is $23.00. The empowerment evaluator average hourly rate is $40.00. The hourly wage rates for the executive director, project coordinator/prevention manager, and empowerment evaluator are based on the approved budgets of the DELTA FOCUS awardees. The total estimated annualized cost to respondents is $1,897, as summarized in Table A.12-B.

Table A.12-B. Estimated Annualized Cost to Respondents

Type of respondents		Number of respondents	Average burden per response (in hours)	Average Hourly Wage Rate	Total Respondent cost
State Domestic Violence Coalition Executive Director		10	1	$54.00	$540
State Domestic Violence Coalition Project Coordinator		20	1	$26.00	$520
Coordinated Community Response Project Coordinator		19	1	$23.00	$437
State Domestic Violence Coalition Empowerment Evaluator		10	1	$40.00	$400
	Total				$1,897

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Respondents will incur no capital or maintenance costs.

A.14. Annualized Cost to the Federal Government

Two types of government costs will be incurred: (1) government personnel, and (2) contracted evaluation services. Government personnel include up to two behavioral scientist at 60% time each and one project officer at 30%. GS-13 Step 1 will be used to average the cost for the behavioral scientist and GS-12 Step 1 will be used for the project officer. CDC has submitted a procurement request for an annually severable evaluation contract over five years with an annual cost of $352,598.32 in the fourth year. This data collection accounts for approximately 31% of the evaluation annually. The average annualized cost to the federal government is $237,549.48, as summarized in Table A.14-A.

Table A.14-A. Annualized Cost to the Federal Government
Cost Category	Total
CDC Personnel 60% GS-13@$88,305/year x 2= $105,966 30% GS-12@$74,260/year= $22,278 Subtotal, CDC Personnel	$128,244
Evaluation Contractor 31% of $352,598.32= $109,305.48	$109,305.48
Total	$237,549.48

A.15. Explanation for Program Changes or Adjustments

In the current request for reinstatement, the number of respondents was increased from 49 to 59. This was to account for the fact that a majority of state domestic violence coalitions have more than one project coordinator and it is necessary to collect data from all coalition staff that are affiliated with the DELTA FOCUS project in order to adequately inform areas for program improvement.

Some minor revisions have also been made to the survey, including language to specify the timeframe for respondents (e.g., since the end of year 1 of the DELTA FOCUS project), and some changes to the response options (e.g., reducing the number of response options, modifying the language to clarify the intent of the question). A limited number of items have also been added to the survey to improve evaluation of the DELTA FOCUS program implementation (Attachment F). These revisions do not change the intent or purpose of the data collection and are not expected to impact the burden to participants.

A.16. Plans for Tabulation and Publication and Project Time Schedule

The evaluation contractor is required to propose analytic strategies for data analysis (e.g. frequency analysis, trend analysis, crosstab analysis, and limited qualitative analysis). Most statistical analyses will be descriptive. CDC will have final approval of proposed analytic strategies. Statistical modeling may be included to examine predictors of specified outcomes. All information will be aggregated in evaluation reports.

Project Time Schedule table

Activity Time Schedule
The evaluation contractor will submit to CDC a proposed survey administration schedule for CDC approval	November 4, 2016
Survey administered	Immediately upon OMB approval and no later than February 2017

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is not inappropriate.

A.18. Exceptions to Certification for Paperwork Reduction Act

There are no exceptions to the certification.