NTCP Report Template

Evaluation Reporting Template for the National and State Tobacco Control Program

Att 2 DP20-2001_NTCP_EvaluationReportTemplate_Y1_Y2_v4_20220729.xlsx

OMB: 0920-1390

Document [xlsx]

Download: xlsx | pdf

Overview

Intro
C1-Context
C1-Statewide
C1-Community
C1-PolicyHSC
C1-HSC
C2-Quitline
Appendix 1
Appendix 2

Sheet 1: Intro

Form Approved OMB No. 0920-xxx Exp. Date xx/xx/20xx	Annual Evaluation Reporting Guidance and Template for CDC-RFA-DP20-2001: National and State Tobacco Control Program
CDC estimates the average public reporting burden for this collection of information as 8 hours per response, including the time for reviewing instructions, searching existing data/information sources, gathering and maintaining the data/information needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-XXXX)


Recipient State/Territory:
Performance Year:
Funded Components:
(Component 1 and/or Component 2)



Purpose and Instructions

Purpose of this Document
This document provides guidance and the Evaluation Reporting Template for CDC-RFA-DP20-2001: National and State Tobacco Control Program. Recipients are required to submit an annual evaluation report of their funded activities each performance year, with the first report due after the end of performance year two.
Evaluation reports are an important component of demonstrating the value of comprehensive tobacco control programs. The data that CDC collects from the annual evaluation reports will be used to track progress, identify promising practices, and gather information on lessons learned in tobacco control. To aid in meeting reporting requirements, this document provides guidance for completing the reporting template.
Submission Instructions
Completed templates and any supporting materials should be uploaded to the Awardee Management Platform (AMP) annually. In AMP, navigate to the "DELIVERABLES" sub-tab; under "Other Deliverables" select "Evaluation Report - Year #" for the appropriate year (please note, the first report, due July 27, 2022, is titled "Evaluation Report - Year 2"). Completed excel templates in .xlsx format should be uploaded using the "Files" box at the bottom of the page. The state's abbreviation should be included at the start of the excel template file name (e.g. the state of Montana would rename their document as "MT_DP20-2001_NTCP_EvaluationReportTemplate_Y1_Y2_v1.xlsx"). Supporting materials can be uploaded at the discretion of the recipient and may include relevant campaign materials, publications, fact sheets, data visualizations, etc. Jurisdiction abbreviations should be included at the start of the supporting file name, followed by a brief descriptive title (e.g., "MT_Press_Release.pdf"). Check with your assigned OSH Project Officer for new due dates and any changes to the submission protocol each year.
Notes
Recipients’ Evaluation Reports should reflect each component for which they are funded. For example, recipients funded only for Component 2 should only report work related to Component 2 ("C2") and can skip any tabs labeled "C1".
Recipients must report on strategies, populations of focus, and indicators, as specified in the recipient’s evaluation plan. Any changes made to the recipient’s original evaluation plan should be approved by recipient’s assigned CDC OSH Evaluator prior to completing the Evaluation Report.
Please use the following template to enter the evaluation results for the first two budget years (combined), and each subsequent year of funding. Years one and two should be combined into a single report. Each subsequent year’s evaluation report will build on the prior year’s report. Section headers within each table (in blue on each tab) will indicate whether the section is required or optional for each reporting year.
Suggested word counts are provided as a guide; however, strict adherence is not necessary.
Your assigned CDC OSH Evaluator is available for support in answering questions and can assist with reporting your results. Please reach out to your assigned CDC OSH evaluator if you need help filling in this template or otherwise meeting reporting requirements.
Contents and Navigation

Component One Template Tabs
Tab 1. Contextual Factors........................................................................	C1-Context
Tab 2. Statewide Disparities Requirement..............................................	C1-Statewide
Tab 3. Community-Based Disparities Requirement.................................	C1-Community
Tab 4. Policy or Health Systems Change Requirement............................	C1-PolicyHSC
Tab 5. Health Systems Changes...............................................................	C1-HSC

Component Two Template Tabs
Tab 6. Quitline Requirement....................................................................	C2-Quitline

Appendices
Tab 7. Additional Resources.....................................................................	Appendix 1
Tab 8. DP20-2001 Required Evaluation Questions...................................	Appendix 2

Sheet 2: C1-Context

Component One Template - Tab 1. Contextual Factors
Contextual Factors [Required each reporting period]										Word Count
Summarize contextual factors that have affected, positively or negatively, implementation of each of the strategies being evaluated for Component One. Describe how factors related to social determinants of health (e.g. structural racism, classism, sexism, heterosexism, access to healthcare and preventative services, adequate income, secure employment, education, and good working conditions) have had an impact on the program’s work. For each subsequent reporting year, update and/or build on information provided during prior years. [Suggested word count: 500]										0

Sheet 3: C1-Statewide

Component One Template - Tab 2. Statewide Disparities Requirement

Strategies [Required each reporting period]

Select population focus:

Specify strategy evaluated:

2nd strategy evaluated, if applicable:

Findings [Required each reporting period]

Narrative

Summarize and interpret any findings that address the required and recipient-led evaluation questions. Required evaluation questions are listed in Appendix 2. Focus on outcomes that have resulted from implementation of the NOFO-funded strategies. Enter any additional information related to the evaluation that is critical to understanding key findings or other reported information. [Suggested word count: 250]

Word Count

Indicators

Provide data for all indicators specified in the recipient’s evaluation plan. Refer to pages 9-11 and Appendix C in the “Guidance for the Development and Submission of the DP20-2001 Evaluation Plan” for information about recommended indicators. In addition to the indicators specified in the recipient’s evaluation plan, all recipients are asked to report on tobacco use prevalence for their selected population groups. Use the section directly below for indicators that do not fit into this format or if you have more than ten total indicators. Each indicator should be listed in a new row.

Indicator(s)

Data Sources

Data Collection Time Frame
(mm/dd/yyyy)

Baseline
[optional]

Results Provide results in whichever fields best apply for the indicator.

Measurement Notes / Qualitative Results Note any measurement considerations, data collection methods, or analysis methods necessary for interpretation of data. Note if any relevant attachments will be submitted. Describe any changes to the indicators from the original evaluation plan or previous reporting years. [Suggested word count: 75 per indicator]

Start/T1

End/T2

Numerator or Count

Denominator
(if applicable)

Percent

Qualitative results
[Suggested word count: 75 per indicator]

Additional Indicator Information [Optional]: Use the table above for all indicators, if possible. However, if any indicators do not fit into the format above, list the indicators and use this space to present the results and any measurement notes. Each indicator should be listed in a new row. Contact your assigned OSH evaluator if you have more than five additional indicators to report. [Suggested word count: 100 per indicator]

Lessons Learned [Required each reporting period]

Report on at least one lesson learned from the performance year. We are interested in lessons learned that fall within the following categories: 1) challenges (how the program overcame challenges related to program implementation, including data collection, partnerships, policy adoption, increasing access to cessation treatment, and health systems changes); 2) promising practices (new or innovative approaches to achieving tobacco-related outcomes); or 3) value for money (e.g., findings on cost-benefit or return on investment analyses, or lessons learned about strategic efforts to keep costs down while still implementing programs with fidelity). Contact your assigned OSH evaluator if you have more than five ‘lessons learned’ to report.

Category: Select the best fit

Lesson Learned [Suggested word count: 30 per lesson]

How the lesson has informed or will inform the TCP moving forward [Suggested word count: 150 per lesson]

Health Equity and Health Disparities [Optional for Y1-2, required Y3-5]

Describe how interventions are being targeted and/or tailored to reach and engage intended and intersectional population group(s). Describe how the statewide requirement program activities have advanced health equity and reductions in commercial tobacco-related disparities. Describe the evidence of any changes in commercial tobacco related health equity or reductions in disparities. If relevant data have already been provided in the indicator section above, reference them here and interpret the results. [Suggested word count: 250]

Word Count

Use of Evaluation Data [Optional]

Describe how and when early evaluation findings and lessons learned will be used to inform work for subsequent NOFO years. What specific actions have already been taken to use evaluation findings to inform program improvements? What specific actions will be taken during the next year? [Suggested word count: 250]

Word Count

Sheet 4: C1-Community

Component One Template - Tab 3. Community-Based Disparities Requirement

Strategies [Optional for Y1-2, required Y3-5]

Select population focus:

Specify strategy evaluated:

2nd strategy evaluated, if applicable:

Findings [Optional for Y1-2, required Y3-5]

Narrative

Word Count

Indicators

Indicator(s)

Data Sources

Data Collection Time Frame
(mm/dd/yyyy)

Baseline
[optional]

Results Provide results in whichever fields best apply for the indicator.

Start/T1

End/T2

Numerator or Count

Denominator
(if applicable)

Percent

Qualitative results
[Suggested word count: 75 per indicator]

Lessons Learned [Optional for Y1-2, required Y3-5]

Category: Select the best fit

Lesson Learned [Suggested word count: 30 per lesson]

How the lesson has informed or will inform the TCP moving forward [Suggested word count: 150 per lesson]

Health Equity and Health Disparities [Optional for Y1-2, required Y3-5]

Word Count

Use of Evaluation Data [Optional]

Word Count

Sheet 5: C1-PolicyHSC

Component One Template - Tab 4. Policy or Health Systems Change Requirement

Strategies [Required each reporting period]

Select population focus:

Specify strategy evaluated:

Check box if the strategy is part of the recipient’s DP20-2001 workplan E-cigarette Requirement

2nd strategy evaluated, if applicable:

Findings [Required each reporting period]

Narrative

Word Count

Unintended Consequences

To what extent were there unintended consequences (e.g., exacerbating disparities or disproportionately benefiting population groups, illicit sales of tobacco products)? [Suggested word count: 150]

Word Count

Indicators

Indicator(s)

Data Sources

Data Collection Time Frame
(mm/dd/yyyy)

Baseline
[optional]

Results Provide results in whichever fields best apply for the indicator.

Start/T1

End/T2

Numerator or Count

Denominator
(if applicable)

Percent

Qualitative results
[Suggested word count: 75 per indicator]

Lessons Learned [Optional]

Category: Select the best fit

Lesson Learned [Suggested word count: 30 per lesson]

How the lesson has informed or will inform the TCP moving forward [Suggested word count: 150 per lesson]

Health Equity and Health Disparities [Optional for Y1-2, required Y3-5]

Word Count

Use of Evaluation Data [Optional]

Word Count

Sheet 6: C1-HSC

Component One Template - Tab 5. Health Systems Changes
Change Concepts and Ideas [Required each reporting period]
Report on each ‘change idea’ promoted or implemented as part of your DP20-2001 workplan. For each selected change idea indicate the number of healthcare facilities that have implemented the change during the past performance year. Recipients should only report on change ideas they worked on. Recipients can reference the full list of change concepts and change ideas (pages 1-2) from the Tobacco Cessation Change Package.	Number of healthcare facilities:
	Number of healthcare facilities:
	Standalone Mental Health and/or Substance Use Facilities	Federally Qualified Health Center	IHS, UHI*, Tribal Health Center	Other facilities or large health systems***
IHS: Indian Health Services. UIH: Urban Indian Health. ** The “Other” column should include any health systems that do not clearly fit into another category or any large health systems that encapsulate multiple categories. Describe these facilities or large health systems in the text box below each section. For large health systems that span categories, please note if changes were implemented system-wide or within a specific setting in the large health system. Each column should be mutually exclusive, (i.e., a single health system should not be counted in more than one column).	Standalone Mental Health and/or Substance Use Facilities	Federally Qualified Health Center	IHS, UHI*, Tribal Health Center	Other facilities or large health systems***
Key Foundations
Make Tobacco Cessation a Practice and System Priority
Identify one or two key champions and assemble a multidisciplinary team
As a multidisciplinary group, conduct an assessment of your clinic/system and develop an action plan to address the current gaps
Adopt or update a unit, practice, or system-wide policy to reflect prioritization of tobacco treatment
Create a Supportive Environment for Cessation
Implement and strengthen hospital or clinic-wide tobacco-free or smoke-free campus policies
Leverage mass-reach media campaigns to encourage and normalize quitting (e.g., with media in waiting rooms, throughout clinic or hospital system)
Support employees and their family members in quitting smoking by providing health benefits for tobacco cessation
***If applicable, describe “other” healthcare facilities here:
Equipping Care Teams
Equip All Staff to Engage in Tobacco Cessation Efforts
Adopt a clinician/staff training policy to train and retrain staff
Conduct onboarding and annual trainings on tobacco policies, systems, and procedures
Optimize billing practices by leveraging existing codes to capture all billable services
Provide Clinician and System-Level Feedback on Progress and Impact
Set and communicate specific, measurable performance and quality goals
Make tobacco cessation a quality improvement measure at the clinician or system level
Track clinician, clinic, hospital, and system performance to provide feedback to clinicians and decision-makers
***If applicable, describe “other” healthcare facilities here:
Screening
Make Tobacco Use a Vital Sign: Screen Every Patient for Tobacco Use at Every Visit
Adopt a tobacco use screening protocol
Establish a workflow and determine roles for tobacco use screening and documentation
Embed a tobacco use status prompt in the EHR or other patient record-keeping system
Embed decision support scripts for screening into the EHR or other patient record-keeping system
***If applicable, describe “other” healthcare facilities here:
Treatment
Establish a Tobacco Treatment Protocol
Implement a treatment intervention
Establish a workflow to determine roles for delivering the treatment intervention
Enhance Clinical Decision Support
Embed treatment intervention prompts into the EHR or other patient record-keeping system
Embed decision support scripts for each intervention step into the EHR or other patient record-keeping system
Implement standard order sets for counseling and medication
Adopt tools to guide medication selection
Implement Standardized Approaches to Support Cessation Efforts
Deliver standard patient education regarding cessation medication
Adopt a clinical decision support tool for helping patients plan their quit attempt
***If applicable, describe “other” healthcare facilities here:
Referral and Follow-Up
Establish Protocols to Identify and Connect Patients to Referral Resources
Implement a protocol or workflow to ensure clinician follow-up with patients
Identify and partner with referral services that can serve as an adjunct to care
Set up direct referrals to internal and external resources by creating standard referral orders
Establish two-way communication with referral services to get information on whether referrals were accepted by the patient
Employ Population Management Strategies to Better Identify and Reach Patients Who Use Tobacco
Use tobacco registries or other methods to track patients who use tobacco
Conduct proactive follow-up with patients who use tobacco
***If applicable, describe “other” healthcare facilities here:

Sheet 7: C2-Quitline

Component Two Template - Tab 6. Quitline Requirement

Strategies [Required each reporting period]

Specify strategy evaluated:

2nd strategy evaluated, if applicable:

Contextual Factors [Required each reporting period]

Describe contextual factors that have affected, positively or negatively, implementation of the Quitline requirement during this performance year. For each subsequent reporting year, update and/or build on information provided during prior years. [Suggested word count: 100]

Word Count

Findings [Required each reporting period]

Narrative

Summarize and interpret findings to-date that address the required Quitline Evaluation Questions. Required evaluation questions are listed in Appendix 2. Address which services or combination of services resulted in increased quit attempts and sustained quits and for whom. Focus on outcomes that have resulted from implementation of the NOFO-funded strategies. Enter any additional information related to the evaluation that is critical to understanding key findings or other reported information. [Suggested word count: 250]

Word Count

7 Month Follow Up Data

Provide data in the table below. Recipients are required to collect and report 7-month follow-up data among tobacco users who used Quitline services at least one time during the NOFO performance period. Recipients that routinely collect 7-month follow-up data should report the data as part of their annual evaluation reporting, as available.

Sample Size for Follow-Up (N):

Number of Respondents

Response Rate (%):

Data Collection Timeframe
(mm/yyyy-mm/yyyy)

Among unique tobacco users who received at least one Quitline service, what was the NAQC Standard Quit Rate* (using thirty-day point prevalence abstinence**) at 7 month follow up, overall and among populations experiencing disparities? Provide data in the table below. Optionally, select 1-2 additional service modalities that correspond to services offered through the recipient’s Quitline.

Overall (N, %)
[optional]

Among population experiencing disparity (specify population) (N, %)
[required]

Among population experiencing disparity (specify population) (N, %)
[optional]

*See: https://www.naquitline.org/page/ImpQR#worksheet

**Thirty-day point prevalence abstinence is measured with the Minimal Data Set (MDS) follow-up question: "Have you used any [INSERT TOBACCO TYPE], even a puff or pinch, in the last 30 days?”.

[Required] Among unique tobacco users who received at least one Quitline service

Indicators

Provide data for all indicators specified in the recipient’s evaluation plan. Refer to pages 9-11 and Appendix C in the “Guidance for the Development and Submission of the DP20-2001 Evaluation Plan” for information about recommended indicators. Use the section directly below for indicators that do not fit into this format or if you have more than ten total indicators. Each indicator should be listed in a new row.

Indicator(s)

Data Sources

Data Collection Time Frame
(mm/dd/yyyy)

Baseline
[optional]

Results Provide results in whichever fields best apply for the indicator.

Start/T1

End/T2

Numerator or Count

Denominator
[if applicable]

Percent

Qualitative results
[Suggested word count: 75 per indicator]

Lessons Learned [Required each reporting period]

Category: Select the best fit

Lesson Learned [Suggested word count: 30 per lesson]

How the lesson has informed or will inform the TCP moving forward [Suggested word count: 150 per lesson]

Health Equity and Health Disparities [Optional for Y1-2, required Y3-5]

Describe how interventions are being targeted and/or tailored to reach and engage intended and intersectional population group(s). Describe how the Quitline activities have advanced health equity and reductions in commercial tobacco-related disparities. Describe the evidence of any changes in commercial tobacco related health equity or reductions in disparities. If relevant data have already been provided in the indicator section above, reference them here and interpret the results. [Suggested word count: 250]

Word Count

Use of Evaluation Data [Optional each reporting period]

Word Count

Sheet 8: Appendix 1

Appendix 1. Additional Resources

Conducting Quitline Evaluations – This workbook is a guide to facilitate the evaluation of quitlines and is intended for use by quitline staff and stakeholders, state tobacco control program managers, and evaluators.

Developing an Effective Evaluation Plan – This workbook was developed to help public health program managers, administrators, and evaluators develop an effective evaluation plan. It is intended to assist in developing an evaluation plan and can be used along with other evaluation resources.

Developing an Effective Evaluation Report – A workbook that applies the CDC Framework for Program Evaluation in Public Health to construct a final evaluation report by laying out a six-step process for the decisions and activities involved in planning and conducting an evaluation.

DP20-2001 Notice of Funding Opportunity – This document provides details about the DP20-2001 funding opportunity.

DP20-2001 Performance Measure Guidance – This document provides performance measure definitions, data collection guidance, and example data sources.

Eliminating Exposure to Secondhand Smoke: Outcome Indicators for Comprehensive Tobacco Control Programs – This publication is the third in a series of updates to the 2005 publication Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs. This update provides a revised logic model and set of outcome indicators for Goal Area 2 of the National Tobacco Control Program - Eliminating Exposure to Secondhand Smoke.

Evaluators’ Network – Built for the surveillance and evaluation staff of Tobacco Control Programs across the country, the Network provides an opportunity for peer engagement and provides evaluation support staff and contractors working on tobacco control programs. Check the networking site to find DP20-2001 evaluation support, dues dates, and learning opportunities. If you are not already a member of the Evaluators’ Network, contact ntcpevaluators@cdc.gov

Guidance for the Development and Submission of the DP20-2001 Evaluation Plan – This document describes the requirements for the DP20-2001 evaluation plan and outlines reporting expectations for each reporting year.

Identifying and Eliminating Tobacco-Related Disparities: Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs – This guide supports and complements broader monitoring and evaluation efforts to identify and eliminate tobacco-related disparities. Tobacco prevention and control program managers and evaluators can use the indicators in this document to focus their evaluations, inform the selection of indicators, link these to intended outcomes, and assist in gathering credible evidence.

Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide – This document is a “how to” guide for planning and implementing evaluation activities. The manual, based on CDC’s Framework for Program Evaluation in Public Health, is intended to assist managers and staff of public, private, and community public health programs to plan, design, implement, and use comprehensive evaluations in a practical way.

Practical Strategies for Culturally Competent Evaluation – CDC’s Division for Heart Disease and Stroke Prevention and the National Asthma Control Program developed this guide for program staff and evaluators to promote cultural competence. The guide includes strategies to make sure evaluation efforts have cultural relevance and generate meaningful findings

Preventing Initiation of Tobacco Use: Outcome Indicators for Comprehensive Tobacco Control Programs—2014 - This publication is the first in a series of updates to the 2005 publication Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs. This update provides a revised logic model and set of outcome indicators for Goal Area 1 of the National Tobacco Control Program.

Promoting Quitting Among Adults and Young People: Outcome Indicators for Comprehensive Tobacco Control Programs-2015 - This publication is the second in a series of updates to the 2005 publication Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs. This update provides a revised logic model and set of outcome indicators for Goal Area 3 of the National Tobacco Control Program, which addresses promoting quitting among adults and young people.

Surveillance and Evaluation Data Resources for Comprehensive Tobacco Control Programs - Surveillance and Evaluation Data Resources is an at-a-glance compilation of surveillance and evaluation resources for comprehensive tobacco control programs.

Tobacco Cessation Change Package – This document contains a list of change concepts and change ideas (pages 1-2) that are relevant to specific health systems changes that recipients may be evaluating.

Sheet 9: Appendix 2

Appendix 2. DP20-2001 Required Evaluation Questions

This list of evaluation questions was originally published in the document: Guidance for the Development and Submission of the DP20-2001 Evaluation Plan
Evaluation Focus Areas	Evaluation Questions
Statewide Disparities Requirement (behavioral health or low income) AND Community-based Disparities Requirement (selected population group)	Effectiveness: What evidence-based strategies, promising practices, and/or culturally tailored interventions were effective (and not effective) at reaching and improving positive tobacco-related outcomes among the selected populations affected by tobacco-related disparities? What were lessons learned, promising practices, and unintended consequences? Outcomes: To what extent did recipient efforts improve tobacco-related outcomes, such as increased protection and reduced exposure from secondhand smoke, increased use of evidence-based cessation treatment, increased quit attempts and sustained quits, and reduced tobacco use and dependence among the selected populations affected by tobacco-related disparities?


Policy or Health Systems Change Evaluation Requirement*	Outcomes: What impact did the policy or systems change have on achieving tobacco-related outcomes for one of the following: 1) Decreasing access and tobacco use among youth, including e-cigarette use, 2) Increasing protection and reducing exposure to secondhand smoke, or 3) Promoting cessation, such as increasing use of evidence-based cessation treatment and increasing quit attempts and sustained quits? Intended and unintended consequences: What effect did the policy or health systems change have overall, and as appropriate among population experiencing tobacco-related disparities? To what extent were there unintended consequences (e.g., exacerbating disparities or disproportionately benefiting population groups, illicit sales of tobacco products)?

Quitline Requirement	Effectiveness: What services and modalities and/or combination of services resulted in increased quit attempts and sustained quits at 7-month follow-up? For whom? Outcomes: To what extent did recipient efforts contribute to a measurable change in quit attempts and sustained quits at 7-month follow-up, overall, and among populations experiencing tobacco-related disparities?


*For the policy and health systems change evaluation requirement, recipients are encouraged, but not required, to evaluate a policy or systems change that was implemented as part of the E-cigarette Youth and Young Adult NOFO Requirement.

File Type	application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
File Modified	0000-00-00
File Created	0000-00-00