Matrix of Evaluation Questions and Data Sources

Att 3b. Evaluation Matrix for DP18-1805.docx

Assessment of Technical Assistance and Training Approaches to Accelerate Comprehensive Cancer Control Outcomes

Matrix of Evaluation Questions and Data Sources

OMB: 0920-1193

Document [docx]
Download: docx | pdf











Attachment 3b.

Evaluation Matrix for DP18-1805



Evaluation Matrix for DP18-1805

Evaluation Subquestions

Indicators

Data Sources

Potential Analysis

Evaluation Question 1: How are DP18-1805 TTA providers implementing the components of the NOFO to build capacity among their target audience(s)?

EQ 1A. To what extent did DP18-1805 TTA providers implement the components of the NOFO to build capacity among their target audience?

Description of activities related to the core strategies of the NOFO:

  • Conduct needs assessment

  • Surveillance data analysis and findings

  • Use a capacity-building framework

  • Collaborate with partners

  • # of partners engaged

  • Contribution of partners in TTA activities

  • Develop TTA plan using evidence-based

approaches

  • Training curriculum

  • Webinar topics

  • Product development plan

  • Implement TTA

  • # and type of TTA provided in terms of dose,

  • frequency, intensity, and mode (e.g., in person, group, one on one, virtual, website)

  • # and type of materials and/or products developed (e.g., webinars, presentations,

  • podcasts, guidance documents, fact sheets, newsletters)

  • # and type of channels used to disseminate

  • evidence-based interventions and/or

  • promising practices

  • Monitor implementation of TTA for quality improvement

  • Evaluate TTA

Development of evaluation plan

Collection of evaluation data

  • Program materials (e.g., work plans, progress reports, evaluation reports, program logic models)

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Thematic analysis

EQ 1B. What factors affect TTA providers’ implementation of DP18-1805 NOFO components?

  • Contextual factors

  • Costs/funding

  • Staffing/personnel

  • Amount of time

  • Program maturity/infrastructure

  • Staff capacity

  • Access to those with subject matter expertise

  • Partnerships

  • Organizational factors

  • History

  • Maturity

  • Mission

  • Leadership

  • Qualities inherent to the cooperative agreement

  • TA received

  • Tools/resources made available

  • Access to new partners

  • Program materials (e.g., work plans, progress reports, evaluation reports, program logic models)

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Thematic analysis

Evaluation Questions 2: To what extent did DP18-1805 TTA providers achieve planned short-term outcomes?

2A. To what extent did DP18-1805 funded organizations achieve short-term, TTA provider-specific outcomes?

  • Perceived improvements in TTA resource allocation

  • Evidence of reduced duplication of TTA efforts among TTA provider

  • Evidence of quality and accessibly of TTA materials (e.g., public education trainings, evaluation tools, and resources)

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies


  • Thematic analysis

  • Descriptive analysis

2B. To what extent did DP18-1805 funded organizations’ TTA activities contribute to TTA consumers’ achievement in short-term outcomes?

  • # of CCC programs that report the development of partnerships for cancer prevention, screening, and survivorship

  • # of CCC programs that report implementation

of evidence-based interventions

  • # of CCC programs that report use of health system; policy, systems, and environmental; and community clinical linkage strategies

  • # of CCC programs that report use of rigorous evaluation by NCCCP grantees


  • Program materials review

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Survey of NCCCP program directors, coalition members, and/or partners


  • Thematic analysis

  • Descriptive analysis

Evaluation Questions 3: To what extent did DP18-1805 funded organization TTA activities contribute to NCCCP implementation and achievement in NCCCP priorities and goals?

3A. To what extent did DP18-1805 funded organizations’ TTA activities contribute to TTA consumers’ achievement in intermediate and longer term outcomes?

  • # of CCC programs that report an improved ability to leverage resource and sustained implementation of CCC activities

  • # of CCC programs that report sustained use of evaluation for program improvement

  • # of CCC programs that continued use of evidenced-based approaches for cancer prevention and control


  • Survey of NCCCP program directors, coalition members, and/or partners



EQ 3B. How do key stakeholders from DP18-1805 perceive the effectiveness of TTA in contributing to NCCCP implementation and achievement in NCCCP goals?

  • # and type of individuals’ perceptions of the overall effectiveness of the TTA

  • # and type of individuals’ perceptions of the most effective aspects of the TTA provided

  • # and type of individuals’ perceptions of the least effective aspects of the TTA

  • # and type of individuals’ suggestions for improving the effectiveness of each TTA provider

  • Program materials review

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Survey of NCCCP program directors, coalition members, and/or partners


  • Thematic analysis

  • Descriptive analysis

Evaluation Question 4: What elements of TTA were most effective in building the capacity of NCCCP programs?

4A. Which, if any, of the TTA approaches, strategies or activities implemented under DP18-1805 contributed the most to improved NCCCP programmatic capacity and/or outcomes?

  • #, type, and content of the DP18-1805 funded organizations’ TTA approaches, strategies, and/or activities that result in the most improved outcomes

  • #, type, and content of the DP18-1805 funded organizations’ TTA approaches, strategies, and/or activities

  • #, type, and content of the DP18-1805 funded organizations’ TTA approaches, strategies, and/or activities CDC staff and national organizations

  • consider core/essential

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Survey of CCC program director, coalition chair and up to 3 partners per program


  • Thematic analysis

  • Descriptive analysis


4B. What contextual factors influenced the effectiveness of the TTA?

  • Characteristics related to the design of the DP18-1805 cooperative agreement

  • Organizational characteristics of DP18-1805 funded organizations

  • Characteristics of the NCCCP grantees

  • Characteristics of NCCCP coalition members and Partners

  • Demographic or behavioral characteristics of specific populations/audiences


  • Program materials review

  • Interviews with organizational staff that support the implementation of DP18-1805 conducted via case studies

  • Survey of NCCCP program directors, coalition members, and/or partners


  • Thematic analysis

  • Descriptive analysis




1


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorThummalapally, Sharanya
File Modified0000-00-00
File Created2021-01-14

© 2024 OMB.report | Privacy Policy