Revised Proposed Outcomes, Measures and Data Sources

Attachment B proposed outcomes and measures_5 27 2015.docx

WISEWOMAN National Program Evaluation: Implementation Assessment

Revised Proposed Outcomes, Measures and Data Sources

OMB: 0920-1068

Document [docx]
Download: docx | pdf


ATTACHMENT b

Proposed outcomes, measures and data sources


This page left blank for double-sided copying.


B.1. Proposed outcomes, measures, and data sources

Proposed Outcomes

Proposed Measures

Proposed Data Source(s)



New data collection

Existing data

I.1. What systems and external factors are in place in communities that could help or hinder grantees in improving outcomes? (Environmental Scan: Program Year 1 and ongoing)


  • Partnerships/contracts with LSPs, by type

  • Partnerships for physical activity and healthy food

  • Number and types of LSP partners

None

  • MDEs

  • Grantee applications

  • Number and types of non-partner LSPs available in the community

  • Network survey

  • Site visits

  • Community scan

  • Community-based resources available/referred to

  • Partnerships for smoking cessation

  • Types of referrals made to community-based resources


  • Program survey

  • Network survey

  • Site visits

  • Grantee applications

  • Community scan

  • MDEs

  • Chronic disease programs in the community and collaborated with

  • Types of chronic disease programs in the community (available and collaborated with)

  • Program survey

  • Network survey

  • Site visits

  • Grantee applications

  • Community scan

  • MDEs

  • Partnerships to increase access to other resources/services that support healthy behaviors

  • Types of other resources/services in the community (available and collaborated with)

  • Program survey

  • Network survey

  • Site visits

  • Grantee applications

  • Community scan

  • Resources shared

  • Types of resources shared with the grantee by other organizations, agencies, or programs

  • Program survey

  • Site visits

  • Grant applications and annual program reports

  • Community scan

  • Strength and number of partnerships

  • Number of partnerships

  • Network survey

  • Site visits

  • Program survey

  • Grant applications and annual program reports

  • Strength of partnerships

  • Network survey

  • Site visits

  • Program survey

  • Grant applications and annual program reports









I.2. What are the characteristics of the environment where the program operates and how have changes in environmental factors and policies affected the delivery of services (e.g., effect on coverage of the services, provider case mix, provider caseload, etc.)? (Domain 3)


  • Changes in environment or policy

  • Changes to policies related to provision of screening services and provider requirements

None

  • Community scan

  • Previous evaluation information

  • Changes in provider practice related to delivery of screening, referrals, and LSPs

  • Changes to provider practices related to delivery of screening, referrals, and LSPs

  • Program survey

  • Site visits

  • Clinical guidelines (e.g., JNC-7)

  • Previous evaluation information

II.1. To what extent are grantees implementing team-based care and other program components to address hypertension control according to approved protocols (i.e., with fidelity)? What are the barriers and promising approaches to implementing program models? (Domain 3)


  • Implementation in comparison to approved protocols

  • Types of team-based approaches for hypertension control

  • Program survey

  • Site visits

  • Grant applications and annual program reports

  • Fidelity to and deviations from approved protocols

  • Program survey

  • Site visits

  • Grant applications and annual program reports

  • Promising approaches

  • Types of promising team-based approaches to hypertension control

  • Program survey

  • Site visits

  • Annual program reports

  • Barriers to implementation

  • Types of barriers to team-based hypertension control

  • Program survey

  • Site visits

  • Annual program reports

II.2. To what extent is health coaching being implemented according to approved protocols? What are the barriers and promising approaches to implementing program models? What QI processes are used to improve implementation? (Domain 3)


  • Health coaching implementation in comparison to approved protocols

  • Number of participants referred to health coaching

None

  • MDEs

  • Percentage of referred participants within demographic categories of interest

None

  • MDEs

  • Percentage of referred participants within demographic categories of interest

None

  • MDEs

  • Percentage of referred participants with health risks

None

  • MDEs

  • Average number and duration of health coaching sessions completed

None

  • MDEs

  • Percentage of health coaching sessions within each setting and contact type

None

  • MDEs

  • Fidelity to and deviations from approved health coaching protocols

  • Program survey

  • Site visits

  • Annual program reports

  • Health coaching protocols

  • Strategies used to refer, ensure utilization, and track health coaching attendance

  • Types of strategies used to refer participants to health coaching

  • Program survey

  • Site visits

  • Annual program reports

  • Health coaching protocols

  • Types of strategies used to ensure participant utilization of health coaching

  • Program survey

  • Site visits

  • Annual program reports

  • Health coaching protocols

  • Promising approaches

  • Types of promising approaches to implementation of health coaching

  • Program survey

  • Site visits

  • Annual program reports

  • Health coaching protocols

  • Barriers to implementation

  • Types of barriers to implementing health coaching

  • Program survey

  • Site visits

  • Annual program reports

  • Health coaching protocols

II.3. To what extent are the grantees making referrals to the LSPs according to approved protocols? What are the barriers and promising approaches to implementing program models? (Domain 3)


  • Process for LSP referrals in comparison to approved protocols (for example: appropriateness of the referrals and readiness to change)

  • Number of participants referred to LSPs

None

  • MDEs

  • Percentage of referred participants within demographic categories of interest

None

  • MDEs

  • Percentage of referred participants with health risks

  • Percentage of referred participants by readiness to change status

None

  • MDEs

  • Average number and duration of LSP sessions completed

None

  • MDEs

  • Number of LSP completed sessions by setting and contact type

None

  • MDEs

  • Fidelity to and deviations from approved LSP protocols

  • Program survey

  • Site visits

  • Annual program reports

  • LSP protocols

  • Promising approaches

  • Types of promising approaches to implementation of LSPs

  • Program survey

  • Site visits

  • Annual program reports

  • Barriers to implementation

  • Types of barriers to implementing LSPs

  • Program survey

  • Site visits

  • Annual program reports

II.4. What linkages and referrals do grantees make to resources in the communities (by risk factor) other than LSPs/health coaching? (Domain 4)


  • Referrals and participation in tobacco cessation resources by type and grantee

  • Number of participants referred to tobacco cessation resources, by type of resource

None

  • MDEs

  • Number of participants with completed tobacco cessation resource, by type of resource

None

  • MDEs

  • Use of other community-based referrals, not including LSPs/health coaching and tobacco cessation by grantees

  • Number of participants referred to other community-based resources, by type of resource

  • Number of participants with completed community-based resources, by type of other resource

  • Program survey

  • Network survey

  • Site visits

  • Annual program reports

II.5. What systems do the grantees have in place to track participants, their service utilization, completion of referrals, and other follow-up activities? How well the systems work? (Domains 1 and 3)


  • Data systems used and processes for submitting to CDC

  • Types of data systems used by grantees

  • Program survey

  • Site visits

  • Community scan

  • MDEs

  • Grant applications and annual program reports

  • Capacity assessment

  • Types of data submission processes used by grantees

  • Program survey

  • Site visits


  • Community scan

  • MDEs

  • Grant applications and annual program reports

  • Capacity assessment

  • Error and quality checks

  • Number and type of quality assurance checks (e.g., automated validation checks, provider data reports, chart audits) used by grantees

  • Program survey

  • Site visits

  • Community scan

  • Grant applications and annual program reports

  • Capacity assessment

  • Collection and use of high quality data and information for program improvement, reporting and evaluation

  • Types of non-MDE data elements collected by grantees

  • Processes used by grantees to collect MDE and non-MDE data

  • Program survey

  • Site visits

  • Community scan

  • Grant applications and annual program reports

  • Capacity assessments

  • Number of grantees using data for program improvement, reporting, evaluation and ways in which data is used

  • Program survey

  • Site visits

  • Community scan

  • Grant applications and reports

  • Capacity assessments

II.6. What are key formal and informal partnerships exist in the community to address cardiovascular health (e.g., type of linkages and partnerships)? What is WISEWOMAN’s role in these partnerships? What are the strengths and gaps in the linkages and networks? Which ones are working? (Domains 2 and 4)


  • Continuity of relationship with systems and practices to support cardiovascular health

  • Types of cardiovascular health programs in the community (available and collaborated with)

  • Program survey

  • Network survey

  • Site visits

  • Annual program reports

  • Number of cardiovascular health partnerships

  • Program survey

  • Network survey

  • Site visits

  • Annual program reports

  • Strength of cardiovascular health partnerships

  • Program survey

  • Network survey

  • Site visits

  • Annual program reports

II.7. What are the costs (direct and in-kind) associated with delivering WISEWOMAN?


  • Cost per woman served

  • Average cost per WISEWOMAN participant served

  • Program survey


  • Grant applications and annual program reports

  • MDEs





Evaluation Questions:

III.1. What are the changes in participant behaviors, risk, and CVD outcomes from base year to option year 2/Program Year 3? (Domains 2-4)

III.4. What are the changes in disparities from base year to option year 2/Program Year 3? How did WISEWOMAN programs tailor their efforts to address disparities?
(Domains 2-4)



  • Risk Reduction Counseling (RRC)

  • Percentage of participants that complete RRC

None

  • MDES

  • Average number of completed RRC sessions per participant


  • Percentage of participants receiving RRC that identify nutrition as a priority area


  • Percentage of participants receiving RRC that identify physical activity as a priority area


  • Percentage of participants receiving RRC who smoke that identify smoking cessation as a priority area


  • Percentage of participants receiving RRC that are not taking hypertension medication as prescribed that identify hypertension medication adherence as a priority area


  • Readiness to Change

  • Percentage of participants in the “pre-contemplation” state of change

None

  • MDES

  • Percentage of participants in the “contemplation” state of change


  • Percentage of participants in the “preparation” state of change


  • Percentage of participants in the “action” state of change


  • Percentage of participants in the “maintenance” state of change


  • Hypertension/blood pressure control

  • Percentage of participants hypertensive at baseline screening

None

  • MDES

  • Percentage of participants hypertensive at rescreening


  • Percentage of participants pre-hypertensive at baseline screening


  • Percentage of participants pre-hypertensive at rescreening


  • Percentage of hypertensive participants with controlled hypertension at baseline screening


  • Percentage of hypertensive participants with controlled hypertension at rescreening


  • Percentage of hypertensive participants that monitor their blood pressure at home


  • Cholesterol

  • Percentage of participants with high cholesterol at baseline screening

None

  • MDEs

  • Percentage of participants with high cholesterol at rescreening


  • Percentage of participants with borderline high cholesterol at baseline screening


  • Percentage of participants with borderline high cholesterol at rescreening


  • Percentage of participants with high cholesterol participants with controlled cholesterol at baseline screening


  • Percentage of participants with high cholesterol participants with controlled cholesterol at rescreening


  • Diabetes

  • Percentage of participants with diabetes at baseline screening

None

  • MDEs

  • Percentage of participants with diabetes at rescreening


  • Percentage of participants with pre-diabetes at baseline screening


  • Percentage of participants with pre-diabetes at rescreening


  • Medication Adherence

  • Percentage of hypertensive participants taking medication at baseline screening

None

  • MDEs

  • Percentage of hypertensive participants taking medication at rescreening


  • Percentage of participants with high cholesterol participants taking prescribed medication at baseline screening


  • Percentage of participants with high cholesterol participants taking prescribed medication at rescreening


  • Percentage of with diabetes participants taking prescribed medication at baseline screening


  • Percentage of with diabetes participants taking prescribed medication at rescreening


  • Cardiovascular risk factors

  • Percentage of participants that have been diagnosed by a healthcare provider as having any of these conditions: coronary heart disease/chest pain, heart attack, heart failure, stroke/transient ischemic attack (TIA), vascular disease, or congenital heart defects

None

  • MDEs

  • Diet

  • Percentage of participants currently watching or reducing your sodium or salt intake

None

  • MDEs

  • Average amount of fruit participants eat daily


  • Percentage of participants that eat two servings or more of fish weekly


  • Percentage of participants that eat three ounces or more of whole grains daily


  • Percentage of participants that drink less than 36 ounces (450 calories) of beverages with added sugars weekly


  • Exercise

  • Average amount of moderate physical activity that participants get in a week

None

  • MDEs

  • Average amount of vigorous physical activity that participants get in a week


  • Tobacco use

  • Percentage of participants that smoke tobacco in any form

None

  • MDEs

  • Body Mass Index (BMI)

  • Percentage of participants overweight at baseline screening (BMI between 25.0 and 29.9)

None

  • MDEs

  • Percentage of participants obese at baseline screening (BMI at or above 30.0)


  • Quality of life

  • Average number of days in past 30 days that participants reported their physical health was not good

None

  • MDEs

  • Average number of days in past 30 days that participants reported their mental health was not good


  • Average number of days in past 30 days that participants reported their mental health or physical health prevented them from doing their usual activities, such as self-care, work, or recreation


  • Alert values

  • Percentage of participants with SBP >180 or DBP >110 that have completed workup

None

  • MDEs

  • Percentage of participants with GLUCOSE ≤50 or GLUCOSE ≥250 that have completed workup


III.2. What are the changes in rates of recommended referrals (and completion) to LSPs, health coaching, and tobacco cessation resources in the context of risk from base year to option year 2/Program Year 3? (Domains 2-4)


  • Referrals

  • Percentage of participants that receive referral to LSPs/health coaching resources

None

  • MDEs

  • Average number of referrals to LSPs/health coaching resources received


  • Percentage of participants who smoke that receive referrals to tobacco cessation resources


  • Completed referrals

  • Percentage of participants with completed referral to LSPs/health coaching resources

None

  • MDEs

  • Average number of completed referrals to LSPs/health coaching resources


  • Percentage of participants with completed referral to tobacco cessation resources


III.3. What are some unintended effects from the WISEWOMAN program (such as increased empowerment and social capital among individuals and the community)? (Domains 2-4)


  • Measures of social capital

  • Degree of participant involvement with community-based groups

  • Program survey

  • Network survey


None

  • Responses to questions from the SOCATa trust and cooperation composite (answered by program and network survey respondents based on their experiences with participants)


  • Responses to questions from the SOCATa networks and mutual support organizations composite (answered by program and network survey respondents based on their experiences with participants)


  • Responses to questions from the SOCATa exclusion composite (answered by program and network survey respondents based on their experiences with participants)


IV.1. What are the grantee characteristics that may influence outcomes (for example, WISEWOMAN program components, LSP and health coaching curricula, and organizational properties)? (Domains 2-4)

IV.2. What are the types and characteristics of community-clinical linkages and agency/organizational-level networks that may influence outcomes? (Domains 2-4)



  • The outcomes proposed for evaluation questions III.1 and III.2 can be included as outcomes for evaluation question IV (summative evaluation).

  • We propose to focus on measures for the following outcomes: risk reduction counseling, readiness to change, hypertension/blood pressure control, cholesterol, diabetes, medication adherence, cardiovascular risk factors, diet, exercise, tobacco use, BMI, quality of life, referrals, and completed referrals.

None

  • MDEs

a Social Capital Assessment Tool (SOCAT) is a multifaceted instrument designed by the World Bank to collect cognitive and structural social capital data at the household level, which is crucial to link social capital information with poverty and household welfare outcomes.

MDEs = minimum data elements




File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorEmily Wharton
File Modified0000-00-00
File Created2021-01-26

© 2024 OMB.report | Privacy Policy