RE-AIM Framework

National Diabetes Prevention Program’s (NATIONAL DPP) RE-AIM Evaluation Protocol

Developed 2012 for DP12-1212PPHF12 Grantees

Purpose of framework for evaluation: National DPP to examine process/qualitative and outcome measures that are pertinent to program delivery and program implementation. Data can be correlated with biometric data captured by the Diabetes Prevention Recognition Program (DPRP) to examine which strategies are effective in contributing to positive health outcomes. Also, it’s important to examine those strategies that are less effective and which ones do not achieve the program goals. Strategies that will be examined include, but are not limited to, criteria for program enrollment (i.e., BMI; screening or diagnostic test), referral sources, media and communications approaches, recruitment and retention efforts, program fidelity (e.g., how program is delivered, by whom with what credentials, # of participants, program drop-out rates, completion rates core and post-core, etc.), employer coverage and/or insurance reimbursement for the lifestyle change program (i.e., policy approaches).

Overarching evaluation question: What are the best practices around scaling and sustaining the National DPP?

RE-AIM Evaluation Framework				National DPP Grantee Evaluation	National DPP Site Evaluation
Submitter		Date		Grantee/Organization Name	Grantee/Organization Name
Intervention Name (if applicable)		Year Initiated National DPP		Grant Year Fiscal Year	Year lifestyle change program initiated
Implementing Organization’s Name				Please provide the name and organizational code for all NEW sites recruited or selected in the current grant year.	Site Name (MUST be the Org code that is provided by the DPRP)
INTERVENTION
Brief Description	Briefly describe the NATIONAL DPP lifestyle change program/intervention focus in terms of the target audience (i.e., race/ethnicity, age, geographic area, low SES, low literacy, etc.). Briefly describe the places from which NATIONAL DPP participants were recruited. Briefly describe or list the organizations offering the NATIONAL DPP lifestyle change programs. Briefly describe the recruitment methods (i.e., materials or campaigns, etc.). Prompt: How do you identify and recruit 1. Sites to offer the NATIONAL DPP Lifestyle change program and 2. Participants? Briefly describe any policy (i.e., legislative or organizational) or any reimbursement/financing mechanism (i.e., employer coverage or insurance reimbursement) if applicable.			Among NEW sites that began in the current grant year, how many are currently offering the National DPP lifestyle change classes? How did you or your sub-awardee (if applicable) recruit and/or select sites to offer the lifestyle program (i.e., recruitment methods such as materials or campaigns, working with healthcare groups/practices, etc.) in the current grant year? Please select ALL that apply. Did you engage in any activities to recruit participants directly or connect them to a program site? If so, what activities? Please select ALL that apply. Was there any change in policy resulting in a new financing/reimbursement mechanism? If so, who was the target of the change in policy resulting in a new financing/reimbursement mechanism? Please select ALL that apply. What was the policy that was changed? Please select ALL that apply.	Please select from the list demographic characteristics of PARTICIPANTS. Please select ALL that apply. Please select from the list the places from which National DPP participants were recruited. Please select ALL that apply. What recruitment methods (i.e., materials or campaigns, from healthcare/insurance company patient rosters, etc.) did you use to identify and recruit new participants? Please select ALL that apply. Is there any policy in place that establishes a financing/reimbursement mechanism? If so, what entities have implemented the financing/reimbursement mechanism policy? Please select ALL that apply. What was the policy that was established? Please select ALL that apply.
Resources Required	Please describe all primary resources necessary for program start-up and implementation. Prompts: Partnerships/coalition, in-kind contributions, etc. What others tools or tangible resources were needed? Prompt: Office/classroom space, computer or printing equipment, etc. Do you use volunteer or paid lifestyle coaches? How many lifestyle coaches are there per setting? How many total under your grant? How are lifestyle coaches trained? Prompt: Days of training received, content of training, costs related to training, What are the credentials required of lifestyle coaches by the program (e.g., years/type of education and/or experience)?			For the current grant year, how many lifestyle coaches were trained? For the current grant year, how many master trainer lifestyle coaches were trained? For the current grant year, what was the average number of days of training received for lifestyle coaches? For the current grant year, what was the average number of days of training received for master trainers? Please select from the list who trained NEW lifestyle coaches. Please select ALL that apply. Did lifestyle coaches receive an additional one day of motivational interview training?	Please indicate from the list of primary resources necessary for program start-up and implementation (e.g., partnerships/coalition, in-kind contributions, etc.). Please select ALL that apply. Please indicate how many of your lifestyle coaches volunteers, hourly contractors, or salaried employees are. If none, write "0". Please indicate the number of coaches with each of the listed credential using the categories provided. If none, write "0." What is the funding source for lifestyle coaches? Please select ALL that apply. What are the credentials of the lifestyle coaches? Please indicate the number of coaches with each of the listed credential using the categories provided. If none, write "0."
Reach (The absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative, intervention, or program; RE-AIM.org.)	At what level is the NATIONAL DPP lifestyle change program targeting its audience (e.g., state, county, health care system, clinic/practice, private organization, government or other non-profit)? At what level is the reimbursement or financing mechanism targeted (e.g., state, county, other government or non-profit organization, etc.)? What is the number of participants initially enrolled in the lifestyle change program at your site? What is the average total number of sessions completed by participants at each site each year? What is the number of participants who completed the 16-week core program? From this, calculate a percentage (%) of participants who completed the 16-week core program (# completed/total # of those enrolled). What is the number of participants who completed both the 16-week core program AND the 6 month post-core follow-up at end of year 1? From this, calculate a % of participants who completed both the 16-week core program AND the 6 month post-core follow-up at end of year 1 (i.e., completed both/total # of those enrolled). Do you consider your target population to be disparate or vulnerable? If so, why?			Where or at what level are you targeting participant recruitment efforts? Please select ALL that apply. For current grant year, at what level is the reimbursement or financing mechanism targeted (e.g., state, county, other government or non-profit organization, etc.)? Please select ALL that apply. Is there purposeful targeting of NEW disparate or vulnerable populations at the grantee level? If so, what populations are targeted? Please select ALL that apply.	Data reported to the Diabetes Prevention Recognition Program (DPRP). Data reported to the DPRP. Data reported to the DPRP. Data reported to the DPRP. Is there purposeful targeting of disparate or vulnerable populations during recruitment for program participation at the site level? If so, what populations are targeted? Please select ALL that apply.
IMPLEMENTATION STRATEGIES
Brief Description of Implementation (In short, fidelity and use of intervention strategies; RE-AIM.org.)	If the NATIONAL DPP lifestyle change program takes place in multiple sites, is it offered in the same manner? If no, why not? Prompt: Using the same curriculum, same credentials in terms of lifestyle coaches, same type of reminders, etc. Did NATIONAL DPP lifestyle change program sites use either monetary incentives (e.g., returning participant deposit money, use of participant commitment contracts, etc.) or non-monetary incentives (e.g., giving out weight loss diaries, tools, or other free items, etc.) to encourage participant program completion? What were the credentials of the lifestyle coach/trainer who conducted the intervention? Prompt: Lay professional (i.e., Community Health Worker), Certified Diabetes Educator, nutritionist, etc. Briefly describe the key steps taken to start and implement the policy and/ or reimbursement/financing aspect of the intervention (e.g., employer coverage for participants or insurance reimbursement). Briefly describe barriers to program start-up and implementation and any strategies to address barrier(s). Describe where referrals to NATIONAL DPP lifestyle change programs are coming from (i.e., clinics, employers/worksites, physicians, health centers, private organizations, non-profits, etc.).			What kinds of incentives and/or behavioral techniques, if any, were used to retain program participants? Please select ALL that apply. For the current grant year, please provide the number of employers who were educated about the benefits and cost-savings of the evidence-based lifestyle change program as a covered health benefit for employees. For the current grant year, please provide the number of employer council events attended to educate participants about the benefits and cost-savings of the evidence-based lifestyle change program. For current grant year, please provide the number of health care purchasing coalitions educated about the benefits and cost-savings of the evidence-based lifestyle change program. For current grant year, please provide the number of insurance broker events attended to educate participants about the benefits and cost-savings of the evidence-based lifestyle change program. Please select from the list the barriers to NEW site start-up (if applicable) and implementation in current grant year. Please select ALL that apply. Please select from the list strategies to address barriers listed above at organizational level for Year 3. Please select ALL that apply.	In the current grant year, please list the number of and types of locations where classes are offered, and zip codes for all class locations(e.g., employer worksite, community health centers, clinic/practice, community centers, YMCAs, gyms, church, other non-profit, etc.)? Please select ALL that apply. Which DPP curriculum are you using at this site? How have you adapted the lifestyle change program to address the specific cultural needs or preferences of one or more of your targeted populations? Please select ALL that apply. Are there any participation or completion incentives offered to participants? If so, what are the incentive items? Please select ALL that apply. What are the credentials of the lifestyle coaches? Please indicate the number of coaches with each of the listed credential using the categories provided. If none, write "0." Please select from the list the barriers to program start-up and implementation. Please select ALL that apply. Please select from the list strategies to address barriers listed above at site level. Please select ALL that apply. Please select from the list of referral sources to the lifestyle change programs (i.e., clinics, employers/worksites, state health departments, physicians, health centers, private organizations, non-profits, insurers, etc.). Please select ALL that apply.
Adoption [The absolute number, proportion, and representativeness of settings and intervention agents (people who deliver the program) who are willing to initiate a program; RE-AIM.org.]	In how many settings/organizations is the NATIONAL DPP lifestyle change program being offered (need an absolute number)? What is the representation across states? (I.e., statewide, regional, in a few counties, in one county per state, etc.) Are there other eligible organizations that could use the NATIONAL DPP lifestyle change program in other settings? Prompt: Other healthcare facilities, including hospitals, private clinics/FQHCs; YMCAs or other non-profit organizations, etc. How are these other settings being explored? Did policy or reimbursement/financing achieve intended results in adopted settings (i.e., provide optimal benefit coverage possible, pay for a large percentage of classes, etc.)?			Please provide the name and organizational code for all NEW sites recruited or selected in the current grant year. If yes to policy changes with insurers, how many new insurers and markets are now providing reimbursement or coverage for the lifestyle change program? Please select which type of insurer, and for each insurer please specify the name of the insurer and specify the market (a geographic level indicating the state or region, or county). For current grant year, please provide the number of employers offering the National DPP lifestyle change program as a covered benefit or as a wellness benefit for employees. For the current grant year, please provide the number of employers who offer the National DPP program on-site.	Please select from the list the type of organization that best describes your site. Please list the full address of your site. Were there any NEW places from which National DPP participants were recruited? Please select ALL that apply. Were there any NEW referral sources (i.e., clinics, employers/worksites, state health departments, physicians, health centers, private organizations, non-profits, insurers, etc.)? Please select ALL that apply.
Maintenance/ Sustainability (The extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies; RE-AIM.org.)	What are the primary resources necessary for sustainability? What are the known barriers to sustainability? What are your most successful participant recruitment strategies (i.e., physician/provider referrals; employer referrals; media material; self-referral, etc.)? Explain and link to outcomes where applicable. What are your most successful participant retention strategies? Explain and link to outcomes where applicable. Has the reimbursement/financing aspect of the NATIONAL DPP lifestyle change program been sustained? If so, for how long now? What are your most successful employer/insurance company recruitment strategies? Explain and link to outcomes where applicable. How do you plan to ensure or increase employer coverage and/or insurance reimbursement the NATIONAL DPP lifestyle change program? What are your organizations’ long-term plans regarding offering and continuing the NATIONAL DPP lifestyle change program?			Assessment of correlations between site recruitment methods and participant enrollment and retention rates. Assessment of correlations between types of incentives and participant retention rate. Assessment of correlations between types and number of strategies used to educate employer/insurance company and number of employer/insurance coverage for the National DPP. Have you communicated with any state health departments or other 1422 awardees about collaboration for diabetes prevention program reimbursement?  If so, please describe. Data reported to PGO as annual grantee progress reports.	Do you charge participants for the program? If so, and if you are able to report this data, how much do you charge on average per participant? For the current grant year, was there any change from the previous grant year regarding how lifestyle coaches were paid? Was this site still active in Year 3? If yes, continue to question 6. If no, answer question 5 and stop. If the site was not active in Year 3, please provide reasons. What recruitment methods (i.e., materials or campaigns, from healthcare/insurance company patient rosters, etc.) did you use to identify and recruit new participants? Please select ALL that apply. Is any participation or completion incentive offered to participants? If so, what are the incentive items? Please select ALL that apply. Was there any change in policy resulting in a new financing/reimbursement mechanism? If so, what entities have implemented the change in policy resulting in a new financing/reimbursement mechanism? Please select ALL that apply.
EVALUATION
Efficacy/ Effectiveness (The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes; RE-AIM.org.)	Completion and drop-out rates: Calculate a participant completion rate per course (or site) at the end of Year 1 (see data submitted in REACH section). Calculate a participant drop-out rate per course (or site). Cost calculations: What are the first year organizational start-up costs (i.e., training costs of lifestyle coaches, travel, materials, scales, space, etc.)? What are the organizational costs per year beginning in Year 2 once the program is in place? What is the estimated annual cost of the NATIONAL DPP lifestyle change program per participant? If funding or reimbursement is provided by an employer or insurance carrier, what are their costs per participant? Did your program achieve the expected program outcomes? Prompt: If so, what are the outcomes? Are any related directly or indirectly to the expected outcomes (e.g., % of weight loss, increase in physical activity, lower BMI, no current diagnosis of type 2 diabetes, increase in healthy eating, etc.)?			Calculations of aggregated drop-out rates per site and per grantee using sites’ data reported to DPRP. Please provide the average monetary cost per new lifestyle coach trained. Have you calculated the average cost per participant amongst your program sites? If you are able to report, what is the numeric value of the cost? Calculations of aggregated DPRP requirements outcomes per grantee using sites’ data reported to DPRP.	Data reported to the DPRP. What is the average salary of a lifestyle coach, if you are able to report this data? What is the average salary of a program coordinator, if you are able to report this data? Data reported to the DPRP.

Sources: CDC/RTI Effective Strategies interview protocol for reimbursement, 2011; RE-AIM.org (http://re-aim.org/about-re-aim/what-does-re-aim-mean.aspx).

2