Work@Health Advance SS B_073015

CDC Work@Health^® Advance Program:

Evaluation of Train-the-Trainer and Advanced Technical Assistance Programs

New

Supporting Statement: Part B

Program official/project officer: Jason Lang, MPH, MS

Team Lead, Workplace Health Programs (CDC/NCCDPHP/DPH)

Tel: (770) 488-5597

Fax: (770) 488-5962

Email: jlang@cdc.gov

July 30, 2015

Table of Contents

Section B Collection of Information Employing Statistical Methods

B-1 Respondent Universe and Sampling Methods

B-2 Procedures for the Collection of Information

B-3 Methods to Maximize Response Rates and Deal with No Response

B-4 Tests of Procedures or Methods to Be Undertaken

B-5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Attachments

Attachment A-1. Authorizing Legislation

Attachment A-2. Patient Protection and Affordable Care Act Prevention and Public Health Fund (P.L. 111-148, Section 4002)

Attachment A-3. Public Health Service Act (42 USC 241)

Attachment B-1. Federal Register Notice – 60 Day

Attachment B-2. Summary of Public Comments

Attachment C. Work@Health^® Program Objective – Survey Instrument Crosswalk

Attachment D. Work@Health^® Information Collection Flow Chart

Attachment E. Screenshots of CDC Work@Health^® Accreditation Readiness Assessment Tool

Attachment F. Screenshots of CDC Work@Health^® Advanced Technical Assistance Survey

Attachment G. Screenshots of CDC Work@Health^® Follow-up Accreditation Survey

Attachment H. CDC Work@Health^® Advance Employer Follow-up Survey

Attachment I. Screenshots of Train-the-Trainer Application Form

Attachment J. Screenshots of CDC Work@Health^® Train-the-Trainer Knowledge and Skills Survey

Section B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The Work@Health Advance program will enroll approximately 360 total employers over a three year period (120 annually) to participate in advance level technical assistance. Work@Health^® advanced technical assistance will draw from the existing sample of employers who have participated in the Work@Health^® Program and have successfully completed all Core Curriculum modules as well as all technical assistance milestones.

The Work@Health® Basic Training and Technical Assistance Program (CDC Work@Health Program: Phase 2 Training and Technical Assistance Evaluation,” OMB No. 0920-1006, exp. 1/31/2016) has focused employer outreach and recruitment on small employers. Small employers are less likely to possess the internal knowledge, skills, and capacity to develop and implement effective workplace health programs making the Work@Health® training program appealing and helpful to them as it addresses these weaknesses. In the first year of the basic program, approximately 75% of participants (n=480) were small employers from across the country with less than 250 employees. To have been eligible for the Work@Health® Program, employers must have been in business for at least one year with a valid business license, headquartered in the United States, have at least 20 employees, and provide health insurance to their employees. The employers of the Work@Health® Program represent a variety of sizes, industries, and employer types from all regions of the United States. Additionally, the Work@Health® Program reviewed the employer application form to select candidates who were highly motivated and committed to fully participate and complete the training and technical assistance program.

In the first year of the program (2014-2015), 480 employers were enrolled in Work@Health®. Of this group, 30% (n=144) were lost to attrition due to several factors including leadership and/or staffing changes within the organization; change in organizational priorities shifting time and attention away from Work@Health®; or failure to meet all training and technical assistance requirements including participation in all training activities; meeting regularly with the participant’s technical assistance provider; or submitting all progress reports and completion logs describing progress made in furthering their worksite’s health promotion efforts. The remaining 70% of participants (n=336) met the basic program requirements and received a certificate of completion. This group of participants (n=336) represents the eligible applicant pool for Work@Health® Advance. It is expected that the Work@Health® Program will be continued and a similar number of participants will be eligible for the advanced technical assistance program each year. However, the total number of employer candidates may vary year to year due to demand for the program and employer attrition resulting in failure to complete the core curricula training and technical assistance milestones.

From this applicant pool, the Work@Health® Advance Program plans to select 120 employer participants each year for the advanced technical assistance program. It is expected that approximately 30% will be lost to attrition due to similar unexpected factors experienced during the basic training program (n=100) leaving approximately 236 candidates in the annual eligibility pool. Approximately half of these remaining candidates (n=120) will be selected to participate in Work@Health® Advance using a similar sampling frame from the basic program which provided outreach to and encouraged applications from twice as many employers than were eventually enrolled in the program. The focus is on selecting those candidate who are strongly motivated, committed, and positioned to succeed in the Work@Health® Advance program by fully engaging in the program’s technical assistance activities; likely to follow through with pursuit of a third party workplace health accreditation and/or recognition program which would rely on the skill and capacity built through their Work@Health® experience.

The Train-the-Trainer model of the Work@Health^® Program is designed to prepare approximately 100 qualified individuals annually (300 total over three years) to acquire the knowledge and skills needed to train employers to implement the Work@Health^® Core Curriculum. The individuals who participate in the Train-the-Trainer model will be recruited from health departments, professional organizations, business coalitions, private employers, and non-profit organizations. The Work@Health® Program has the capacity to accommodate 100 Train-the-Trainers each year relying on the number of professional trainers and technical assistance providers supported by the program; the capacity of the online training and technical assistance platform built for exchanges of information and resources; the structure of training classes capped at 25 participants per class; and the financial resources available to the program. Although the demand from prospective participants exceeds this capacity, the Work@Health® is committed to providing training opportunities to as many Train-the-Trainersas possible given limited resources and the commitment to a deliver high quality training experience As before, CDC will publicize the opportunity to participate and encourage eligible participants to enroll in the Train-the-Trainer model by reaching out to employers and organizations that support employer worksite health program efforts identified from a compiled list. The 100 participants selected to participate in the Train-the-Trainer model will have: 1) a referral from state or local health department, or other qualifying organization; 2) evidence of worksite health program knowledge and skills; 3) training skills and experience; and 4) experience with implementing worksite health programs.

Based on attrition trends observed in the Work@Health® Basic Program, the Train-the-Trainer Application has been revised to better reflect motivation, engagement, and commitment of the applicants. Approximately 15% (n=-20) of 120 Train-the-Trainer participants were lost to attrition primarily due to an inability to identify and train five employers as part of their final technical assistance milestone. The Train-the-Trainers either were unable to recruit all five employers, or those employers who were enrolled did not meet one or more of the program’s eligibility requirements such as having less than the minimum 20 employees or not offering health insurance to their employees. The revised Train-the-Trainer application form focuses on identifying more experienced candidates who are better connected to the employer community and will be able to fulfill the program’s training expectations. The Work@Health® Advance Program feels the initial attrition among Train-the-Trainers can be corrected with better scrutiny prior to enrollment allowing the program to train 100 candidate per year. CDC will use these application questions to select only the most motivated and committed applicants, therefore minimizing attrition from the program. To ensure that the sample size is maintained, CDC will create a waiting list of applicants who can be invited to participate in the training if a selected applicant is not able to participate.

2. Procedures for the Collection of Information

Overview

CDC plans to collect information from employers who have previously completed the Work@Health^® Core Curriculum and technical assistance milestones to assess readiness for accreditation of their worksite health program and their need for additional technical assistance using the CDC Work@Health Accreditation Readiness Assessment Tool (Attachment E); to obtain trainees’ reactions to the advanced technical assistance using the CDC Work@Health Advanced TA Survey (Attachment F); and to document their experiences applying for and/or receiving accreditation of their worksite health program using the CDC Work@Health Follow-up Accreditation Survey (Attachment G). A short telephonic follow-up survey will be administered to provide information about program maintenance and sustainability for all employer participants of the Work@Health^® Advance Program (120 employers annually). The purpose of the CDC Work@Health Advance Employer Follow-up Survey (Attachment H) is to determine to what extent each employer is continuing to implement the Work@Health^® elements, what changes have been made, what barriers have been encountered, and what lessons were learned. CDC will administer the survey to a representative from each employer, such as the wellness committee champion or human resources staff, approximately 12 months after the formal program participation ends.

CDC also plans to collect information needed to select the individuals who will participate in the Train-the-Trainer model with the Train-the-Trainer Application Form (Attachment I); and to assess changes in trainees’ knowledge and skills before and after their participation in the training using the CDC Work@Health Train-the-Trainer Knowledge and Skills Survey (Attachment J). Those interested in participating in the Train-the-Trainer model of the Work@Health^® Program will be instructed to view the Work@Health^® website and fill out an online Train-the Trainer Application Form (Attachment I).

An outline of the timing and purpose of each instrument is presented in the Crosswalk of Instruments (Attachment C) and an outline of the timing of information collection for each instrument is presented in the Information Collection Flow Chart (Attachment D). Respondents and their respective information collection assessments are categorized as follows:

A. Employers. Employers who successfully completed training through the Hands-on, Online, or Blended models of the Work@Health^® Program and all technical assistance milestones will be invited to participate in the advanced technical assistance of Work@Health^®. Employers who are interested in participating in the advanced technical assistance will complete the CDC Work@Health Accreditation Readiness Assessment Tool (Attachment E). This form will collect information from employers to evaluate their readiness to apply for accreditation from an external body. This form will be administered online to all interested employers prior to participating in the Work@Health^® advanced technical assistance and again after the close of technical assistance, 7 months after baseline.

The CDC Work@Health Advanced Technical Assistance Survey (Attachment F) will collect information from the employers participating in the Work@Health^® advanced technical assistance to assess how much participants used the technical assistance and their perceptions of the utility of the technical assistance they received through the course of the program. Trainees will be asked to assess how useful different aspects of the technical assistance (e.g., topical webinars, interactive discussions with peers and facilitators) were to their ability to transfer what they learned to their worksites’ process for accreditation. This survey will be administered twice online; once 3 months after the start of technical assistance and once approximately 4 months later, at the close of technical assistance.

The CDC Work@Health Follow-up Accreditation Survey (Attachment G) will collect information on the employers’ current accreditation application status, barriers to application, and future plans to apply for accreditation or recognition. The survey will be administered online once 10 months post-baseline.

B. Employer Graduates of Work@Health. Employers who participate in the Work@Health^® Advance Program participating employers will complete a telephonic follow-up survey (Attachment H) at approximately12 months following their formal program participation phase to gauge program sustainability.

C. Train-the-Trainer Participants. Individuals interested in participating in the Train-the-Trainer model of the Work@Health^® Program will be asked to complete a Train-the-Trainer Application Form (Attachment I). The form will be used to assess applicants’ background experience in worksite health programs and training facilitation. CDC anticipates that approximately 200 individuals will complete the form. CDC will select 100 qualified employers, trainers, and facilitators to participate in the Train-the-Trainer model of the Work@Health^® Program.

The CDC Work@Health Train-the-Trainer Knowledge and Skills Survey (Attachment J) will collect information from the 100 selected Train-the-Trainer participants to assess changes in trainees’ facilitation skills, knowledge about worksite health programs, and ability to train others using the Work@Health^® curriculum. The survey will be collected online twice: one month before training and 6 months post-baseline.

3. Methods to Maximize Response Rates and Deal with No Response

The Work@Health® Program changed the structure of the delivery of the program to minimize program attrition and non-response. A new user online platform has been created to allow for better interaction between trainer/technical assistance providers and program participants. This enhanced communication makes it easier to notify participants when surveys will be collected, the nature of the data collection, and the methods to submit information. Additionally, the system has created a calendar and notification system to improve communication and the Work@Health Advance Program team has also developed issue notices to also be more proactive in communicating the program’s evaluation needs. Better scrutiny applied to prospective employers for the advanced technical assistance program and the Train-the-Trainer program prior to enrollment including discussions of time commitments, program expectations, and barriers to completion have been incorporated into the program to minimize attrition. However, with a free, voluntary program, attrition cannot be eliminated completely and the program has based its enrollment figures on the experienced rate of program attrition (30% for employers and 15% Train-the-Trainers).

For all instruments being used in the Work@Health^® Program, CDC designed the procedures for collecting information to minimize the burden to respondents and to the government, to maximize convenience and flexibility, and to ensure the quality of the information collected. The Work@Health^® Program will seek to identify eligible, motivated, engaged, and committed participants through the information collected on the Train-the-Trainer Application forms. This includes an awareness and understanding of program requirements, including information collection.

CDC’s implementation contractor, ASHLIN Management Group, was selected in part because of their experience and expertise in planning and managing similar training initiatives and working successfully with content and technical experts of the type required for the current Work@Health^® initiative. Specific methodologies and strategies associated with each collection tool are described below.

For all online surveys related to participating in the Work@Health^® advanced technical assistance, the evaluation contractor will actively encourage trainees to complete the surveys and will provide detailed instructions to ensure accurate responses. The surveys are designed to be easy to complete and will provide respondents with an opportunity to think about their worksite’s readiness to apply for accreditation and how well the Work@Health^® advanced technical assistance has prepared them for the accreditation process. Trainees will receive reminders at regular intervals about completing the surveys by the deadline. The opportunity to give information about their experiences and feedback about the TA they received should also encourage trainees to complete the survey.

The Work@Health^® Program’s outreach team will work with champion organizations to inform employers and organizations that support employers in worksite health about the project, encourage them to visit the Work@Health^® website, and invite them to complete the Train-the-Trainer Application Form (Attachment I). All interested employers and trainers must complete this form online. Completion of the CDC Work@Health Train-the-Trainer Knowledge and Skills Survey (Attachment J) is required as part of the program’s enrollment process; therefore, CDC anticipates that all individuals who are interested in participating in the Work@Health^® Program will welcome the opportunity to complete this assessment as part of the process of learning how to deliver the Work@Health^® Core Curriculum and technical assistance to employers. Reminders will be sent to individuals who do not complete the Train-the-Trainer Knowledge and Skills Survey and access to the online training and technical assistance portal will be granted only after the survey has been completed.

The CDC Work@Health Advance Employer Follow-up Survey (Attachment H) will be conducted by telephone. The evaluation contractor will actively encourage participants to complete the surveys and will provide detailed instructions when providing the survey links to ensure accurate responses. Outreach to employer graduates will include invitations to ongoing educational webinars, postings, and announcements through the Work@Health^® online community.

Participation of employers and trainees in the Work@Health^® Program is strictly voluntary. Employers and trainees may withdraw from Work@Health^® at any time simply by notifying the implementation or evaluation contractor staff.

4. Tests of Procedures or Methods to be Undertaken

The Work@Health^® team, including subject matter experts from CDC, ASHLIN Management Group (the implementation contractor) and its team leaders - including the evaluation team leader and the training and curriculum team - provided input on the content of the required information and assessment tools to adequately capture the information required for implementation and evaluation of the different training models, as well as the frequency of information capture.

The CDC Work@Health Advanced Technical Assistance Survey (Attachment F), Train-the-Trainer Application Form (Attachment I), and CDC Work@Health Train-the-Trainer Knowledge and Skills Survey (Attachment J) are revised versions of instruments already used in the Work@Health^® Program (OMB No. 0920-1006). Where previously fielded forms have been revised for the Work@Health Advance Program, the burden per response has been adjusted.

CDC also pre-tested the survey materials for clarity, organization, and timing with a group of external employers who were representatives the target audience of the Work@Health^® Advance training (employers), and instructors with experience in conducting trainings, including Train-the-Trainer participants who participated in the initial Work@Health^® program. The CDC Work@Health Accreditation Readiness Assessment Tool was tested by 9 employers, the CDC Work@Health Follow-up Accreditation Survey was tested by 7 employers, the Train-the-Trainer Application Form was tested by 5 trainers/instructors, and the CDC Work@Health Train-the-Trainer Knowledge and Skills survey was tested by 7 trainers/instructors. Table 8-a contains the names and affiliations of persons who completed the pre-tests. The CDC Work@Health Advanced Technical Assistance Survey was not pilot tested because the format and content of this instrument is similar to the TA Survey that was used in the initial Work@Health^® program. The previously-approved TA Survey was revised to reflect the new webinars and other content developed for Advanced TA.

The pre-tests confirmed that the estimated time to complete each instrument in the burden table was adequate. No survey items were identified as being confusing or inappropriate. Pre-testers gave some helpful suggestions for formatting some questions and clarifying instructions.

In addition, the scoring algorithm developed for the CDC Work@Health Accreditation Readiness Assessment Tool (Attachment E) was pre-tested with 9 employers who are graduates of the Work@Health Basic training program and candidates for the Advance Program. These employers were asked to complete the CDC Work@Health Accreditation Readiness Assessment Tool and their overall scores were generated using the scoring algorithm. The scores were entered into a computer adapted tool that creates a technical assistance roadmap to be used to the employer and their technical assistance provider during the Work@Health Advance program. The program team and subject matter experts validated the prioritization and guidance provided in the roadmap against the recommendations that would be made from their interpretation of the CDC Work@Health Accreditation Readiness Assessment Tool results.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

CDC will provide overall program management for the Work@Health^® Program, directing regular planning and coordination meetings with the contractor staff including the information collection plan and reporting to participating employees, employers, and in the aggregate.

Work @Health® participants will be trained and supported by CDC’s implementation contractor, ASHLIN Management Group. Public Health Management Corporation (PHMC), ASHLIN’s evaluation sub-contractor, is charged with the evaluation of the Work@Health® Program.

The implementation contractor, ASHLIN Management Group, Inc., will provide operational management of the worksite health promotion training program and coordinate program activities. Public Health Management Corporation (PHMC) has responsibility for collecting and analyzing results from all instruments. Information collected by PHMC will be aggregated and de-identified before being shared with ASHLIN Management Group, Inc.

The principal contacts for each organization are listed below:

Staff from CDC	Contact Information
Jason Lang Team Lead, Workplace Health Programs CDC/ONDIEH/NCCDPHP	Phone: (770) 488-5597 Email: jlang@cdc.gov
Implementation and Evaluation Contractors
Linda Botts ASHLIN Management Group	Phone: (301) 345-8357 Email: lbotts@ashlininc.com
Suzanne Hemphill ASHLIN Management Group	Phone: (301) 345-8357 Email: shemphill@ashlininc.com
Jennifer Lauby Public Health Management Corporation	Phone: (215) 985-2556 Email: jennifer@phmc.org
Mary Harkins-Schwarz Public Health Management Corporation	Phone: (215) 985-2082 Email: mharkins@phmc.org
Livia Fortunato Public Health Management Corporation	Phone: (267) 765-2336 Email: lfortunato@phmc.org
Brandon Alston Public Health Management Corporation	Phone: (215) 985-2552 Email: balston@phmc.org
Nicholas Hobar ASHLIN Management Group/Learning Front	Phone: (443) 255-4944 Email: nickhobar@learningfront.com

12