Observation Tool

Information Session Observation Tool

INSTRUCTIONS

Purpose of the Tool

The purpose of this Tool is to observe and document how select enrollment activities are conducted at each lifestyle change program (LCP) site and will be completed by trained Abt evaluation team members.

How to Complete the Tool

Step 1: Complete the information on page 4 noting the observation date, name of the lifestyle change program (LCP), location of the observation (address), observer name, anticipated start date of the LCP and observation start and end time.

Step 2: Read the questions prior to starting the observation and review the instructions for each section. Definitions are provided on pages 2-3. Be sure to review the definitions, instructions and questions before the time of the observation.

Section 1 may be completed prior to the site visit. If possible, request organizations send, fax or email a copy of their enrollment materials prior to the time of the site visit. If this is not possible, obtain a copy of these materials during the site visit.

Step 3: At the time of the observation, sit in a location that allows you to observe the activities without disturbing the coaches and attendees. Complete the observation form while trying to be as objective and discrete as possible.

Step 4: Review the observation form within 24 hours to ensure the form is complete and any details you were not able to capture in the moment are recorded. It is important to do this as soon as possible, while details of the observation are fresh in your mind.

Step 5: Keep the observation (and other site visit materials) with you during your travels. Do not pack site visit documents in checked bags if traveling by air. When returning to the office, turn observation forms in to the evaluation team lead for data entry and storage.

OBSERVATION TOOL DEFINITIONS

The sections below provide an overview of key definitions for observers. These definitions will be reviewed in group trainings but are included in this document for reference while onsite.

Section I

1. Enrollment materials – enrollment materials include items such as brochures, post-it notes, factsheets, checklists or other materials used to create awareness of and increase enrollment to lifestyle change programs.

Section III

1. Clarity – The extent to which the coach’s explanation, instructions, and/or guidance was clear.

   Not clear [1]	Most attendees did not understand the instructions and could not actively participate; many clarifying questions were asked.
   Somewhat clear [3]	About half of the group understood, while the other half asked questions for clarification.
   Very clear [5]	90 to 100% of attendees participated in the activities/discussions with no hesitation. Clarifying questions were not asked.

2. Understanding – The extent to which attendees seemed to understand the information (judge based on discussion and conversations)

   Little understanding [1]	Less than 25% of attendees seemed to understand.
   Some understanding [3]	About half of attendees seemed to understand.
   Good understanding [5]	75 to 100% of attendees seemed to understand.

3. Participation – The extent to which attendees actively participated in activities and discussions (judge based on discussions and conversations)

Low participation [1]	Less than 25% of attendees participated
Medium participation [3]	About half of attendees participated
High participation [5]	75 to 100% of attendees participated

4. Information delivery – The quality with which the coach delivered the information

   Poor [1]	Appeared nervous or hurried. Did not have good eye contact with the attendees.
   Average [3]	Appeared somewhat hurried during some activities and had good eye contact with some, but not all attendees.
   Excellent [5]	Did not hesitate to promptly address concerns. Was well organized and calm.

5. Rapport and communication – Ability of the coach to communicate and develop rapport with attendees

   Poor [1]	Didn’t address attendees by their names. Didn’t seem to “connect” with attendees and seemed distant or unfriendly.
   Average [3]	Connected with some attendees, but not all attendees.
   Excellent [5]	Got attendees talking and excited. Very friendly and used names when appropriate.

6. Effectiveness – The extent to which the coach was effective in addressing attendee questions and/or concerns

   Poor [1]	Engaged in “power struggles” with attendees. Responded negatively to comments. Gave inaccurate information/ answers. Didn’t direct attendees elsewhere for more information.
   Average [3]	Struggled with some questions and/or concerns, less so with other questions and/or concerns.
   Excellent [5]	Answered questions with accurate information. Validated or addressed comments. Was honest when s/he didn’t know an answer and directed attendees elsewhere for more information.

7. Session Quality – The overall quality of the Information Session

Poor [1]	Reading from the manual, stumbling over content, poor connection with class, uninvolved attendees, engaging in power struggles, judgmental responses to questions/ comments, flat affect and lack of enthusiasm, unorganized and distracted and/or bad time management.
Average [3]	Seemed to be very familiar with some activities but less familiar with others; connected with some attendees but not all; completed the session on time but rushed through the content at the end.
Excellent [5]	Attendees are actively engaged in activities/discussion, non-judgmental responses to questions/comments, positive demeanor and connected with class, adequate pacing—not too fast and did not drag on, appropriately avoided power struggles, answered questions with the right information or provided alternative ways to find the information, organized and focused and/or good time management.

Observation Date
LCP Name
Anticipated LCP Start Date
Observation Location
Observer Name
Observation Time	Start:	End:

Section I: Enrollment Materials

This section of the observation tool may be completed prior to the site visit. To complete this section, please obtain any lifestyle change program (LCP) enrollment materials used by the organization prior to the site via mail, email or fax and complete questions 1-2 below.

Do brochures:	Yes	No	Comments:
explicitly state why it is important to delay or prevent type 2 diabetes?	□	□
explicitly state that it is important for the attendee to enroll in a LCP to delay or prevent type 2 diabetes?	□	□
have a place for LCP staff to sign and/or include their contact information?	□	□
have a place for a person’s type 2 diabetes risk-test information to be included?	□	□
Do enrollment materials (e.g., post-it notes, fact sheets, checklists, or other materials) :	Yes	No	Comments:
include a space for people to write in the date and time they plan to take action?	□	□
include phrases like “Now is the time,” or “Take 5 minutes now,” or “Put a reminder on your phone?”	□	□
use words that show the recipient has been specifically selected or considered to participate in an LCP?	□	□
highlight action steps so they are difficult to miss?	□	□
present information in way that is easy to understand (reading level)?	□	□

Section II: LCP Information Session Components

This section of the observation tool will be completed during the Information Session. Remember to sit in a location that allows you to observe the activities without disturbing the coaches and attendees. The table below lists the expected main components of an Information Session. Below each component are activities. Check the box for any activity you observe being addressed during the Information Session. Notes/observations can be added in the far right column.

If you observe something that does not fall into components 1-9 below, it can be entered as “Other Component” at the bottom of this table. Provide a name and brief description of the session component in the box provided.

Information Session Component or Activity	Occurred	Notes/Observations:
COMPONENT 1: INTRODUCTION AND GOALS
Session lead introduced themselves (and other staff members)	□
Past or current lifestyle change program participants introduced	□
Session lead acknowledged attendees have successfully taken a critical step in coming to the session	□
Attendees briefly introduced themselves	□ None □ Some (25%) □ Most (75%) □ All
Goals of the Information Session presented	□
COMPONENT 2: VALUES AFFIRMATION ACTIVITY
Handout distributed as part of the exercise	□
Attendees asked to write/do an activity about an action that connected to their top values?	□
Attendees were engaged in this exercise/actively participating?	□ None □ Some (25%) □ Most (75%) □ All
COMPONENT 3: DEFINE PREDIABETES & TYPE 2 DIABETES
Session lead defined/discussed prediabetes?	□
Session lead explained the consequences of prediabetes are reversible and the LCP can help?	□
Attendees asked questions/engaged in discussion (about prediabetes and/or reversing prediabetes)?	□ None □ Some (25%) □ Most (75%) □ All
COMPONENT 4: DESCRIBE THE NATIONAL DIABETES PREVENTION PROGRAM
Session lead discussed why the LCP is worth joining (even if people are on medication)?	□
Session lead mentioned the LCP is proven to be more effective than medication or individual effort alone?	□
Session lead mentioned important components related to the LCP effectiveness are the individualized support and knowledge from the coach and other group members?	□
Session lead discussed how much the program will cost in terms of money?	□
Session lead discussed how much the program will cost in terms of time?	□
Attendees raised questions or concerns about the costs? If yes, list in the notes/observations column.	□ None □ Some (25%) □ Most (75%) □ All
COMPONENT 5: VIDEO OR IN-PERSON TESTIMONIALS
Session included a testimonial from a current or past LCP participant?	□ Video □ In-person	Number of testimonials:
The testimonial addressed: Why the person decided to enroll? What they got from being part of the program? Why they would recommend it to others?	□ □ □
Those giving testimonials represented the pool of Information Session attendees? (If yes, please explain in the notes/observations column.)	□ None □ Some (25%) □ Most (75%) □ All
COMPONENT 6: SMALL GROUP DISCUSSION
Involved a discussion prompt provided by the session lead? If yes, what was the prompt (include in the notes/observation column)?	□
One member of each small group reported back to the full group about the small group discussion?	□
COMPONENT 7: WRAP UP
Session lead left time to address attendee’s questions at the end of the session?	□
Session lead let attendees know that the last 20 minutes/last minutes were for enrolling or talking with staff or current/past participants?	□
Attendees actively approached others to ask questions or enroll?	□ None □ Some (25%) □ Most (75%) □ All
Session lead handed out promotion material about the lifestyle change program (obtain a copy at the end of the session)?	□
Promotional material emphasized the time and location of the first session?	□
Session lead left a slide up during the last 20 minutes of the presentation highlighting the main questions for undecided attendees to ask themselves, such as: Do I want to grab this opportunity to improve my health and reduce my risk of type 2 diabetes? Do I want to pass up the opportunity for expert support and leave my type 2 diabetes risk to grow?	□
COMPONENT 8: ENROLLMENT
There was a process in place for attendees to be able to sign up for a LCP (e.g., table with forms and a person to assist)?	□
Staff spoke one-on-one with attendees who stayed but did not enroll/seemed uncertain?	□
OTHER COMPONENT (specify): _______________________________________
Briefly describe:
OTHER COMPONENT (specify): _________________________________________
Briefly describe:
OTHER COMPONENT (specify): _________________________________________
Briefly describe:

Section III: Overall Information Session Assessment

After observing the entire Information Session, answer the questions below. In answering questions 2-9, think about the Information Session in its entirety and refer back to the definitions at the beginning of the document when selecting your response.

1. Did the lead coach or member of the team provide Information Session attendees with a check sheet, magnet or other material showing they have taken the first step? Check all that apply:

• Check sheet

• Magnet

  

• Other (specify):

For the table below, circle only one response for each question.

Following the Information Session, reflect on your observations and answer questions 7-9 below. It is important to complete this section within 24 hours of the observation.

Question	Response
In general, how clear was the coach’s explanation of the activities during the Information Session?	1 Not Clear	2	3 Somewhat Clear	4	5 Very Clear
To what extent did the attendees seem to understand the information? (Judge based on discussion and conversations)	1 Little understanding	2	3 Some understanding	4	5 Good understanding
How actively did the attendees participate in activities and discussions? (Judge based on discussions and conversations)	1 Less than 25% of attendees participated	2	3 About half of attendees participated	4	5 75 to 100% of attendees participated
Use the following scale to rate your observation of how the lifestyle change coach ran the Information Session:	Poor		Average		Excellent
delivery of the Information Session	1	2	3	4	5
rapport and communication with Information Session attendees	1	2	3	4	5
effectively addressed questions/concerns	1	2	3	4	5
Overall, rate the quality of how the Information Session was run	1 Poor	2	3 Average	4	5 Excellent

Additional Observations:

7. Briefly describe any challenges or problems in running the Information Session that you observed. Be sure to note any noticeable changes/adaptations to what is provided in the Implementation Guide and Example Materials.

8. Note anything that you felt was done well or particularly effective during the Information Session.

9. Other comments. (discuss additional comments and indicate if something might have affected your observation)

1