Instrument 16: Follow-up classroom observation protocol (teacher burden)

VIQI Follow-up Classroom Observation Protocol

April 2021

FOLLOW-UP CLASSROOM OBSERVATION PROTOCOL

The purpose of the Follow-up Observation Protocol is to give teachers information about the observation and to collect classroom-level data on quality and teacher practices. The observational data collected at follow-up point will be used to assess the impact of the interventions on quality and teacher practices and to examine the nature of the quality to child outcomes relationship.

Overview: During the second half of school year, trained field staff will conduct observations in all classrooms participating in the VIQI Project. Three-hour observations will take place on three days for the follow-up observations in the pilot study and in the impact evaluation and process study. Field staff will follow the protocol outlined below to conduct activities prior to the start of observations: (1) Observation Scheduling, (2) Pre-Observation Protocol, and (3) Post-Observation Protocol. If due to the COVID-19 pandemic, we cannot do in-person observations, we will attempt remote observations.

A. Observation Scheduling

Grantee/Agency Program Administrator Communications: Field staff will call, email, or send a letter to Program Administrators to give them information about the study. The following information will be included in communications with Program Administrators to schedule observations:

• Introduction to the VIQI Study: study background, importance of the study, and sponsors

• Link to the VIQI Study website

• Toll free study number and study email account for questions

• If available, the letter of support from sponsors

Center Administrator Communications: Field staff will email and call Center Administrators to give them information about the study and data collection procedures. During the phone call, they will ask Center Administrators to provide two Target Weeks during which field staff can plan to conduct observations over two days, as well as a current Center calendar with holidays listed. The following points will be included in communications with Center Administrators to schedule observations:

• Reminder about participation in the VIQI Study

• Upcoming data collection schedule and itinerary of study activities

• Request to identify two Target Weeks during which they can block their schedule for observations

• Request for Center calendar

• Request that they let selected teachers know about the observation

• If available, a VIQI Study brochure

Teacher Communications: Field staff will email, call, or send a letter to teachers to give them information about the study and data collection procedures. During the call or email, they will ask teachers to confirm the Target Weeks for observations. The following points will be included in their communications with teachers to schedule observations:

• Introduction to the VIQI Study: study background, importance of the study, and sponsors

• Description of data collection procedures in each teacher’s classroom; including the plan to observe on two separate days

• Confirm the Target Week dates for observations

• Toll free study number and study email account for questions

B. Pre-Observation Protocol

Overview: On the first day of observations, field staff will meet with teachers to go over information about the observation, or hand them a pamphlet with the following information, and answer any questions the teacher may have. Field staff will use the following talking points to convey information about the observation to the teacher:

• Overview: We will conduct observations in your classroom for the VIQI Study. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0508 and the expiration date is XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to [Contractor Contact Name]; [Contractor Contact Address].

• Observation Purpose: These observations will be used to help us build evidence about how to best improve the quality of early child care and education.

• Observation Experience: The observations should not disrupt your classroom at all. I will be a silent observer and will stay out of your way as much as possible. At the end of the observations, I will ask you a few questions.

• Post-Observation Questions Length: The questions I will ask you before and after the observation will take about 18 minutes.

• Privacy: The research team will do everything possible to keep your identity private, to the extent permitted by law. Only the research team will view the observation data. Teacher and classroom names will never be named in reports or other publications or presentations.

• Voluntary: The observations and questions afterwards are voluntary. You may choose whether to participate or not, and may opt out at any point in the study. Opting out will have no job implications.

ASK LEAD TEACHER THESE QUESTIONS PRIOR TO STARTING THE OBSERVATION

1. I have a few questions about the staff assigned to the classroom. For each position:

1. How many [staff position] are assigned to the classroom on a typical day?

2. How many days per week does [person in position] work in this classroom?

3. Does [staff position] work in the classroom for the full day or part of the day?

4. Is [name of person in position] permanently assigned to this classroom?

5. Has [name of person in position] been working in this classroom since September? If no, In what month were they assigned to this classroom?

6. Is the [staff position] present today? If no, Is there a substitute?

NOTE: ASK TEACHER TO POINT OUT WHO IN THE ROOM HOLDS EACH POSITION.

Staff Position	# staff assigned	# days a week	Full-day?	Permanent?	Month assigned	Absent?	Sub present?
Teacher(s)							
Assistant Teacher(s)							
Aide(s)							
Volunteer(s)							
Floater teacher							
Other							

2. What is your general schedule for today?

3. If there is a new lead or assistant teacher/para in the classroom, record their name and their contact information:

	First Name	Last Name	Contact info (e.g., email)
Lead Teacher
Assistant Teacher/Para

OBSERVATION MEASURES

Overview: During the pilot year and full-scale implementation years, there will be two days of observations using a combination of observational tools aimed at capturing teacher practices and classroom processes as a whole as well as related to specific instructional practices in domain-specific areas. These measures, which may be slightly adapted for the study, are outlined below:

• Classroom Assessment Scoring System (CLASS): The CLASS (Pianta, La Paro, & Hamre, 2008) assesses the quality of teacher-child interactions in preschool classrooms. The CLASS includes three domains of teacher-child interactions that support children’s learning and development: Emotional Support, Classroom Organization, and Instructional Support.

• Early Childhood Environment Rating Scale-Revised (ECERS-R) or Early Childhood Environment Rating Scale-3 (ECERS-3): The ECERS-R/ECERS-3 (Harms, Clifford, & Cryer, 1998; Harms, T., Clifford, R. M., & Cryer, D. (2014)) is a measure assessing different aspects of the classroom through seven subscales: (1) space and furnishings, (2) personal care routines, (3) language-reasoning, (4) activities, (5) interaction, (6) program structure, and (7) parents and staff.

• Language Interaction Snapshot (LISn): The LISn (Atkins-Burnett et al. 2011) is an observational measure for assessing language interactions in linguistically diverse early childhood programs. Observers code interactions between teachers and children, marking the language(s) in which each type of interaction occurs.

• Global Indicators of Quality: This is a study-created observational tool designed to collect information on aspects of fidelity of implementation (e.g., dosage, adherence, quality) across all participating classrooms. It will allow us to see the degree to which unique aspects of the interventions being tested are present in classrooms assigned to the other intervention and/or control.

Post-Observation Protocol

Overview: Following the observation, field staff will ask teachers a series of questions about their classroom and activities.

ASK LEAD TEACHER THESE QUESTIONS AFTER FINISHING THE OBSERVATION

1. Aside from my presence in the classroom, on a scale of 1 to 5 with 1 being not typical and 5 being very typical, would you say that today was a typical day in the classroom?

Not typical								Very typical
O1		O2		O3		O4		O5

2. Did you use any curricula today during the observation? Which one(s)? (check all that apply)

No curriculum used OWL Building Blocks Creative Curriculum Connect4Learning

Preschool Pathways to Science Pyramid Model for Social Competence [Insert other curricula] Other (specify: _________________)

1. [If yes, to any of the above, ask]: What unit and week did you teach today?

Curriculum or Source Name	Unit #/Name	Week #
1)
2)
3)
4)
5)

2. Do today’s activities or lessons incorporate a theme or focal question?

• Yes; If yes, ask the following: Please describe the theme(s) or focal question(s):

• No [If no, go to #4:]

2. Which of the following was most influential in determining what content you taught today? Pick one.

• What your children know and are able to do

• Your curriculum or lesson plans

• What you think the children should know

THIS QUESTION WILL BE ASKED DURING ONE OF THE OBSERVATION DAYS:

2. What languages did children speak in your classroom and about what proportion of children speak these languages?

	Do children in your classroom speak this language?	If yes… What proportion of children speak this language?
English	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%
Spanish	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%
Haitian Creole	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%
Mandarin	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%
Portuguese	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%
[INSERT ADDITIONAL OPTIONS FOR MOST COMMON LANGUAGES IN LOCALITIES WHERE VIQI WILL BE CONDUCTED]	O1 Yes O2 No	O1 Less than 25% O2 Between 25%-50% O3 Between 51%-75% O3 Between 75%-100%

8