Assessing the Implementation and Cost of High Quality Early Care and Education: Comparative Multi-Case Study, Phase 1

Implementation interview protocol

coaching and technical assistance MATHEMATICA POLICY RESEARCH

module 3: Coaching and technical assistance

A. Ongoing Coaching and Technical Assistance

We’d like to collect information about each coaching, mentoring or ongoing consultation that has taken place over the past 12 months to help staff improve skills or gain new skills in working with children or make other quality improvements.

[Interviewer: Use information collected during pre-visit phone call on coaching to tailor this section as necessary. Collect information on all topics below for each training. ]

1. On what topics is/was this assistance focused? (for example, improving instructional practice, child assessment, or the use of learning materials)

2. Did lead teachers receive this coaching or technical assistance?

1 □ YES, ALL lead teachers GO TO A3

2 □ YES, SOME lead teachers

0 □ NONE of the lead teachers GO TO A3

2a. How were lead teachers chosen to receive coaching or technical assistance?

3. Did assistant teachers receive this coaching or technical assistance?

1 □ YES, ALL assistant teachers GO TO A4

2 □ YES, SOME assistant teachers

0 □ NONE of the assistant teachers GO TO A4

3a. How were assistant teachers chosen to receive coaching or technical assistance?

4. Did aides receive this coaching or technical assistance?

1 □ YES, ALL aides GO TO A5

2 □ YES, SOME aides

0 □ NONE of the aides GO TO A5

4a. How were aides chosen to receive coaching or technical assistance?

5. Did education specialists receive this coaching or technical assistance?

1 □ YES, ALL education specialists GO TO A6

2 □ YES, SOME education specialists

0 □ NONE of the education specialists GO TO A6

5a. How were education specialists chosen to receive coaching or technical assistance?

6. Did any administrators or management team members receive this coaching or technical assistance?

1 □ YES, ALL administrators or management team members GO TO A7

2 □ YES, SOME administrators or management team members

0 □ NONE of the administrators or management team members GO TO A7

6a. How were administrators or management team members chosen to receive coaching or technical assistance?

7. How was the assistance initiated?

1 □ By the center based on an identified need

2 □ As part of a specific program (such as through Head Start) (specify program)

3 □ Based on participation in a Quality Rating and Improvement System

4 □ Other (specify)

8. Who provides this coaching or technical assistance?

MARK ONE ONLY

1 □ Coach/TA from a specific program or quality initiative (for example, Head Start or QRIS) at no cost to center. (specify source)

2 □ External coach/independent consultant paid or contractor (specify)

3 □ Internal coach employed by the center

4 □ Other (specify)

9. On what criteria was this individual or entity selected?

[Ask only of centers that reported hiring a coach/mentor/consultant.] What criteria were considered in making the selection made and by whom?

10. How is this coaching or technical assistance funded or paid for?

MARK ONE ONLY

1 □ Funded through a program or quality initiative and free to center (for example, Head Start or QRIS

2 □ Paid for by the center (including compensation for external coaches/independent consultants or internal coaches employed by the center)

3 □ Other (specify)

11. How was this coaching or technical assistance delivered?

MARK ALL THAT APPLY

1 □ Individualized

a □ Electronic (email or online)

b □ Telephone

c □ In-person consultation

d □ Coaching in the classroom

e □ Direct observation and feedback

f □ Other (specify)

2 □ Group

a □ Email

b □ Conference calls

c □ Webinars

d □ Online and regional trainings

e □ In-person

f □ Other (specify)

12. When did this assistance begin and for how long did/will it occur?

Start Date: | | | / | | | / | | | | |

Month Day Year

End Date: | | | / | | | / | | | | |

Month Day Year

13. Did this coaching or technical assistance include any of the following?

MARK ALL THAT APPLY

1 □ Coach or TA discussed the theory, philosophy, and values behind the skills or practices

2 □ Coach or TA demonstrated key skills

3 □ Trainees created a plan for conducting the new practice in their classroom/program

4 □ Trainees participated in behavioral rehearsals to practice new skills

5 □ Trainees were observed using new skills in the classroom setting

6 □ Trainees were required to meet a specific goal or benchmark indicating skill knowledge or acquisition

7 □ Coaching or TA was completed before the teacher was required to implement the new initiative or practice

8 □ Coaching or TA was linked to a prior training or professional development session

15. What information was used to identify the need for this type of assistance or to direct the content of the assistance?

16. What was the total number of hours of this coaching or technical assistance?

| | | HOURS OF ASSISTANCE

17. Did this coaching or technical assistance typically occur during the normal work day?

1 □ Yes

0 □ No GO TO A18

17a. Are accommodations made to allow teaching staff time during normal working hours to receive assistance or feedback that may be necessary outside of the classroom?

1 □ Was another caregiver required?

1 □ Yes

How was that arranged?

0 □ No

18. If not during normal work hours, when did it happen?

How was that time arranged?

Were staff compensated for their time?

1 □ Yes

0 □ No

19. What, if any, follow-up activities were conducted after this coaching or technical assistance was received?

LIST ACTIVITIES

20. Do you track any indicators to gauge the success of this coaching or technical assistance?

1 □ Yes

(specify what is tracked)

0 □ No

21. Are staff asked to evaluate the assistance?

1 □ Yes

0 □ No

B. Supporting Coaching

1. Is a specific plan in place for the delivery of coaching or technical assistance?

1. Does the plan specify responsibilities of all persons involved, the purpose of the assistance, and towards what goal?

2. Who is involved in developing the plan?

3. Is adherence to the plan regularly reviewed? If so, with what frequency?

2. How have you used information from follow-up activities, tracking of indicators, or staff evaluations about the success of the coaching or TA?

1. Is information shared with coaches or supervisors to target competency development?

2. Is information used in selecting and hiring staff with the skills and competencies needed?

3. Is information used to adjust training priorities as a whole or plans for individual staff?

3. How do you keep track of the coaching and technical assistance your staff receives?

1. Do you have an electronic tracking system?

2. If so, who enters data into the coaching and technical assistance tracking system?

4. Do you have specific professional development goals for staff tied to the coaching and technical assistance they receive? Can you please give me some examples of those professional development goals?

5. Has the center requested specific coaching or technical assistance over the past 12 months and not received it? What did you request and not receive?

C. Peer Learning and Classroom Observations

1. Do staff who work with children ages 0-5 participate in peer learning communities or communities of practice? (These are defined as a group of educators that meets regularly, shares expertise, and works collaboratively to improve teaching skills. These are different from group supervision because facilitation is usually conducted by peers rather than by a supervisor.)

1. If so, please describe the frequency, location, and extent of staff participation.

2. Also describe how the center may supports staff participation in these learning communities.

2. In the past 12 months, have staff who work directly with children ages 0-5 visited other classrooms to observe new practices or skills? [If no, go to question #3. If yes, ask the questions below.]

1. For what purpose?

2. Which staff observed other classrooms?

3. Where did the observations occur?

Select one

• Within same center

• At a different center within same program

• At a different center, different program

• Other, specify

4. With what frequency did the observations occur?

5. What happened after the observation(s)?

6. How was classroom coverage arranged for staff to participate in observations?

7. Were staff paid for their time spent observing other classrooms?

3. Are observations of staff who work directly with children ages 0-5 conducted?

1 □ Yes

0 □ No GO TO END

3a. For what purpose are the observations conducted?

MARK ALL THAT APPLY

1 □ To provide supervision

2 □ To provide feedback on a specific practice

3 □ As part of the overall performance assessment process

4 □ Some other reason (specify)

For each purpose specified, ask the questions that follow.

3b. Who conducts the observation?

3c. What tool is used to conduct the observations?

MARK ONE ONLY

1 □ Commercially developed and standardized tool (such as the CLASS or ERS), specify

2 □ Tool developed by the center or program

3 □ Other (specify)

3d. What is the frequency of observation? Does it vary by staff position, or classroom? [specify]

| | | TIMES

SELECT TIME PERIOD

1 □ Per week

2 □ Per month

3 □ Per year

4 □ Other (specify)

3e. What happens after observation of staff who work directly with children ages 0-5 is conducted?

• What discussion occurs or how is feedback provided to staff?

• If a standardized tool is used, are staff provided with scores to identify areas of strength and areas needing attention?

Cognitive Interviewing Questions for Module

1. Please tell me about your overall experience answering questions in the “Coaching and Technical Assistance” section.

2. [Ask only if perceived difficulty in responding]: I noticed that you paused when responding to one question in particular. The question reads as follows [repeat question]. Was this question difficult to understand? If so, why?

3. Were any other questions in this section not clear to you or difficult to respond to for some other reason?

1. If so, can you think of other ways that the question might be worded so that it is easier to understand or easier to respond to?

2. What additional information would you need to answer the question?