Assessing the Implementation and Cost of High Quality Early Care and Education: Comparative Multi-Case Study, Phase 1

Implementation interview protocol

staff orientation, training, and professional activities MATHEMATICA POLICY RESEARCH

MODULE 2: staff orientation, Training, and professional activities

Now I’d like to switch gears and talk about any training that occurs.

Let’s start with training that may happen for newly hired staff.

A. Training New Staff

1. How does your center orient new staff to the center and to general procedures?

2. Who is involved in orienting new staff to the center?

3. Is this a standard process across all staff?

1 □ No

0 □ Yes GO TO K4

1. Does if differ by type of staff?

2. Does it differ by age of children cared for?

4. How long does the orientation process typically take for the majority of new staff?

| | |

SELECT TIME PERIOD

1 □ Hour/s

2 □ Day/s

3 □ Week/s

4 □ Other (specify)

5. How long has this process been in place?

| | |

SELECT TIME PERIOD

1 □ Month/s

2 □ Year/s

3 □ Other (specify)

6. Beyond a general orientation for all staff, are there ways that staff who work directly with children ages 0-5 are oriented to classroom practices and activities or the supports available for children and their families?

1. Who is involved with orienting new teaching staff?

2. Do new teaching staff shadow existing staff or observe the children and routines in their assigned classrooms before fully assuming their responsibilities?

3. Do new teaching staff have a specified probationary period? If so, what criteria must be met within the probationary period?

4. Is this process the same for all new teaching staff or does it differ by position or age of children cared for? (for example, teachers, assistants, aides, or for teachers in infant/toddler classrooms vs. classrooms for preschool age children)?

7. What else do staff need to be trained on before they enter the classroom?

[If specific training required before they enter the classroom, then ask series of questions that follow. Otherwise, proceed to section B.]

1. Why is this training required, and by whom? For example, is the training required to teach a specific curriculum, use a specific assessment too, or required by a specific program or funder with which the center is affiliated?

2. How long has this training requirement been in place?

| | |

SELECT TIME PERIOD

1 □ Week/s

2 □ Month/s

3 □ Year/s

4 □ Other (specify)

3. How is this required training conducted?

MARK ALL THAT APPLY

1 □ In-person, on-site

2 □ In-person, off-site

3 □ Online, non-interactive (such as a self-paced training module)

4 □ Online, interactive (such as via webinar)

5 □ Other (specify)

4. Who conducts the training?

5. How is the training provider selected? Are there certain criteria that are used?

6. How long does the training last?

| | |

SELECT TIME PERIOD

1 □ Day/s

2 □ Week/s

3 □ Month/s

4 □ Other (specify)

7. Does the training include any of the following?

MARK ALL THAT APPLY

1 □ Do trainers discuss the theory, philosophy, and values behind the skills or practices?

2 □ Do trainers demonstrate key skills?

3 □ Do trainees participate in behavioral rehearsals to practice new skills?

4 □ Are trainees observed using new skills in the classroom setting?

5 □ Are trainees required to meet a specific goal or benchmark indicating skill knowledge or acquisition?

8. Is a pre- and post-test of knowledge and skills conducted in connection with the training?

1 □ Yes, please describe

0 □ No

B. Setting Priorities and Identifying Needs for Training and Professional Development

Now let’s talk about training and professional development for current staff.

1. Do you have current priorities for training and professional development of staff who work directly with children ages 0-5 at your center?

1. What are the priorities?

2. Who set the priorities?

3. How were the priorities set?

4. Do these priorities differ across the staff? If so, in what ways? (for example, based on the age of the children they work with or the education level of staff)

2. In what ways, if any, do staff who work directly with children ages 0-5 have input into setting priorities for training and professional development?

1. Which staff contribute input?

2. How do staff contribute (for example, through surveys, focus groups, informal discussions)?

3. In what ways, if any, do you use information from the following sources to identify training needs and make decisions about training? [Interviewer: tailor based on response to question B.2.c from the pre-visit implementation call.]

• Information from the interview process

• Staff supervision and performance assessment

• Formal or informal classroom observations

• Regulatory or other requirements

• Other, specify

4. How do staff who work directly with children make a request to pursue training, professional development, or educational opportunities? (For example, is there a formal process or form or is it done informally?)

5. What information, if any, do you require from staff to track progress in or completion of a training or professional development activity?

6. When new training or development opportunities come to your attention what criteria, if any do you use to decide whether to pursue them and for which staff?

7. In the past 12 months, have you been able to address the training priorities and needs identified?

1. If some were not met, why not? Select all that apply

• Were you unable to find services/providers that meet your needs?

• Did you not have the funds to cover the costs of the needed training?

• Did you not have time to release staff for a particular activity?

• Other, specify

8. If resources are limited, how do you prioritize which needs you address?

C. Ongoing Training and Professional Development

We’d like to collect information about each training session, workshop, or other professional development activity that has taken place over the past 12 months for management, supervisory staff, and staff who work directly with children ages 0-5.

[Interviewer: Use information collected during pre-visit phone call on training to tailor this section as necessary. Collect information on all topics below for each training. ]

1. What was the main topic of the training?

MARK ALL THAT APPLY

1 □ Health and safety in the classroom

2 □ General child development (including cognitive development such as early reading or mathematics; social, emotional, and behavioral growth; behavior; and physical development and health)

3 □ Assessment of children’s development or progress monitoring

4 □ How to work with families

5 □ Serving children with special physical, emotional, or behavioral needs

6 □ Working with children who speak more than one language

7 □ Planning activities that meet the needs of the whole class

8 □ Learning about a specific curriculum

9 □ Leadership and management

10 □ Other (specify)

2. What was the impetus of the training?

MARK ALL THAT APPLY

1 □ In response to training plan

2 □ To respond to a program deficiency

3 □ To support a new initiative or practice

4 □ Ongoing skill development

5 □ Other (specify)

3. Which staff participated in the training?

MARK ONE ONLY

1 □ All staff

2 □ All administrators

3 □ All education specialists

4 □ All teaching staff

5 □ A subset of staff (specify)

4. What was the format?

MARK ONE ONLY

1 □ Group

2 □ Individual

3 □ Other (specify)

5. How was this training conducted?

1 □ Online

2 □ In-person, on-site

3 □ In-person, off-site

4 □ Online, interactive (such as via webinar)

5 □ Other (specify)

6. Who provided the training?

MARK ONE ONLY

1 □ Staff employed by the center

2 □ External trainer paid for by the center (specify)

3 □ External trainer provided without cost to the center (such as through Head Start), specify

4 □ Other (specify)

7. On what criteria was this individual or entity selected?

8. How many sessions were included in the training?

| | | NUMBER OF SESSIONS

9. What was the total number of training hours required?

| | | HOURS OF TRAINING

10. Has it been completed or is it ongoing?

MARK ONE ONLY

1 □ Completed

2 □ Ongoing

11. Did the training include any of the following?

MARK ALL THAT APPLY

1 □ Trainers discussed the theory, philosophy, and values behind the skills or practices

2 □ Trainers demonstrated key skills

3 □ Trainees created a plan for conducting the new practice in their classroom/program

4 □ Trainees participated in behavioral rehearsals to practice new skills

5 □ Trainees were observed using new skills in the classroom setting

6 □ Trainees were required to meet a specific goal or benchmark indicating skill knowledge or acquisition

7 □ Training was completed before the teacher was required to implement a new initiative or practice

12. What, if any, follow-up activities were conducted after the training?

List activities:

13. Is a pre- and post-test of knowledge and skills conducted in connection with the training?

1 □ Yes, please describe

0 □ No

14. Do you track any other indicators to gauge the success of training?

1 □ Yes (specify what is tracked)

0 □ No

15. Are staff asked to evaluate the training?

1 □ Yes, please describe

0 □ No

16. [If training included staff who work with children ages 0-5] Did staff who work with children ages 0-5 receive assistance with direct costs of training or education, such as tuition or registration fees?

1 □ Yes

0 □ No GO TO 17

16a. What is the source of the assistance?

MARK ONE ONLY

1 □ Center or affiliated organization

2 □ Local or state agency

3 □ Private funding source

4 □ College or university

5 □ Other (specify)

17. [If training included staff who work with children ages 0-5] Did staff who work with children ages 0-5 receive help with other costs of training or education, such as travel or child care?

1 □ Yes (specify)

0 □ No GO TO D

17a. What is the source of the assistance?

Select one

1 □ Center or affiliated organization

2 □ Local or state agency

3 □ Private funding source

4 □ College or university

5 □ Other (specify)

D. Supporting Training

1. Is a system in place that allows staff who work directly with children ages 0-5 to be released from time in their classrooms to participate in training or professional development activities? If yes, then ask the following questions:

1. Does the center arrange classroom coverage so that care can be provided to children while staff participate in these activities?

   1. Who provides coverage (for example, other staff members, substitutes who are contracted for this purpose)?

2. What expenditures, if any, are made by the center to provide this coverage?

   1. Are funds budgeted specifically for this purpose?

2. How have you used information from follow-up activities, tracking of indicators, or staff evaluations about the success of training?

1. Is information shared with coaches or supervisors to target competency development?

2. Is information used in selecting and hiring staff with the skills and competencies needed?

3. Is information used to adjust training priorities as a whole or plans for individual staff?

4. Does the information feed into the performance appraisal process for individual staff?

3. How do you determine whether the training your staff receives meets your program goals and their professional development needs?

E. Professional Activities

1. How many of each type of staff have a membership in professional associations focused on caring for children (such as the National Association for the Education of Young Children, National Head Start Association, a religiously identified child care organization, or a similar organization) among the following staff in your center?

	SELECT ONE RESPONSE PER ROW
	ALL	MOST	FEW	NONE	DON’T KNOW
a. Administrators	1 □	2 □	3 □	4 □	d □
b. Education specialists	1 □	2 □	3 □	4 □	d □
c. Lead teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
d. Assistant teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
e. Aides who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □

1a. Please list some of the professional associations of which staff are members:

1b. Does the center (or larger organization with which the center is affiliated) pay for these memberships?

1 □ Yes

0 □ No

2. How many of each type of staff have a membership in a union (such as Service Employees International Union, American Federation of Teachers, American Federation of State, County and Municipal Employees (AFSCME) or the Teamsters)?

	SELECT ONE RESPONSE PER ROW
	ALL	MOST	FEW	NONE	DON’T KNOW
a. Administrators	1 □	2 □	3 □	4 □	d □
b. Education specialists	1 □	2 □	3 □	4 □	d □
c. Lead teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
d. Assistant teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
e. Aides who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □

2a. Please list some of the unions of which staff are members:

3. Over the past 12 months, how frequently have the following staff attended a national-level meeting or conference of a professional organization (such as ZERO TO THREE, National Association for the Education of Young Children)?

	SELECT ONE RESPONSE PER ROW
	NEVER	ONE TIME	TWO TIMES	THREE OR MORE TIMES	DON’T KNOW
a. Administrators	1 □	2 □	3 □	4 □	d □
b. Education specialists	1 □	2 □	3 □	4 □	d □
c. Lead teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
d. Assistant teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
e. Aides who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □

3a. Please provide some examples of the organizations that sponsored these meetings:

3b. Were these meetings attended during work hours?

1 □ Yes

0 □ No

3c. Were staff compensated for their time and travel (when applicable)?

1 □ Yes

0 □ No

na □ Not applicable, staff attended voluntarily on their own time

4. Over the past 12 months, how frequently have the following staff attended a meeting or conference sponsored by a local or state entity that focused on any aspect of quality education and care for young children? [Note: this may include state or local-level meetings organized by chapters of national organizations]

	SELECT ONE RESPONSE PER ROW
	NEVER	ONE TIME	TWO TIMES	THREE OR MORE TIMES	DON’T KNOW
a. Administrators	1 □	2 □	3 □	4 □	d □
b. Education specialists	1 □	2 □	3 □	4 □	d □
c. Lead teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
d. Assistant teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
e. Aides who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □

4a. Please provide some examples of these meetings or conferences:

4b. Were these meetings attended during work hours?

1 □ Yes

0 □ No

4c. Were staff compensated for their time and travel (when applicable)?

MARK ONE ONLY

1 □ Yes

0 □ No

na □ Not applicable, staff attended voluntarily on their own time

5. Over the past 12 months, how frequently have the following staff attended a meeting of a local service or professional organization to discuss community issues or to network with colleagues in similar positions?

	SELECT ONE RESPONSE PER ROW
	NEVER	ONE TIME	TWO TIMES	THREE OR MORE TIMES	DON’T KNOW
a. Administrators	1 □	2 □	3 □	4 □	d □
b. Education specialists	1 □	2 □	3 □	4 □	d □
c. Lead teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
d. Assistant teachers who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □
e. Aides who work directly with children ages 0-5	1 □	2 □	3 □	4 □	d □

5a. Please provide some examples of these meetings or conferences:

5b. Were these meetings attended during work hours?

1 □ Yes

0 □ No

5c. Were staff compensated for their time and travel (when applicable)?

1 □ Yes

0 □ No

F. Cognitive Interviewing Questions for Module

1. Please tell me about your overall experience answering questions in the “Staff orientation and training” section.

2. [Ask only if perceived difficulty in responding]: I noticed that you paused when responding to one question in particular. The question reads as follows [repeat question]. Was this question difficult to understand? If so, why?

3. Were any other questions in this section not clear to you or difficult to respond to for some other reason?

1. If so, can you think of other ways that the question might be worded so that it is easier to understand or easier to respond to?

2. What additional information would you need to answer the question?