Assessing the Implementation and Cost of High Quality Early Care and Education: Comparative Multi-Case Study, Phase 1

Implementation interview protocol

selection & use of tools to support key functions MATHEMATICA POLICY RESEARCH

module 4: Selection and Use of Tools to support key functions

Now we’d like to discuss any resources or tools you may use to support certain activities in your center.

A. Center Administration

Let’s start with resources or tools you may use to assist with administration of the center.

1. Does your center have written information for staff about topics such as benefits or performance appraisal (for example, a staff handbook)?

1 □ Yes

0 □ No GO TO H2

1a. What does it include?

MARK all that apply

1 □ Expectations for staff (such as hours and conduct)

2 □ Benefits for staff (such as health insurance and paid time off)

3 □ Policies or procedures for staff development and performance appraisal’

4 □ Purpose or mission statement

5 □ Other (specify)

1b. How often is this information distributed to staff?

MARK all that apply

1 □ At time of hire

2 □ Annually

3 □ Made available upon request

4 □ Other (specify)

1c. How often is this information updated?

MARK ONE ONLY

1 □ Annually

2 □ Every other year

3 □ As needed

4 □ Other (specify)

2. [If answer to question above is “no”] In what ways, if any, is information about benefits or the performance appraisal process communicated to staff?

Probe on formal or informal methods of communicating to staff

3. Does your center have written operating procedures?

1 □ Yes

0 □ No GO TO H3

3a. What are the main topics included?

MARK ALL THAT APPLY

1 □ Hours of operation

2 □ Child to staff ratios and group size limits

3 □ Health and safety procedures

4 □ Other (specify)

3b. How often is this information distributed to staff?

MARK all that apply

1 □ At time of hire

2 □ Annually

3 □ Made available upon request

4 □ Other (specify)

3c. To which staff is information on operating procedures distributed?

MARK all that apply

1 □ Administrators / managers / supervisors

2 □ Staff who work directly with children ages 0-5

3 □ Specialists

4 □ Other (specify)

3d. How often is this information updated?

MARK ONE ONLY

1 □ Annually

2 □ Every other year

3 □ As needed

4 □ Other (specify)

4. How are staff in this center made aware of requirements or standards they must follow for: [include all that apply]

• Licensing

• Head Start Program Performance Standards

• As a state preK-funded program

• For participation in [insert QRIS name]

• As an accredited center

1. Is there written guidance? [If yes, ask a and b. If no, determine how they are made aware, then go to question 9.]

2. How are requirements or standards distributed to staff?

3. How often are requirements or standards distributed to staff?

5. What does your center do to monitor compliance with licensing?

1. How much of a concern is monitoring compliance and what challenges do you face?

2. Do you have the tools you need?

3. Is the monitoring process helpful to you? How is the information used for program administration?

6. Are there any other programs or funding sources not yet discussed for which the center must adhere to certain requirements or standards beyond licensing?

[If yes, then ask]

1. How long have these standards been in place in your center?

2. In what ways, if any, do the process or tools for monitoring compliance with these standards differ from those already discussed?

7. Are there any other specific reporting tools or processes staff in this center who work directly with children ages 0-5 must use to track or document compliance with requirements?

1. Name of tool

2. How long has tool been in place?

3. How often is the tool/process reviewed or updated?

C. Curriculum planning and child assessment

Now I’d like to discuss what processes or tools help staff who work directly with children in the classroom decide what activities to do and how.

Instructional planning and child assessment for infants/toddlers.

Let’s start by focusing on what is used in rooms with infants and toddlers (less than 36 months). [If the center does not serve infants or toddlers then skip to question C7.]

1. What information or resources, if any, are provided to staff who work directly with infants and toddlers (less than 36 months) to help them decide what activities to do with the children? [Use information from the pre-visit call to tailor this question.]

1. How were these resources developed?

2. How long have they been in place?

3. Are they used within all rooms with infants and toddlers? By all staff who work directly with infants and toddlers?

[If not used consistently across rooms or teachers] In what proportion of the rooms with infants and toddlers is this resource used? What are the reasons for the variation in use?

2. What, if any, curriculum is currently used for infants and toddlers (less than 36 months) in this center? [Tailor based on worksheet completed by the site in advance, if applicable.]

[Ask questions a – f below for each curriculum mentioned either in response to general question (#1 above) or as follow-up to known curriculum from worksheet.]

1. [If not used in all classrooms, ask] In what proportion of the rooms with infants and toddlers is this curriculum used? What are the reasons for the variation in use?

2. How was this curriculum selected (or developed internally) and by whom?

3. Who is trained on the curriculum? Select one and explain, if applicable

• All staff who work with infants and toddlers

• Just some staff, explain

4. How are staff trained on the curriculum and by whom?

5. Is training provided to keep people current on the curriculum?

[If yes] If there is ongoing training, how frequently does it occur?

Centers screen and assess children for a range of purposes, for example, screening for developmental delays or assessing children’s progress to help teachers individualize instruction and children’s experiences.

We are interested in learning about the full range of tools or processes used by staff to inform the care and instruction of infants and toddlers (less than 36 months) in your center.

3. What tools or processes, if any, do staff use to inform the care and instruction of infants and toddlers (children less than 36 months)? These can include anything used to conduct screening, monitoring, or assessment for a range of purposes.

[Tailor based on worksheet completed by the site in advance, if applicable. If not completed, ask the respondent to take a few minutes to complete the worksheet, and assist as needed]

[Ask questions a – f below for each curriculum mentioned either in response to general question (#1 above) or as follow-up to known tool from worksheet.]

1. [If not used in all classrooms, ask] In what proportion of the rooms with infants and toddlers is this tool used? What are the reasons for the variation in use?

2. How was this tool selected (or developed internally) and by whom?

3. Who is trained on the process or tool? Select one and explain, if applicable

• All staff who work with infants and toddlers

• Just some staff, explain

4. How are staff trained on the process or tool and by whom?

5. Is training provided to keep people current on the process or tool?

[If yes] If there is ongoing training, how frequently does it occur?

3. In what ways, if any, are staff who work with infants and toddlers supported in interpreting and using information they gather about children to guide what they do in the classroom or inform how they interact with children?

1. Do staff have access to written guidance or support materials?

2. Do staff work collaboratively with each other, or with someone else, to assess results and plan activities? [If yes, then ask questions below]

• Who works together?

• How frequently?

3. What other standardized tools, templates, or forms are used to help staff who work directly with infants and toddlers plan activities, or track and report on child progress? For example, are there written daily reports to parents, or other forms that provide information to parents about child activities or progress?

1. What is the name and purpose of the tool/template/form?

2. Who developed it?

3. Is it supported by an automated system?

4. [If applicable, based on answers to questions above about curriculum/assessment] Is it linked with the curriculum or child assessments in use?

5. For how long has it been used?

6. What is the frequency of its use?

7. Is it used within all rooms that serve infants and toddlers? If not, why not?

8. Is it used by all staff who work with infants and toddlers? If not, why not?

Instructional planning and child assessment for preschoolers

Now let’s talk about what is used in rooms with preschool children (ages 3-5). [If the center does not serve preschoolers then skip to the next section.]

REPEAT FULL SET OF QUESTIONS C1-C4 ABOVE FOR PRESCHOOL CHILDREN

6. What other standardized tools, templates, or forms are used to help staff who work directly with preschoolers plan activities, or track and report on child progress? For example, are there lesson plan templates or forms that provide information to parents about child activities or progress?

1. What is the name and purpose of the tool/template/form?

2. Who developed it?

3. Is it supported by an automated system?

4. [If applicable, based on answers to questions above about curriculum/assessment] Is it linked with the curriculum or child assessments?

5. For how long has it been used?

6. What is the frequency of its use?

7. Is it used within all rooms that serve preschoolers? If not, why not?

8. Is it used by all staff who work with preschoolers? If not, why not?

Development screening and assessment

6. [Based on answers provided above, determine if development screening and assessments are conducted for children ages 0-5. Ask the following questions, as needed, for clarity.] What process, if any, do you have in place to identify children with development delays and connect them with the necessary services?

1. Do you conduct development screening in the center? If not, are children referred somewhere else for screening?

2. Do you conduct development assessments if a child screens positive for a possible delay, or do you make a referral for further assessment?

3. When a child is determined to have a development delay, what role, if any, does the center play in connecting or providing him or her with the necessary services?

Planning and coordination

6. How do staff who work directly with children ages 0-5 work together or with other staff (such as specialists or managers) to plan or coordinate instruction or other classroom activities focused on quality improvement?

1. On what topics do staff coordinate?

2. Who is involved in these interactions?

3. What is the format of this coordination?

1. Is there planned group meeting time for specific staff?

2. Does it occur on more of an ad-hoc basis?

3. Is there an automated system that supports planning and coordination?

4. With what frequency does this occur?

5. Does this type of coordination occur consistently across staff who work with children ages 0-5, or does it vary (for example, based on the age of children served or type of staff such as lead teachers or aides)? Please explain.

6. Do staff who work directly with children ages 0-5 get paid planning time, when teachers are not responsible for children?

1. Is this provided to all staff or only some staff? Please explain differences.

2. Approximately how much time is provided? Specify for different staff if it varies.

D. Child and family support

Next, let’s discuss tools used to provide children and their families with supports or services.

[Ask the questions below for each type of service recorded in pre-visit call. Or, ask the respondent to take a few minutes to complete the worksheet and provide assistance as needed]

1. What is the process for connecting and providing children with this support or service?

1. Are teaching staff trained to identify, connect, or provide children with this service or are specialized staff involved in the process?

2. Are specific tools used to identify, connect, or provide children with this service (for example, electronic systems or hard copy forms)?

2. How is information about the support or service provided to children used to inform classroom practice or other activities at the center?

3. Is the support or service available to all children or is it based on some criteria

MARK ONE ONLY

1 □ Available to all children

2 □ Based on age

3 □ Based on family income

4 □ Based on specific screener

5 □ Based on some other criteria (specify)

4. What proportion of infants and toddlers (children less than 36 months) received this support or service in the past 12 months?

| | | | %

5. What proportion of preschool children (ages 3-5) received this support or service in the past 12 months?

| | | | %

6. Who provides this service?

MARK all that apply

1 □ Staff employed by the center

2 □ External consultants paid for by the center

3 □ External consultants provided without cost to the center (such as through Head Start)

4 □ Other (specify)

7. [If service is provide at the center] How long has this support or service been provided at this center?

| | | ENTER NUMBER

SPECIFY TIME PERIOD

1 □ Weeks

2 □ Months

3 □ Years

8. [If service is provided at the center] How is the support or service for children tracked?

MARK ONE ONLY

1 □ An electronic system

2 □ Specific form

3 □ Some other way (specify)

9. [If the service is referred out] Is there a mechanism to track whether the connection to the service was made and the child and family receive the support or service?

1 □ Yes

0 □ No

10. [If the service is referred out] How is the support or service for children tracked?

MARK ONE ONLY

1 □ An electronic system

2 □ Specific form

3 □ Some other way (specify)

E. Information systems and use of technology

1. Are there electronic systems or technology that you use in this center to support program operations or the care of young children beyond those we have already discussed (such as for curriculum, assessment, or planning)?

Cognitive Interviewing Questions for Module

1. Please tell me about your overall experience answering questions in the “Selection and Use of Tools to Support Key Functions” section.

2. [Ask only if perceived difficulty in responding]: I noticed that you paused when responding to one question in particular. The question reads as follows [repeat question]. Was this question difficult to understand? If so, why?

3. Were any other questions in this section not clear to you or difficult to respond to for some other reason?

1. If so, can you think of other ways that the question might be worded so that it is easier to understand or easier to respond to?

2. What additional information would you need to answer the question?

13