In-Depth Project Team Interview

Appendix III
In-Depth Interview for Evaluation Team

In-Depth Interview
Evaluation Team (59 minutes)
May 9, 2007
Paperwork Burden Statement
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information unless such collection displays a valid OMB control number. The valid OMB control number for this
information collection is 1875-NEW. The time required to complete this information collection is estimated to
average 90 minutes per response, including the time to review instructions, search existing data resources, gather
the data needed, and complete and review the information collection. If you have any comments concerning
the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S.
Department of Education, Washington, D.C. 20202-4700. If you have comments or concerns regarding the
status of your individual submission of this form, write directly to: Policy and Program Studies Service, U.S.
Department of Education, 400 Maryland Avenue, SW, Room 6W259, Washington, D.C. 20202.

Introduction (2 minutes)
Thank you for taking the time to talk with me. My name is ________ and I work for
Child Trends, an independent research organization. We are conducting a study for the
U.S. Department of Education to understand better how the Early Childhood Educator
Professional Development Program (ECEPD) projects are functioning in the field. We
have already learned a great deal about your ECEPD program from an interview with the
Project Director of your program, ___________________. We asked your Project
Director if we could do a follow-up interview with key members of the evaluation team
so that we could go into greater detail on certain aspects of the evaluation. S/he gave us
your names and contact information as key members of the team.
I want to assure you that the information you provide today will be kept anonymous.
Specific comments will not be attributed to individuals or programs, nor will we share
your comments with anybody outside the research team. We do, however, need to
caution that because of the small number of ECEPD sites (18 in total), it is possible that
even without attributing information to individuals or sites, some people who are very
familiar with the ECEPD programs will be able to identify the information you give us as
related to your site. The information we gather from the ECEPD programs across the
country will be used to write a report for the U.S. Department of Education. Our report
will describe the range of program and evaluation activities being implemented through
the ECEPD grants.
We would like to record our interview with you so that we can transcribe your responses
and capture accurately the information you share with us. Do we have your permission to
record this interview? [Interviewer: Confirm respondent’s permission]
Do you have any questions before we get started?
Evaluation Team Role and Structure (10 minutes)

In-depth Interview – Evaluation Team

1

1. Please could you each identify yourselves and indicate your roles on the evaluation
team?
2. Are there others involved in the evaluation? If so, what are their roles?
3. How are the activities of the team coordinated?
Probe: Is there a leader to the team, or set of leaders for different components of the
evaluation?
Probe: Is there a regular schedule for meetings or communication, or does that happen on
an as-needed basis?
4. What attempts, if any, are made to coordinate the activities of the evaluation team
with the activities of those implementing your ECEPD program?
Research Design and Analyses (10 minutes)
5. We have read the written reports of your evaluation that have been sent to the U.S.
Department of Education. Since [insert date of last report], have there been any
changes to the research questions, the research design, the measures you are using, or
analyses you are carrying out since the last report?
Probe: Please could you describe these to us?
Probe: If possible, please could you tell us the reasons you decided to make these
changes?
6. ECEPD program evaluations are using various methods to track fidelity of
implementation of the activities, curricula, or techniques taught as part of their
ECEPD programs. Could you please tell us if you are using any of the following
methods to track fidelity?
Do you perform classroom/center observations?
Do you conduct a survey of staff?
Do you conduct a survey of parents?
Do you conduct assessments of the children?
Do you conduct performance reviews?
Is there oversight of the training staff?
Do participants keep written records of their progress?

In-depth Interview – Evaluation Team

Yes
Yes
Yes
Yes
Yes
Yes
Yes

No
No
No
No
No
No
No

2

[For any items checked yes, please probe to learn more about how fidelity was tracked
using this method]
7. What are the key results from the fidelity analyses you have conducted?
Outcomes for Early Childhood Educators (6 minutes)
8. What, if anything, has been most surprising to you about the results you’ve been
finding in terms of educator knowledge and skills?
Outcomes for the Children of Early Childhood Educators Participating in the
ECEPD Program (6 minutes)
9. What, if anything, has been most surprising to you about the results you’ve been
finding in terms of child outcomes?

10. What aspects of the evaluation have been most challenging?
Next Steps in the Evaluation (15 minutes)
11. What are the next steps for your evaluation?
12. What advice would you give to other ECEPD teams as they design and conduct their
evaluations?
13. If there were no constraints on evaluation design or methods, what aspects of the
evaluation, if any, would you want to change?
Promising Practices (10 minutes)
14. Given everything you have learned thus far from the ECEPD project you evaluated,
what would you say are the professional development strategies that are most likely
to improve early childhood educator knowledge and practice as well as child
outcomes?
15. What patterns in the findings, if any, would you describe as promising early
indications or trends that you want to look at more closely over time?

In-depth Interview – Evaluation Team

3

Thank you very much for completing this interview and helping us to understand
your ECEPD program evaluation in greater depth!

In-depth Interview – Evaluation Team

4