District staff interviews

National Study of Education Data Systems and Decision Making

District Interview Protocol

Introduction

The following is a suggested introduction. When you begin the interview, make these points, but use your own words for a more personal introduction:

The National Study of Educational Data Systems and Decision Making is a large-scale project sponsored by the US Department of Education to look at the ways that states, districts and schools are using data in their educational decision making. The project looks at the prevalence and quality of both the data systems that are in use nationwide and the supports in place for educational decision makers to use them. In particular, we are interested to learn about how teachers are using data from these systems to make decisions about their instruction, and what types of professional development and support teachers are receiving to learn about the systems and their applications.

We want you to know that this project does not evaluate you or your district; we’re looking to learn from your experiences to get a better understanding of how districts across the country use data systems.

Before we start, I need to have you review and sign this consent form.

AFTER THEY REVIEW CONSENT FORM: Do you have any questions before we get started?

NOTE: It may be necessary to ask these questions of several district staff. For example, the district technology coordinator may be the person to talk to about the data system, while you’ll need to talk to a district professional development coordinator to obtain answers to questions about the training on DDDM.

Background

1. Could you tell me a little about your role here at the district overall and how it relates to supporting schools’ use of data to improve instruction?

District Context and Goals for DDDM

This section asks about the district’s goals for integrating data into instruction.

2. How did your district first become involved in data-driven decision making to support teaching and learning? What are the district goals for using data to support teaching and learning? Has the district adopted a comprehensive plan to implement data-driven decision making?

3. What are the specific features of the district’s data-driven decision making activities that you are involved with (e.g., professional development, regular data meetings, data driven school improvement plans)?

4. Have your district’s goals around data-driven decision making been impacted by NCLB? (If yes) What changes were made? Did your district maintain a student-level data system prior to the NCLB accountability requirements?

5. To what extent would you say your district’s use of data systems is in support of NCLB accountability requirements versus other goals for improving instruction?

6. Has the district received any support around the implementation of data-driven decision making (e.g., from state-sponsored regional technical assistance staff, vendors)?

Data System

The purpose of this section is to learn more about the primary data system used in this district and its interoperability with other systems.

7. Are all schools in the district using or supposed to be using [name of district’s primary data system]?

[If NO] What other systems, if any, are schools using? Can data be moved efficiently between these other systems and [district system name]?

8. What types of district data are housed in [name of data system]?

Check “yes” or “no” for the following types of information:

Type of Information Does the system have…	Yes	No
a. Student demographic data?
b. Student standardized test score data?
c. Student attendance data?
d. Student grades?
e. Program participation (including special education) information?
f. Student performance on district benchmark tests?
g. Student course enrollment histories?
h. Samples of student work?
i. Teacher certification/education data?
j. Teacher professional development data?
k. Model lessons/instructional resources?
l. Other? (describe):

9. Who has access to some or all of the data in the district system?

Write “yes” or “no” for whether the following people have access. If there are limits on access, please describe in the appropriate column.

Stakeholder Do [ ] have access to the system?	Has Access	Limits on Access (describe)
a. School administrators
b. Teachers
c. District technical staff (eval/assess/accountability staff)
d. District policymakers
e. Researchers
f. Parents
g. General public
h. Other:

10. In your view, what are the barriers to system use by teachers and non-technical district staff us? How could access and usability be improved?

11. Is there an efficient process for exchanging data between [name of district system] and your state’s student information system? Can you describe the process of data exchange between the two systems?

12. Are you confident in the accuracy of your district’s data in [system name]? Is there a process in place to ensure data accuracy?

13. Have you had problems with discrepancies in data? What measures were taken to deal with these kinds of discrepancies?

Support for DDDM

This section deals with the level of support the district provides for data-driven decision making activities in its schools.

14) What staff does the district have dedicated to supporting data-driven decision making at the district level?

1. Technical staff who handle questions and problems with [name of district data system]?

2. Instructional support for teachers to learn how to use data from [name of data system] in their instructional practice.

15) What resources (funds, staff, equipment) have been allocated to schools in this district to meet the increasing need for data collection, entry and reporting?

16) What is your district doing to increase teachers’ skills in data-driven decision making to support improved instructional practice? [Probe on] What kinds of professional development on data-driven decision making or training to use [name of data system] has your district provided teachers in recent years? Are you satisfied with this level of professional development and, if not, what would you like to do differently? How do you assess teachers’ professional development needs for integrating data-driven decision making in their teaching practice? Who makes the decision about what and how much professional development to offer on data-driven decision making?

17) Do you feel that teachers in this district have any unmet needs with regard to professional development in the use of data for instructional purposes? If so, what are they?

a. What are the major barriers to these needs being met?

18) How much and what kind of professional development is provided to school non-teaching staff responsible for data entry, including during the summer and any on-line offerings?

19) What has been your experience been with respect to costs for developing a data system and implementing data-driven decision making (e.g., what are the main cost elements, soft costs such as time and additional training)? Any unanticipated costs?

20) What have been the biggest challenges as you have worked to implement the data-driven decision making initiative? [Probe on past and current challenges] What factors have facilitated the implementation of your data-driven decision making initiative? [Probe on factors at the district and school level]

21) Has your district developed benchmarks to track the progress and report results of the implementation of the [system name] and data use practices?

District Data Use

In this section we are looking for details about how teachers and administrators are using data in the district.

22. How often do district staff use data from [system name] for each of these purposes (which features account for the most use)?

Choose one for each activity.

Activity	Multiple times a year	At least once a year	Rarely or Never
a. Track student test scores by school
b. Looking at student test scores for subgroups across schools.
c. Tracking student graduation rates by school.
d. Tracking student promotion rates by subgroup across schools.
e. Tracking other key measures of student progress (describe).
f. Inform student placement in courses or special programs.
g. Inform parents about student progress.
h. Inform curriculum changes.
i. Evaluate promising classroom practices.
j. Decide what professional development to offer
k. Inform resource allocation decisions (describe)
l. Evaluate teacher performance
m. Other: (describe)

23) Which features of the [system name] account for the most use by school staff?

24) Do you find variation among schools in the level of data use/integration with instruction? In your experience what school factors influence this variation?

25) What do you feel has been the impact of teachers’ data use on students’ educational experiences in this district?

26) Do you believe that faculty have benefited in any way from participation in data-driven decision making activities? [IF YES] How and to what do you attribute this? [IF NO] Why not?

Closing

27) What advise would you have for other districts who want to use data to improve instruction?

28) Are there any other things you would like us to know about your work in the district with DDDM that we have not already covered?

Before you leave, be sure to ask for copies of any documents such as student handbooks, curriculum frameworks, standards, teacher professional development materials, and school improvement and accountability materials.

D-5