Alternative student outcomes for growth measures case studies

OMB Supporting Statement Part B: Alternative Student Growth Measures for Teacher Evaluation: Case Studies of Early Adopters

August 6, 2013

Appendix C: District Recruitment Conversation Guide

This document provides guidance on how to structure initial telephone calls with districts to recruit their participation in the study. Prior to the initial call, districts will receive the notification letter and a study summary.

Introduction

My name is ____________, and I’m calling on behalf of the Regional Educational Laboratory (REL) for the Mid-Atlantic Region, which is funded by the U.S. Department of Education (ED). Your district has been selected to participate in a national study that will examine the use of alternative measures of student growth—including Student Learning Objectives (SLOs) and either end-of-course exams or nationally normed tests applied in growth models—to assess teacher performance. You are one of nine districts across the country selected for this study. The study will provide valuable information about the implementation of student growth measures that do not rely on state assessments. My organization, Mathematica Policy Research, is conducting this study for REL Mid-Atlantic. A letter inviting your district to participate in the study was recently sent to you and the superintendent.

If you have 15 minutes I would like to discuss the study and the possible participation of [District Name], or we can schedule another time that is convenient for you.

If need to reschedule: What is a date and time that work well for you?

If agree to talk now, continue with conversation guide.

Study Overview

We are hoping that [District Name] will participate in this study, which will examine the implementation of alternative measures of student growth—that is, SLOs and either end-of-course exams or nationally normed tests applied in growth models—to assess teacher performance. Your participation would help us understand a high priority issue for policymakers and districts in the Mid-Atlantic Region and throughout the country: whether and how measures of student growth that do not use state standardized assessments can be effectively and efficiently implemented to assess teacher performance. Many states are beginning to require the use of measures of student growth in teacher evaluation systems. These alternative measures can be applied to teachers in subjects and grades not covered by state assessments, but there is currently little information available to help inform districts interested in implementing them. Ultimately, the U.S. Department of Education hopes that the study will provide valuable information on the features and uses of these measures, the process of implementation, and the challenges encountered during implementing.

[District Name] is among a select group of districts invited by the U.S. Department of Education to participate in the study. [District Name] is a good fit for this study because of its experience implementing [insert alternative measures of student growth currently in use].

Do you think that [District Name] might be interested in participating?

We know that many districts are operating under tight budget constraints—so this study is designed to be low burden for your staff. We will conduct interviews with district and school staff over a short time period and can be flexible in conducting either telephone or onsite interviews. The study will not require any interaction with students. Participation in the study is voluntary.

Do you have any questions about the study at this point?

It might be helpful to describe how the study works in more detail. This study aims to fill the gap in information available to districts and policymakers on measures of student growth that do not use state standardized tests via qualitative case studies of up to nine districts that are using alternative measures of student achievement growth in teacher performance ratings. The study’s scope will encompass three categories of alternative student growth measures: (1) end-of-course curriculum-based assessments used in growth models, (2) nationally normed assessments such as the Iowa Test of Basic Skills used in growth models, and (3) Student Learning Objectives (SLOs). We will examine what alternative outcome measures are used, how the alternative growth measures are implemented, challenges and obstacles in implementation, how the measures are being used, and, where possible, the distribution of teacher performance on the measures, as compared with the distribution of teacher performance on conventional value-added measures that are based on state assessments. Districts participating in the study will not be identified in published reports.

Study Benefits

Through their participation in the study, districts can make an important contribution to our understanding of alternative measures of student growth as tools for measuring teacher performance in multiple contexts. By providing information on the implementation process, the effectiveness of the measures in differentiating teacher performance, and the perceived costs and benefits of the measures from stakeholder perspectives, this study has the potential to inform states and districts in the REL Mid-Atlantic region (and throughout the country) in deciding which measures are promising for use in evaluation, which are of doubtful value, and how to move forward with implementation.

Do you have any questions about the study?

Study Requirements

Participation in the study involves assisting the study team with scheduling a site visit and/or a series of telephone interviews with school and district staff in the fall of the 2013-14 school year. The study team will conduct one-on-one interviews with up to ten staff members—including at least one district administrator, two or three principals, two or three teachers, and one teachers’ union/association representative. During these interviews, the study team will gather information on district- and school-level implementation of alternative growth measures, applications of the measures, the distribution of teacher performance on the measures, and the perceived costs and benefits of the measures from the perspectives of those interviewed.

Next Steps

Do you have any questions about the study? Do we need to include anyone else in discussions about the study? [Identify relevant district staff and schedule follow-up call as needed.]

Who would be responsible for approving the district’s participation in the study?

OMB Package: Alternative Student Outcomes for Growth Measures Case Studies 1