Study of the Turnaround Schoool Leaders Program

Contract Number: ED-PEP-11-O-0088/TO28

Task Order 28

Implementation Study of the
Turnaround School Leaders Program

Study Instruments

Survey Instrument

Appendix C. Training Partner Survey

September 28, 2016

Prepared for:

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

Prepared by:

Westat

1600 Research Boulevard

Rockville, MD 20850-3129

(301) 251-1500

and

Policy Studies Associates

1718 Connecticut Avenue, NW, Suite 400

Washington, DC 20009

(202) 939-9780

Survey

Introduction

The Implementation of the Turnaround School Leaders Program study (TSLP) is examining how grantees recruit, develop, support, and retain leaders for turnaround schools. This study is being conducted by Westat and its partner, Policy Studies Associates, for the United States (U.S.) Department of Education, Policy and Program Studies Service.

This survey is part of the study. Your responses are critical to drawing lessons about the implementation of the Turnaround School Leaders Program.

The survey includes four sections:

1. The roles project partners play in implementing projects and achieving project goals.

2. How grantees are adapting to challenges in implementing their projects.

3. How projects recruit, select, place, develop, and support leadership candidates

4. How grantees are measuring the success of their TSLP projects.

Responses to this data collection will be used only for research purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific individual. We will not provide information that identifies you to anyone outside the study team, except as required by law.

Participation in this study is required under the conditions of the Turnaround School Leadership grant program (34 CFR 75.591). According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is: XXXX-XXXX. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.

1. Partner Roles

I-1. Which of the following roles does your organization play in the project?

Project function	Role
	Not involved	Participate in discussions about this function	Assist other partners with primary responsibility	Share responsibility for this function equally with other partners	Primarily responsible for function with help from others	Solely responsible for this function
Conceptualizing the overall project
Identifying turnaround leadership skills or competencies
Recruiting leaders or aspiring leaders
Developing selection criteria for accepting applicants
Making the decision to select an applicant into the turnaround leader training
Developing training content
Delivering training
Providing coaching or mentoring to project participants
Developing criteria for assessing participant performance
Evaluating participant performance
Deciding who will be graduated from the training
Providing ongoing support for placed leaders
Deciding how to measure the success of the TSLP project
Evaluating the overall success of the TSLP project

I-2. Is there a memorandum of understanding (MOU) or other written agreement documenting the roles and responsibilities of each partner organization?

• Yes

• No

• Other ___________________________________

I-3. What were your two biggest successes in making your partnership work?

Text box for open-ended response

I-4. What were your two biggest challenges in making your partnership work?

Text box for open-ended response

2. Project Changes and Sustainability

II-1. Has your TSLP project changed in any of the following areas since you received the grant?

1. Recruitment of current or potential leaders

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
a.1 Developed better methods
a.2 Change in state or district policies or priorities
a.3 Changes in capacity or commitment of other partners
a.4 Input from program participants
a.5 Financial constraints
a.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
a.7 Other (please describe) _____________________

2. Methods of selecting project participants

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
b.1 Developed better methods
b.2 Change in state or district policies or priorities
b.3 Changes in capacity or commitment of other partners
b.4 Input from program participants
b.5 Financial constraints
b.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
b.7 Other (please describe) _____________________

3. Content of turnaround leadership professional development

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
c.1 Developed better methods
c.2 Change in state or district policies or priorities
c.3 Changes in capacity or commitment of other partners
c.4 Input from program participants
c.5 Financial constraints
c.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
c.7 Other (please describe) _____________________

4. Mentoring or coaching provided to current or potential leaders

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
d.1 Developed better methods
d.2 Change in state or district policies or priorities
d.3 Changes in capacity or commitment of other partners
d.4 Input from program participants
d.5 Financial constraints
d.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
d.7 Other (please describe) ____________________

5. Incentives for current or potential leaders to participate

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
e.1 Developed better methods
e.2 Change in state or district policies or priorities
e.3 Changes in capacity or commitment of other partners
e.4 Input from program participants
e.5 Financial constraints
e.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
e.7 Other (please describe) ____________________

6. Change in the population of schools being served

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
f.1 Developed better methods
f.2 Change in state or district policies or priorities
f.3 Changes in capacity or commitment of other partners
f.4 Input from program participants
f.5 Financial constraints
f.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
f.7 Other (please describe) _____________________

7. Changing professional development or support providers

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
g.1 Developed better methods
g.2 Change in state or district policies or priorities
g.3 Changes in capacity or commitment of other partners
g.4 Input from program participants
g.5 Financial constraints
g.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
g.7 Other (please describe) _____________________

h. Use the text box below to describe any other major changes not covered above and the reasons the changes were made.

Text box for open-ended response

II-2. Do you expect that your organization will continue to work with the partner districts on preparing turnaround school leaders after the TSLP grant expires?

• Yes

• No (skip to question III-1)

II-3. Do you expect the partners to continue to work together in any of the following ways? (Please check all that apply)

• To identify or recruit potential turnaround leaders

• To provide input on the training/preparation needs of turnaround school leaders

• To select or place leaders for turnaround schools

• To provide training or development opportunities specifically targeted to turnaround school leaders

• To provide coaching or mentoring support to turnaround school leaders

• To help districts assess the training/development needs or performance of turnaround school leaders

• Other (Please describe) _________________________________________________

3. Recruitment, Selection, Development, Placement, Evaluation, Support, and Retention

III-1. Does your organization do any of the following to identify potential candidates for turnaround school leadership before they become school leaders or apply to participate in your TSLP project?

• Seek referrals from peers or existing school leaders

• Collect and analyze information on prior performance of existing staff

• Develop or review skills inventories

• Obtain lists of those currently in teacher leader roles or other positions along the leadership pipeline

• Use historical data patterns to identify the initial career paths of successful school leaders

• Use historical data patterns to identify where successful turnaround leaders were recruited in the past

• Other __________________________________________________________

III-2. About how much training time does your project allocate to each of the following topics when preparing TSLP participants?

Topic	Much less than most other topics	Somewhat less than most other topics	About the same as most other topics	Somewhat more than most other topics	Much more than most other topics
Developing/communicating a school vision or strategy
Developing a positive school culture
Teacher recruitment, selection, retention
Monitoring/evaluating instruction
Selecting instructional models or curricula
Culturally relevant/responsive pedagogy
Use of data for instructional and project decisions
Parent communication/engagement
Community engagement/partnership
Communication/interpersonal skills
Student behavior/engagement
Resource allocation and budgeting
Technology use
Other______________________________
Other___________________________

III-3. Does your organization provide participants with a mentor or coach during the TSLP training?

• Yes

• No (skip to III-5)

III-4. About how often do mentors or coaches:

Activity	Not expected	Weekly	Twice a month	Monthly	Several times per year	Once per year
Maintain regular contact with the participants?
Provide social/emotional support?
Suggest resources, strategies, or techniques to improve performance?
Observe the leaders performing leadership activities (e.g., conducting staff meetings, parent meetings, teacher evaluations)?
Advise/assist participants with work assignments?
Model appropriate leadership behavior?
Provide input into a summative evaluation of the participants’ performance?

III-5. How often are the following learning practices/instructional strategies used as part of the coursework in your TSLP project’s leadership training?

Practice/strategy	Not used	Used once	Used 2 or 3 times	Used 4 times or more
Field-based projects in which aspiring turnaround leaders apply ideas in the field
Analysis and discussion of field-based problems or problem-based- learning approaches
Action research or inquiry projects
Analysis and discussion of case studies
A portfolio demonstrating aspiring turnaround leaders’ learning and accomplishments

III-6. Does the coursework in your TSLP project’s leadership training include a capstone or culminating project?

1. Required

2. Optional

3. Not included

4. Other (Specify_________________________)

III-7. To evaluate participants, does your organization use: (Select NA in the appropriate column if your project has no participants to whom the column applies.)

	For participants who are currently school leaders	For participants who are not currently school leaders
	Yes No NA	Yes No NA
Course grades?
Observation of practice by mentor, supervisor, or project staff?
Self-assessment?
Review of artifacts such as school improvement plans, budgets, or meeting agendas?
Student surveys?
Staff surveys?
Parent surveys?
Student achievement growth?
Other student outcomes?
Other ______________________________

III-8. What roles does your organization play in deciding where turnaround leaders who complete the TSLP training are placed? (Check all that apply)

• Not involved in the placement decision <Skip to question III-11>

• Screen or assess qualifications for leading a turnaround school

• Participate in deliberations about candidate fit

• Recommend hiring or placement

• Make final hiring/placement decision (subject to district governing board or committee approval)

III-9. When are aspiring turnaround leaders (those not currently leading a SIG, SIG-eligible, or turnaround school) matched with a leadership position in a turnaround school?

• Not applicable; our project includes only leaders who currently lead turnaround schools <Skip to question III-11>

• Before they apply to the project (e.g., applicant or sponsor designates school of placement)

• At the time they apply to or are accepted into the turnaround leader training

• During their training

• After the completion of their training

• Other (Please describe in text box below.)

III-10. In general, how important is each of the following when making placement decisions?

	Not important	Somewhat important	Important	Very important
Prior experience in the potential placement school
Prior experience in a similar school
Performance during the development project
Prior experience in the placement district
Experience/expertise with specific school improvement/turnaround strategies
Other forms of leadership experience
Other ___________________________
Other ___________________________

Evaluation of Turnaround Leaders

III-11. To evaluate participants, does your organization use: (Select NA in the appropriate column if your project has no participants to whom the column applies.)

	For participants who are currently school leaders	For participants who are not currently school leaders
	Yes No NA	Yes No NA
Course grades?
Observation of practice by mentor, supervisor, or project staff?
Self-assessment?
Review of artifacts such as school improvement plans, budgets, or meeting agendas?
Student surveys?
Staff surveys?
Parent surveys?
Student achievement growth?
Other student outcomes?
Other ______________________________

Ongoing Support for Leaders

III-12. After completion of the turnaround leader training, does your organization provide any of the following supports to participants placed as turnaround leaders?

	Yes	No
Periodic group meetings or seminars with other turnaround leaders
Assistance with work products like school improvement plans, budgets
Assistance with individual professional development planning
Individual mentoring or coaching

<If respondent selects “Yes” to III-12d, present questions III-13 and III-14; if respondent selects “No” to III-12d, skip to question IV-1>.

III-13. How long after completing the turnaround leader training is the mentor or coach available to work with the placed or aspiring leaders?

• Less than 1 year

• 1 year

• 2 years

• More than 2 years

• As long as the leader remains in a turnaround school

III-14. About how often do mentors or coaches:

Activity	Not expected	Weekly	Twice a month	Monthly	Several times per year	Once per year
Maintain regular contact with the participants?
Provide social/emotional support?
Suggest resources, strategies, or techniques to improve performance?
Observe the leaders performing leadership activities (e.g., conducting staff meetings, parent meetings, teacher evaluations)?
Advise/assist participants with work assignments?
Model appropriate leadership behavior?
Provide input into a summative evaluation of the participants’ performance?

4. Measuring the Success of Your Project

IV-1. In addition to your project’s GPRA Indicators, is your organization using any locally-developed metrics to measure the project’s performance?

• Yes

• No (Skip to the end of the survey)

IV-2. Which of the following are included in your locally developed metrics?

	Yes	No
Percentage of participants receiving license, certification, endorsement, or degree
Performance of leaders on competency assessments or performance evaluation instruments designed for the project
Performance of placed leaders as evaluated by state/district school administrator evaluation process (i.e., ratings of performance or effectiveness)
Participants’ perceptions of the project
Supervisors’ perceptions of participants
School climate/culture in leaders’ schools
Achievement gaps in leaders’ schools
Retention of participants remaining as a leader of a high-need or SIG school after training or placement
Other _______________________________________________
Other ________________________________________________