Study of the Turnaround Schoool Leaders Program

Contract Number: ED-PEP-11-O-0088/TO28

Task Order 28

Implementation Study of the
Turnaround School Leaders Program

Study Instruments

Survey Instrument

Appendix A. Project Director Survey

September 28, 2016

Prepared for:

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

Prepared by:

Westat

1600 Research Boulevard

Rockville, MD 20850-3129

(301) 251-1500

and

Policy Studies Associates

1718 Connecticut Avenue, NW, Suite 400

Washington, DC 20009

(202) 939-9780

Survey

Introduction

The Implementation of the Turnaround School Leaders Program study (TSLP) is examining how grantees recruit, develop, support, and retain leaders for turnaround schools. This study is being conducted by Westat and its partner, Policy Studies Associates, for the United States (U.S.) Department of Education, Policy and Program Studies Service.

This survey is part of the study. Your responses are critical to drawing lessons about the implementation of the Turnaround School Leaders Program.

The survey includes four sections:

1. How programs recruit, select, place, develop, and support leadership candidates.

2. The roles project partners play in implementing projects and helping grantees achieve project goals.

3. How grantees are adapting to challenges in implementing their projects.

4. How grantees are measuring the success of their TSLP projects.

Responses to this data collection will be used only for research purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific individual. We will not provide information that identifies you to anyone outside the study team, except as required by law.

Participation in this study is required under the conditions of the Turnaround School Leadership grant program (34 CFR 75.591). According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is: XXXX-XXXX. The time required to complete this information collection is estimated to average 60 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.

1. Recruitment, selection, development, placement, evaluation, support, and retention

Recruitment

I-1. Does your project do any of the following to identify potential candidates for turnaround school leadership before they become school leaders or apply to participate in your TSLP project? (Check all that apply.)

• Seek referrals from peers or existing school leaders

• Collect and analyze information on prior performance of existing staff

• Develop or review skills inventories

• Obtain lists of those currently in teacher leader roles or other positions along the leadership pipeline

• Use historical data patterns to identify the initial career paths of successful school leaders

• Use historical data patterns to identify where successful turnaround leaders were recruited in the past

• Other __________________________________________________________

I-2. To determine whether a potential candidate is eligible for participation in your TSLP project, do you consider:

	For those who were not school leaders before participation		For those who were school leaders before participation
	Yes	No	Yes	No
Teaching experience?
Teacher leadership experience?
School administrator experience?
Existing administrator credential or certification?
Employment in partner district?
Employment in SIG school?
Level of performance (e.g., effectiveness) in current position?

I-3. To recruit potential turnaround leaders, does your project use:

	Yes	No
Job postings by district/partner districts?
Referrals/nominations?
Job fairs?
Recruitment alliances?
District website(s)?
Project website(s)?
University placement offices/services ?
Development provider website?
Advertising?
Other ____________________________________________

I-4. How important was each of the following in identifying the turnaround leader competencies your project seeks to develop?

	Not important	Somewhat Important	Important	Very Important
Research on effective school leadership
National or state leadership standards
Activities or strategies from SIG school turnaround models
Specific district school turnaround programs or strategies
Leadership models of training providers
District leadership models
Other ___________________________

I-5. To assess leadership candidates, did your project use:

	Yes	No
One-on-one interviews?
Panel interview?
Personality tests?
Situational judgment tests?
Resume reviews?
Role plays?
Written exercises or reflections?
Leadership ability questionnaires or surveys?
Observations or video clips of observable performance?
Review of written examples of work?
Other ___________________________________

Development

I-6. Using the table below, enter the number of potential participants in your project who were not leaders in turnaround schools before beginning the project. If you did not begin a cohort for one of the time periods listed, place a check in the first column.

Cohort beginning	Not applicable	Applied	Accepted	Began program	Completed program	Was placed as a turnaround school leader within 1 year of project completion	Remained leader of a turnaround beyond the initial year of placement
Fall 2014
Spring 2015
Summer 2015
Fall 2015
Spring 2016
Summer 2016
Fall 2016

I-7. Using the table below, enter the number of potential participants in your project who were leaders in turnaround schools before beginning the project. If you did not begin a cohort for one of the time periods listed, place a check in the first column.

Cohort beginning	Not applicable	Applied	Accepted	Began program	Completed program	Was employed as a turnaround leader within 1 year after project completion	Remained leader of a turnaround beyond the initial year of placement
Fall 2014
Spring 2015
Summer 2015
Fall 2015
Spring 2016
Summer 2016
Fall 2016

I-8. What percentage of project completers are placed in turnaround schools within 1 year of completion? ____ %

I-9. What is the expected duration of the training and development for:

A participant aspiring to be a turnaround school principal or assistant principal	_____ months
A participant who entered as a school leader, but was not leading a turnaround school	_____ months
A participant who entered as a leader of a turnaround school	_____ months

I-10. How much is the total tuition or fee for one candidate to complete the turnaround leader training? (If there is no tuition or fee, please enter “0”).

$ ___________ <Skip to Question I-12 if value=0>

I-11. Who typically pays each participant’s tuition? (Check all that apply)

• Federal grant funds

• Participant

• School district

• Other ____________________________________________________

I-12. About how much training time does your project allocate to each of the following topics when preparing TSLP participants?

Topic	Much less than most other topics	Somewhat less than most other topics	About the same as most other topics	Somewhat more than most other topics	Much more than most other topics
Developing/communicating a school vision or strategy
Developing a positive school culture
Teacher recruitment, selection, retention
Monitoring/evaluating instruction
Selecting instructional models or curricula
Culturally relevant/responsive pedagogy
Use of data for instructional and project decisions
Parent communication/engagement
Community engagement/partnership
Communication/interpersonal skills
Student behavior/engagement
Resource allocation and budgeting
Technology use
Other______________________________
Other___________________________

I-13. Is graduate coursework required as part of the leadership preparation training in your TSLP project (NOT including on-site internship/residency)?

• Yes

• No <Skip to Question I-15>

I-14. How often are the following learning practices/instructional strategies used as part of the coursework in your TSLP project’s leadership training?

Practice/strategy	Not used	Used once	Used 2 or 3 times	Used 4 times or more
Field-based projects in which aspiring turnaround leaders apply ideas in the field
Analysis and discussion of field-based problems or problem-based- learning approaches
Action research or inquiry projects
Analysis and discussion of case studies
A portfolio demonstrating aspiring turnaround leaders’ learning and accomplishments

I-15. Do aspiring turnaround leaders participate in a supervised internship/residency (either in the job they held on entry or in a new position) as part of their leadership preparation training?

(Note: by “residency/internship” we mean the time that aspiring school leaders spend practicing leadership skills in a school setting as part of the requirements of the project. This could include activities added on to the participant’s full time job as a teacher, instructional coach, assistant principal, or placement in a new setting for an extended period (often a full school year) as an apprentice leader.)

• Yes

• No <Skip to Question I-20>

I-16. How long is the supervised internship/residency?

• Less than half of a school year

• Half of a school year

• One school year

• More than one school year but less than two

• Two school years or more

I-17. Which of the following best describes how the internship/residency is structured?

• Participants remain in their current positions for the duration of the residency or internship

• Participants are placed at a different school for part of the residency or internship period

• Participants are placed at a different school for all of the residency or internship period

• Other (specify____________________________________________________________)

I-18. Does the supervised internship/residency experience include:

	Yes	No
Supervision and assistance by a mentor or coach knowledgeable about school turnaround?
Responsibilities for leading, facilitating, and making decisions typical of a turnaround leader?
Regular evaluation of performance by project staff?
Opportunities to network with other interns/residents?

I-19. If the internship or residency includes a mentor or coach, about how often do mentors or coaches:

Activity	Not expected	Weekly	Twice a month	Monthly	Several times per year	Once per year
Maintain regular contact with the participants?
Provide social/emotional support?
Suggest resources, strategies, or techniques to improve performance?
Observe the leaders performing leadership activities (e.g., conducting staff meetings, parent meetings, teacher evaluations)?
Advise/assist participants with work assignments?
Model appropriate leadership behavior?
Provide input into a summative evaluation of the participants’ performance?

Placement

I-20. When are aspiring turnaround leaders (those not currently leading a SIG, SIG-eligible, or turnaround school) matched with a leadership position in a turnaround school?

1. Not applicable; our project includes only leaders who currently lead turnaround schools <Skip to question I-23>

2. Before they apply to the project (e.g., applicant or sponsor designates school of placement)

3. At the time they apply to or are accepted into the turnaround leader training

4. During their training

5. After the completion of their training

6. Other (Please describe in text box below.)

I-21. Are turnaround school staff involved in any of the following stages of the placement process?

	Yes	No
Initial review of candidate qualifications
Candidate interviews
Discussions about who should be hired or placed
Making the recommendation to the appointing authority about who should be hired or placed

I-22. In general, how important is each of the following when making placement decisions?

	Not important	Somewhat important	Important	Very important
Prior experience in the potential placement school
Prior experience in a similar school
Performance during the development project
Prior experience in the placement district
Experience/expertise with specific school improvement/turnaround strategies
Other forms of leadership experience
Other ___________________________
Other ___________________________

Evaluation of Turnaround Leaders

I-23. To evaluate participants, does your project use: (Select NA in the appropriate rows if your project has no participants to whom the column applies.)

	For participants who are currently school leaders	For participants who are not currently school leaders
	Yes No NA	Yes No NA
Course grades?
Observation of practice by mentor, supervisor, or project staff?
Self-assessment?
Review of artifacts such as school improvement plans, budgets, or meeting agendas?
Student surveys?
Staff surveys
Parent surveys
Student achievement growth
Other student outcomes
Other ______________________________

Ongoing Support for Leaders

I-24. After completion of the turnaround leader training, does your project provide any of the following supports to the TSLP participants who are placed as turnaround school leaders?

	Yes	No
Periodic group meetings or seminars with other turnaround leaders
Assistance with work products like school improvement plans, budgets
Assistance with individual professional development planning
Individual mentoring or coaching

<If respondent selects “Yes” to I-24d, present questions I-25 and I-26; if respondent selects “No” to I-24d, skip to question I-27>

I-25. How long after completing the turnaround leader training is the mentor or coach available to work with the placed leaders?

• Less than 1 year

• 1 year

• 2 years

• More than 2 years

• As long as the leader remains in a turnaround school

I-26. About how often do mentors or coaches:

Activity	Not expected	Weekly	Twice a month	Monthly	Several times per year	Once per year
a. Maintain regular contact with the participants?
b. Provide social/emotional support?
c. Suggest resources, strategies, or techniques to improve performance?
d. Observe the leaders performing leadership functions (e.g., conducting staff meetings, parent meetings, teacher evaluations)?
e. Advise/assist participants with work assignments?
f. Model appropriate leadership behavior?
g. Provide input into a summative evaluation of the participants’ performance?

Incentives (Financial and Nonfinancial)

I-27. Are any financial incentives provided for:

	Yes	No
Participating in the program?
Being placed in a turnaround school on completion of the preparation?

<If the respondent says yes to I-27b, present I-27c>

<If the respondent says no to both I-27a and I-27b,skip to I-29>

I-27c. Does your project provide leaders placed in turnaround schools with:

	Yes	No
A one-time bonus upon being placed or hired as a leader?
A special salary increase for taking a job as a turnaround leader?
A retention bonus or stipend after each year of leading a turnaround school?
A performance-based bonus or stipend specifically for leaders of turnaround schools?
A performance-based bonus or stipend similar to that for which other school leaders in the district would be eligible?
Other (please describe)

I-28. If project participants receive financial incentives, who pays the cost? (Check all that apply)

• The project, through federal funds

• The project, through other grant funds

• The district

• The state

• A private foundation

• Other _______________________________________________________

I-29. Are any nonfinancial incentives provided or available for:

	Yes	No
Participating in the program?
Being placed in a turnaround school on completion of the preparation?

<If respondent selects yes to a or b above, show text box> Please list the non-financial incentives.

2. Partner Roles

II-1. Which of the following roles does your organization play in the project? (Please check one box in each row.)

Project function	Role
	Not involved	Participate in discussions about this function	Assiste other partners with primary responsibility	Responsible for this function equally with other partners	Primarily responsible for function with help from others	Solely responsible for this function
Conceptualizing the overall project
Identifying turnaround leadership skills or competencies
Recruiting leaders or aspiring leaders
Developing selection criteria for accepting applicants
Making the decision to select an applicant into the turnaround leader training
Developing training content
Delivering training
Providing coaching or mentoring to project participants
Developing criteria for assessing participant performance
Evaluating participant performance
Deciding who will be graduated from the training
Providing ongoing support for placed leaders
Deciding how to measure the success of the TSLP project
Evaluating the overall success of the TSLP project

II-2. Is there a memorandum of understanding (MOU) or other written agreement documenting the roles and responsibilities of each partner organization?

• Yes

• No

• Other _______________

II-3. What were your two biggest successes in making your partnership work?

Text box for open-ended response

II-4. What were your two biggest challenges in making your partnership work?

Text box for open-ended response

3. Project Changes and Sustainability

III-1. Has your TSLP project changed in any of the following areas since you received the grant?

1. Recruitment of current or potential leaders

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
a.1 Developed better methods
a.2 Change in state or district policies or priorities
a.3 Changes in capacity or commitment of other partners
a.4 Input from project participants
a.5 Financial constraints
a.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
a.7 Other (please describe) _________________

2. Methods of selecting project participants

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
b.1 Developed better methods
b.2 Change in state or district policies or priorities
b.3 Changes in capacity or commitment of other partners
b.4 Input from project participants
b.5 Financial constraints
b.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
b.7 Other (please describe) _________________

3. Content of turnaround leadership professional development

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
c.1 Developed better methods
c.2 Change in state or district policies or priorities
c.3 Changes in capacity or commitment of other partners
c.4 Input from project participants
c.5 Financial constraints
c.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
c.7 Other (please describe) _________________

4. Mentoring or coaching provided to current or potential leaders

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
d.1 Developed better methods
d.2 Change in state or district policies or priorities
d.3 Changes in capacity or commitment of other partners
d.4 Input from project participants
d.5 Financial constraints
d.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
d.7 Other (please describe) _________________

5. Incentives for current or potential leaders to participate

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
e.1 Developed better methods
e.2 Change in state or district policies or priorities
e.3 Changes in capacity or commitment of other partners
e.4 Input from project participants
e.5 Financial constraints
e.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
e.7 Other (please describe) _________________

6. Change in the districts or schools being served

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
f.1 Developed better methods
f.2 Change in state or district policies or priorities
f.3 Changes in capacity or commitment of other partners
f.4 Input from project participants
f.5 Financial constraints
f.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
f.7 Other (please describe) _________________

7. Changing professional development or support providers

• Yes

• No

• Don’t know

If yes, how important were the following?

Potential reason	Not a factor	Contributing factor	Main factor
g.1 Developed better methods
g.2 Change in state or district policies or priorities
g.3 Changes in capacity or commitment of other partners
g.4 Input from project participants
g.5 Financial constraints
g.6 Change in the demand for turnaround leaders (e.g., more or fewer vacancies than expected)
g.7 Other (please describe) _________________

8. Use the text box below to describe any other major changes not covered above and the reasons the changes were made

Text box for open-ended response

III-2. Do you expect to the partners to continue to work together in any of the following ways after the end of the grant? (Please check all that apply.)

• To identify or recruit potential turnaround leaders

• To provide input on the training/preparation needs of turnaround school leaders

• To select or place leaders for turnaround schools

• To provide training or development opportunities specifically targeted to turnaround school leaders

• To provide coaching or mentoring support to turnaround school leaders

• To help districts assess the training/development needs or performance of turnaround school leaders

• Other (Please describe) _________________________________________________

III-3. What aspects of your project do you expect to continue after the end of the grant? (Check all that apply.)

• The methods of recruiting turnaround leaders

• The methods of selecting potential or current leaders for training

• The training content

• Mentoring or coaching of aspiring or placed leaders

• The methods of evaluating the performance of aspiring or placed leaders

• Incentives for recruiting or retaining placed leaders used by the project

Measuring the Success of Your TSLP Project

IV-1. In addition to your project’s GPRA indicators, are you using any locally developed metrics to measure your project’s performance?

• Yes

• No (skip to Question IV-3)

IV-2. Which of the following are included in your locally developed metrics?

	Yes	No
Percentage of participants receiving license, certification, endorsement, or degree
Performance of leaders on competency assessments or performance evaluation instruments designed for the project
Performance of placed leaders as evaluated by state/district school administrator evaluation process (i.e., ratings of performance or effectiveness)
Participants’ perceptions of the project
Supervisors’ perceptions of participants
School climate/culture in leaders’ schools
Achievement gaps in leaders’ schools
Retention of participants remaining as a leader of a high-need or SIG school after training or placement
Other _______________________________________________
Other ________________________________________________

IV-3. Who is responsible for analyzing and summarizing data measuring the performance of your project? (Check all that apply.)

• Project staff

• Development providers

• Partner districts

• An external evaluator

• Other __________________________________________________________